OSS API traffic visibility in K8s clusters

Full disclosure - I am a Tyk employee, and using Tyk API Gateway in this blog

APIClarity is “an open source cloud native visibility tool for APIs”. What does that mean though? You mean the super-long title isn't clear enough?!

API Clarity is an app that monitors REST API requests in your Kubernetes cluster and compares them against your OAS (OpenAPI specification) documentation, and then reports discrepancies. ie. if your APIs are being used in ways that are NOT expected.

If your APIs do not have OAS documentation, APIClarity generates OAS documentation by aggregating API metadata based on consumption.

Let's play with Tyk OSS + APIClarity and see what we get.

Hands On

For this, we'll be using a GitHub repo that deploys OSS Tyk & APIClarity in a few commands. (If you'd like to try the integration step-by-step, check out the APIClarity documentation here)

We follow the README to run Tyk & APIClarity
port-forward to both services so we can access them locally.
Now, let's generate some API requests against Tyk using the included script:

$ ./generate-apicalls.sh
Generating load against two APIs.
Finished generating load against two APIs.

Open up APIClarity on our browser via http://localhost:9999

We see that Tyk has reported telemetry data to APIClarity from two APIs.

Let's upload the included OAS spec in “./oas/jsonplaceholder.json” into APIClarity

This loads all our endpoints for this API into APIClarity. APIClarity will now compare this with realtime API traffic and alert us to unusual behaviour.

And just like that, we added anomaly detection to our APIM.

APIClarity, like all good OSS software, is aligned with OSS standards.

This means that your teams are being rewarded for using best-practice OSS standards such as OAS for your REST APIs documentation.

OAS Reconstruction

Let's pretend we don't have an OAS spec for HTTPBin.

Here's how we can have APIClarity generate OAS documentation from our API traffic:

That's all we had to do to generate OAS documentation! We can now visit the Swagger sandbox right from the UI and view it in real-time.

API Clarity even generated our Response objects and the field types

Real-time Anomaly Detection

Now, let's re-generate more API requests and see what insights APIClarity gathers for us.

$ ./generate-apicalls.sh
Generating load against two APIs.
Finished generating load against two APIs.

After refreshing the APIClarity page, we see some alerts:

Here we can see that APIClarity has identified 5 API requests that fall outside of the OAS spec for the Jsonplaceholder API. We can even click into an item for details on that request:

All-in-all, very powerful stuff.

Conclusion

APIClarity delivers pretty powerful features with a very small footprint by sitting inside our k8s cluster and giving us more visibility and insight into our API traffic.

Moreover, an insider scoop tells me that the APIClarity functionality will be expanded with more testing and analytics tools that have their basis in OWASP API top 10 issues.

From an AppSec or SecOps perspective, this is quite low-hanging fruit!

Keeping in mind that the API Gateway pattern involves all traffic passing through a singular “front-door” into your ecosystem, you've added another layer of protection globally by integrating APIClarity with Tyk. Another bodyguard to bolster the ones already in place!

Though, am I telling you to RUN and install this in your production environments? Of course not. There are many factors to consider, such as:

How is the telemetry data handled?
Is it blocking or non-blocking?
How does that affect latency?
Is this set up compliant with highly regulated environments?

And so on.

This is intended to get the brain juices running on an otherwise very lightweight and powerful piece of Open Source Technology, something I'm a huge advocate for.

Is this something you would use?

-Sedky