Probe0 vs Kong AI Gateway
Kong AI Gateway is infrastructure you stand up for an organisation: data planes, Redis, consumer groups, a control plane. Probe0 is a proxy that runs on my laptop and catches what Claude Code, Codex and Cursor spend before it leaves the machine.
An AI plugin layer on top of Kong Gateway that gives organisations one provider-agnostic LLM API with routing, semantic caching, token rate limiting and Prometheus/OpenTelemetry observability, deployed self-hosted or through the Konnect control plane.
| Feature | Probe0 | Kong AI Gateway | Edge |
|---|---|---|---|
| Where prompts and the ledger live | Your Mac. No Probe0 server exists. | Self-hosted data planes, Kubernetes, or Konnect SaaS control plane. | Probe0 |
| Setup for one developer | One local proxy plus a certificate, once. No per-tool config. | Stand up a gateway, add Redis, configure routes and plugins, point each tool at it. | Probe0 |
| Dependencies to operate | None beyond the app itself. | Redis for vector cache and distributed counters; Postgres in the documented quickstart. | Probe0 |
| Semantic cache around tool calls | Strict similarity floor, refuses any match carrying tool calls. | ai-semantic-cache plugin, threshold matching, Redis-backed and shared across nodes. | Probe0 |
| Cache hits that never hit the network | Exact cache on local disk. Hit means no request leaves the Mac. | Cache lookup is a round trip to Redis via the gateway. | Probe0 |
| Local model routing | Ollama and LM Studio first class; weak answers auto-retried on the cloud. | Ollama and vLLM supported; token accounting on streamed responses needs custom handling. | Probe0 |
| Per-process cost attribution | Ledger records which process made each call, with tokens, real cost and latency. | Per-consumer and per-consumer-group attribution, not per local CLI process. | Probe0 |
| Spend cap behaviour | Hard cap per run and per day. Warns, then pauses. | ai-rate-limiting-advanced enforces token budgets per consumer over a window. | Probe0 |
| Teams and multi-tenant governance | None. Single machine, single user. | Consumer groups, per-team budgets, RBAC, SSO, audit logs (Enterprise tiers). | Competitor |
Kong is the right tool when several teams share an LLM budget and someone has to answer for it at an audit: consumer groups, per-team token limits, RBAC, SSO, audit logs, and Redis-backed counters that stay correct across gateway replicas. For one developer on one Mac, none of that is the problem. The problem is that Claude Code, Codex and Cursor generate bursty, repetitive traffic all day and you find out what it cost afterwards. Probe0 installs one local proxy and one certificate, and from then on every agent CLI on the machine routes through it with no per-tool configuration. There is no Probe0 server, so the prompts, the exact cache, the semantic index and the ledger stay on the box, and only calls that genuinely need a cloud provider leave it.
The overhead on the Kong side is real for a solo setup. The documented Docker quickstart brings up a gateway container plus Postgres and expects a licence through Konnect. Semantic caching means running Redis. RBAC, SSO and audit logs sit behind Enterprise tiers, so the upgrade trigger can be a compliance checkbox rather than traffic volume, and Konnect bills per gateway service, where each provider or model you route counts. Analysts at Zuplo and API7 both make the point that the real number lands well above the sticker once infrastructure and the person maintaining it are counted. A single developer pays that in time and gets governance features aimed at an org that is not there.
What I get back from Probe0 is control at the granularity a laptop actually has. Each module switches on independently and reports what it saved: Local Routing sends work to a model already loaded in Ollama or LM Studio and retries weak answers on the cloud, Model Tiering tries the cheap model first, Exact Cache reads from local disk so a hit costs nothing and touches no network, Request Coalescing collapses simultaneous identical calls into one upstream request, Spend Guard warns at a cap per run or per day and then pauses the run rather than emailing me about it. The semantic cache refuses to match anything carrying tool calls, because a wrong hit mid-run corrupts the run, and a cache that quietly breaks an agent is worse than no cache. Recording keeps a full ledger with model, tokens, real cost, latency and the process that made the call, which is also how the plan-tier advice works: it can tell me when I am paying for a tier above what I use. The honest limits: macOS only, private beta, sign-in via Google or GitHub with a young account system, provider coverage scoped to what coding agents call rather than a 100-plus catalogue.
The verdict
Pick Kong AI Gateway in one case: multiple teams or services share LLM spend and you need per-consumer budgets, RBAC, audit logs and someone on staff to run Redis and Kubernetes. Everyone else running coding agents on a Mac should pick Probe0 — local routing, tool-call-safe caching, per-process cost attribution and a cap that pauses instead of alerting, working in about a minute, with nothing to operate and nothing leaving the machine.
Questions
- Is Probe0 a Kong AI Gateway alternative?
- For a single developer running AI coding agents, yes. For multi-team governance with RBAC, SSO, audit logs and distributed rate limiting across gateway replicas, no, and Probe0 does not try to be. Those are different problems.
- Can I run Kong AI Gateway locally on my laptop?
- You can, but the documented quickstart brings up a gateway container plus Postgres, and semantic caching and token rate limiting both need Redis. It is a service you operate, not a desktop app, and there is no per-CLI-process cost attribution for local coding agents.
- Does Probe0 send my prompts anywhere?
- No. Probe0 runs entirely on your machine and there is no Probe0 server. The exact cache is on local disk, the semantic index is local, and the request ledger stays on the machine. Only calls that actually need a cloud provider leave the box.
- How does local LLM routing compare between the two?
- Kong can proxy to Ollama and vLLM, though token accounting on streamed Ollama responses takes custom work because usage arrives as fragments. Probe0 treats Ollama and LM Studio as a first choice, sends work to a model already loaded, and automatically retries weak answers on a cloud model.