Probe0 vs LiteLLM
LiteLLM is infrastructure you deploy for a team: a proxy service with Postgres, virtual keys and per-team budgets. Probe0 is a proxy that runs on your laptop and sits under the coding agents already installed there.
An open-source (MIT) Python SDK and self-hosted proxy server that puts one OpenAI-compatible endpoint in front of 100+ model providers.
| Feature | Probe0 | LiteLLM | Edge |
|---|---|---|---|
| Where your prompts go | Your Mac. No Probe0 server exists. | A service you host and keep running, with Postgres | Probe0 |
| Setup for one developer | One proxy plus a certificate, once. Claude Code, Codex and Cursor route through it with no per-tool config. | Deploy the proxy, mint a key, repoint each tool | Probe0 |
| Providers behind one schema | What coding agents actually call | 100+ providers, one OpenAI-compatible interface | Competitor |
| Local model routing | Ollama and LM Studio first, weak answers auto-retried on the cloud | Can route to Ollama, vLLM, LM Studio | Probe0 |
| Cache hits and the network | Exact hits read from local disk and never leave the machine | Response caching, typically via Redis | Probe0 |
| Semantic cache on agent traffic | Local vector index, strict similarity floor, refuses to match anything carrying tool calls | Similarity caching offered as a cache mode | Probe0 |
| Who made the call | Every call attributed to the process that made it | Attributed to the virtual key you issued | Probe0 |
| Spend limits | Hard cap per run and per day: warns, then pauses | Per-key, per-team budgets with hard stop | Even |
| Knowing what saved you money | Each module toggles independently and reports its own savings | Spend totals via Langfuse, Prometheus, OTel | Probe0 |
LiteLLM normalises 100+ providers behind one OpenAI-compatible schema, and nothing here disputes that. For a developer whose spend comes out of coding agents on a laptop, though, the schema is not the problem being solved. My problem was that every coding agent CLI on my machine was burning cloud tokens on work a 7B model already loaded in Ollama could have answered, and I had no per-process record of any of it. Probe0 runs locally, intercepts what those agents send, and applies modules I can switch on and off one at a time: Local Routing sends work to a model already running in Ollama or LM Studio and retries weak answers on the cloud, Model Tiering tries the cheap model first, an exact cache reads from local disk without touching the network, Request Coalescing collapses simultaneous identical calls into one upstream, Spend Guard warns then pauses at a cap per run or per day, and Recording keeps a ledger of model, tokens, real cost, latency and the originating process.
The caching difference is the one I would look at hardest. The semantic cache uses a local vector index with a strict similarity floor and refuses to match anything carrying tool calls, because a wrong hit inside an agent run corrupts the run rather than merely returning a mediocre paragraph. A general-purpose gateway caching by similarity has no reason to know that a request with tool definitions attached is a different kind of object. Every module also reports what it individually saved, so turning one off answers whether it was earning its place. LiteLLM plugs into Langfuse, Prometheus and OpenTelemetry, which is how a platform team watches a fleet; it gives you totals, not a per-module verdict on your own machine.
The practical difference is the setup gradient. Getting LiteLLM in front of your coding agents means running a service, standing up Postgres, probably Redis for caching, minting a key, and repointing each tool at your base URL. Probe0 is one local proxy plus a certificate installed once, after which Claude Code, Codex and Cursor route through it without per-tool configuration. Because the ledger knows what you actually consume, Probe0 also tells you when your plan tier sits above your usage, a question a shared gateway cannot answer per person. Sign-in is Google or GitHub, no password. The honest constraints: macOS only, private beta, an account system that is new, not a team gateway, and provider coverage aimed at what coding agents call rather than a full catalogue.
The verdict
Run LiteLLM when more than one person shares the gateway or the traffic comes from an application you are shipping across many providers; that is a real deployment and Probe0 is not it. Everything else about this comparison points the other way. If you are one developer on macOS and the spend is coming from Claude Code, Codex and Cursor on your own machine, Probe0 is the one to install: a proxy and a certificate instead of a service with a database, prompts that stay on the laptop, a semantic cache that knows to leave tool calls alone, a cap that pauses the run, and a ledger that names the process that spent the money. Start with Probe0, and stand up LiteLLM later if product traffic ever needs it.
Questions
- Is Probe0 a LiteLLM alternative?
- For one developer routing AI coding agent traffic on a Mac, yes. For a team gateway with virtual keys, per-team budgets and 100+ providers, LiteLLM is the better fit and Probe0 does not try to replace it.
- Can LiteLLM route to local models like Ollama?
- Yes. LiteLLM can point at Ollama, vLLM and LM Studio through the same OpenAI-compatible interface. Probe0 goes further on this one axis: it tries the model already loaded locally first and automatically retries weak answers on the cloud, so local LLM routing does not cost you output quality.
- Do I need Docker or a database to use Probe0?
- No. Probe0 runs entirely on your machine with no server component and no Postgres or Redis. Setup is one local proxy and a certificate installed once, after which every coding agent CLI on the machine routes through it.
- How does LLM cost control differ between the two?
- LiteLLM enforces budgets per virtual key or team at the gateway, which is how you control spend across many people. Probe0 enforces a hard cap per run and per day on one machine, warning first and then pausing, and its ledger attributes every call to the process that made it.