Probe0 vs Docker Model Runner
Docker Model Runner runs models on your machine. Probe0 sits in front of every coding agent on your machine and decides which model each request should go to, then records what it cost. They solve adjacent problems, and they compose better than they compete.
A local model serving and distribution layer bundled with Docker Desktop, running GGUF and safetensors models on host hardware behind an OpenAI-, Anthropic-, and Ollama-compatible API.
| Feature | Probe0 | Docker Model Runner | Edge |
|---|---|---|---|
| What it is | Local LLM proxy for coding agents | Local model serving and distribution runtime | Even |
| Runs the inference engine | No, assumes Ollama or LM Studio is already up | Yes: llama.cpp, vLLM, vllm-metal, GPU detection, idle unload | Competitor |
| Local LLM routing with cloud fallback | Routes to a locally loaded model, retries weak answers on the cloud | Routes by model format to the right engine; no provider fallback | Probe0 |
| Cheap-model-first tiering on hosted APIs | Cheap model first, expensive one only when needed | No hosted provider path, so no tiering | Probe0 |
| Response caching | Exact cache on local disk; hits never touch the network | Caches model weights as OCI artifacts, not responses | Probe0 |
| Repeat work that is worded differently | Local vector index, strict similarity floor, refuses tool-call traffic | No semantic layer | Probe0 |
| Spend caps and LLM cost control | Hard cap per run and per day: warns, then pauses | None; local inference is free so there is nothing metered | Probe0 |
| Which process spent the money | Ledger of model, tokens, real cost, latency, calling process | Prometheus /metrics endpoint and a raw request dump | Probe0 |
| Setup for one developer, every agent CLI | One proxy plus one certificate; Claude Code, Codex and Cursor covered | Settings toggle in Docker Desktop, then a base URL per tool | Probe0 |
Docker Model Runner ships the real thing on the serving side: llama.cpp for GGUF, vLLM for safetensors, vllm-metal on Apple Silicon, engine selection by model format, GPU detection, idle unload, weights packaged as OCI artifacts you can pin by digest, and a Compose provider service so an integration test gets deterministic local inference. That is a different job from the one that costs you money. If you run coding agents on a Mac, the spend is in hosted API calls, and Probe0 is the layer that sees them: one proxy and one certificate installed once, after which Claude Code, Codex and Cursor all route through it with nothing configured per tool. Every module switches on and off independently and reports what it saved, so you can judge each one on its own numbers instead of on a marketing claim.
My cost problem was never local inference, which is free. It was the Claude Code session that quietly burned through a day of budget refactoring a test file, and the Codex run that re-asked the same question eleven times because a retry loop looked stateless from the outside. Docker Model Runner has no answer there, deliberately: no spend cap, no per-run budget, no response cache, no semantic cache, no cost attribution, no routing or fallback across hosted providers. Its observability story is a Prometheus endpoint and a raw request dump, both useful for debugging what an agent actually sent, neither of them an answer to how much yesterday cost and which process spent it.
Probe0 answers that on the axis Docker Model Runner leaves empty. Local Routing sends work to a model already loaded in Ollama or LM Studio and retries weak answers on the cloud. Model Tiering tries the cheap model first. The Exact Cache lives on local disk and a hit never touches the network, and the Semantic Cache holds a strict similarity floor and refuses to match anything carrying tool calls, because a wrong hit in the middle of an agent run corrupts the run. Request Coalescing collapses simultaneous identical calls into one upstream call. Spend Guard warns and then pauses at a hard cap rather than mailing you a notice after the fact. Recording keeps the ledger the metrics endpoint does not: model, tokens, real cost, latency, and which process made each call, which is also how Probe0 can tell you when you are paying for a plan tier above what you actually use. All of it stays on the machine. There is no Probe0 server for it to go to.
The verdict
Pick Probe0 if the models already run and the money is going to hosted APIs through your coding agents on a Mac. One proxy and one certificate cover Claude Code, Codex and Cursor, cache hits stay off the network, the cap pauses instead of alerting, and the ledger names the process that spent each dollar. The one case where Docker Model Runner is the better pick is when you need the engine itself: weights pinned by digest in a registry, a model declared in Compose, inference on Windows, Linux, CI or Kubernetes. Run both if that is you, and let Probe0 decide when a request should go to the local model instead of a frontier API.
Questions
- Is Docker Model Runner an LLM proxy?
- Not in the gateway sense. It serves models locally behind an OpenAI-, Anthropic- and Ollama-compatible API, and it routes between inference engines based on model format, but it does not proxy third-party provider APIs, do fallback chains, or make cost-based model choices.
- Does Docker Model Runner have spend limits or cost tracking?
- No. There is no budget, quota, virtual key or per-user cost attribution in it, because its cost position is that local inference is free. Docker Offload compute is metered by Docker as a platform feature, not by Model Runner. If you need LLM cost control over hosted API spend, that has to come from somewhere else.
- Can I use Probe0 and Docker Model Runner together?
- Yes, and it is the setup I would suggest. Docker Model Runner serves the local model on your machine; Probe0 sits in front of your AI coding agents, decides which requests go local versus cloud, caches what repeats, and logs the real cost of what is left.
- What is the best Docker Model Runner alternative for controlling coding agent costs?
- They are not really substitutes. If your goal is local LLM routing plus cost control across Claude Code, Codex and Cursor on a Mac, Probe0 is built for exactly that. If your goal is running and shipping model weights across machines and CI, nothing on the proxy side replaces Docker Model Runner.