Probe0 vs AnythingLLM
AnythingLLM is an application you talk to. Probe0 is a local LLM proxy that sits underneath the coding agents you already use. If you want a chat window with your documents in it, that is AnythingLLM. If you want Claude Code and Codex to cost less without changing how you work, that is Probe0.
An open-source (MIT) all-in-one AI application from Mintplex Labs: document ingestion, RAG, workspaces, a chat UI, and agents wrapped around someone else's inference.
| Feature | Probe0 | AnythingLLM | Edge |
|---|---|---|---|
| Coding agent CLIs you already run | Claude Code, Codex and Cursor route through it with no per-tool config | Its own chat UI and REST API; agent CLIs do not benefit | Probe0 |
| Setup for one developer | One local proxy and one certificate, installed once | Desktop app, Docker server, or hosted cloud, then per-workspace setup | Probe0 |
| Response caching | Exact cache on local disk plus a semantic cache with a strict similarity floor | Router classifier cache only; general response caching not shipped | Probe0 |
| Cache safety during an agent run | Semantic cache refuses to match anything carrying tool calls | No prompt-level cache, so nothing to match against | Probe0 |
| Spend control | Hard cap per run and per day: warns, then pauses | Daily message count limit per user; no dollar budgets | Probe0 |
| Cost attribution | Ledger records model, tokens, real cost, latency, calling process | Per-chat token and speed metrics, audit logs in multi-user mode | Probe0 |
| Knowing what each feature saved | Every module toggles independently and reports its own savings | Token data tracked internally, not surfaced as savings | Probe0 |
| Local model support | Routes to Ollama or LM Studio, retries weak answers on the cloud | Bundled local engine plus Ollama, LM Studio, LocalAI, llama.cpp GGUF | Even |
| Chat over your own documents | None of this, by design | Ingestion, chunking, citations, bundled vector DB, agents, MCP | Competitor |
AnythingLLM is the better product for document chat, and I will not argue that point: ingestion, chunking, a bundled vector database, citations, an agent flow builder and MCP support, all MIT licensed and free for desktop and self-host. What it does not do is make the coding agents on your machine cheaper, and that is the job I care about. A prompt from Claude Code never enters AnythingLLM, so none of its 30-plus provider integrations, sticky sessions or fallback rules touch the traffic that actually drains a credit balance. Probe0 works the other way round: one local proxy and one certificate, then every agent CLI on the Mac routes through it without being told to. No prompt leaves the laptop unless it was already headed to a provider, because there is no Probe0 server to send it to.
The gap is what happens to the money. AnythingLLM has no dollar budgets, no per-key spend ceiling and no cost attribution. The control it does have is a daily message count per user, which is a throttle on messages rather than on spend, and the self-hosted build has no usage caps at all by design. Token data is tracked internally but not surfaced, which is what the open issues asking for per-user token visibility are about. There is no general response cache either: the caching in the Model Router exists to avoid re-running the classifier, and KV or cache-augmented generation is an unshipped community request. This is why integration pages for Portkey, TrueFoundry and LiteLLM exist alongside it. Governance gets delegated to a gateway you run separately, which is a reasonable architecture for an application and a strong reason to have something like Probe0 underneath it.
I built Probe0 for the specific problem of an agent CLI burning through credits while I am not watching it. Each module switches on and off independently and reports what it saved: local routing to a model already loaded in Ollama with automatic cloud retry when the answer is weak, a cheap model tried first, an exact cache on local disk whose hits never touch the network, a semantic cache that refuses to match anything carrying tool calls because a wrong hit corrupts a whole run, coalescing of simultaneous identical calls, a spend guard that warns and then pauses at a hard cap, and a ledger recording which process made each call and what it actually cost. Because it knows real usage it will tell you when your subscription tier sits above what you use. The honest limits: macOS only, private beta, sign-in is Google or GitHub with a new account system, no team gateway, and provider coverage is what coding agents call rather than a 100-provider catalog.
The verdict
Install AnythingLLM if your actual need is a chat product with your documents in it, with citations, agents and MCP in one place; nothing in Probe0 replaces that. For everything downstream of a coding agent, Probe0 is the pick: it is the only one of the two that caches your prompts, routes them to a local model, records the real cost per process and stops spending when the cap is hit. Run both if you like. The one that changes your bill is Probe0.
Questions
- Is Probe0 an AnythingLLM alternative?
- Only for the cost and routing part. AnythingLLM is an application with a chat UI, document RAG and agents; Probe0 is an LLM proxy with no UI for end users. If you came to AnythingLLM for document chat, Probe0 does not replace it.
- Does AnythingLLM have spend limits or budgets?
- Not in dollar terms. It offers an admin-set daily message limit per user, which caps message count rather than cost, and self-hosted installs have no usage caps by default. Probe0 enforces a hard cap per run and per day, warning first and then pausing.
- Can AnythingLLM cache responses to cut token spend?
- There is a classification cache inside the Model Router so the classifier is not called on every message, but no general response or semantic cache for user prompts. Probe0 runs an exact cache on local disk and a local vector semantic cache with a strict similarity floor.
- Will either one make Claude Code or Codex cheaper?
- Probe0 will, because those CLIs route through it once the local proxy and certificate are installed and it applies caching, local LLM routing and model tiering to their traffic. AnythingLLM is a destination you send requests to, so an agent CLI pointed at a provider gets no benefit from it.