Using Ollama with Claude Code and Cursor to cut cloud costs

Short answer: you can send a meaningful share of coding-agent traffic to a model already running in Ollama, and the share that works is the boring share. Commit messages, file summaries, naming, single-file explanations, quick classification, the small "is this relevant" judgments an agent makes dozens of times per run. Long multi-step tool loops, large repo-wide refactors, and anything where a wrong answer sends the agent down a bad path are still worth cloud money. Route by task shape, not by wholesale switching your default model.

The reason this matters is that an agent run is not one request. It is a chain, and the chain is mostly small calls. If you look at a real ledger of what Claude Code or Cursor actually sent during an afternoon, the token-heavy calls are the ones you remember and the request-heavy calls are the ones you forgot happened. The forgotten ones are the ones a local model can absorb.

What local models are actually good at inside an agent

I spent a while being wrong about this. My first attempt was pointing Claude Code entirely at a local endpoint, watching it produce a plausible plan, and then watching it fail to hold a tool-calling loop together over fifteen steps. Local coding models have improved a lot at emitting well-formed tool calls, but the failure mode is not "produces bad JSON". The failure mode is that the model loses the thread of a long trajectory: it re-reads a file it already read, forgets a constraint from step three, or declares success without running the test. Each individual response looks fine. The run does not converge.

So the useful split is by how much the answer is load-bearing for the next step. Things I have found reliably local-friendly:

• Commit message and PR description drafting from a diff you already have in hand
• Summarising a single file or a log dump so a bigger model does not have to read all of it
• Classification: is this file relevant to the task, is this error a build error or a test failure, does this changed line touch public API
• Naming, docstrings, comment cleanup, small mechanical rewrites where you will read the output anyway
• Repeat questions you have already asked this week in slightly different words
• Anything you would have grepped for if the agent had not been there

Things I stopped sending local: multi-file edits with cross-file invariants, anything where the model is choosing which tool to call next in a long chain, and any step whose output is silently consumed rather than read by a human. The last one is the real rule. If nobody looks at the answer before it becomes input to the next step, a mediocre answer is worse than an expensive one.

The mechanics, briefly

Ollama serves an OpenAI-compatible API on localhost, which is why this is even possible without writing an adapter per tool. LM Studio does the same. Cursor lets you point at a custom OpenAI-compatible base URL, and Claude Code respects a base URL environment variable, so in principle you can flip either one at a local server and watch what happens.

In practice that all-or-nothing flip is the thing that makes people give up on local models after an afternoon. You get one switch with two positions: everything cloud, or everything local. Everything local is too weak for the hard steps. Everything cloud is what you were already paying for. There is no position on that switch for "the easy 60 percent local, the hard 40 percent cloud, and put it back on the cloud automatically if the local answer is junk."

There is also a hardware reality nobody enjoys stating. Keeping a capable coding model resident costs you RAM you were using for your build, your simulator, and forty browser tabs. If you have to cold-load the model on every request, the latency wipes out the appeal for exactly the small fast calls that were supposed to be the win. Local routing is worth doing when the model is already loaded and already warm. If you are booting Ollama to answer one question, you did not save anything, you just moved the cost to your own wall clock.

Quality control is the whole problem

Any local-routing scheme lives or dies on what happens when the local answer is bad. If your setup silently returns a weak answer into a running agent, you will spend more time untangling the run than you saved on tokens, and you will conclude local models are useless when what was actually useless was the routing.

So the design constraint is: detect a weak local answer and retry it on the cloud, before the agent acts on it. Refusals, truncation, empty or degenerate output, malformed tool calls, answers that ignore the requested format. Catching those cheaply is not glamorous work but it is the difference between local routing being a real saving and being a science project. When retry-on-cloud works, the worst case is you paid for one local inference you threw away, which is free, plus the cloud call you were going to make anyway.

The second constraint is measurement. If you cannot say what the local route saved this week, you cannot defend keeping it on, and you will quietly turn it off the first time something feels slow.

Where Probe0 fits

I built Probe0 because I wanted the split without maintaining a per-tool config. It is a local proxy that runs on your machine, with a certificate installed once, and every coding agent CLI on the machine routes through it: Claude Code, Codex, Cursor. There is no Probe0 server, so none of this is a question of trusting me with your prompts.

Local Routing is one module of several, and it does the two things above. It sends eligible work to a model already running in Ollama or LM Studio, and it retries weak answers on the cloud rather than letting them into your run. Each module is switchable on its own and reports what it saved, so if local routing is not earning its keep for your workload you will see that instead of guessing.

The other modules matter here because local routing is not the only way to avoid a cloud call. Exact Cache answers repeats from local disk without touching the network. Semantic Cache does the same for near-duplicates with a strict similarity floor, and it refuses to match anything carrying tool calls, because a wrong cache hit inside an agent run corrupts the run in a way that costs more to debug than the call cost to make. Request Coalescing collapses simultaneous identical calls into one upstream. Model Tiering tries a cheap model before an expensive one. Spend Guard is a hard cap per run and per day that warns and then pauses. Recording keeps a full ledger: model, tokens, real cost, latency, and which process made each call, which is how you learn that the process burning your budget is not the one you assumed.

The ledger is the part that changed my behaviour, not the routing. Seeing which process made each call is how you find out your money is going somewhere you never looked.

Honest limits. Probe0 is macOS only and in private beta. It is not a team gateway, it does not do multi-user deployment or shared org policy, and provider coverage is what coding agents actually call rather than a catalogue of a hundred providers. If you need a hosted gateway with team controls and provider breadth, tools like LiteLLM, Portkey and OpenRouter are aimed at that and you should use one of them. Sign-in is Google or GitHub, no email and password, and the account system is new.

How to try this without wasting a weekend

Start by measuring before you route. Run a normal day of work with a full ledger on and look at the distribution of calls, not the total. If most of your requests are small and repetitive, local routing has room to work. If your traffic is a handful of enormous context-heavy calls, local routing will barely move the number and caching or tiering is the better lever.

Then turn local routing on for the easy categories only, keep the model warm, and check the retry rate. A high retry rate means the local model is not good enough for the category you assigned it, which is useful information and takes one glance to get. A low retry rate with a real saving means you found the slice, and you can widen it carefully from there.

One last thing worth doing while you are in the numbers: check what plan tier you are actually using. Plenty of people pay for a tier well above their real consumption, and once you know your usage that is a decision you can make in five minutes rather than a vague suspicion you carry around for a year.