When is a local model good enough for coding, and when isn't it

Short answer: a local model is good enough when you can verify its output faster than you could have written the code yourself, and the blast radius of a wrong answer is small. It is not good enough when the task needs multi-file reasoning, touches an unfamiliar API surface, or the cost of a subtle bug slipping through is higher than the time you saved routing it locally. That is the whole rule. Everything below is just how to apply it without guessing.

Most advice on this topic is a benchmark table: model X scores Y on HumanEval, model Z scores Y+2 on SWE-bench. None of that tells you what to do at 2pm on a Tuesday when you are about to fire off a request and have to pick a model in under a second. The benchmarks measure the model. They do not measure your task, your hardware, or what it costs you when the answer is wrong.

The real cost is verification time, not tokens

A wrong answer from a local 8B model costs you almost nothing in dollars. It can still cost you twenty minutes if you don't notice it's wrong until three files later. The number that matters isn't inference cost, it's how long it takes you to confirm the output did what you asked. Autocomplete-shaped work has near-zero verification time because you read the line and either accept or reject it. A refactor that touches a shared interface across twelve files has verification time measured in a full read-through plus running the suite. Route by that number, not by how the task feels in the moment.

Where local models genuinely hold up

• Inline autocomplete and next-edit suggestions where you're going to read every character anyway
• Commit message and PR description generation from a diff you already understand
• Mechanical renames, import fixes, and formatting-adjacent edits with a small, obvious blast radius
• Test stubs and table-driven test skeletons for functions with an unambiguous contract
• Docstring and comment generation for code you just wrote yourself
• Boilerplate CRUD, config file scaffolding, and repetitive glue code you've written a hundred times before

The common thread isn't task category, it's that you already hold the correct answer in your head and the model is just typing faster than you. That is the setting where a 7B-32B class model running on a Mac at good speed wins outright, because the frontier model's extra reasoning depth buys you nothing you weren't going to supply yourself.

Where they fall apart

• Multi-file architectural changes where the model has to hold invariants across files it hasn't all seen
• Anything touching auth, payments, permissions, or other code where a plausible-looking wrong answer is dangerous
• Work against an unfamiliar library or API where the model is likely to hallucinate a method that doesn't exist
• Long-context reasoning across a large diff or a sprawling codebase, past what the local model's effective window actually holds
• Ambiguous specs that need a clarifying question before any code should be written at all

Local models don't fail loudly here. They fail by producing code that compiles, looks reasonable, and is subtly wrong in a way that costs you the twenty minutes mentioned above. That is the actual danger, not that the model refuses or crashes.

A decision rule you can run in your head

Three questions, answered honestly before you send the request:

• Can I verify the output in under a minute? If yes, local is fine even when it's occasionally wrong, because the cost of catching a miss is cheap.
• Is the blast radius small — one file, easily reverted, not on a path to production without review? If yes, local is fine.
• Does the task need context or reasoning depth beyond what the local model can actually hold and use well? If yes, send it to the cloud regardless of the first two answers.

Yes, yes, no: local. Anything else: cloud. The third question overrides the first two on purpose, because a model that can't hold the context can't give you a verifiable answer no matter how cheap verification would otherwise be.

The question was never which model is smarter. It's which model you can afford to be wrong from, on this specific task, right now.

How I built Probe0 around this rule

I built Probe0 because I kept manually eyeballing this decision for every request and getting it right maybe half the time. Probe0 is a local proxy that sits in front of Claude Code, Codex, and Cursor on macOS — one proxy, one certificate installed once, no per-tool config, no server of mine in the loop. Its Local Routing module sends eligible work to whatever you already have running in Ollama or LM Studio, and if the local model's answer looks weak, it auto-retries on the cloud instead of quietly shipping a bad response. That retry path is the part I actually care about: it means you can be aggressive about routing to local without eating the tail-risk of a silently wrong answer sitting in your codebase.

It's not a replacement for the verification-time judgment above. It's a way to stop making that judgment by hand for every single request, and to get a running record — in the Recording module — of what actually got answered locally versus what got escalated, so you can see whether your own routing instincts were right.

The honest ceiling

On consumer hardware — an M-series Mac with 16 to 64GB of unified memory — you're realistically running models in the 7B to 32B active-parameter range at usable speed. That ceiling is real. On hard multi-step reasoning, unfamiliar-API work, or anything requiring genuine architectural judgment, these models lag frontier cloud models by a wide and noticeable margin, not a rounding error. If your task lives in that territory, don't route it locally to save a few cents. Ollama and LM Studio are excellent at what they do — running models fast and simply on your own machine — and neither claims to close that gap either. The rule above exists precisely because there's a large, useful zone where local is not just adequate but strictly better, and a real zone where it isn't, and pretending the second zone doesn't exist is how you end up debugging a hallucinated method call at 11pm.