How to actually raise your cache hit rate with a coding agent

Short answer, so you can stop reading if this is all you needed: provider prompt caching is prefix caching. The model provider stores a prefix of your request and reuses it only if the next request starts with byte-identical content in the same order. So the levers that raise your hit rate are the ones that keep the front of the request still. Stop editing your agent config mid-session, stop letting tool definitions and MCP servers churn between calls, keep anything volatile (timestamps, git status, random file dumps) out of the system prompt, and finish a task inside one session instead of restarting a fresh one every ten minutes. Everything else is detail.

The second answer is that a chunk of your spend is not cacheable by the provider at all, because it is the same request being asked twice in a slightly different shape, or by two processes at once. That part you fix locally, in front of the provider, which is the part I will get to at the end.

Why a coding agent should have a very high hit rate already

A coding agent is close to the ideal caching workload. Every turn resends the whole conversation: the system prompt, your project instructions, the tool schemas, every file the agent read, every diff it produced. By turn twenty the request is mostly a replay of turn nineteen with a few thousand new tokens on the end. That is exactly the shape prefix caching was built for.

The economics follow from that. On Anthropic's API, writing a cache entry costs a premium over normal input tokens, and reading one costs a fraction of the normal input rate. OpenAI applies a discount to cached input automatically without a write premium. Either way the arithmetic is the same: a long session that keeps hitting is dramatically cheaper per turn than a short session that keeps missing, and a session that thrashes the cache can cost more than not caching at all, because you pay the write premium repeatedly and never collect the read discount.

Anthropic's cache entries live about five minutes by default, refreshed each time they are read, with a longer-lived option available at a higher write price. That five minute number is the one people underestimate. It means the cache is a property of your working rhythm. If you send a turn, read the diff, go make coffee, come back and send the next turn, you paid full price for that turn and paid the write premium again on top.

The things that silently break your prefix

Almost every miss I have investigated came down to something at the front of the request changing when the developer thought nothing had changed. In rough order of how often I see them:

• Editing CLAUDE.md, AGENTS.md, or .cursorrules while the agent is running. These are loaded into the prefix. One character invalidates every downstream token for the rest of the session.
• Adding or removing an MCP server. Tool definitions sit ahead of the conversation, and the set of tools is part of the cached prefix. Turning on a new MCP mid-task is a full cache reset, and a large tool catalog also inflates the prefix you are paying to write.
• Anything time-varying injected into context. A timestamp, a git branch summary, a file listing that includes modified times, an environment block that renumbers on each run. All of them look harmless and all of them guarantee a miss on every single turn.
• Switching models. Cache entries are per model. Bouncing a task between a big model and a small one to save money can cost more than it saves if each switch triggers a fresh cache write of a large context.
• Compaction and context trimming. When the agent summarises and drops the middle of the conversation, the prefix after the summary is new. This is unavoidable and correct, but it is a real cost event, and it is a reason to finish work before the window fills rather than after.
• Starting a new session per question. Every fresh session pays the write price for the full project preamble. Three sessions in one hour on the same repo is three full writes of the same content.

What to actually change

Keep the volatile material at the end. If you want the agent to know the current date or the state of the working tree, put it in your message, not in the persistent instruction file. Message content near the end of the request costs you nothing in cache terms because it sits after the cached prefix.

Keep the instruction file small and stable. A long project instruction file is not free even when it hits, and it is expensive every time you edit it. I treat it like a header file: change it deliberately, in between sessions, not while the agent is mid-run.

Prune the tool surface. Enabling ten MCP servers because they might be useful puts ten schema blobs in front of every request you will ever send. Turn on the ones the current task needs.

Batch your turns. The five minute window rewards conversational momentum. If you know you are stepping away, it is cheaper to end the task cleanly than to leave a half-finished session that you will resume cold ten minutes later at full write price.

Then measure it, because none of this is worth arguing about in the abstract. Every provider returns cache read and cache write token counts on the response. If you are not looking at those numbers per session, you are guessing.

The misses the provider cache cannot catch

Prefix caching only helps when the request continues something the provider already saw. Two other patterns burn money and are invisible to it. The first is repeated identical calls: the same lint explanation, the same commit message prompt, the same small classification the agent runs on every file in a loop. The second is concurrent duplicates, where two agent processes or two parallel subagents fire the same request within milliseconds of each other and both pay in full.

Those are local problems and they want a local answer, which is where Probe0 comes in. It is a proxy that runs on your machine, with a certificate installed once, and every coding agent CLI on the box routes through it. Claude Code, Codex and Cursor all go through the same place with no per-tool configuration.

Three of its modules target exactly this. Exact Cache keeps identical requests on local disk and answers them without touching the network. Semantic Cache uses a local vector index with a strict similarity floor, and it refuses to serve a match for anything carrying tool calls, because a near-miss on a tool-calling turn corrupts the whole run and no cost saving is worth that. Request Coalescing collapses simultaneous identical calls into one upstream request. On top of that, the Recording ledger stores every request with its model, tokens, real cost, latency and the process that made the call, so you can see which agent and which loop is generating the repeats.

The cache you control locally is the one that answers before a packet leaves the machine. The provider cache only helps once you have already decided to pay for the round trip.

Honest limits, because this is not a universal answer. Probe0 is macOS only and in private beta. It is a single-developer tool, not a team gateway, and it covers what coding agents actually call rather than a hundred-provider catalog. If you need shared caching across a team, a hosted gateway like Portkey or Helicone, or a self-hosted LiteLLM proxy, is the right shape for that job and I would point you there without argument.

But if the question is why your solo coding-agent bill is higher than it should be, start with the prefix. Keep the front of the request still, finish tasks in one sitting, and put a local cache in front of the duplicates the provider will happily charge you for twice.