Deduplicating simultaneous identical LLM requests in an agent workflow

If you run a coding agent that spins up parallel subagents, you've probably seen this in your logs: three or four requests with the identical model, identical system prompt, identical messages, firing within milliseconds of each other. A cache doesn't catch this. Caching works by matching a new request against a previous response, but these requests aren't sequential, they're simultaneous. The first one hasn't returned yet when the second, third, and fourth go out. Nothing to match against. All four hit the provider, all four get billed, and you paid for the same answer three extra times.

The fix isn't a bigger cache. It's a coalescing layer sitting in front of the cache that holds a request open, checks whether an identical one is already in flight, and if so, waits for that one to finish and hands back its result to everyone who asked. This is the answer up front: deduplicate concurrent identical requests by tracking in-flight calls keyed on a hash of the request, not by relying on a response cache that can't exist until a response comes back.

Why parallel subagents produce this pattern

Most agent frameworks that fan out subagents don't coordinate what those subagents ask the model. If you have five subagents each independently reading the same file and asking the model to summarize it, or a planner and a critic both re-deriving the same context because neither knows the other already has it, you get identical prompts issued in parallel. It's not a bug in your code so much as a structural gap: the framework parallelizes execution but doesn't dedupe intent. I've watched a single Claude Code session with three subagents ask for a project's dependency tree three separate times inside a two-second window, same tokens, same model, same answer coming back three times.

This is distinct from the caching problem people usually solve for. Exact-match and semantic caches both assume there's a completed response sitting somewhere to compare against. Coalescing is about the seconds before that response exists, while several callers are all waiting on the same unfinished work.

The coalescing pattern

The mechanics are the same shape as any request-deduplication problem you've solved at the HTTP layer, just with an LLM request as the unit of work:

• Hash the request: model, full message array, tool definitions, temperature, and any other parameter that affects output. Two requests are only the same call if every one of these matches.
• Before dispatching, check a table of in-flight hashes. If the hash is present, attach the new caller as a listener on the existing promise instead of sending a new request.
• If the hash is absent, register it, dispatch to the provider, and resolve every attached listener when the response lands.
• Remove the hash from the in-flight table once the response resolves, successfully or not, so the next distinct call for that same prompt goes through normally.
• Propagate errors to every waiter, not just the original caller. If the upstream call fails, everyone who coalesced onto it needs to know, not just the request that happened to trigger the dispatch.

The subtle part is streaming. If you're proxying streamed completions, coalescing means fanning a single upstream stream out to multiple downstream consumers, each of whom may be reading at a different pace. That's more state to manage than a simple promise-sharing pattern for non-streamed responses, and it's the part people skip when they build this in an afternoon and then wonder why it only works for non-streaming calls.

Where this pattern breaks down

Coalescing is safe for read-shaped, deterministic-intent requests: summarize this file, classify this ticket, answer this question given this context. It gets risky the moment a request carries side effects or depends on external state that changes between the time you'd hash it and the time it actually executes. If two subagents are both about to call a tool that mutates state, you don't want to collapse those into one call just because the prompt text matches, because the second one may need to see the result of the first. In practice this rarely bites you on the coalescing side, because true concurrency windows are short, but it's worth being deliberate about which call sites you allow to dedupe. Anything already carrying tool calls in its context is the case to be most careful with, since a wrong assumption there corrupts the run rather than just wasting a request.

The other failure mode is scope. Coalescing only catches requests that overlap in time. If your five subagents don't happen to fire within the same window, you're back to caching or, more realistically, back to redesigning the workflow so subagents share context instead of re-deriving it. Coalescing is a safety net for accidental duplication, not a substitute for giving subagents a shared read of the state they all need.

How I handle this in Probe0

I built Probe0 as a local proxy that every coding agent CLI on your Mac routes through: Claude Code, Codex, Cursor, one proxy, one certificate installed once, no per-tool config. Request Coalescing is one of the modules, and it does exactly the pattern above, sitting ahead of Exact Cache and Semantic Cache in the request path. When two subagents issue the same call inside the same window, the second one collapses onto the first instead of going upstream a second time, and the module reports what it actually saved so it isn't a guess.

It's deliberately conservative about what it will coalesce. The same caution that keeps Semantic Cache from matching anything carrying tool calls applies here: if there's ambiguity about whether two in-flight requests are really asking for the same thing, the module lets them both through rather than gambling on a merge that could hand one subagent the wrong result. Everything runs on your machine against a local disk and a local index, nothing about a coalesced request leaves your laptop to get deduplicated somewhere else.

A cache saves you from asking twice. Coalescing saves you from asking at the same time.

Building it yourself vs. not

If you're running a single agent framework in one process, an in-memory map keyed on a request hash with promise-sharing gets you most of the value in under a hundred lines, and that's a reasonable thing to just write. Where it stops being a weekend project is the moment you have more than one process or more than one CLI tool hitting the same provider, because now the in-flight table has to live somewhere shared, and you're back to running infrastructure to save yourself infrastructure. That's the case a local proxy actually earns its keep: every tool on the machine routes through the same coalescing table because they're routing through the same proxy, not because you wired each one up separately.

Either way, the fix for duplicate concurrent calls is not a smarter cache. It's tracking what's already in flight and making the second caller wait for the first instead of paying for its own copy.