Probe0 vs AWS Bedrock (with LLM Gateway pattern)
Bedrock is the upstream provider, and the LLM gateway is infrastructure you deploy in front of it to get quotas, tagging and chargeback that Bedrock does not enforce natively. Probe0 is an LLM proxy that runs on one developer's machine and sits under every AI coding agent CLI already installed there.
Amazon Bedrock is a fully managed AWS service offering API access to 100+ foundation models, usually fronted by a gateway layer that teams deploy themselves (API Gateway + Lambda, or the open-source aws-samples/bedrock-access-gateway) to get per-team quotas and chargeback.
| Feature | Probe0 | AWS Bedrock (with LLM Gateway pattern) | Edge |
|---|---|---|---|
| Where your prompts go | Stay on your Mac. No Probe0 server exists. | AWS-hosted models, plus a gateway you deploy and operate. | Probe0 |
| Setup for one developer | One local proxy plus a certificate. Claude Code, Codex and Cursor route through it with no per-tool config. | SDK or endpoint config per tool, plus CloudFormation for the gateway. | Probe0 |
| Hard spend cap | Per-run and per-day cap that warns, then pauses the run. | No native cap blocks inference. Gateway usage plans throttle by API key. | Probe0 |
| Cache hits that skip the network | Exact hits served from local disk. No request, no tokens. | Prefix cache lives in the inference layer. Request still goes to AWS, cached tokens still billed at a discount. | Probe0 |
| Cache safety around tool calls | Semantic cache holds a strict similarity floor and refuses to match anything carrying tool calls. | Prefix reuse only, so no semantic near-miss to guard. | Even |
| Local model routing | Routes to a model already running in Ollama or LM Studio, weak answers auto-retried on the cloud. | None on your hardware. Custom Model Import runs your weights on AWS. | Probe0 |
| Cheap-first model tiering | Tries a cheap model first, escalates only when needed. | Intelligent Prompt Routing picks between two models in the same family, English-optimized. | Probe0 |
| Which process spent the money | Ledger with model, tokens, real cost, latency and the process that made the call. | Tag-based attribution, per usage-type per day, tags up to 24h late and not retroactive. | Probe0 |
| Serving an organisation | macOS only, private beta. Stops when the laptop sleeps. | Always-on and region-distributed, with IAM, KMS, CloudTrail and versioned Guardrails across teams, CI and production. | Competitor |
Bedrock's prompt caching happens inside the inference layer, so the model skips recomputation and you pay a discounted rate on cached tokens, and a local proxy cannot reproduce that. What a local proxy can do is not send the request at all: a Probe0 exact-cache hit is served off your disk, costs nothing, and never leaves the machine. That is the shape of the whole comparison for a developer running coding agents on a Mac. Bedrock is the provider and the gateway is infrastructure you own, while Probe0 sits under the CLIs you already have and answers the three questions that actually come up during a run — what did this cost, which process spent it, and can I stop it before it spends more.
The gap opens on enforcement and on granularity. There is no hard spend limit anywhere in native Bedrock that stops a request from going out. Application inference profiles, Projects and Workspaces are attribution mechanisms, tag-based and reported per usage-type per day, with tags taking up to 24 hours to appear and not applying retroactively. Per-request metadata tagging is explicitly not enforced by the service: requests that omit it still succeed, and AWS's own guidance is to set it in a shared client or LLM gateway. Routing is limited to two models within one family and is documented as English-optimized. Invocation logging covers the bedrock-runtime endpoint only, so Responses API calls on bedrock-mantle are not captured. Every piece of real-time enforcement, cross-provider routing and per-request spend control lives in the gateway, and the gateway is your CloudFormation, your Lambda cold starts, your CloudWatch retention, and a second line on the bill.
I built Probe0 for the case where all of that is overhead you will never recover. Spend Guard pauses at a cap rather than reporting the overspend tomorrow, and the local ledger carries real cost and latency per call along with the process that made it, so a runaway Cursor session is identifiable rather than an unexplained bump in a daily total. Local Routing sends work to a model already loaded in Ollama or LM Studio and retries weak answers on the cloud. The semantic cache refuses to match anything carrying tool calls, because a near-miss on a tool call corrupts an agent run and that is a worse outcome than paying for the request. Every module switches off individually and reports what it saved, so you can judge each one on its own numbers. Probe0 is macOS only, in private beta, and is not a team gateway.
The verdict
If you are one developer on a Mac and the problem is what your coding agents spend, pick Probe0. You get a cap that pauses instead of a report that arrives a day late, a per-request ledger that names the process, cache hits that never touch the network, and routing to models already loaded on your machine, with no infrastructure to deploy or keep running. Bedrock with a gateway is the better pick in one situation, and it is a real one: serving an organisation, where multiple teams, CI, production traffic and compliance obligations need policy that applies whether or not an application cooperates. Short of that, the gateway is a bill and a CloudFormation stack standing in for something Probe0 does locally, and running Probe0 in front of coding agents that call Bedrock is a reasonable arrangement.
Questions
- Is AWS Bedrock an LLM proxy?
- No. Bedrock is the upstream provider, a managed AWS service that serves the models. The proxy or gateway layer is something teams build in front of it, commonly API Gateway plus Lambda or the open-source aws-samples/bedrock-access-gateway, because Bedrock does not enforce per-team quotas or chargeback on its own.
- Can AWS Bedrock stop a request when I hit a spend limit?
- Not natively. Bedrock's cost features are attribution and reporting: tag-based cost allocation at per usage-type per day grain, with tags appearing up to 24 hours later and not retroactively. Enforcement comes from the gateway you deploy, typically API Gateway usage plan quotas and IAM policies restricting which models a role may invoke.
- Does Bedrock support running models on my own hardware?
- No. Custom Model Import lets you bring open-weight fine-tunes in safetensors format, but they run on AWS-managed serverless infrastructure and are billed per five-minute window plus storage. Probe0's Local Routing is the opposite shape: it sends work to a model already running in Ollama or LM Studio on your machine and auto-retries weak answers on the cloud.
- Do I need a gateway if I only want LLM cost control for my own coding agents?
- Probably not. A gateway earns its keep when several teams share one account and someone needs chargeback. For a single developer looking for an AWS Bedrock (with LLM Gateway pattern) alternative that handles AI coding agent traffic, a local LLM proxy with a per-request ledger and a hard pause is far less to operate. Probe0 is macOS only and in private beta, so check that fits before you plan around it.