Prompt Caching: Cut AI Agent Bills 45% in 2026

The practical 2026 guide to prompt caching: how the cache became the single biggest lever on what your AI agents cost to run.

On September 1, 2026, Anthropic cut cache reads on its flagship by 75%, from $1.00 to $0.25 per million tokens, and told customers it would make highly agentic workloads up to 45% cheaper - VentureBeat. Nine days later, DeepSeek shipped a model whose cache reads cost $0.003 per million tokens, close enough to free that the number reads like a typo - DeepSeek. In one week, every major lab moved the same lever in the same direction, and it was not the base price. It was the cache.

The reason is structural, and once you see it you cannot unsee it. An AI agent is a machine for re-reading the same context over and over. Every turn of an agent loop resends the system prompt, the tool definitions, and the entire growing conversation, then adds a few hundred new tokens on the end. The expensive part of an agent bill is not the thinking. It is the re-reading. Prompt caching is the one technique that charges you full price for the re-reading exactly once, and pennies on the dollar every time after.

But here is the problem: caching is quietly easy to get wrong. A single timestamp in the wrong place, a tool list that reorders itself, an effort setting that flips mid-conversation, and your cache hit rate silently drops to zero while the dashboard still says everything is fine. Teams routinely believe they are caching when they are not, and pay the uncached rate on a workload engineered to be 95% cacheable.

This guide breaks down exactly how prompt caching works across every major provider, the real per-token numbers as of September 2026, the structural reason agents are the ideal caching workload, and the specific engineering patterns that turn a theoretical discount into a 45% smaller bill. It covers where caching wins, where it fails, and how it composes with the other cost levers (routing, batching, semantic caching) that got cheaper the same month. The audience is anyone who pays an LLM bill and suspects it is bigger than it needs to be.

Contents

1. Why agent bills explode: the token math nobody warns you about
2. What prompt caching actually is
3. The September 2026 price war that made cache the dominant lever
4. The caching scorecard: every major provider, ranked
5. Anthropic Claude: the deepest cache economics
6. OpenAI: automatic caching, premium base
7. Google Gemini: implicit, explicit, and the storage fee
8. DeepSeek: the near-free cache floor
9. xAI Grok, Amazon Bedrock, and the delivery channels
10. Provider caching versus semantic caching: two different tools
11. The agent caching playbook: how to actually cut 45%
12. Where caching fails: the silent-invalidation traps
13. The 45% claim, pressure-tested
14. The future: stateless protocols, payments, and the cache as infrastructure
15. Conclusion: a decision framework

1. Why agent bills explode: the token math nobody warns you about

Start from first principles, not from the pricing page. When you buy an LLM API call, what are you actually paying for? You are paying for compute spent turning input tokens into output tokens. The naive assumption is that a smarter, longer answer is the expensive part. For a single question and answer, that is roughly true. For an agent, it is almost exactly backwards. The output is cheap. The input is the bill. Understanding why is the whole game, because it tells you which lever to pull.

An agent is not one request. It is a loop. The model proposes an action, a tool runs, the result comes back, and the model reads everything so far and proposes the next action. Because the Messages API is stateless, there is no server-side memory of the conversation. You resend the entire history on every single turn - Anthropic. A ten-turn agent session with a 5,000-token system prompt reprocesses that same system prompt ten separate times, plus a conversation that grows longer with each step. The static scaffolding (the instructions, the tool schemas, the reference documents) is billed again and again, and it dwarfs the handful of new tokens each turn actually produces.

This is why input-to-output ratios for agents run heavily lopsided. A chatbot might send 500 tokens and receive 500 back. An agent on turn eight might send 40,000 tokens of accumulated context to produce a 200-token tool call, a ratio of 200 to 1. Independent research on long-horizon agents makes the same point from the data: prompt caching benefits scale linearly with both prompt size (from 500 to 50,000 tokens) and tool-call count (from 3 to 50), precisely because the resent prefix grows with both - arXiv. The bigger and longer the agent, the more of its bill is repetition.

Now the leverage becomes obvious. If most of your input is the same bytes you sent last turn, then the correct question is not "how do I use a cheaper model" but "why am I paying full price to reprocess bytes the provider already processed a second ago." That is the exact question prompt caching answers. It is not a discount you negotiate. It is a discount that exists because recomputing an identical prefix is genuinely wasteful, and providers would rather charge you a little to reuse it than a lot to redo it. We break down the full cost structure of agent workloads in our guide to the true cost of LLM inference in 2026, but the one sentence version is this: for agents, you are paying for memory you do not have, one turn at a time.

Put real numbers on it. Take a ten-turn agent on Claude Sonnet 5 with a 5,000-token system prompt, 3,000 tokens of tool definitions, and a conversation that grows by roughly 2,000 tokens a turn. Without caching, the request drags an 8,000-token frozen prefix through all ten turns, so you pay to process that prefix ten separate times: about 80,000 tokens of pure repetition per session, billed at the full $2 per million input rate. With the prefix cached, the first turn writes it once and the next nine read it at $0.20 per million, a tenth of the price. The repetition does not disappear, because the agent still needs its context on every turn. What disappears is the bill for reprocessing bytes that never moved, and across millions of sessions a month that gap is the line between a viable product and an unshippable one.

The practical implication is that caching is not a niche optimization for RAG pipelines with giant documents. It is the default cost posture for anything that loops. Once your workload re-reads context, the money is in the re-reading, and the re-reading is the most cacheable thing in the entire request. Every section that follows is downstream of this single fact.

2. What prompt caching actually is

Prompt caching is often explained as "the provider remembers your prompt," which is close enough to be misleading. What actually gets stored is not your text. It is the model's internal computation over your text. When a transformer processes a prompt, it computes a large set of intermediate tensors, the key-value pairs of the attention mechanism, for every token. That computation is deterministic: the same tokens in the same order always produce the same tensors. Prompt caching stores those computed tensors so the next request with an identical prefix skips the recomputation and starts generating from where the cache ends - Flexera.

Two properties follow directly from this and govern everything else. First, caching is a prefix match, not a fuzzy match. The cache is valid only up to the first byte that differs. If your system prompt is identical but you inserted one new word at the top, everything after that word is a cache miss, because the attention tensors for every later token depend on that word. Second, the output is byte-identical to what you would have gotten without the cache. Unlike a response cache that returns a stored answer, prompt caching only reuses the input-side computation, then the model generates fresh. Quality does not change, because nothing about the generation changes.

This is why placement is the entire craft. The provider renders your request in a fixed order (for Anthropic that order is tools, then system, then messages) and caches a contiguous prefix from the start - Anthropic. To maximize what gets cached, you put stable content first and volatile content last. The diagram below shows the anatomy of a well-structured agent request: a large frozen prefix that repeats every turn, a cache breakpoint, and a small volatile tail that changes each time.

There are three prices in a caching system, not one, and confusing them is the most common early mistake. A cache write stores a new prefix and costs slightly more than a normal input token, because the provider has to compute and persist the tensors. A cache read reuses an existing prefix and costs a small fraction of a normal input token. A plain uncached input token is the full price you pay when there is no cache to hit. The economics work only when reads vastly outnumber writes, which for an agent loop they almost always do. We treat cache design as one pillar of broader context engineering for agents, because deciding what belongs in the stable prefix is the same discipline as deciding what belongs in context at all.

The last thing to internalize is that caches expire. They live for a short window (commonly five minutes, extendable to an hour) measured from the request that touches them, and every hit resets the timer - Caylent. An actively used agent keeps its cache warm indefinitely. An agent that goes idle loses the prefix and pays a fresh write on its next turn. This time dimension, invisible on the pricing page, is why bursty and steady workloads see very different real-world savings from the same nominal discount.

3. The September 2026 price war that made cache the dominant lever

For most of the modern LLM era, the headline number was the base price per million tokens, and the competition happened there. In the second half of 2026 the competition moved. Every major lab shipped a model within a few weeks of each other, and the differentiator was increasingly the cache read, not the base rate. This is not a coincidence. Once the whole industry understood that agents are cache-heavy, the cache read became the price that actually determines what a production agent costs, and so it became the price worth cutting.

Look at the sequence. Anthropic released Claude Fable 5.1 and Claude Mythos 5.1 on September 1 at the same $10 and $50 per million as their predecessor, but with cache reads cut 75% - gHacks. Google shipped Gemini 3.8 Flash on September 2 at an introductory $0.75 and $3.75 per million, with cached input at just $0.075 - Google. OpenAI launched GPT-6 Astra on September 3 at a premium $10 and $50, with cached input a tenth of that - Layer3Labs. DeepSeek followed on September 10 with V4.1-Flash and its near-free cache-hit rate - DeepSeek. Grok 4.6 had already landed on August 12 at $2 and $6 with a 75% cache discount - Morph. Five labs, one lever.

The same weeks saw the other half of the story: agents stopped being a research demo and became a product line, which is precisely what makes cost a board-level concern rather than a hobbyist footnote. Salesforce launched seven named, job-ready Agentforce agents on September 11, each with months of memory, timed just ahead of Dreamforce - EnterpriseDNA. OpenAI put its managed Agents API into public beta on September 10, packaging the Codex harness behind a single call - OpenAI. And GitHub shipped Project HydraFusion on September 4, a Copilot feature that routes each coding task across multiple models. We track the full shortlist of frontier options in our best LLM for AI agents ranking, updated as the models ship.

HydraFusion is worth pausing on, because it shows the second big cost lever (routing) working alongside caching rather than against it. In GitHub's own offline evaluations, the strongest configuration cut estimated workflow cost by 67% on Terminal-Bench 2.1, 36% on DeepSWE, and 65% on CheckpointBench relative to a single frontier model - GitHub. The honest caveat, which VentureBeat highlighted, is that it cut cost in every benchmark but only matched quality in one - VentureBeat. The architecture below shows how it decides.

Routing and caching are complements, not substitutes, and the distinction matters for how you spend your engineering time. Routing sends each request to the cheapest model that clears the quality bar; caching makes each request to any model cheaper by not reprocessing its prefix. You can do both, and the best-run agent stacks do. We cover the routing half in depth in our guide to AI model routing to cut agent costs 60%; this guide is about the caching half, which is the one that requires no second model, no classifier, and no quality tradeoff. HydraFusion's own published numbers make the routing lever concrete.

The pattern in those bars is the recurring lesson of 2026 cost engineering: the cheapest configuration is rarely one model doing everything, and the savings arrive with a quality asterisk you have to measure rather than assume. Caching is the friendlier cousin of the same idea, because it lowers cost with no quality asterisk at all, since the output is byte-identical whether the prefix was cached or not. The chart below shows where the cache-read floor sits across the models that shipped this season.

The spread is more than two orders of magnitude, from a dollar per million on the most premium flagship down to fractions of a cent on the cheapest flash model. That range is the opportunity. For a workload that is 90% repeated context, the cache-read column is close to the real per-token cost, and choosing where to sit on that column is one of the highest-leverage decisions a team makes.

4. The caching scorecard: every major provider, ranked

Before the per-provider detail, here is the whole field in one view. The table scores the six caching implementations a builder would realistically choose from, weighted by what actually matters when you are running an agent in production rather than reading a launch post. Each cell carries the score and the evidence behind it. The table is sorted by final score, highest first.

The four criteria are chosen from first principles, not from a generic feature checklist. Discount depth (30%) is how cheap a cache read is relative to a fresh token, because that ratio is what converts repetition into savings. Ease and automation (20%) is whether caching happens automatically or demands manual breakpoints, because setup friction is why teams leave the discount on the table. Agent economics (25%) is the net cost of running a cache-heavy loop, blending base price with cached rate. Flexibility (15%) covers TTL options, storage fees, and breakpoint control. Ecosystem and isolation (10%) covers SDK and gateway support plus how caches are kept private.

DeepSeek tops the table on raw economics: when a cache hit costs $0.003 per million tokens, the cached portion of your bill effectively disappears, and for an agent whose input is mostly cached, the total collapses toward the output-only cost - DeepSeek. Anthropic sits a close second on the strength of control and ecosystem: four breakpoints, two TTLs, no storage fee, and the deepest discount of any Western lab on its flagship. The gap between them is a genuine tradeoff, not a ranking artifact. DeepSeek is cheaper and simpler; Anthropic is more controllable and more broadly integrated, which for a regulated enterprise can outweigh a lower sticker price.

The middle of the table is where the nuance lives, and it rewards reading past the final score. Google and OpenAI tie at 7.8 for opposite reasons: OpenAI wins on effortlessness (caching is fully automatic with zero code) but loses on the premium base price of its flagship, while Google wins on a rock-bottom Flash base but carries a per-hour storage fee on explicit caches that no other provider charges. Grok lands last here not because it is a weak model but because its cache discount is 75% where rivals offer 90% or more, and it is not yet available on the enterprise channels where much agent spend actually happens. Your own ranking should reweight these criteria for your workload: a startup optimizing pure cost will weight discount depth higher, while a bank will weight isolation and ecosystem.

To make that concrete: a consumer startup burning venture money on a high-volume chat agent might weight discount depth at 45% and ecosystem at 5%, which pushes DeepSeek and Gemini Flash further ahead and drops Bedrock down the list. A regulated enterprise running the same agent might weight isolation and ecosystem at 30% combined and discount depth at just 15%, which lifts Anthropic and Bedrock above the cheaper open-weight options, because per-workspace isolation and enterprise contracts matter more to it than a fraction of a cent per token. The scores here are not a verdict; they are a starting point you recompute against your own constraints, and the fact that the order changes under reasonable reweightings is the sign that the criteria are doing real work rather than rubber-stamping a favorite. The rest of this guide is the evidence behind each row.

5. Anthropic Claude: the deepest cache economics

Anthropic introduced prompt caching in August 2025 and built the most controllable implementation in the market, which is why it rewards the most careful engineering. The model is explicit: you mark a content block with a cache_control directive, and Anthropic stores the encoded state of everything from the start of the request up to and including that block - Anthropic. You get up to four cache breakpoints per request, which lets you cache sections that change at different frequencies: your tool definitions rarely change, your system prompt changes occasionally, and your conversation grows every turn, and a separate breakpoint on each means a change to one does not invalidate the others.

The pricing has three tiers per model, and the multipliers are consistent. A cache write on the default five-minute TTL costs 1.25x the base input rate; a write on the one-hour TTL costs 2x; a cache read costs 0.1x for standard models - Anthropic. The special case that made headlines is Fable 5.1 and Mythos 5.1, where the read multiplier is not 0.1x but 0.025x, four times deeper than the standard discount. That is the mechanism behind the $0.25 per million cache read on a model whose base input is $10. The table below shows the current lineup.

All prices are per million tokens, from Anthropic's official pricing documentation - Anthropic. The strategic read of this table is that Sonnet 5 is the quiet workhorse for cached agent workloads: at a $2 base and a $0.20 cache read, a cache-heavy loop runs at close to twenty cents per million on the input side, which is competitive with models several tiers cheaper on paper. We put Sonnet 5 through a full cost breakdown in our Claude Sonnet 5 guide, and the caching numbers are a large part of why it punches above its base price for production agents.

The chart below puts the fresh and cached rates side by side, and the gap between the two bars for each model is the discount you forfeit if your prefix is not actually caching.

Read the pairs, not the absolute heights. GPT-6 Astra and Fable 5.1 share a $10 base, but Fable's cached bar is a quarter of Astra's because of its deeper 0.025x read rate, which is why a cache-dominated workload can be dramatically cheaper on Fable despite the identical sticker price. Sonnet 5 and Gemini 3.8 Flash sit lowest on both bars, which is what makes them the default choices for high-volume agents where every cached token counts and the base rate still has to stay sane.

There are three mechanics that determine whether you actually get these prices, and all three trip teams up. First, there is a minimum cacheable prefix: 512 tokens for Fable 5.1, Mythos 5.1, and Opus 5; 1,024 for Sonnet 5; and up to 4,096 for Haiku 4.5. A prefix shorter than the minimum is silently processed without caching, and no error is returned, so a small-but-repeated system prompt can look cached in your code and cost full price in your bill. Second, caches are isolated per organization, and per workspace on the first-party API, so you never share a cache with another customer and you cannot accidentally leak one. Third, the write premium means caching only pays off above a modest reuse threshold, which we quantify in the pressure-test section.

The one-hour TTL deserves a specific note because it is underused. It costs 2x to write instead of 1.25x, but the read price is identical, so for a stable prefix that gets hit across a long-running session (a coding agent working a task for twenty minutes, a support agent handling a slow conversation) the extra write cost is recovered almost immediately and the cache survives the quiet stretches that would kill a five-minute entry. If you are choosing between Fable 5.1 and Opus 5 for an agent, the cache economics are part of the decision, which we work through in Claude Fable 5.1 vs Opus 5 for agents. The short version: Fable's deeper 0.025x read rate rewards workloads that are overwhelmingly cached, while Opus 5's lower base rewards workloads with more fresh input per turn.

6. OpenAI: automatic caching, premium base

OpenAI made the opposite design choice from Anthropic, and it is a genuinely different philosophy rather than a better or worse one. Where Anthropic gives you explicit control and asks you to place breakpoints, OpenAI caches automatically with no code change at all. Any prompt over roughly 1,024 tokens is eligible, the system detects the repeated prefix on its own, and cached tokens are billed at the discounted rate without you lifting a finger - Helicone. For a team that wants the savings without the engineering, this is the lowest-friction caching in the market.

The current flagship, GPT-6 Astra, launched September 3 at a base of $10 input and $50 output per million tokens, positioning it as a premium reasoning model rather than a cost play - CloudZero. Cached input drops to $1.00 per million, a tenth of the fresh rate, and multiple pricing trackers report a cache-write premium of $12.50, mirroring Anthropic's 1.25x structure - eesel. There is also a long-context surcharge: requests above 272,000 input tokens are billed at $20 input and $75 output, so very large cached contexts cross into a higher tier that partially offsets the cache discount - Layer3Labs.

Two service-tier facts change the arithmetic for non-interactive workloads and are easy to overlook. Astra runs at half price on the Batch and Flex tiers ($5 input, $25 output) and at double on Fast mode ($20 input, $100 output) - CloudZero. Batch and caching are the two levers that stack cleanly here: if your agent work is not latency-sensitive, running it through the batch tier and letting automatic caching handle the repeated prefix compounds two independent discounts on the same tokens. We put Astra's real per-task cost under a microscope in our GPT-6 Astra pricing analysis, and the headline is that the base price makes it expensive for high-volume agents unless caching and batching are both switched on.

The automatic model has a subtle cost of its own, which is the flip side of its convenience: you have less control over exactly what gets cached and when. You cannot place a breakpoint to protect a stable prefix from a volatile suffix the way you can with Anthropic, so if your prompt structure is not already cache-friendly, you get whatever the automatic detector finds rather than what you engineered. In practice this rarely bites, because a well-ordered prompt caches well under automatic detection too, but it does make the failure harder to diagnose: with Anthropic you can point at a missing breakpoint, whereas with OpenAI you are reverse-engineering what the detector decided. The mitigation is the same discipline either way, stable content first, and the prompt_cache_key parameter gives you a lever to group requests that should share a cache, which matters when many distinct users hit the same system prompt and you want their traffic to warm one shared prefix rather than fragment into many cold ones. OpenAI also folded a managed agent runtime into the same ecosystem with the Agents API beta on September 10, which runs the Codex harness and hosts the loop for you - MarkTechPost. We cover that runtime in shipping a long-running agent on the OpenAI Agents API; for cost purposes, the relevant point is that a managed harness resends context on your behalf, so its bill is shaped by the same caching dynamics as a loop you write yourself.

7. Google Gemini: implicit, explicit, and the storage fee

Google splits caching into two mechanisms, and the split is the most important thing to understand about it because it is the only major provider that charges you rent. Implicit caching is automatic and on by default for the current Gemini models: the system detects repeated prefixes and passes the savings through with no code and no storage fee. Explicit caching uses the CachedContent API, where you deliberately store a block of context and get a guaranteed discount on reads, but you pay a per-hour storage charge for as long as the cache lives - Google.

The numbers on Gemini 3.8 Flash make the tradeoff concrete, and they come with an expiry date that every planner should mark. Through December 31, 2026, input is $0.75 per million and output is $3.75, with cached input at $0.075, a clean 90% discount on the cached portion - apidog. Explicit-cache storage costs $0.50 per million tokens per hour over the same period. On January 1, 2027, all of it doubles: input to $1.50, output to $7.50, cached input to $0.15, and storage to $1.00 per million per hour - Google. The introductory pricing is the same promotional pattern Google used for earlier Flash releases, so treat the low rate as a window, not a baseline.

The storage fee changes when explicit caching is worth it, and this is the calculation Gemini forces that no other provider does. With Anthropic or OpenAI, you pay a one-time write premium and then reads are cheap forever until the cache expires. With Gemini's explicit cache, you pay by the hour whether or not anyone reads it, so a cache that sits warm but idle is a running meter. The break-even is a function of read frequency: if you hit a cached 100,000-token context often enough that the read savings exceed roughly five cents per hour of storage, explicit caching wins; if the context is read rarely, implicit caching (no storage fee, smaller guaranteed savings) is the better default. Gemini 3.8 Flash is genuinely cheap for agents on a per-task basis, which we quantify in our Gemini 3.8 Flash cost analysis, and the storage-fee math is the one thing that separates a good Gemini caching setup from a wasteful one.

Make the rent concrete. A 100,000-token explicit cache costs $0.50 per million tokens per hour to store, which is $0.05 an hour, or about $1.20 a day if you keep it warm around the clock. Each read of it avoids fresh input worth roughly $0.0675 for that 100,000-token context (the $0.075 fresh cost minus the $0.0075 cached read). So a context that gets read even a few times an hour repays its storage many times over, while one that sits warm but idle is a meter running against nothing. The discipline Gemini forces, and that providers with free storage do not, is lifecycle management: create explicit caches for genuinely hot contexts, size their lifetime to real read frequency, and delete them the moment the work is done rather than letting the meter run.

For most teams, the right posture on Gemini is to lean on implicit caching for ordinary agent loops and reserve explicit caching for a small number of large, hot, shared contexts (a giant system prompt or a reference corpus that every request touches within the hour). That gives you the automatic savings everywhere and the deeper guaranteed savings exactly where the storage fee is paid back many times over. Treating explicit caching as a default, the way you might on a provider with free storage, is the classic Gemini overspend.

8. DeepSeek: the near-free cache floor

DeepSeek did something the other labs have not: it made cache reads so cheap that for a cache-heavy workload, input tokens nearly stop mattering. The mechanism, which DeepSeek calls Context Caching on Disk, is fully automatic and requires no code or interface change; it caches repeated prefixes on a distributed disk array, matches from the very first token, and charges nothing for storage - DeepSeek. The only content that does not cache is a prefix under 64 tokens, which is irrelevant for any real agent.

The current numbers on V4.1-Flash, which launched September 10, are the aggressive part. A cache hit on input costs $0.003 per million tokens off-peak and $0.006 peak; a cache miss costs $0.15 off-peak and $0.30 peak; output is $0.60 off-peak and $1.20 peak - DeepSeek. A cache hit is therefore roughly 50 times cheaper than a miss, an order of magnitude beyond the 10x that Western providers offer, and it is cheap enough that the cached portion of an agent's bill rounds to zero. The launch graphic below lays out the full peak and off-peak structure.

There are two structural reasons DeepSeek can price this low, and both are worth understanding because they explain why the number is real rather than a loss-leader that will vanish. First, V4.1-Flash uses a compressed key-value cache: DeepSeek reports its cache needs about a quarter of the high-bandwidth memory and an eighth of the SSD storage of the prior generation, and compressing the cache is what makes storing it nearly free - DeepSeek. Second, DeepSeek runs a time-of-day pricing model: off-peak rates are exactly half of peak, with peak hours defined as 01:00-04:00 and 06:00-10:00 UTC on weekdays and everything else off-peak - DeepSeek. A batch agent workload scheduled into the off-peak window stacks the time discount on top of the cache discount.

The one honest caveat is that the eye-catching $0.003 is the off-peak number, and a plain "$0.003 cache reads" claim is misleading for half the day, so always state the window. Even at the peak $0.006, though, DeepSeek is the cheapest cache floor on the market by a wide margin. The tradeoffs are the usual ones for an open-weight lab: fewer enterprise delivery channels and org-level rather than per-workspace isolation. We work the full arithmetic, including how the off-peak window interacts with cache warmth, in cutting your DeepSeek agent bill 50% with off-peak scheduling, and profile the model family in our DeepSeek V4 guide.

The open-weight nature cuts both ways, and it is worth naming honestly rather than burying under the price. On one side, an open-weight model with a near-free cache means you are not locked to a single vendor's pricing curve: if the rates move against you, the weights run elsewhere, and self-hosting is a credible fallback in a way it never is with a closed flagship. On the other side, some enterprises cannot route data to a China-based API for regulatory or procurement reasons, and org-level cache isolation is a coarser boundary than a bank's security team may accept. The caching economics here are the best in the market, but caching economics are not the only axis, and for a regulated workload the data-residency and delivery-channel questions can outweigh a lower cache-read rate. For a cost-first agent where you control the model and the data path, DeepSeek is the aggressive answer.

9. xAI Grok, Amazon Bedrock, and the delivery channels

Two more caching stories round out the field, and they matter more for where you run an agent than for which lab trained it. xAI's Grok 4.6, released August 12 at $2 input and $6 output per million, offers cached input at $0.50, a 75% discount rather than the 90% that has become the industry norm - eesel. The shallower discount is the main reason Grok scores lower on caching specifically, even though its base price is competitive. There is also a long-context tier: past 200,000 tokens, the whole request reprices to $4 input, $1 cached, and $12 output - Morph. Grok is capable and cheap on fresh input, and we rank it among the value options in Grok 4.6 as the cheapest frontier LLM for agents, but on the cache-read axis it is a step behind.

The more important distinction is that caching behaves differently depending on the channel you buy through, and the biggest channel is Amazon Bedrock. Bedrock has offered prompt caching for Claude models since April 2025, and in January 2026 it added the one-hour TTL that first-party callers already had, extending the default five-minute window for longer agentic sessions - AWS. The discount mirrors the first-party economics: cached tokens cost about a tenth of regular input, a 90% reduction on the cached portion - Caylent.

Bedrock also surfaces a number that every caching decision should be measured against: the break-even hit rate. Because a cache miss (a write) costs more than a plain uncached token, caching is a net loss if your prefix is almost never reused, and Bedrock's own guidance puts the break-even at roughly a 12% cache hit rate on the five-minute TTL - Niklas Palm. For an agent loop, where the same prefix is reused every single turn, you clear that bar on the second request. For a fleet of one-off classification calls with no shared prefix, you would fall below it, and caching would cost you money rather than save it. The channel also affects isolation: Bedrock and Google Cloud use organization-level cache isolation, while the first-party Claude API and Microsoft Foundry isolate per workspace - Anthropic.

The practical lesson is that "which model" and "which channel" are two separate caching decisions, and the channel can lag the model. Grok 4.6, for instance, was not yet on Bedrock or Azure at launch, with those platforms still offering an earlier version, so an enterprise standardized on Bedrock could not get the newest Grok caching even if it wanted to - eesel. If your infrastructure is committed to a cloud, verify that the model and the caching feature you are counting on are actually available there before you build a cost model around them.

10. Provider caching versus semantic caching: two different tools

The word "caching" hides two genuinely different techniques, and conflating them is how teams either double-count their savings or miss half of them. Everything above is provider prompt caching: an exact-prefix, key-value cache that reduces the input-side cost of a request that still runs through the model. Semantic caching is a different animal entirely: it sits in front of the model, embeds the incoming query, and if a past query is similar enough, it returns the stored answer and never calls the model at all - Redis. One reduces the cost of running the model; the other avoids running it.

The economics differ in a way that determines which to reach for. Provider prompt caching only touches input tokens, but it applies to every request, including novel ones, because even a unique question shares the stable system-and-tools prefix. Semantic caching bypasses the model completely on a hit, saving both input and output tokens, but it only fires when a query is close to one it has seen before. In production, semantic caches report cutting costs 30% to 70%, with Redis LangCache claiming up to 73% on high-repetition workloads and typical deployments serving 20% to 45% of traffic straight from cache - Spheron. The diagram below shows how the layers stack.

The failure modes are where the two diverge most sharply, and they are not symmetric. Provider prompt caching cannot give a wrong answer, because the output is byte-identical to the uncached call; the worst case is a silent cache miss that costs you money but corrupts nothing. Semantic caching can absolutely give a wrong answer: if the embedding model maps two genuinely different questions to nearby vectors, a false cache hit returns a stored response to a query it does not actually match, and the user gets a confident, wrong, hallucinated answer - NeuralTrust. The quality of the embedding model matters more than any other tuning choice, and a poorly chosen similarity threshold is a correctness bug, not a cost bug.

Picture the concrete failure. A user asks "what is our refund policy for damaged items?" and a past query, "what is our refund policy for late items?", sits in the semantic cache with a high similarity score. Set the threshold a hair too loose and the cache returns the late-items answer to the damaged-items question, confidently and instantly, and the user acts on wrong information that no log flags as an error because, mechanically, nothing failed. This is why semantic caching demands an evaluation harness that measures false-hit rate on real traffic before it goes near production, and why the safe rollout order is provider prompt caching first (zero correctness risk) and semantic caching second (high ceiling, real risk), with the similarity threshold tuned against measured harm rather than guessed comfort.

The right mental model is that these are complementary layers, not competitors, and a mature agent stack often runs both. Semantic caching catches the paraphrased repeats and takes them off the model entirely; provider prompt caching makes everything that does reach the model cheaper. Open-source tooling like GPTCache from Zilliz and managed services like Redis LangCache implement the semantic layer, while the provider caching layer is a directive on your API call - Redis. For agents specifically, provider prompt caching is the safer first move because it carries no correctness risk, and semantic caching is the higher-ceiling second move that demands careful evaluation of false-hit rates before you trust it in production. It pairs naturally with a well-designed agent memory architecture, where deciding what to store and recall is the same discipline as deciding what to cache.

11. The agent caching playbook: how to actually cut 45%

Theory does not lower a bill; structure does. The single most important rule, and the one the research on agentic caching keeps returning to, is stable content first, volatile content last. Because the cache is a prefix match, anything that changes between requests must live after everything that does not, or it invalidates the reusable prefix. The order that maximizes cache hits is tool definitions, then the system prompt, then long-lived reference context, then the growing conversation, with the volatile new turn on the very end. Get this ordering wrong and no amount of cache_control will help you.

The most rigorous public study of this, an evaluation across OpenAI, Anthropic, and Google over more than 500 agent sessions, found that prompt caching cut API costs by 41% to 80% and improved time to first token by 13% to 31%, but only when placement was strategic - arXiv. Its central warning is in the title, "Don't Break the Cache," and its finding is counterintuitive: naive full-context caching can paradoxically increase latency, because caching everything (including content that changes every turn) forces constant rewrites. The winning strategy was to place dynamic content at the end of system prompts, avoid classic function-calling constructs inside cached sections, and exclude dynamic tool results from the cached prefix. Here is a concrete Anthropic-style structure that follows those rules.

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=4096,
    system= [
        {"type": "text", "text": STABLE_SYSTEM_PROMPT,
         "cache_control": {"type": "ephemeral", "ttl": "1h"}},  # frozen, 1h TTL
    ],
    tools=STABLE_TOOL_DEFINITIONS,  # deterministic order, cached with system
    messages= [
        *prior_turns,                 # grows each turn, sits after the breakpoint
        {"role": "user", "content": new_user_turn},  # the only new tokens
    ],
)
# Verify the cache is working, every deploy:
print(response.usage.cache_read_input_tokens)   # should be large after turn 1
print(response.usage.cache_creation_input_tokens)  # should be small after turn 1

The verification step in that snippet is not optional, and skipping it is the difference between believing you cache and actually caching. Every provider exposes usage fields that tell you the truth: on Anthropic, cache_read_input_tokens should be large and cache_creation_input_tokens small on every request after the first - Anthropic. If the read count is zero across repeated requests with the same prefix, a silent invalidator is at work and you are paying full price. Wire these numbers into a dashboard or a test, because caching regressions are invisible until you look at them, and they cost real money the whole time they go unnoticed.

Four structural habits turn the theory into a durable discount, and each one addresses a specific way agents leak cache hits.

• Freeze the prefix. Keep the system prompt and tool schemas byte-stable across turns; never interpolate a timestamp, a request ID, or a UUID into them.
• Order tools deterministically. A tool list that serializes in a different order invalidates the entire cache, so sort it once and never rebuild it per request.
• Use the one-hour TTL for long sessions. The 2x write premium is recovered in a turn or two, and the longer window survives the quiet stretches that expire a five-minute cache.
• Cache the history, not the moment. Append new turns after the breakpoint so each turn reuses the whole prior conversation rather than rewriting it.

Applied together, these habits are what convert a nominal 90% cache-read discount into a real 40% to 45% cut in a production agent bill, because they raise the share of input that actually hits the cache toward the ceiling the pricing implies. Gateways make this easier to roll out across a fleet: LiteLLM lets you specify cache-control injection points in configuration, including caching tool definitions, so you can add breakpoints without editing every call site - LiteLLM. This is the same discipline we teach for long-running coding agents, where a task can span dozens of turns and an unbroken cache is the difference between a viable feature and an unaffordable one.

A worked example shows the size of the prize. Picture a support agent handling one million sessions a month, each a five-turn loop on Claude Sonnet 5 with a 6,000-token cached prefix (system prompt, tools, and account context). Uncached, that prefix is 6,000 tokens times five turns times a million sessions, or 30 billion input tokens a month at $2 per million: $60,000 just to reprocess the same scaffolding. Cached, the first turn writes the 6,000-token prefix once per session (6 billion write tokens at $2.50 per million, about $15,000) and the remaining four turns read it (24 billion tokens at $0.20 per million, about $4,800). The prefix line falls from $60,000 to roughly $19,800, a 67% cut, and that is before you touch model choice, routing, or the batch tier. The write premium is real, but on any loop longer than two turns it is rounding error against the read savings.

12. Where caching fails: the silent-invalidation traps

Caching has a specific and dangerous failure mode: it does not error when it stops working. A misconfigured cache does not throw an exception or return a warning; it simply reprocesses your prefix at full price while every other signal looks normal. This is why caching bugs can run for weeks in production before anyone notices a bill that crept up 30%. Knowing the invalidators by name is the only defense, because you cannot monitor for a failure you do not know exists.

The invalidation hierarchy is the foundation, and it follows the render order. Anthropic caches in the sequence tools, then system, then messages, and a change at any level invalidates that level and everything after it - Anthropic. Changing a tool definition therefore blows away the entire cache, including the system prompt and conversation, because tools come first. Changing the system prompt invalidates system and messages but leaves tools cached. Appending a new turn to the end invalidates nothing before it, which is exactly why the append-only pattern is so cache-friendly. The diagram below shows what each kind of change costs you.

Beyond the obvious prefix edits, there are subtler invalidators that catch teams who think their prefix is frozen. Toggling web search, citations, or the speed setting invalidates the system and message caches. Changing the tool_choice parameter or adding or removing an image invalidates the message blocks. And changing the reasoning effort mid-conversation invalidates message blocks on most models, which is a real trap for adaptive agents that dial effort up for a hard step and back down for an easy one, because each flip silently rewrites the cache - Anthropic.

A concrete failure looks mundane, which is exactly why it survives for weeks. A team ships an agent whose system prompt opens with "Today is {date} and the current time is {timestamp}." Every request now has a unique first line, so the prefix never matches, the cache-read count sits at zero, and the workload runs at the full uncached rate: it was engineered to be 95% cacheable and realizes none of it. Nothing errors, the agent answers correctly, and the only symptom is a line on an invoice nobody is watching. The fix is a single move (push the timestamp to the end of the message, after the cache breakpoint) and it can halve the input bill the moment it ships. This class of bug is common precisely because the code looks right and the output is right, and only the accounting is wrong. The most insidious invalidator of all is a dynamic value hidden in a supposedly static prompt: a current date, a "you are helping user 12345" line, or an unsorted JSON blob that serializes differently each time. Any of these makes the prefix change every request, and your cache read count sits at zero while your bill sits at full price.

The multi-provider caveats compound the problem. Cross-model cascades, the routing pattern that saves money by sending easy work to cheap models, forfeit cache reuse across the models they route between, because caches are model-scoped and a request sent to a different model starts cold. The economics can still favor routing, but the two levers partially fight each other, and you should measure the net rather than assume both discounts apply. Semantic caching adds its own failure mode, the false hit that returns a wrong stored answer, which is a correctness risk rather than a cost one. The unifying discipline is to treat cache hit rate as a monitored production metric, alert on it when it drops, and test for it in CI, because the whole category of failure is defined by being invisible until you deliberately measure it. This is one more reason to standardize prompt structure across an agent fleet, the way disciplined teams standardize Claude Code subagents so every worker shares a cache-friendly prefix.

13. The 45% claim, pressure-tested

A headline like "cut agent bills 45%" deserves suspicion, so let us build it from the arithmetic rather than accept it on faith. The number originates with Anthropic, which said the Fable 5.1 cache-read cut reduces effective cost by around 25% for typical workloads and up to roughly 45% for highly agentic workloads where cached context is a large share of token usage - VentureBeat. The critical qualifier is "highly agentic," and understanding why the number lives there tells you whether it applies to you.

Work the input side first. Suppose an agent's request is 90% cached prefix and 10% fresh tokens, a realistic split for a deep loop. Under the standard 0.1x read multiplier, the input cost relative to no caching is (0.10 fresh at full price) plus (0.90 cached at a tenth), which is 0.10 + 0.09, or 0.19: an 81% cut on the input side. Under Fable 5.1's deeper 0.025x rate, it is 0.10 + 0.0225, or about 0.12: an 88% cut. The chart below shows how input cost collapses as the cached share rises, for both the standard and the deep discount.

Now reconcile the 81% to 88% input cut with the 45% total figure, because the gap is where the honesty lives. Total cost is input plus output, and caching does nothing for output tokens. If output is a meaningful slice of the bill, the total saving is diluted below the input saving. For a workload where input is, say, 70% of spend and you cut that input by 80%, the total falls by about 56%; where input is a smaller share, or where the cache hit rate is below 90%, the total lands nearer the 25% to 45% band Anthropic quotes. The 45% is an upper bound for genuinely cache-dominated agents, not a promise for every workload. That framing is confirmed independently: the multi-provider study found real cost reductions of 41% to 80% across agent workloads, which brackets Anthropic's claim rather than contradicting it - arXiv.

Two adjustments keep the estimate honest in the other direction. First, the write premium: the first request pays 1.25x or 2x to create the cache, so on a session with very few turns the amortization is incomplete and the realized saving is lower than the steady-state math suggests. Second, cache warmth: a bursty workload that lets caches expire between bursts pays repeated writes and never reaches the ceiling, which is why the same nominal discount produces very different bills for steady versus spiky traffic. The break-even is low (Bedrock puts it near a 12% hit rate) so almost any looping agent comes out ahead, but "ahead" and "45% ahead" are different claims - Niklas Palm. The defensible statement is that a well-structured, steady agent on a deep-discount model can realistically cut its bill 40% to 45% from caching alone, and stacking batch and off-peak levers on non-interactive work can push it further. We keep a living ledger of these compounding levers in our guide to cutting LLM costs.

14. The future: stateless protocols, payments, and the cache as infrastructure

Zoom out to the structural forces, because they explain why caching gets more important from here, not less. The clearest signal is the Model Context Protocol's July 2026 revision, which rewrote the protocol core to be stateless: it deleted protocol-level sessions, the session-ID header, and the initialize handshake, so that every request now carries its own context and capabilities - MCP. Statelessness is wonderful for scaling and routing, but it has a direct cost consequence: if there is no session state on the server, there is nothing to remember your context between calls, so you resend it every time. A stateless world is a world where the resent prefix is unavoidable, and caching is the only thing that makes resending it affordable. We compare the transport tradeoffs in MCP versus A2A for agents.

The money flowing into the agent stack points the same way. Modal raised a $355M Series C in May 2026 at a $4.65B valuation on the back of AI-inference demand, and Cognition reached a $26B valuation for its Devin coding agent - FinSMEs. Capital is betting that agents run constantly and at scale, and anything that runs constantly at scale lives or dies on unit economics. Meanwhile the emerging payment rails (the Linux Foundation's x402 for internet-native agent payments, OpenAI and Stripe's checkout protocol, Google's agent payments standard) assume a future where agents transact autonomously, which means an agent's own margin, and therefore its token cost, becomes a first-class business metric - Linux Foundation. When an agent pays its own way, every cached token is margin. We map that rail in our agent payments infrastructure guide.

There is a capability arc underneath all of this that raises the stakes further. Models are getting powerful enough that their runtime behavior is a security and cost concern at once: Epoch AI documented a spike to roughly 3.5 times the prior monthly record of high- and critical-severity CVEs after autonomous vulnerability-discovery models arrived, and climbing - Epoch AI. More capable agents run longer, hold more context, and loop more, which is exactly the shape of workload where caching pays the most. The trajectory is toward agents that are more autonomous, more persistent, and more expensive to run naively, and caching is the lever that keeps that trajectory economically sane.

This is also where managed platforms enter the picture as a legitimate alternative to hand-tuning everything yourself. Gateways like Cloudflare AI Gateway, Portkey, and Helicone add caching, routing, and observability as a layer in front of the providers, so a team can get most of the discount without owning the plumbing - AgentsCamp. At the far end of that spectrum are full autonomous-operation platforms such as O-mega, where you describe the work and the platform runs the agents, absorbing the caching, routing, and context-engineering decisions so the operator never touches a cache_control directive. That is a different tradeoff from the raw API (less control, less to manage), and it belongs on the menu next to the DIY approach rather than above or below it. The right choice depends on whether caching is a core competency you want to own or a cost you want handled.

15. Conclusion: a decision framework

Strip away the model names and the pricing tables, and the argument of this guide is one structural claim: agents are machines that re-read, the re-reading is the bill, and caching is the discount on re-reading. Everything else is implementation. Once you accept that agents are cache-dominated workloads, caching stops being an optimization you get to and becomes the default posture you start from, because the alternative is paying full price to reprocess bytes the provider handled a second ago.

The decision framework is short. First, confirm your workload actually loops or reuses a large prefix; if it does (any agent, any RAG pipeline, any multi-turn assistant) caching is worth it, because the break-even hit rate is near 12% and a loop clears that on turn two. Second, choose your posture by how much control you want: automatic caching on OpenAI or Gemini's implicit mode for zero effort, explicit cache_control on Anthropic for maximum control, or DeepSeek for the cheapest floor if you can run an open-weight model. Third, structure the prompt correctly (stable content first, one breakpoint, volatile content last) and verify the cache-read count on every deploy, because the failure mode is silent. Fourth, stack the complementary levers (batch tiers, off-peak windows, routing, semantic caching) on the traffic where each one fits, and measure the net rather than assuming the discounts add.

The honest bottom line on the 45% is this: it is real for a steady, well-structured, cache-dominated agent on a deep-discount model, achievable for most looping workloads with disciplined prompt structure, and diluted for anything with a low hit rate or a heavy output share. The number that matters is not the sticker discount but your measured cache-read ratio, and that ratio is an engineering outcome you control. Whether you build the caching yourself against a raw API, buy it through a gateway, or hand the whole loop to a managed platform like O-mega, the leverage is the same: in 2026, the cheapest token in your agent is the one you already paid to compute.

This guide reflects the AI agent and prompt-caching landscape as of September 2026. Model names, per-token prices, and cache multipliers change frequently, and several rates cited here carry explicit expiry or repricing dates, so verify current details on each provider's own pricing page before committing to a cost model.