I turned on caching and my bill went up. What happened?

Almost certainly a cached prefix that changes between turns. A cache write costs more than a normal input token, so if your prefix is invalidated every turn you pay that write premium repeatedly and never collect the cheap read discount. The result is strictly more expensive than not caching. Check your cache-write tokens against your cache-read tokens: if writes are as large as reads, the prefix is unstable. Verify the exact multipliers against your provider's current pricing, since they change.

What write-to-read ratio means caching is helping?

You want reads to dwarf writes. Healthy caching writes a prefix once and reads it back on many subsequent turns, so a ratio of, say, one write for every ten-plus reads is a strong signal it's working. When writes approach or exceed reads, you're rewriting the cache faster than you're reusing it and likely losing money. There's no universal threshold; measure your own split per model rather than assuming. The direction of the imbalance is what matters.

How many turns until caching pays for itself?

For a stable prefix on the short-lived tier, caching pulls ahead after roughly two turns. Turn one is a write at about 1.25x base input; each later turn is a read at about 0.1x instead of the 1.0x you'd pay uncached. Cumulative cost is 1.25 plus 0.1 per additional turn, which drops below the uncached line almost immediately and stays there. This only holds if the prefix actually stays stable across those turns. Confirm the rates on your provider's pricing page before relying on the exact crossover.

What invalidates a cached prefix?

Any change to the cached span, however small. A live timestamp near the top, a per-user or per-request detail, a tool list that reorders between calls, a rolling counter, even whitespace differences can break the match and force a fresh write. The cache keys on the exact prefix, so 'mostly the same' is not the same. Keep everything volatile below the cache breakpoint and everything static above it, so the stable part gets reused and only the small tail changes.

Should I use the 5-minute or 1-hour cache tier?

Match the tier to how often you actually reuse the prefix. The short tier costs about 1.25x base input to write and suits tight loops that hit the same prefix within minutes, which covers most interactive agent work. The long tier costs about 2x to write and only earns that heavier premium back if you reuse the prefix across a longer gap, like repeated queries against a big document over an hour. Overshoot the retention window and you pay for storage you don't use; undershoot it and the cache expires before your next read. Check current tier pricing on your provider before choosing.

Does caching output tokens help?

No, caching applies to input, not output. Prompt caching stores and replays a static input prefix, so it never touches the tokens the model generates. Output tokens are the most expensive line item on most providers and there's no cache discount on them. If output is your dominant cost, caching won't move it; you'd look at shorter completions, a cheaper model for the task, or fewer turns instead. Caching is a lever on the re-sent input prefix specifically.

Prompt Caching Read vs Write: When Caching Costs More Than It Saves

TL;DR

Prompt caching has two prices. A cache read is cheap, roughly a tenth of a normal input token. A cache write costs a premium, about 1.25x base input for the short-lived tier and 2x for the long-lived one.
Caching pays off only when reads outnumber writes. You pay the write premium once to populate the cache, then collect the cheap-read discount on every turn that reuses it.
If your cached prefix changes every turn, the entry is invalidated and you pay the full write premium again with zero read discount. That is net-negative caching: it costs more than not caching at all.
The diagnostic is one ratio. Compare your cache-write tokens to your cache-read tokens. Writes much larger than reads means you are paying the premium without the payoff.
The fix is measurement, not a flag. Look at your own read/write token split per model before you assume caching is helping.

You turned on prompt caching, added cache_control to the big static prefix, and waited for the bill to drop. It didn’t. Maybe it went up a little. This is a common and confusing outcome, because the sibling advice (“enable caching, you’re leaving a discount on the table”) is correct, and yet you did the thing and got nothing back. The missing piece is that caching is not one price. It is two, and one of them is a penalty. Whether caching helps or hurts comes down to how often you read the cache versus how often you rewrite it.

Why does a cache write cost more than a cache read?

Because writing to the cache does extra work, and the provider charges you for it up front. A cache read reuses tokens the provider already stored, so it is cheap. A cache write asks the provider to process and persist a fresh prefix, so it carries a premium over a normal input token.

On one major provider the numbers are precise. A cache read costs about 0.1x a base input token. A cache write to the short-lived tier costs about 1.25x base input, and the long-lived tier costs about 2x. So the two operations sit on opposite sides of the base rate: reads well below it, writes above it.

Horizontal bar chart of per-token cost relative to base input: cache read 0.1x, base input 1.0x, cache write 5-minute 1.25x, cache write 1-hour 2.0x.

That gap is the whole game. A write costs you 1.25 to store a prefix once. Each later read of that same prefix costs 0.1 instead of the 1.0 you’d pay to re-send it uncached. You are trading a one-time premium of 0.25 for a recurring discount of 0.9 per reuse. Do that trade enough times and you come out far ahead. Fail to reuse the prefix and you just paid the premium for nothing.

When does prompt caching actually lose money?

When you write the cache more than you read it. The clean case is a stable prefix reused across turns. The failure case is a prefix that changes every turn, so nothing ever gets reused.

Walk the stable case first. You have a 10,000-token system prompt and history that you re-send on every turn of an agent loop. Turn 1 writes it to the cache at 1.25x. Turns 2 through N read it back at 0.1x each. Cumulative cost after N turns is 1.25 + 0.1·(N−1). Uncached, you’d pay 1.0 per turn, so 1.0·N. The cached line crosses below the uncached line around turn 2 and then stays flat while the uncached line climbs. By turn 10 the cached path costs about 2.15 against 10.0 uncached. That is the win everyone means when they say “turn caching on.”

Now break the prefix. Suppose something near the top changes every turn: a live timestamp, a per-request user detail, a reordered tool list, a rolling counter. Each change invalidates the cached entry, so every turn is a fresh write at 1.25x and never a read. Cumulative cost is 1.25·N, which is above the uncached 1.0·N at every single turn. You didn’t just fail to save. You made it worse, permanently, by exactly the write premium.

Cumulative cost over turns 1 to 10 for three strategies: uncached climbs linearly to 10, cached with a stable prefix flattens near 2 after turn 2, cached but invalidated each turn climbs to 12.5 and stays above uncached the whole way.

How do you tell if your caching is net-negative?

Compare your cache-write tokens to your cache-read tokens. That one ratio tells you which line of the chart you’re on. Healthy caching produces reads that dwarf writes, because you write a prefix once and read it back many times. Broken caching produces the opposite: writes as large as reads, or larger, because the prefix keeps getting rewritten.

Most cost tooling collapses these into a single “cached tokens” number, which hides the exact thing you need to see. You want them logged separately. Then the question answers itself. Reads much greater than writes: caching is working, leave it on. Writes roughly equal to or greater than reads: the prefix is unstable and you’re paying the premium for nothing.

This is where measuring your own traffic beats any rule of thumb, because “does my prefix actually stay stable” depends on your prompts, not the docs. tj optimize cache reads the cache-read and cache-write token split straight out of your local session history and shows the ratio per model, so you can see whether caching is paying off on your real usage rather than in theory. It flags the candidate and shows the estimated split; it doesn’t claim a change is safe on its own. Once you can see the ratio, the fix is usually obvious: find what’s mutating in the prefix and move it below the cache breakpoint.

Does the 5-minute vs 1-hour cache tier change the math?

Yes, and in a way that can flip a marginal case either direction. The short-lived (5-minute) tier costs about 1.25x base input to write. The long-lived (1-hour) tier costs about 2x. So the long tier’s write premium is four times larger (1.0 extra vs 0.25 extra over base input), and you need far more reads to earn it back.

The tiers exist for different reuse rhythms. The short tier fits a tight loop where the same prefix gets hit again within minutes, which is most interactive agent work. The long tier fits a prefix reused across a longer gap, say a large document or codebase you’ll query repeatedly over an hour. Pick the long tier for a fast loop and you overpay on every write for a retention window you never use. Pick the short tier for a prefix you touch once an hour and the cache expires before your next read, so you rewrite from scratch and pay the write premium again. Matching the tier to how often you actually reuse the prefix is the difference between the two write bars in the first chart.

Common questions

I turned on caching and my bill went up. What happened?: Almost certainly a cached prefix that changes between turns. A cache write costs more than a normal input token, so if your prefix is invalidated every turn you pay that write premium repeatedly and never collect the cheap read discount. The result is strictly more expensive than not caching. Check your cache-write tokens against your cache-read tokens: if writes are as large as reads, the prefix is unstable. Verify the exact multipliers against your provider's current pricing, since they change.
What write-to-read ratio means caching is helping?: You want reads to dwarf writes. Healthy caching writes a prefix once and reads it back on many subsequent turns, so a ratio of, say, one write for every ten-plus reads is a strong signal it's working. When writes approach or exceed reads, you're rewriting the cache faster than you're reusing it and likely losing money. There's no universal threshold; measure your own split per model rather than assuming. The direction of the imbalance is what matters.
How many turns until caching pays for itself?: For a stable prefix on the short-lived tier, caching pulls ahead after roughly two turns. Turn one is a write at about 1.25x base input; each later turn is a read at about 0.1x instead of the 1.0x you'd pay uncached. Cumulative cost is 1.25 plus 0.1 per additional turn, which drops below the uncached line almost immediately and stays there. This only holds if the prefix actually stays stable across those turns. Confirm the rates on your provider's pricing page before relying on the exact crossover.
What invalidates a cached prefix?: Any change to the cached span, however small. A live timestamp near the top, a per-user or per-request detail, a tool list that reorders between calls, a rolling counter, even whitespace differences can break the match and force a fresh write. The cache keys on the exact prefix, so 'mostly the same' is not the same. Keep everything volatile below the cache breakpoint and everything static above it, so the stable part gets reused and only the small tail changes.
Should I use the 5-minute or 1-hour cache tier?: Match the tier to how often you actually reuse the prefix. The short tier costs about 1.25x base input to write and suits tight loops that hit the same prefix within minutes, which covers most interactive agent work. The long tier costs about 2x to write and only earns that heavier premium back if you reuse the prefix across a longer gap, like repeated queries against a big document over an hour. Overshoot the retention window and you pay for storage you don't use; undershoot it and the cache expires before your next read. Check current tier pricing on your provider before choosing.
Does caching output tokens help?: No, caching applies to input, not output. Prompt caching stores and replays a static input prefix, so it never touches the tokens the model generates. Output tokens are the most expensive line item on most providers and there's no cache discount on them. If output is your dominant cost, caching won't move it; you'd look at shorter completions, a cheaper model for the task, or fewer turns instead. Caching is a lever on the re-sent input prefix specifically.