Do cache writes really cost more than normal input tokens?

Yes, on the providers that offer prompt caching. On one major provider a cache write costs roughly 1.25x a normal input token for the short-lived tier and more for the longer-lived tier, while a cache read costs about a tenth of a normal input token. Caching only pays off when reads substantially outnumber writes. If your cached prefix changes between turns, you keep paying the write premium and never collect the read discount. Confirm the exact ratios against your provider's current pricing page, since they change.

Why is my agent bill so much higher than the single call I made?

Because you were billed for the loop, not the call. Three multipliers stack. The full conversation, system prompt plus growing history plus every tool result, is re-sent on each turn, so a 30-turn session re-pays for its own transcript many times over. Fan-out spawns sub-agent conversations that each carry their own full bill. And failed steps trigger retries that cost tokens with nothing to show. The call you typed is the top of a tree; the bill is the whole tree.

What are the four token counts I should be logging?

Input tokens, output tokens, cache-read tokens, and cache-write tokens, logged separately for every API call. Collapsing cache reads and writes into one number hides the single most common caching mistake, paying the write premium without earning the read discount. Once you have all four, attribute them by agent, session, and tool so you can see which surface is actually driving spend.

Does trimming my system prompt actually save much?

In an agent loop, yes, because the system prompt is re-sent every turn. Cutting 1,000 tokens saves 1,000 input tokens once in a chat app, but 30,000 across a 30-turn session, plus the downstream cost of those tokens riding along in later cache writes. Static, re-sent-every-turn content gives the best return on trimming. Verify against your own per-turn input token counts before and after rather than assuming a fixed percentage.

How do I stop tool calls from ballooning my context?

Filter tool outputs before they enter the transcript. A raw API response, a full file, or a verbose log gets re-read on every subsequent turn, so a 2,000-token result added on turn 3 is still being billed on turn 20. Return only the fields the agent needs, summarize long results, and drop stale tool outputs from context once they are no longer relevant. The goal is to keep the accumulated transcript lean, since every turn re-pays for all of it.

Is fan-out cost something I can measure, or just estimate?

You can measure it, but only if you attribute spend below the top-level request. Tag every sub-agent conversation with a parent identifier and log its tokens independently, then roll them up per top-level task. Without that tagging, a fan-out tree looks like one cheap call at the top and a mystery total on the invoice. The measurement is straightforward; the discipline of tagging every spawned conversation is the part teams skip.

What Actually Costs Money in an Agent Loop

If you have watched an agent run for an hour and then opened the bill, the number rarely matches your intuition. A single request went in. Somehow it cost dollars. The gap is the loop. An agent does not make one API call; it makes a sequence of them, and each call re-pays for most of what came before. This post walks the actual cost drivers, turn by turn, so the bill stops being a surprise.

What gets billed on a single agent turn?

Three token buckets, and they are priced differently. Input tokens are everything you send to the model: the system prompt, the conversation history, tool definitions, and the latest message. Output tokens are what the model generates, and they are the most expensive per token on every major provider, usually several times the input rate. Cached tokens are input tokens the provider has already seen and stored, split into two very different line items.

The trap is treating a turn as a one-shot inference. In a loop, each turn carries the weight of every turn before it. That is where the money goes.

Why do cache writes cost more than cache reads?

Prompt caching lets a provider store a static prefix of your prompt and reuse it, so you are not re-billed at the full input rate for content that did not change. It has two operations, and they are not symmetric.

A cache read is cheap. On one major provider, reading a cached token costs about a tenth of a normal input token. That is the win everyone talks about.

A cache write costs more than a normal input token. On that same provider, writing a token into the short-lived cache runs about 1.25x the base input rate, and the longer-lived tier costs more again. This is the part most people miss. Caching is not free to turn on; you pay a premium to populate the cache, and you only come out ahead when reads outnumber writes by enough to amortize that premium.

This matters in a loop for a specific reason. If anything in your cached prefix changes between turns, a timestamp, a reordered tool list, a per-user detail slipped near the top, the old cache entry is invalidated and you pay the write premium again with none of the read savings. A prompt that looks slightly different every turn is the worst case: full write price, every time, no discount. TokenJam tracks cache reads and cache writes as separate fields precisely because collapsing them hides this. If your caching “isn’t saving anything,” the write-to-read ratio is usually why.

How does context bloat multiply across a loop?

The system prompt is re-sent on every turn. So is the history. Neither is billed once.

Say your system prompt and tool definitions total 4,000 tokens. On a single call that is a rounding error. Across a 30-turn agent session it is 120,000 input tokens before the agent has done anything useful, and that is only the static part. The conversation history grows on top of it. By turn 20, the model is re-reading turns 1 through 19 every single step. The transcript is the tax, and it compounds.

Agent transcript growth over five turns. Each turn re-sends the system prompt plus the full accumulated history and tool results, so the billed context grows with every step.

This is why prompt discipline pays off more for agents than for chat. Trimming 1,000 tokens from a system prompt saves you 1,000 tokens once in a chat app. In a 30-turn loop it saves 30,000 input tokens, plus whatever those tokens would have cost as they rode along in later cache writes. Small, static, re-sent-every-turn content is where the quiet spend lives.

Why do tool calls re-pay for the whole conversation?

Each tool result gets appended to the context and sent back to the model so it can reason about what the tool returned. That is the whole point of tool use, and it is also a cost multiplier.

Walk it through. The agent calls a search tool. The result, maybe 2,000 tokens of retrieved text, is added to the transcript. On the next turn the model reads the entire transcript, including that 2,000-token result, to decide the following action. It calls another tool. Now the next turn re-reads both results. A loop that makes ten tool calls does not pay for ten results once each; it re-pays for the accumulated pile on every subsequent step. Large tool outputs, verbose logs, full file contents, unfiltered API responses, are the ones that hurt, because they ride along in context for the rest of the session.

What is the fan-out multiplier?

One prompt does not always mean one billing stream. Many agent architectures spawn sub-agents or run parallel branches: a planner delegates to workers, a coordinator fans a task out to specialists, a loop kicks off several exploratory attempts at once. Each of those is an independent conversation with its own system prompt, its own history, and its own bill.

So the real cost scales with the branching factor, not the single top-level call you typed. One prompt that spawns eight sub-agent conversations, each running a handful of turns, is eight loops billed in full. This is the most common reason a bill looks an order of magnitude larger than expected: the person estimating cost is pricing the one call they see, and the system is running the tree underneath it. If you attribute spend only at the top level, fan-out is invisible until the invoice arrives.

Fan-out: one prompt spawns multiple sub-agent conversations, and each one is an independent bill, so the real cost scales with the branching factor rather than the single top-level call.

Where do retries and re-planning waste tokens?

The clean version of an agent loop assumes every step works. Real loops fail steps. A tool returns malformed output, a dependency times out, the model produces something that does not validate. The agent notices, reasons about the failure, and tries again. Every one of those cycles is billed at full price, and it produces nothing you keep.

Re-planning is the quieter cousin. An agent that loses its thread and re-derives its plan, or a fleet of short-lived worker sessions that each re-plan the same task from scratch because none of them can see the others’ work, spends real tokens generating output that gets discarded and regenerated. The tokens are gone whether or not the plan was any good. Runaway retry loops are the extreme case, a stuck agent can burn a month of budget in an afternoon, but steady low-grade retry and re-plan waste is more common and harder to spot, because no single session looks alarming.

The honest takeaway: you can’t cut what you can’t see per turn

Every driver above is invisible if you only look at the monthly total. A bloated system prompt, a cache that writes more than it reads, tool outputs riding along for twenty turns, a fan-out tree, a fat tail of retry-heavy sessions, all of them roll up into one number that tells you nothing about which lever to pull.

The fix is not a clever trick. It is measurement at the right grain. Log the four token counts separately for every call, input, output, cache-read, and cache-write, and attribute them by agent, session, and tool. Then the questions answer themselves. Is caching helping? Compare read tokens to write tokens. Is a prompt bloated? Look at input tokens per turn over time. Are loops the problem? Sort sessions by cost and check whether the top few percent dominate. That per-turn, per-attribute breakdown is exactly what TokenJam’s tj optimize reads out of your own local session history, so the finding is grounded in your traffic rather than a generic rule of thumb. It flags candidates and shows the estimated recoverable spend; it never claims a change is safe on its own. Measurement first, then decide.

Common questions

Do cache writes really cost more than normal input tokens?: Yes, on the providers that offer prompt caching. On one major provider a cache write costs roughly 1.25x a normal input token for the short-lived tier and more for the longer-lived tier, while a cache read costs about a tenth of a normal input token. Caching only pays off when reads substantially outnumber writes. If your cached prefix changes between turns, you keep paying the write premium and never collect the read discount. Confirm the exact ratios against your provider's current pricing page, since they change.
Why is my agent bill so much higher than the single call I made?: Because you were billed for the loop, not the call. Three multipliers stack. The full conversation, system prompt plus growing history plus every tool result, is re-sent on each turn, so a 30-turn session re-pays for its own transcript many times over. Fan-out spawns sub-agent conversations that each carry their own full bill. And failed steps trigger retries that cost tokens with nothing to show. The call you typed is the top of a tree; the bill is the whole tree.
What are the four token counts I should be logging?: Input tokens, output tokens, cache-read tokens, and cache-write tokens, logged separately for every API call. Collapsing cache reads and writes into one number hides the single most common caching mistake, paying the write premium without earning the read discount. Once you have all four, attribute them by agent, session, and tool so you can see which surface is actually driving spend.
Does trimming my system prompt actually save much?: In an agent loop, yes, because the system prompt is re-sent every turn. Cutting 1,000 tokens saves 1,000 input tokens once in a chat app, but 30,000 across a 30-turn session, plus the downstream cost of those tokens riding along in later cache writes. Static, re-sent-every-turn content gives the best return on trimming. Verify against your own per-turn input token counts before and after rather than assuming a fixed percentage.
How do I stop tool calls from ballooning my context?: Filter tool outputs before they enter the transcript. A raw API response, a full file, or a verbose log gets re-read on every subsequent turn, so a 2,000-token result added on turn 3 is still being billed on turn 20. Return only the fields the agent needs, summarize long results, and drop stale tool outputs from context once they are no longer relevant. The goal is to keep the accumulated transcript lean, since every turn re-pays for all of it.
Is fan-out cost something I can measure, or just estimate?: You can measure it, but only if you attribute spend below the top-level request. Tag every sub-agent conversation with a parent identifier and log its tokens independently, then roll them up per top-level task. Without that tagging, a fan-out tree looks like one cheap call at the top and a mystery total on the invoice. The measurement is straightforward; the discipline of tagging every spawned conversation is the part teams skip.