Cache
Measure how much of your input is served from cache, and where to place Anthropic breakpoints.
Cache surfaces prompt-caching opportunities. It has two findings under one name. cache measures how well you’re caching today. cache-recommend suggests where to add breakpoints.
tj optimize cache
tj optimize cache-recommendcache — measure current caching
This finding reads aggregate input and cache tokens from spans in the window and computes the share of input served from cache per (provider, model). It flags a row when input is at least 100,000 tokens over the window (below that, savings are negligible whatever the ratio) and cache efficacy is under 30%.
No content capture is required. It works on any data already in your DB.
Per-provider support varies:
| Provider | Support | Notes |
|---|---|---|
| Anthropic | Full | Cache-read and cache-creation tokens are populated by backfill, OTel patches, and the OTLP logs route. Numerically accurate. |
| OpenAI | Best-effort | Prompt caching is implicit and per-call hit data isn’t exposed consistently. The aggregate approximates cache-eligible input; the renderer surfaces a caveat. |
| Google Gemini | Best-effort | Some models report cache data, others don’t. Confidence varies per model. |
| Bedrock, LiteLLM, Cohere, others | Unsupported in v1 | Marked support = unsupported and never flagged. |
cache-recommend — suggest breakpoints (Anthropic only)
This finding walks captured Anthropic prompts, hashes the opening ~2000 characters of each, and flags prefixes shared by three or more calls. Each shared prefix is a candidate for a cache_control breakpoint.
It needs [capture] prompts = true in your config. Without captured content there’s no prefix to compare, and the analyzer exits with a hint. Non-Anthropic spans in the window are counted but not analyzed; multi-provider breakpoint detection is deferred beyond v1.
Each candidate carries the number of calls sharing the prefix, the average input size on those calls, a rough estimate of tokens that would become cache reads, and a short sample of the prefix for identification.
How to read the output
For cache, the headline is a share: what fraction of your input is already served from cache per model, and which rows fall below the 30% floor. For cache-recommend, the output is a ranked list of prefix candidates with the occurrence count and cacheable-token estimate.
Honesty caveat
Both findings carry a structural confidence level. They detect a pattern in the captured data. They don’t validate that enabling caching produces the same model output — for Anthropic the cache is content-addressed so the output is identical, but that isn’t a general guarantee. The cache framing is deliberately “you’re getting X% of available caching,” a measured share rather than a promised saving.