Stop Paying Frontier-Model Prices for Work a Cheaper Model Handles

Here is the shape of the waste. You wired your agent with one default model, the strongest one, because getting an answer wrong was the expensive failure early on. Then the agent went to production and started making thousands of calls a day. Many of them are trivial: classify this ticket, pull the order ID out of this blob, apply a one-line patch the diff already spells out. Every one of those hits the frontier model at the frontier price. The bill is real and variable, and most of it buys reasoning the task never used. The hard part is not believing this. The hard part is knowing which calls, and what changing them is worth, without guessing.

The waste is in the shape of the call, not the vibe of it

I keep seeing the same pattern in agent traces. A call that takes 400 input tokens, emits 60 output tokens, makes zero or one tool call, and finishes in a single turn is not doing frontier-grade reasoning. It is doing a mechanical job. When a codebase full of those calls all route to claude-opus-4-8 or gpt-4o or gemini-2.5-pro, the per-call premium is invisible until you sum it across a month.

tj optimize downsize reads that structure directly. It walks every LLM call in your captured trace history and scores each one on the features that correlate with task difficulty: how much context went in, how much came out, how many tools it touched, whether it was one turn or a long back-and-forth. Calls that match a small-task shape get flagged as candidates for a cheaper model in the same class or a smaller provider tier. Then it does the part that turns an observation into a decision: it prices the delta in dollars, at your actual usage, so you see what the swap is worth before you touch anything.

# Instrument your agent once. The SDK captures every call locally.
from tokenjam import patch_anthropic, patch_openai

patch_anthropic()
patch_openai()

pipx install tokenjam    # or: pip install tokenjam
tj onboard
tj optimize downsize

$ tj optimize downsize
Scanning 1,204 calls across 318 sessions (last 30d)...

  Candidates: 41% of calls match a smaller-model shape
    • 494 of 1,204 calls: single-turn, <800 input tokens, 0-1 tool calls
    • Current models: gpt-4o, claude-opus-4-8, gemini-2.5-pro
    • Suggested target class: gpt-4o-mini / claude-haiku-4-5 / gemini-2.5-flash
    • Priced delta: $612/mo -> $58/mo on flagged calls (est. -$554/mo)

  ! Structural heuristic. These are candidates, not safe swaps.
    Validate with TokenJam Bench before changing a default.

The number that matters is the last line of the finding, the priced delta. It answers the question a cost dashboard leaves open: not “here is your bill” but “here is the specific slice of it you could move, and here is what moving it saves.” That framing comes straight out of agent token economics, where the unit that drives spend is the call, not the day.

It flags candidates. It does not tell you a swap is safe

This is the line I will not cross. A structural heuristic reads the shape of a call. It cannot know whether the cheaper model would have produced an answer that held, because the only way to know that is to run the cheaper model on the same task and check the result. A short, single-turn call can still be one where the small model quietly gets the logic wrong. Downsize flags it as a candidate and prices it. It never prints “safe,” “certified,” or “guaranteed.”

That test is TokenJam Bench. You take the exact candidate model Downsize flagged, replay it against your original on a task suite that mirrors the flagged work, and read the per-suite verdict: HOLDS or REGRESSED, with a Wilson interval and a McNemar test so a real drop reads differently from sampling noise. In a live Opus-to-Haiku run, the math suite held at a 2.5-point drop (noise) while the code suite regressed 37.5 points at p=0.0001. Same swap, two verdicts. Downsize finds the candidate from your traces. Bench decides whether it survives contact with your actual tasks.

The structural pass is what ships in the open-source CLI today. The tiers that replay flagged sessions automatically and fold in your own accept-or-reject history are Pro and roadmap.

Not autorouting, and not the quota reclaim

Two neighbors in this space do related things, and it is worth being exact about the difference.

Autorouting decides at runtime. It inspects each request as it arrives and picks a model for that one request, live, before the real call runs. Downsize decides after the fact. It analyzes your own past traces to tell you where your defaults are wrong, so you can change them once instead of predicting difficulty on every request forever. One is a live dispatcher. The other is an audit of history that tells you what the dispatcher should learn, or whether you even need one.

The other neighbor is quota reclaim on a Claude Code subscription, where the unit that runs out is your plan window, not dollars. That is a narrower question for a single subscriber managing one session. This post is for the API builder working in real dollars across multiple providers, with an analyzer that reads your whole trace history rather than one session’s worth of turns.

Three ways to attack over-provisioned model spend, and where each one acts.

Feature	Downsize	Autorouting	Opus quota reclaim
When it decides	After the fact, on past traces	At runtime, per request	After the fact, on one session
What it optimizes	Dollar spend across providers	Dollar spend per request	Subscription quota
Input it reads	Your captured multi-provider traces	The live incoming request	A single Claude Code session
Output	Candidate calls plus priced delta	A routed model choice	Whether Opus was needed

Where to start

Instrument your agent with the SDK so calls get captured, run tj onboard once, then run the analyzer against the history you already have:

pipx install tokenjam
tj onboard
tj optimize downsize

Everything runs locally. The traces stay on your machine, and the priced findings are computed against your own usage, not a vendor leaderboard. Read the candidate list, pick the findings with the biggest priced delta, and send those exact candidate models through Bench before you change a single default. Measure, validate, then move.

Common questions

Is a call that downsize flagged safe to downgrade?

No, and the tool will not tell you it is. A flagged call is a candidate, chosen by structural shape, that a cheaper model would likely handle. 'Likely' is not 'proven.' No static pass can know the cheaper model's answer would have held without actually running it. The honest way to close that gap is to take the exact candidate model and replay it against your original in TokenJam Bench on a suite that mirrors the flagged work. If Bench returns HOLDS, you have evidence. If it returns REGRESSED, downsize just saved you from shipping a quiet quality drop.

How is this different from autorouting? Don't they both pick cheaper models?

They act at opposite ends. Autorouting decides at runtime, inspecting each request and choosing a model for that one call live. Downsize analyzes your past traces after the fact and tells you where your existing defaults are over-provisioned, so you change a default once. One is a live per-request dispatcher; the other is an audit of history. You can use the audit to decide whether you even need the dispatcher.

My agent bill jumps around every month and I can't tell where it's going. Where do I start?

Instrument the agent with the SDK so calls are captured, run tj onboard, then run tj optimize downsize against your last 30 days. It groups your calls by structural shape, flags the fraction small enough that a cheaper model would likely handle them, and attaches a dollar figure to each finding. Now you have a ranked target list instead of a bill you can only stare at.

Does downsize work across providers, or just one?

Across providers. If you patch the SDK for Anthropic, OpenAI, and Google, the analyzer reads all of it and suggests a target in the same class per provider (a smaller Haiku, a mini, a flash tier). The priced delta is computed against each provider's real rates from your traces.

Does running this send my traces anywhere?

No. The SDK captures locally and tj optimize downsize reads that local history. The findings are computed on your machine. Nothing about your prompts or usage leaves unless you choose to export it.

What actually ships today versus what's paid or planned?

The structural heuristic, the Level 1 pass that scores call shape and prices the delta, ships in the open-source CLI. The tiers that automatically replay flagged sessions through the candidate model and learn from your own accept-or-reject decisions are Pro and roadmap. The read-only get_optimize_report is available over MCP if you want the findings inside another agent.