• Token cost scales with conversation length, not task count.
• Multi-turn agents re-send a growing transcript at every step.
• Self-attention scales quadratically with sequence length, so longer context costs more than linearly.
• Cache stable prefixes, trim context to the working set, and route easy turns to cheaper models.
• Trimming too aggressively triggers silent retries that cost more than they saved.

Verdict: the levers ship, naive context slips

You shipped an agent that worked in the demo. Now the bill scales faster than usage, and nobody can point to a single expensive line. That is the trap: token cost does not grow with the number of tasks — it grows with the length of every conversation. Verdict: use context discipline, caching, and routing from day one; skip the habit of stuffing the whole history into every call. The levers are unglamorous and they hold. The naive full-context pattern is what quietly kills the margin.

Why the meter runs faster than you think

Two mechanisms compound. First, agents are multi-turn. The AgentBench framework [s1] evaluates models as agents across several interactive, multi-step environments, and every step re-sends a growing transcript — tool calls, observations, retries. Turn ten carries the weight of turns one through nine. Second, attention is not linear. The transformer's self-attention scales quadratically with sequence length [s2], so a context that doubles can cost far more than double to process. AgentBench [s1] also reports a significant gap between leading commercial models and open-source ones on these multi-step tasks, which means the expensive model is often doing work a cheaper one could finish.

The levers that hold — and the one that doesn't

Context discipline first: send the step what it needs, not the entire history — summarize old turns, drop stale tool output, pin only the working set. Caching second: stable prefixes (system prompt, tool definitions, few-shot) can be cached so repeated tokens are not re-billed at full rate; structure prompts so the constant part comes first. Routing third: send easy or well-structured turns to a smaller model and reserve the frontier model for the hard step — the AgentBench [s1] performance spread is exactly the signal that not every turn needs your most expensive model. The honest caveat: caching pays off only when prefixes are genuinely stable, and aggressive context trimming can drop a fact the agent needed, causing a silent retry that costs more tokens than it saved. Measure per-request token growth over a full session before you trust any of it. Untested at scale, a lever is a guess.