Most agent systems get budgeted like chatbots. One blended token price, multiplied by an expected number of calls, plus a margin for error. Then the bill arrives at three to five times the estimate, and nobody can point at the line that went wrong.
Nothing went wrong. The estimate was built on a fiction: that a token is a token.
Three prices, not one
On the major APIs, an output token costs roughly four to five times an input token. A cached input token, one the provider has seen before and kept warm, costs about a tenth of the fresh input price. Lay those side by side and the real price ladder spans a factor of forty between the cheapest token in your system and the most expensive one.
Most agent designs are written as if the ladder were flat. Teams trim prompts with great care, shaving a few hundred input tokens at a time, and then let the model write without limits. The instinct is exactly backwards. Context is the cheap part. The expensive token is the one the model writes.
The loop pays for verbosity twice
A chatbot pays for its answer once. An agent pays for its words, and then pays again to read them.
Every step of an agent loop feeds the previous step's output back in as input. The long reasoning trace, the verbose tool result quoted back in full, the status report written in flowing prose: all of it is generated at output prices, and all of it is re-read at input prices on the next turn, and the turn after that. A retry that regenerates a full answer pays the output cost again for the privilege of discarding the first one.
This is why two agents with identical call counts can have bills that differ by an order of magnitude. The difference is rarely the model. It is how much the system talks to itself.
Caching turns prompt structure into a budget decision
Prompt caching sounds like an infrastructure detail. In practice it makes the shape of your prompt a financial decision.
A cache hit depends on an exact prefix match. Everything before the first changed token is served at the cached price; everything after it is billed fresh. So the order of your prompt is a pricing surface. The stable parts, the system prompt, the tool definitions, the standing context, belong at the front, frozen. The volatile parts, the user request, the timestamp, the request ID, belong at the back. Put a timestamp near the top of the prompt and you silently bust the cache on every call. Plenty of teams do exactly that, and their cache hit rate sits near zero while the dashboard tells them caching is enabled.
Caching also has a break-even point. Writing to the cache costs more than a fresh input token, so a prefix has to be reused, roughly twice or more, before caching saves anything at all. Cache a one-shot prompt and you pay a premium to save money you never spend.
And the cache has a lifetime, measured in minutes. An agent that pauses for a human approval and resumes after the cache has expired pays full price to re-warm its context. A fast human loop is cheaper than a slow one in the most literal sense. Response time is a line item.
Orchestrators multiply everything
In a multi-agent system, every term in this equation gets multiplied by the topology.
Each sub-agent starts cold. The system prompt, the tool schemas, the assembled context: all of it is paid again for every spawn. A fan-out to ten sub-agents pays the preamble ten times, unless the platform deliberately shares one cached prefix across all of them.
A verbose sub-agent taxes the whole graph. Its report becomes the orchestrator's input. The orchestrator's summary, built partly from that report, gets fed into the next sub-agent. One agent that writes in paragraphs instead of bullet points raises the cost of every agent downstream of it.
The systems that keep this under control share a few habits. Sub-agents return structured summaries and references, pointers into shared storage, rather than transcripts and copied documents. The preamble is designed once, as a platform asset, and kept stable so it stays cacheable. And nobody quotes per-call cost anymore, because per-call cost has stopped meaning anything. The governed unit is the cost of one completed workflow, start to finish, however many agents it took.
The compaction tension
Long-running loops eventually face context limits, and the standard answer is compaction: summarize the history, rewrite it shorter, continue. Compaction is usually framed as a context-management decision. It is also a billing decision, and the two goals pull against each other.
Appending to a growing history keeps the prefix stable, which keeps the cache warm and the input cheap per token, right up until the context window or the accuracy gives out. Rewriting that history busts the cache in one stroke and re-bills the entire context at fresh input prices. Sometimes that trade is clearly worth it. Sometimes a loop pays the re-warm cost every few steps because compaction was set to an aggressive default nobody examined. The right answer depends on loop length, cache lifetime, and how much of the history is actually load-bearing. Most frameworks make the decision for you, silently, in the direction that demos well.
How to avoid the trap
None of this requires exotic tooling. It requires treating token flow as an architectural surface, the way you would treat latency or permissions.
Budget the workflow, not the call. Instrument cost per completed workflow, including retries and abandoned runs. That number is the one worth governing, and the one worth showing a finance partner.
Design the prefix once, and freeze it. Stable content first, volatile content last. Treat a change to the shared preamble the way you treat a schema change: reviewed, versioned, and rare.
Cap the output. Set length budgets per step. Prefer structured outputs over prose wherever a machine is the reader, which in an agent system is nearly everywhere. Let deterministic code do the formatting, the joining, and the summarizing of known shapes. Spend model tokens on judgment.
Make sub-agents report like they are on a meter. Summaries and references back to the coordinator. Transcripts are for debugging, sampled and stored, not passed hand to hand through the graph.
Route the janitor work down. Summarization, compression, classification, and extraction belong on small, cheap models. The frontier model should see the steps where its judgment is the product.
Count output tokens in your evals. Next to accuracy and latency, track output tokens per workflow. A change that buys one percent of accuracy at three times the output cost is a pricing decision. Make it on purpose, with the number in front of you.
The invoice is an architecture document
Read your bill that way and it stops being a surprise. Every line on it traces back to a design decision: a preamble that changed on every call, a sub-agent that reported in prose, a retry policy that regenerated instead of resuming, a compaction default nobody chose. Most of those decisions were made by accident, by people optimizing for the demo in front of them.
Designing as if every token costs the same is the most expensive assumption in the stack. The fix is not a cheaper model. The fix is noticing that in an agent system, the architecture is the pricing model.