Cap the whole run, not only the model request

Count every kind of work

An agent run may make several model calls, retrieve documents, query business systems and attempt mutations. Some tools trigger downstream jobs whose cost appears after the conversation ends. A token cap on one model request covers only a fraction of this work.

Record model usage, tool attempts, elapsed time and expensive downstream operations against the run identifier. Separate estimates from confirmed usage where a provider reports costs later. The accounting needs enough detail to explain an unexpectedly expensive run.

Use different limits for different capabilities. A hundred small reads and a hundred external mutations do not have equivalent consequences, even if their network cost is similar.

Reserve budget before concurrent work

If several tool calls can run at once, a simple check of the current total can overshoot the limit. Each worker may see available budget and proceed. Use a shared reservation or atomic accounting mechanism for the resource being bounded.

For example, reserve a mutation slot before accepting an operation, then record its final status. Decide whether failed or denied attempts consume the relevant budget. Attempt limits often should count them, because repeated invalid work still consumes capacity.

The exact accounting depends on the system, but the invariant should be clear: concurrent workers cannot collectively authorise more work than the run permits.

Stop without losing the result

When a limit is reached, prevent new work and account for operations already accepted. Do not abruptly discard a successful mutation because the final response budget ran out. Store the result and give the user a concise status through a deterministic path if needed.

Distinguish a recoverable pause from a cancelled task. A user may choose to continue a legitimate long-running job, but that should be a deliberate extension with current authority and a known remaining plan.

Inspect loops before increasing limits. Repeated calls with unchanged inputs often indicate a permanent error or missing information, not insufficient budget.

Choose limits from observed tasks

Measure ordinary and difficult legitimate runs, including retries and recovery. Set initial limits with room for normal variation while containing runaway behaviour. Document what users see when a task exceeds them.

Review cost per completed useful task alongside abandonment and operator intervention. An extremely low cap can make the product cheap per run but expensive to use because staff repeatedly restart it. The goal is predictable total work with an honest stopping state, not the smallest possible model request.