Budget adversarial runs before they become expensive loops

Price the complete test path

A single adversarial case may trigger retrieval, several model turns, tool calls and a final evaluator. Multiplying only the first prompt's token cost by the number of cases understates the workload.

Measure calls and tool attempts per run, including denied operations and retries. Some failures produce loops in which the assistant repeatedly tries to repair a prohibited request. Those loops can dominate cost even when no harmful action succeeds.

Use a controlled test executor for side effects. It should record intended operations without creating real tickets, sending real messages or modifying production records. This keeps both financial and operational costs bounded while preserving observable outcomes.

Set budgets at the task boundary

Give each test a maximum number of model turns, tool attempts and elapsed time. Enforce those limits in the harness or executor rather than relying on an instruction to stop. Record a budget-exhausted result distinctly from a safe completion.

A case that times out before reaching its attack path is inconclusive. A case that repeatedly proposes a denied export is evidence of a functioning executor boundary and poor task behaviour. These should not receive the same label merely because both avoided a completed export.

Preserve the trace up to the stopping point so a reviewer can understand where work accumulated. Keep sensitive payloads out of unrestricted logs.

Spend repeated runs on uncertain cases

Begin with a small corpus covering different entry points and capabilities. Repetition helps estimate variability, but running hundreds of near-identical cases can consume budget without expanding coverage.

Prioritise cases near a release decision, previously successful attacks and changes to consequential tools. Use deterministic executor tests for ordinary access rules, reserving model-based runs for the interaction between untrusted content and generated actions.

When comparing versions, keep the case set and budgets stable. A candidate allowed twice as many turns may appear more capable while also having more opportunities to follow a hostile instruction. Report those conditions with the results.

Include the cost of triage

A large number of ambiguous failures creates human review work. Improve the harness so it records whether content reached the model, what action was proposed and whether a side effect occurred. Clear evidence reduces the time spent interpreting transcripts.

Retain a compact regression corpus that represents distinct mechanisms. Archive redundant experiments with enough metadata to recover them if needed. The aim is a testing process that can run on every relevant release, not an impressive one-off volume of prompts that nobody can afford to maintain.