An average score can hide the expensive failure

Work through the denominator

Imagine a test set with ninety routine lookup questions and ten approval-rule questions. A candidate answers all routine questions correctly but misses four approval exceptions. Its overall correctness is 96 percent. That number does not tell a reviewer that it failed four out of ten cases in the consequential group.

The figures are illustrative, but the arithmetic shows why composition matters. An aggregate score reflects the mix of cases, not just the quality of the system. Adding more easy questions can improve the headline without fixing a single important failure.

Report counts and rates by task category and relevant consequence. Keep the overall result for context, but do not let it replace the breakdown.

Separate different properties

For a grounded-answer system, factual support, completeness and relevance answer different questions. An answer can contain only supported statements while omitting the exception needed to act correctly. A refusal can avoid unsupported claims while failing a request the system should handle.

For an agent, task completion, target selection and prohibited effects also need separate measures. A tool call with a successful status code may have changed the wrong record.

Choose assertions that expose these distinctions. Avoid combining them into one weighted score until the team understands what each component means and which failures cannot be traded away.

Account for uncertainty in the measurement

Publish sample sizes and repeated-run conditions. One failure in a small group deserves inspection, while zero observed failures does not prove the failure is impossible. Use more cases or repeated runs where variability affects a consequential decision.

Track evaluator disagreement and harness failures. If a model-based judge is uncertain or inconsistent on a particular category, a precise-looking average can conceal weak measurement.

Keep reviewed examples beside the metrics. They help a release owner understand whether a numerical change represents a minor wording issue, a missing qualification or an unauthorised action.

Decide before optimising the score

Set category-specific release criteria based on the product's intended use. Some failures may require a blocker, while others can be accepted with a visible limitation or escalation path. Make that policy explicit before comparing candidates.

Review regressions individually even when the average improves. The release decision should explain what became better, what became worse and why the remaining behaviour is acceptable. That is a stronger basis for shipping than a single percentage detached from the work users need to complete.