AI abstention design
Keep examples of justified and unjustified refusals
The next team needs to know both when the assistant must stop and when it should help. Preserve the reasoning behind those expectations as the product evolves.
In this article
Build a balanced fixture set
Include supported questions, clarifiable requests, genuine evidence gaps and authority denials. Add cases the system previously refused unnecessarily and explain what evidence made them answerable.
Keep source versions and required claims with each fixture. A refusal that was correct before a policy was added may become a defect afterwards. The expected outcome depends on the available approved information.
Avoid storing only the current model's exact response. Preserve the intended behaviour so future models can phrase it naturally while meeting the same rule.
Transfer the outcome policy
Document the answer, clarification, limited-answer and escalation states the product supports. Explain how permission denials remain separate from uncertainty.
Name the owner of unresolved source gaps and the team that receives escalations. Support should not have to invent a destination whenever the assistant cannot answer.
Keep user-facing commitments accurate. If the product creates a draft summary rather than a submitted ticket, the handover should preserve that distinction in both code and copy.
Rehearse a full clarification and handoff
Ask the receiving team to resolve an ambiguous fixture by supplying one detail, then verify the final answer uses the correct source. Follow with an unanswerable case and inspect the escalation package.
Check that the recipient is authorised to see the evidence and can understand the unresolved point without rereading an entire conversation. Record any missing context that forces repeated work.
Include an access-denied case to ensure the team does not treat every refusal as a threshold-tuning problem.
Maintain the boundary with evidence
Review refusal patterns after source, model and retrieval changes. Track unnecessary refusals alongside unsupported answers and escalation workload.
When changing an expected outcome, record whether the source changed, the product scope changed or the old label was wrong. This keeps historical evaluations explainable.
A useful handover preserves the judgement behind stopping and continuing. The receiving team can then improve helpfulness without erasing the reasons the assistant sometimes needs to say that the available evidence or authority is insufficient.
Primary sources
Microsoft Learn: retrieval and answer evaluatorsOWASP: authorisation guidanceReferences checked 11 September 2026.