# Measure recovery by the final business state

A restarted worker is not a recovered task until its effects are correct and its status is understandable. Measure duplication, unresolved work and operator effort together.

By Cobnex editorial. Published 2026-09-10. Updated 2026-09-11.

## Define the expected result for each fault

For a lost response after a successful write, the desired outcome is a confirmed result without another business effect. For a crash before dispatch, it may be one eventual mutation after authority is rechecked. A generic "resume succeeded" metric cannot express both.

Create a fault matrix with interruption points and expected target states. Include pending approval, external timeout, expired permission and incompatible workflow version. These cases exercise different recovery decisions.

Keep the expected user-facing outcome too. A correct database state with a message inviting the user to repeat the action can still cause operational duplication.

## Count effects and unresolved operations

Track duplicate effects, missing effects and operations whose result remains uncertain. Measure how long uncertainty lasts and how many cases require manual reconciliation.

Separate retry attempts from business effects. Repeated network calls can be valid under an idempotent contract, while a duplicate mutation is a correctness failure. Conversely, making only one call does not prove success if the task remains unfinished.

Include abandoned and permanently failed runs in the report. Excluding them from recovery-time statistics can make performance appear better precisely when difficult cases are being left unresolved.

## Measure the cost of recovering

Record repeated model work, tool calls and operator time. A workflow that always restarts from the beginning may eventually reach the right state but waste substantial effort and encounter changed source data along the way.

Break recovery time into waiting for the dependency, automated reconciliation and human intervention. These intervals suggest different improvements. More workers will not fix a missing downstream status endpoint or a proposal awaiting an authorised reviewer.

Use representative workload sizes. A single short synthetic run establishes a mechanism, while larger scenarios reveal checkpoint volume, contention and backlog behaviour.

## Judge releases against the same interruptions

Run candidate versions against a stable set of fault cases and compare final states. Include old paused workflows when changing schemas or execution code. A new release that handles fresh runs well can still strand existing ones.

Set acceptance criteria according to the consequence of the operation. Document any cases that deliberately stop for manual review rather than forcing automatic completion. Recovery is dependable when the system reaches a correct, explainable state and keeps uncertainty visible until it is resolved.

## Sources

- [AWS: workflow redrive](https://docs.aws.amazon.com/step-functions/latest/dg/redrive-executions.html)
- [AWS Builders' Library: idempotent APIs](https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/)