Start with one representative failed message

Identify the failure group

Group messages by bounded error category, consumer version and relevant time window. Avoid assuming every message in one queue failed for the same reason.

Choose a representative operation and inspect its trusted scope and history through authorised tools. Keep sensitive payloads out of broad incident channels.

Check retention deadlines early so evidence is not lost while the investigation continues.

Establish what already happened

Look for local effects and external operation results. A worker may have completed the action before failing to acknowledge or record it.

If the outcome is unknown, use the application's reconciliation path. Do not treat a missing local completion flag as proof that replay is harmless.

Verify whether the intended action is still valid against current business state.

Consider a synthetic dispatch command whose courier request succeeded before the worker crashed. The local order still says dispatch pending. Use the stored request identity to locate the courier result and repair the local mapping through the approved path. Replaying a fresh create request would arrange another collection.

If the courier lookup returns two plausible records, stop at review rather than choosing the newest. The recovery record should preserve that uncertainty and identify the person who can reconcile it. A clean queue is less important than an accurate account of the external effect.

Fix and test the cause

Resolve the dependency, code or contract defect in a safe test. Use the original failing shape where it can be handled under the data policy.

A malformed message may require a reviewed repair or rejection rather than replay through unchanged code.

Keep the consumer version and recovery decision associated with the selected operation.

Replay a bounded scope

Start at a controlled rate and inspect actual outcomes before expanding. Monitor current traffic and downstream limits.

Know how to stop further movement and which messages have already entered the destination. Cancelling a redrive does not necessarily retrieve work already moved.

Close each selected operation through its confirmed result or explicit unresolved state. Record the remaining categories and owners rather than ending the incident when a bulk movement task reports completion.