# Simulate a network partition, not only a stopped server

The harder failover case is an old region that some clients can still reach. Test how the design prevents conflicting writes under partial visibility.

By Cobnex editorial. Published 2026-09-10. Updated 2026-09-11.

## Define who loses connectivity

Use a safe environment and specify which paths are interrupted: health observers, application clients, replication or administrative access. Keep the rest of the topology explicit.

Stopping every primary component at once can make failover look simpler than it is. A partition leaves some components alive and creates disagreement about whether the service is available.

Choose a controlled fault that can be reversed and record the expected authority state before introducing it.

## Observe the failover decision

Check which signal causes the recovery process to begin and who or what makes the decision. A single failed probe should be interpreted according to the design's documented policy.

Verify the mechanism that prevents the old writer from continuing independently. Use the platform's supported fencing or coordination behaviour rather than assuming traffic routing provides it.

Attempt writes through the remaining reachable paths in the test environment. The expected outcomes should preserve one coherent history under the chosen model.

## Track ambiguous operations

Create a synthetic operation near the partition boundary and lose its response. Determine whether it committed before retrying from the recovery path.

Check replication state and the operation ledger or provider evidence available. The recovery region may not yet know about an effect that occurred elsewhere.

Verify that duplicate protection survives the regional transition. A region-local idempotency store that was not replicated may not recognise the original operation.

## Restore connectivity carefully

When the old region becomes reachable again, confirm that it does not automatically resume authority over stale data. Follow the documented rejoin or rebuild procedure.

Reconcile the synthetic operations and complete a user workflow from the active region. Record any data gap and the time required to establish the final state.

Keep the test's exact partition model in the evidence. Passing one scenario does not prove every possible network failure is handled, but it gives the team a concrete understanding of the authority mechanism under partial failure.

## Use a specific operation to expose disagreement

An illustrative dispatch request is useful because its effects are easy to distinguish. Give it one operation identity, let the primary accept it and interrupt the response path. Then submit that identity through recovery. The acceptable result is the documented status or reconciliation path, not a second dispatch created because the local operation table has no record. Inspect both local state and the safe test carrier endpoint to determine what actually happened.

## Sources

- [AWS: disaster recovery strategies](https://docs.aws.amazon.com/whitepapers/latest/disaster-recovery-workloads-on-aws/disaster-recovery-options-in-the-cloud.html)