Articles

Replication and the mistakes it replicates

Synchronous and asynchronous replication, and why a replica faithfully copies corruption.

Reading: 4 minServer & Virtualization

Article cover: Replication and the mistakes it replicates

Replication keeps a second copy of a volume on another system and updates it as the primary changes. Data replication is “Continuously maintaining a copy of data” that provides high availability and redundancy, and it “may be used for disaster recovery and business continuance” (SNIA); every mode copies the current state, and none keeps a version history. What a failure costs is decided by one question: when the write is acknowledged.

        write from the application
                    |
        +-----------+-----------+
        |                       |
   SYNCHRONOUS             ASYNCHRONOUS
        |                       |
   primary log             primary log
        |                       |
        | (network)             +--> ack to the application
        |                       |         (the write is done)
   replica log                  |
        |                       +--(network, later)--> replica log
        v                                                  |
   ack to the application                                  v
        |                                       the window = data still
   write completes                              only at the primary
        |
   achievable RPO = 0                       achievable RPO = the window

   both paths copy every write, including the wrong ones: a deletion,
   a corruption or an encryption is replicated as faithfully as the data.

The write, and when it is acknowledged

The two modes differ in one place: where the write is acknowledged. In synchronous replication the data “must be committed to stable storage at both the primary site and the secondary site before the write is acknowledged to the host” (SNIA): the source does not answer the I/O immediately — “the originating storage doesn’t acknowledge the I/O immediately” — and the changes reach the destination and return first. It gives zero data loss at the file system level, but the round trip sits inside the application’s I/O path; the same documentation asks for “an average of 5-ms roundtrip latency or less for synchronous replication”.

Asynchronous replication commits at the primary alone: data “must be committed to storage at only the primary site and not the secondary site before the write is acknowledged to the host”, then is “forwarded to the secondary site as the network capabilities permit” (SNIA), and the engine “captures the write and immediately acknowledges it to the application”.

The gap is the difference. A synchronous pair reports a recovery point objective (RPO) of zero, because the write was not complete until both copies held it; an asynchronous pair has a higher-than-zero RPO, equal to the window itself. NIST SP 800-209 notes that the delay and frequency of asynchronous replication “are dictated by the enterprise’s disaster recovery policy”; the mode sets what the infrastructure can reach, and setting the objective is a separate job.

Consistency: the order the writes are replayed

Copying blocks is not enough if two volumes are applied in a different order on the two sides: the replica may hold a combination of states the application never produced. The consistency group prevents this — “A collection of replication sets grouped together to ensure write order consistency across all the replication sets’ primary volumes” (SNIA) — and an operation on the group applies to every member. Storage Replica states it operationally: write ordering means “applications like SQL Server can write to multiple replicated volumes and the data is written on the destination server sequentially”. It works at the partition layer and replicates the Volume Shadow Copy Service (VSS) snapshots Windows or backup software creates; crash consistency is guaranteed, but application consistency depends on the snapshot the application was asked to quiesce.

The snapshot as the base for an incremental transfer

A transfer over distance needs a shared starting point, and ZFS makes this explicit: zfs send “Creates a stream representation” of a snapshot, and “By default, a full stream is generated”; add -i and it will “Generate an incremental stream from the first snapshot (the incremental source) to the second snapshot (the incremental target)”, which the target applies with zfs receive. The stream carries only what changed since the snapshot. Where the provider hides the mechanism the shape is the same: an Amazon EBS snapshot is “an incremental backup”, saving “only the blocks on the volume that have changed since the most recent snapshot”, and its data “is automatically replicated across all Availability Zones in the Region”. Microsoft’s Storage Replica takes the opposite route, replicating continuously rather than from checkpoints, so “the delta of changes tends to be far lower”.

What the replica copies as faithfully as the data

This is the caution the title names: a replication engine does not judge the write, it copies it. Microsoft states the consequence — “Storage Replica replicates all changes to all blocks of data on the volume, regardless of the change type. If a user deletes all data from a volume, Storage Replica replicates the deletion instantly to the other volume, irrevocably removing the data from both servers” — and draws the conclusion: “Don’t use Storage Replica as a replacement for a point-in-time backup solution.”

The window decides how fast the damage arrives, not whether it does: under synchronous replication a deletion, a corruption or an encryption is committed to both copies before the application is told the write succeeded; under asynchronous replication it travels as the next delta. Either way the replica holds the present, and the present is damaged — only a copy made before the event and held apart can go back. NIST SP 800-209 therefore lists replication among the things an attacker targets: privilege escalation “can also compromise backup copies of the data (e.g., synchronous/asynchronous copies, snapshots) or the generation of future backups”, and when the copies cannot be reached the copying process itself is poisoned. CISA reaches the same point operationally: it is “critical to maintain offline, encrypted backups of data”, because ransomware variants “attempt to find and delete any accessible” copies.

Availability and continuity, not a versioned copy

Replication’s job is to keep a service running and a site interchangeable — availability and continuity, not recovery to an earlier point. SNIA pairs the copy with “high availability and redundancy”, and Microsoft notes that a higher-than-zero RPO makes asynchronous replication “less suitable for high-availability solutions like failover clusters”. The mode sets the achievable RPO — zero on a synchronous pair, the window on an asynchronous one — but it never delivers a version: there is one copy of the current state, and once an error has been replicated there is no copy of the state before it. A replica is for availability and continuity; it does not replace a copy that is isolated, versioned, and can be taken back to a point before the damage.

Level and prerequisites

L3 — mechanism, consistency and trade-offs: it explains how the two modes work and what RPO each can reach, without designing a topology or setting the objectives themselves. Prerequisites are the L1 storage model of blocks and volumes and the L2 sheets on local storage and on snapshots; how RPO and RTO are decided, owned and verified belongs to the governance sheet of the same class, and the ZFS pool and the backup chains are separate sheets.

Where to go next

  • Knowledge — the index.
  • Server and virtualization — the area this sheet belongs to.
  • Storage — this sheet’s node.
  • Companion sheets, still in draft: snapshots and their limits; full, incremental and differential backup; retention, immutability and separate copies; RPO and RTO, which owns the objectives.

References

Nodrius Field Kit — cover

Get the Nodrius Field Kit

Enter your email and we'll send you the complete Field Kit: eight technical resources in one download.

Your email address is sent to Nodrius and used to deliver the Field Kit to you by email. Marketing messages are separate and optional: the checkbox does not affect your download, and you are only added to our updates list if you tick it. Privacy Policy.

Privacy & cookies

This site does not use cookies, analytics or tracking, and it does not profile you. There is nothing technical to switch off, so Accept and Reject change nothing about how the site works: either choice lets you browse everything normally.

The only difference is what your browser remembers: two entries in local storage — your choice here, so this notice is not shown again, and a marker that the Field Kit offer has already been made — so the offer is presented once and not on every page. They contain no identifier, they are not shared with anyone, and the site sets no cookie for either.

What we do with your data
How your email address is used when you ask for a Resource, how long it is kept and which rights you have — in the Privacy Policy.
What the site stores in your browser
Two entries in local storage — your choice on this notice and whether the Field Kit offer has been shown — and no cookies, no session storage, no third-party content: in the Cookie Policy.