Articles

ZFS: pools, datasets, snapshots and scrub

Pools, datasets, copy-on-write snapshots and scrub: detecting and repairing corruption.

Reading: 5 minServer & Virtualization

Article cover: ZFS: pools, datasets, snapshots and scrub

ZFS fuses the volume manager and the filesystem, and adds three properties that decide whether data survives: every block carries a checksum stored by its parent, nothing in use is overwritten in place, and a snapshot is a reference to old blocks, not a copy. So corruption is detectable on every read, often repairable from a redundant copy, and found by the periodic scrub even where no read ever touched it.

The model: a pool, its vdevs and the datasets on top

A pool (zpool) is storage assembled from virtual devices (vdevs); the datasets — file systems, volumes and snapshots — divide the pool’s space. A virtual device is a single device or a collection of devices, organized according to certain performance and fault characteristics, and a pool’s top-level vdevs are its root vdevs.

pool (zpool)                    one pool, any number of root vdevs
 ├─ root vdev  mirror sda sdb   redundancy lives in the vdev, not the pool
 ├─ root vdev  raidz2 sdc..sdg
 └─ datasets on top of the pool
      ├─ pool/home          file system (the default dataset type)
      ├─ pool/home/alice    child: inherits properties from pool/home
      └─ pool/vm/disk0      volume (zvol): a block device, not a file system

$ zpool create tank mirror sda sdb mirror sdc sdd   # two mirrored root vdevs

Redundancy lives in the root vdev, not the pool. A mirror replicates data in an identical fashion across two or more devices; a raidz group is distributed parity, similar to RAID-5/6, surviving one, two or three failures by its parity count. “Virtual devices cannot be nested arbitrarily” — a mirror or raidz vdev is built from disks, not other mirrors — and losing a root vdev means losing the pool.

Datasets form a hierarchy that mirrors their names, and most properties are inherited down it. A dataset is a file system (the default), a volume (a block device a hypervisor attaches) or a snapshot; datasets share the pool’s free space, and quotas and reservations bound that.

What copy-on-write changes

Almost everything distinctive about ZFS follows from one rule: “a block that is in use is never overwritten”; changing data means writing it elsewhere and switching to the new version. The pool is one tree whose interior nodes hold block pointers, each recording where a child lives and its checksum — stored in the parent, not next to the block, so a damaged block cannot vouch for itself. A write changes the block, then its parent, up to the root block, and finally a new uberblock; until then the old tree still describes a complete, consistent pool, so the switch is atomic.

There is no half-updated state to replay, so no journal recovery and no mount-time consistency check — there is no fsck. A write goes to new blocks, so it consumes free space even when overwriting data, and a pool with none cannot be written to; the previous version of every touched block stays whole until nothing references it.

From a checksum to a repair

All metadata and data is checksummed, and ZFS repairs bad data from a good copy when corruption is detected. The checksum is calculated when the block is written, so it identifies a block as the one ZFS stored. End-to-end checksums — “a key feature of ZFS” — separate ZFS from older RAID and filesystems: they detect corruption on reading from media, and a corrupt block is repaired automatically when possible, from RAID protection or redundant copies.

A checksum error is not a disk that reported a failure — it is a disk that returned data expected to be correct but which was not, an instance of silent data corruption. The repair has one precondition: a good copy must exist. A block can be rebuilt from raidz parity or a mirrored replica, but only if the pool uses some form of redundancy; without it, a single case of bit corruption can render all the data unavailable.

Scrub: what it does, and what it cannot do

Corruption that is never read is never caught. A normal scrub examines all data in the pool and verifies each block’s checksum, and for replicated devices repairs any damage it discovers. It differs from resilvering, which examines only data ZFS knows to be out of date, as after a replacement disk is attached: a scrub looks at all data for silent errors due to hardware faults or disk failure.

The limits follow from the mechanism: a scrub checks every block against the checksum ZFS itself wrote and repairs from redundancy, so it cannot

  • repair a block when no good copy exists — a pool with no redundancy, or a raidz2 group already missing too many disks;
  • see logical damage: a file overwritten with wrong content, encrypted by ransomware or simply deleted was written correctly, so the checksums match.

A scrub is a media-integrity check, not a recovery tool.

A snapshot is a dataset property, not a copy

A snapshot is a read-only copy of a file system or volume at a point in time, created atomically as a consistent image of a dataset. But it is not a copy of the data: because ZFS never modifies a block in place, keeping an old version costs nothing more than a reference to the blocks that were live at that moment, and a snapshot’s name belongs to its dataset’s namespace — dataset@name — so it is a property of that dataset.

It consumes space only as the live dataset diverges, when blocks only the snapshot still references can no longer be freed. There this sheet stops: the copy-on-write chain, how snapshots accumulate against the pool’s free space, and why a snapshot in the same pool is not a backup belong to the sibling sheet on snapshots and their limits.

Limits and the common error

The common error is to treat ZFS as a guarantee of safety. ZFS makes corruption detectable on every read and repairable when a good copy exists; it does not create that copy, nor judge whether the data was ever right.

  • Checksums detect; redundancy repairs. A single-device pool has the first without the second.
  • A scrub covers media, not intent. Deletion, a bad overwrite and ransomware all leave valid checksums.
  • A snapshot is not a backup. It lives in the same pool, depends on the same vdevs, and goes with the dataset.

Level and prerequisites

L3 — the model, copy-on-write and corruption handling: how a pool is composed, why nothing is overwritten in place, and what a scrub can and cannot repair. It is beyond L2, which stops at composing volumes and filesystems and resizing them, and short of L4, where allocation and tuning begin. Prerequisites: the L1 sheet on RAID, the L1 sheet on storage media and the L2 sheet on local storage and volume management.

Where to go next

The sibling sheet on snapshots and their limits continues the copy-on-write chain and the pool space it holds. Replication and the mistakes it replicates, and the sheet on full, incremental and differential backup, cover taking a copy off the pool, where zfs send and zfs receive live. Retention, immutability and separate copies takes the property of the copy itself.

References

Nodrius Field Kit — cover

Get the Nodrius Field Kit

Enter your email and we'll send you the complete Field Kit: eight technical resources in one download.

Your email address is sent to Nodrius and used to deliver the Field Kit to you by email. Marketing messages are separate and optional: the checkbox does not affect your download, and you are only added to our updates list if you tick it. Privacy Policy.

Privacy & cookies

This site does not use cookies, analytics or tracking, and it does not profile you. There is nothing technical to switch off, so Accept and Reject change nothing about how the site works: either choice lets you browse everything normally.

The only difference is what your browser remembers: two entries in local storage — your choice here, so this notice is not shown again, and a marker that the Field Kit offer has already been made — so the offer is presented once and not on every page. They contain no identifier, they are not shared with anyone, and the site sets no cookie for either.

What we do with your data
How your email address is used when you ask for a Resource, how long it is kept and which rights you have — in the Privacy Policy.
What the site stores in your browser
Two entries in local storage — your choice on this notice and whether the Field Kit offer has been shown — and no cookies, no session storage, no third-party content: in the Cookie Policy.