Articles

Mounts and filesystems: attaching storage and making it survive a reboot

How Linux attaches a filesystem, and why one wrong /etc/fstab line can stall the boot.

Reading: 5 minServer & Virtualization

Article cover: Mounts and filesystems: attaching storage and making it survive a reboot

A filesystem becomes reachable only when it is mounted on a directory, and a mount made by hand is gone at the next reboot. On a systemd host that persistence lives in /etc/fstab, which is not read by a boot script but translated into native mount units; a required entry that cannot be mounted stops the boot from continuing. This sheet is about attaching storage, and about the failure that turns “one data disk is missing” into “the server does not start”.

The three objects: device, filesystem, mount point

Three separate things are involved, and most confusion comes from treating them as one.

  • The block device is the disk or partition, such as /dev/sda1.
  • The filesystem is the structure written on it (ext4, XFS) that carries files and directories. Creating it with mkfs is a different operation from the mount that exposes it.
  • The mount point is a directory in the running tree where the filesystem is attached.
device                 filesystem                 mount point
/dev/sdb1  --mkfs-->   ext4, XFS  --mount-->      /data

/                 the root filesystem
├── etc
├── var
└── data          while /dev/sdb1's filesystem is mounted here,
                  whatever /data held before is hidden, not merged

mount joins the filesystem to the mount point; umount detaches it. While a filesystem is mounted, the original content of the mount-point directory is not accessible: the mounted filesystem covers it, it does not merge with or delete it. The relationship between a device and a filesystem is also not always one-to-one — one filesystem can span several devices — which is why findmnt and lsblk carry a plural column (SOURCES, MOUNTPOINTS) instead of assuming a single device. Mounting requires privilege; the user option is the exception for one listed filesystem.

Attaching storage by hand

Identify the device and its filesystem, create the mount point, then mount.

# umount /mnt/data

mount normally works out the filesystem type by itself and only needs -t when detection fails. When the filesystem is already in /etc/fstab, naming only the device or the mount point is enough: mount reads the table to fill in the rest.

Making it survive a reboot: /etc/fstab

Each line of /etc/fstab has six whitespace-separated fields: the device, the mount point, the filesystem type, the mount options, the dump backup field, and the fsck check order.

UUID=ea74bbec-536d-490c-b8d9-5b40bbd7545b  /data  xfs  defaults,nofail  0  0

The last field is the boot-time check order: the root filesystem is given 1, other filesystems 2, and 0 disables the check. The dump field is legacy, and RHEL 9 no longer ships the dump utility, so it stays 0. The options field accepts the same options as mount: defaults expands to rw,suid,dev,exec,auto,nouser,async; noauto keeps a line out of mount -a and therefore out of the automatic boot mount; nofail means “do not report an error if this device is not there”.

On a systemd host, /etc/fstab is converted into mount units by systemd-fstab-generator early at boot and again whenever the manager reloads its configuration. After editing the file, run systemctl daemon-reload, or the running manager keeps the old picture while a bare mount may already work.

UUID and label, not the device name

/dev/sda, /dev/sdb and their partitions are named in the order the kernel detects disks, so adding or removing a disk can shift the letters. An /etc/fstab line that names /dev/sdb1 can therefore point at a different filesystem after a hardware change. The manual recommends a persistent identifier instead — UUID= or LABEL= for the filesystem, PARTUUID= or PARTLABEL= for a GPT partition — and lsblk --fs prints the filesystem’s own UUID and label for you to copy.

What a wrong entry does at the next boot

systemd’s boot ordering is what makes this matter. systemd-fstab-generator adds a Before=local-fs.target ordering to every mount unit for a local mount point, and local-fs.target is the target the rest of the boot waits on; a mount unit that cannot start takes the target with it, and boot-up cannot continue. The emergency shell is the state systemd uses when a required filesystem cannot be brought up. nofail is what changes this: it removes the ordering and makes the mount merely wanted rather than required, so a missing disk produces a warning and a boot that finishes without it. That is why the classic mistake is damaging: a hand-written /dev/sdb1, or a UUID typed with one character wrong, for a data disk without nofail, converts a missing disk into a server stopped at the console. Nothing validates /etc/fstab for you: the file is read, never written, and keeping it correct is the administrator’s job.

Verifying without rebooting

  • findmnt prints the mounted filesystems as a tree, and findmnt /data answers the narrower question “what is mounted here”.
  • lsblk --fs lists devices, partitions, filesystem type, label, UUID and the current mount point in one view.
  • df -h reports used and free space per mounted filesystem in human-readable units.
  • mount -a mounts everything the table expects, respecting noauto; note that mount(8) calls mount -a bad practice for checking /etc/fstab and points to findmnt --verify for that.

Read them together: device present, filesystem identified, mount point above it — a mistake in any layer shows up as a mount that silently did not happen.

Limits and the common error

Mounting on a non-empty directory does not merge the contents; the directory’s own files are hidden until the filesystem is unmounted. This produces the “disk is full but I deleted everything” trap, when a mount silently fails and an application keeps writing to the underlying directory. Weaker but real: the same filesystem mounted twice, or two mount points nested so that one hides the other.

What not to expect: nofail does not make the data available, it only lets the boot finish — the application then runs against an empty mount point. And defaults is not “no options”: it is the explicit set above, suid, exec and dev included.

Level and prerequisites

L2 — operational: attach a filesystem, make it persistent, and diagnose why it did not survive. Prerequisites are the L1 block-device model and basic shell use. Partition tables, volume groups and filesystem creation belong to the storage sheet.

Where to go next

References