Local and shared storage for virtualized workloads
Local disks and shared storage seen from the hypervisor: what each gives a workload.

Every virtual machine has at least one disk, and the guest sees only a block device. Where the bytes live is the datastore, and on a cluster the datastore decides which cluster functions are even possible. A hypervisor stores a VM disk in one of several shapes, and libvirt’s domain format names them: “Representation of the disk source depends on the disk type attribute value as follows” — a file (“The file attribute specifies the fully-qualified path to the file”), a host device (“The dev attribute specifies the fully-qualified path to the host”), or a network address, where “The protocol attribute specifies the protocol to access to the” image. The guest sees a SCSI or a virtio device either way. Proxmox VE states the split plainly: VM images “can either be stored on one or several local storages, or on shared storage like NFS or iSCSI (NAS, SAN)”. This sheet is about the guarantees a datastore must give a virtualized workload — and why a local disk cannot give the shared ones.
What a datastore has to guarantee
Five properties decide what a datastore can do for a cluster: who can see the bytes, who may write them, what the write path costs, how large the allocation unit is, and where capacity comes from. They are easiest to read side by side.
GUEST one block device — the guest never knows where the bytes are
│
HYPERVISOR <disk type='file'|'block'|'network'> ──▶ the datastore
│
LOCAL SHARED
visible to one host every host in the cluster
locking a pointer to that host cluster-wide lock (shared=1)
host fails the disk goes with it bytes stay; HA can restart
HA / migration no / copy the whole image yes / no copy, so faster
I/O path the host's own disk host stack + network
capacity that one device a pool several hosts consume
A local disk is not a worse datastore — it is a datastore whose scope is
one host, and that scope is exactly what forbids HA, migration and
maintenance without an outage.
Shared visibility: one set of bytes
The first guarantee is that every host sees the same content. A cluster filesystem or a distributed pool provides it: in Storage Spaces Direct “all the servers can see all of each other’s local drives”, and the Cluster Shared Volume layer “unifies all the ReFS volumes into a single namespace accessible through any server. To each server, every volume looks and acts like it’s mounted locally.” Proxmox VE keeps one configuration for every node, because “Sharing storage configuration makes perfect sense for shared storage, because the same “shared” storage is accessible from all nodes.”
But the configuration is not the storage: the shared flag only “marks an already shared storage as such”, and a local type kept in a shared configuration “is available on all nodes, but it is physically different and can have totally different content.” Two nodes can then carry the same storage id and different bytes — the trap behind a guest that starts on the wrong host.
Locking: one writer, and who arbitrates
Visibility is not coordination. When several hosts can write the same blocks, something must decide who may. A file-level datastore arbitrates on the server — “Some storage operation need exclusive access to the storage, so proper locking is required” — and that locking “does not work between different clusters.” A block datastore must be told it is shared: “The backend implements proper cluster-wide locking if the storage is marked as shared in the configuration.” Without a shared locking layer the hypervisor falls back on the guest’s owner, keeping its configuration on one node so that “This concept enables the usage of local locks instead of expensive cluster-wide locks for preventing concurrent guest configuration changes.” The lock is cheap because it is local — and that is why it fails together with the node.
The write path and its latency
The third guarantee is where the bytes travel. On the local path a virtual disk is served by the host’s own storage stack, close to the media. On the shared path every I/O crosses the network to the node that owns the media, which is why the fabric can be ordinary Ethernet, replacing, in Microsoft’s words, “costly and restrictive Fibre Channel or Shared SAS cabling”. Latency is then the sum of more hops — the same media can be slower to a guest — and a datastore that stops answering stalls a guest whose I/O is synchronous with its disk.
The allocation unit and the declared size
A datastore hands out space in units. On block media “Disk devices typically store data in fixed size blocks (e.g., 512 byte blocks, or 4,096 byte blocks)”. The hypervisor adds its own unit: a modern virtual hard disk is created with “the optimal 4 KB alignment”, and the guidance is to “match block size to the allocation patterns of the workloads using the disk”, because a bigger block “increases the amount of space the VHD uses on the host”. The declared size of a disk is not the space the datastore commits; that gap is the sibling sheet Virtual disks: thin and thick provisioning. What matters is that the datastore sizes its unit and its capacity for many guests at once, because when it runs out the failure is shared: “If a storage runs full, all guests using volumes on that storage receive IO errors.”
Why a local disk rules out HA, migration and maintenance
A local disk is visible to one host only, so nothing that needs the disk to outlive that host is available. Microsoft states it directly: “Standalone Hyper-V with local disks doesn’t support automatic failover of virtual machine (VM) workloads due to the lack of shared storage”. With shared storage the same workload “can move VMs to any node in the cluster using Live Migrate or automatically restart after a failure (failover)”. Proxmox VE gives the reason to share: it allows one to “live-migrate running machines without any downtime, as all nodes in the cluster have direct access to VM disk images”, and “There is no need to copy VM image data, so live migration is very fast in that case.” Maintenance is the same property: draining a host before a reboot is live migration repeated, and with a local disk the honest options are an outage or a copy of the whole image. Even with shared disks, after the owning node fails a regular migration is impossible “because such a local lock on the (offline) owning node is unobtainable”, and guests “with local disks (or other local resources which are only available on the offline node) are not recoverable like this.” Restoring it needs a distributed lock service and an HA stack, which belong to the cluster sheets.
Level and prerequisites
L4 — infrastructure and reliability: what a datastore must guarantee a virtualized workload, and which cluster functions a local disk forbids. Prerequisites: the L3 sheets NAS and SAN and Virtual disks: thin and thick provisioning, and the L2 sheet Local storage. The distributed systems that replace a shared LUN are the sibling sheet Shared and distributed storage.
Where to go next
- Server & Virtualization — the area this sheet belongs to, whose Storage node is storage.
- The sibling sheet Shared and distributed storage.
- The cluster sheets Clusters and quorum, High availability and failure handling, and Live migration and its prerequisites.
- Capacity planning and host maintenance, where shared storage turns a host reboot into a migration.
References
- Proxmox Server Solutions — Proxmox VE Administration Guide — https://pve.proxmox.com/pve-docs/pve-admin-guide.html
- Proxmox Server Solutions — Storage (Proxmox VE wiki) — https://pve.proxmox.com/wiki/Storage
- Microsoft — Storage Spaces Direct overview — https://learn.microsoft.com/en-us/windows-server/storage/storage-spaces/storage-spaces-direct-overview
- Microsoft — Hyper-V storage architecture — https://learn.microsoft.com/en-us/windows-server/virtualization/hyper-v/storage-architecture
- Microsoft — Hyper-V storage I/O performance — https://learn.microsoft.com/en-us/windows-server/administration/performance-tuning/role/hyper-v-server/storage-io-performance
- libvirt — Domain XML format — https://libvirt.org/formatdomain.html
- SNIA — The SNIA Dictionary — https://www.snia.org/dictionary