Articles

Virtual disks: thin and thick provisioning

Fixed and dynamic virtual disks, thin pools and overcommit: where the capacity risk sits.

Reading: 4 minServer & Virtualization

Article cover: Virtual disks: thin and thick provisioning

A virtual disk presents a guest with a block device of a declared size. Behind it the hypervisor keeps a file, or a set of blocks, on real media. The size the guest sees and the space the host has committed are different numbers, and provisioning is the study of the gap between them. An image utility reports the host-side figure as the “size reserved on disk which can be different from the displayed size”: the file can occupy far less than the guest believes it owns, and it will not release that space on its own.

Two numbers, not one

The first number is the declared capacity, or virtual size: what the guest partitions and formats, fixed at creation. The second is the allocated space — the bytes the host is actually holding. The SNIA Dictionary defines assigned capacity as “The amount of space on a system or data container which has been allotted to be written”: on a thin system it is “a promise of the amount of space that is available on demand”, on a fully provisioned system “usable capacity must be committed at the same time the container is allocated”. Thin provisioning “allocates the physical capacity of a volume or file system as applications write data, rather than preallocating all the physical capacity at the time of provisioning”; thick provisioning is the other half of that sentence.

A raw image on a filesystem with holes reserves only the written sectors; a measure operation reports the size a fresh image needs against the fully allocated size. Cloud block storage repeats the pattern: a volume is created at a chosen size within a published range.

The three provisioning states

Three states matter: thick and eager, thick and lazy, and thin. They differ at creation, on the first write, and on delete.

State At creation On first write On delete in the guest Typical mechanisms
Thick, eager the whole declared size is allocated at creation the write goes straight through, no allocation step the block stays allocated raw with preallocation=full; a fixed VHD
Thick, lazy the size is reserved on the host but not zeroed the block is zeroed before first use the block stays allocated raw with falloc; metadata preallocation; a fully provisioned LVM LV
Thin nothing is allocated; only the declared size is recorded a block is allocated and zeroed on demand the block stays allocated until a discard arrives qcow2 and dynamic VHDX; an LVM thin LV; a Storage Spaces virtual disk
GUEST VIEW        declared capacity, e.g. 500 GiB
                  |------------------------------------|
                  the guest formats and fills this whole range

HOST, THICK       allocated and zeroed at creation: 500 GiB
                  |####################################|
                  every byte reserved up front

HOST, THIN        allocated on demand: e.g. 30 GiB
                  |#####...............................|
                  only written blocks occupy the datastore

POOL              declared sizes may exceed real space
                  ten 500 GiB disks declared in a 2 TiB pool

At creation the difference is immediate. A fixed virtual hard disk is fully materialised: “When you create a fixed VHD file, the system allocates space for it”, and it “has the lowest CPU overhead of the three file options because read and write operations don’t need to look up the mapping of the block” — but a “large fixed-size VHD can take extra time to create”. The image formats offer the same choice through preallocation modes: “full mode preallocates space for image by writing data to underlying storage” is eager, and falloc only reserves space. A thin disk records only the declared size: “Thin LVs are thinly provisioned from a thin pool, and are created with a virtual size rather than a physical size”, and a dynamic VHDX allocates space “on-demand”.

Writes, deletes and the space that does not come back

On the first write to an unallocated block, a thin disk must find space and update metadata before the data lands: “This allocation increases the number of disk I/Os required for the write, increasing CPU usage.”

Deletion is the more deceptive half. When a guest deletes a file, its filesystem frees the blocks, but the virtual disk keeps them allocated; before unmap, “the storage space remained abandoned” on the host. Since Windows Server 2012, “Hyper-V supports unmap notifications”, and a VHDX file can report deleted data so the stack reclaims it — but the command reaches the host only through specific controller types, and trimming in Hyper-V also “requires pass-through or SCSI disks and trim-compatible hardware”. On Linux the mechanism is discard: “ignore causes the thin pool to ignore discards”, while passdown processes them and passes “the discards to the underlying device”. Without one of those paths, space released inside the guest never returns to the datastore.

The thin pool and overcommit

A thin pool backs many thin disks from one physical allocation, so their declared sizes are unbounded by its size: thin volumes “can be much larger than physically available space”. SNIA calls it oversubscription: “more services are requested than are able to be delivered”. An image tool states it: “virtual size (-V) and actual size (-L) together creates a sparse LV”; a Proxmox VE guide gives the case: a virtual machine with a 32 GB disk, after an install holding 3 GB of data, writes only 3 GB, “even if the guest VM sees a 32GB hard drive”. Storage Spaces puts the intent plainly: virtual disks “allocate storage capacity dynamically as data grows, rather than pre-allocating the entire capacity upfront”.

When the pool runs out it stops everything that depends on it: “If a storage runs full, all guests using volumes on that storage receive IO errors.” With error-when-full enabled, the layer will “immediately return an error when a thin pool is full and an I/O request requires space”; disabled, it queues “these I/O requests for a period of time to allow the thin pool to be extended”. The practical rule is to “avoid over-provisioning of your storage resources, or carefully observe free space”, and to watch the metadata pool.

When thick is the right choice

Thick provisioning buys predictability. Space is committed at creation and there is no first-write allocation. A fixed disk is also recommended when “storage on the hosting value isn’t actively monitored”: if no one watches free space, a thin or lazy disk can exhaust its backing store silently, while a thick disk cannot. The price is the space itself — a thick disk occupies its declared size even if the guest never fills it.

Limits and the common conceptual error

The error to avoid is reading the guest’s own numbers as if they were the host’s. Free space inside the guest is not free space on the datastore, and deleting files there does not return space until a discard reaches the storage layer. The second error is treating a thin disk’s declared size as a reservation: it is a maximum the disk can grow into, not space set aside.

Level and prerequisites

L3 — provisioning states, allocation timing, pool failure: this sheet assumes the reader knows what a virtual machine is. Prerequisites: the L1 sheet on the hypervisor and workload split, the L2 sheet on local storage and volumes, and the sibling L3 sheets Hosts, VMs, vCPU, vRAM and virtual devices and Volumes, partitions and filesystems.

Where to go next

Snapshot mechanics and the copy-on-write space they consume belong to Snapshots and their limits; differencing disks and linked clones to Templates, clones and snapshots. The storage underneath a pool is developed in ZFS: pools, datasets, snapshots and scrub, and in NAS and SAN. CPU and memory oversubscription, passthrough and live migration are higher-level capacity topics, outside this sheet.

References

Nodrius Field Kit — cover

Get the Nodrius Field Kit

Enter your email and we'll send you the complete Field Kit: eight technical resources in one download.

Your email address is sent to Nodrius and used to deliver the Field Kit to you by email. Marketing messages are separate and optional: the checkbox does not affect your download, and you are only added to our updates list if you tick it. Privacy Policy.

Privacy & cookies

This site does not use cookies, analytics or tracking, and it does not profile you. There is nothing technical to switch off, so Accept and Reject change nothing about how the site works: either choice lets you browse everything normally.

The only difference is what your browser remembers: two entries in local storage — your choice here, so this notice is not shown again, and a marker that the Field Kit offer has already been made — so the offer is presented once and not on every page. They contain no identifier, they are not shared with anyone, and the site sets no cookie for either.

What we do with your data
How your email address is used when you ask for a Resource, how long it is kept and which rights you have — in the Privacy Policy.
What the site stores in your browser
Two entries in local storage — your choice on this notice and whether the Field Kit offer has been shown — and no cookies, no session storage, no third-party content: in the Cookie Policy.