Clusters and quorum: who declares a failure
How a cluster agrees who is alive, what quorum protects, and what its loss costs.

A virtualization cluster is a set of hosts that manage workloads as one domain, and the first thing they must agree on is who is alive. The answer is not a roll call — a silent host is not voting no, it is absent — and not unanimity either. A cluster counts votes and acts only while it holds a quorum, “the minimum number of votes that a distributed transaction has to obtain in order to be allowed to perform an operation in a distributed system”. That count is not about peak availability: it stops two halves of the same cluster from both believing they are the cluster. This sheet is about that decision: who holds a vote, what a witness adds, and what losing quorum costs.
five nodes, one vote each, quorum = three
[n1]---[n2]---[n3]---[n4]---[n5]
1 1 1 1 1 five votes -> quorate
the link between n3 and n4 fails
{ n1 n2 n3 } | { n4 n5 }
3 votes | 2 votes
>= quorum | < quorum
keeps serving | stops; it cannot form a second cluster
A vote, not a roll call
A cluster manager groups hosts and runs a distributed protocol underneath. Proxmox VE uses the Corosync Cluster Engine for group communication and replicates its configuration through a database-backed file system; Windows uses Failover Clustering for the same role. What matters is the arithmetic. “Proxmox VE assigns a single vote to each node by default”, and the status tool exposes the sum: a four-node cluster reports four expected votes, four total votes and a quorum of three. Three votes are a majority of four; one or two are not. The difference from a roll call is direction. Unanimity asks whether every host answered; quorum asks whether a majority did, and counts a silent host as a missing vote rather than a veto.
The majority the cluster acts on
The property a majority buys is exclusivity: two subsets cannot each hold more than half of a set, so at most one partition can be quorate at a time. So the cluster stays a single decision-making body even when its network does not. Proxmox states the consequence — “In case of network partitioning, state changes requires that a majority of nodes are online. The cluster switches to read-only mode if it loses quorum.” Read-only is a decision, not a fault: the cluster does not guess which side is right or let both act; it withdraws state changes until the votes are counted again. Availability is what it gives up to keep its members from diverging.
A witness: a vote where there is no host
Another failure is arithmetical: an even number of nodes has no majority without all of them, so a two-node cluster whose link is down has no quorate side. The remedy is a vote that is not a full member — a witness. Windows Failover Clustering offers “three types of quorum witnesses” — disk, file share or cloud — and notes that modern clusters “automatically manage the witness vote and their nodes vote with Dynamic Quorum”. A disk witness is “The disk used for the witness must be a shared disk”; a file share witness is an SMB share, and Microsoft asks that “The file share should be physically separate from the cluster nodes or cluster sites”, because a witness on the same power or network can vanish with the nodes. A cloud witness is the same idea off-site: a blob holding “the blob file required for voting arbitration”. Proxmox’s QDevice is the same, a daemon that supplies votes on an external arbitrator’s verdict, which “can see all nodes and thus choose only one set of nodes to give its vote”.
What a lost quorum costs
When a node stops taking part, the cluster does not merely shrink; it decides. A node that leaves the quorum becomes unknown, and the cluster stops accepting the state changes a lone member might otherwise forge. The requirement of “at least three cluster nodes (to get reliable quorum)” says the same from the other side: two nodes run workloads but cannot guarantee a majority when one disappears. Losing quorum is paid for in service, on purpose. What happens to the workloads afterwards — the restart, the isolation of the failed node, the timeouts — belongs to the companion sheet on high availability and failure handling; the claim here is that a cluster without quorum first refuses to act.
Split brain: the failure the vote exists to prevent
A partition is a network split; a split brain is both sides keeping on serving. If two partitions each ran the same virtual machine against the same shared storage, both would write and the copies would diverge beyond reconciliation — Microsoft warns that a replicated file system “can cause a partition-in-space or partition-in-time, where the cluster nodes are running independently of each other and could cause data loss”. The vote is what makes a partition decidable: only the side that keeps a majority may continue, and the minority must stop rather than become a second cluster. The term also covers a storage-level failure, where replicas on two hosts both take the primary role; that belongs with the sheets on shared and distributed storage, not with the membership vote counted here.
Sizing votes and the witness is a design decision
The rule to keep: quorum protects the integrity of the cluster, not the availability of the service. Every vote and every witness buys a way to survive a failure and costs something — hardware, a network path, a dependency; there is no neutral default. “For smaller 2-node clusters, the QDevice can be used to provide a 3rd vote”, but a three-host cluster already reaches “at least three nodes for reliable quorum” with no witness, and an even cluster with a witness scores no better than an odd one without. The witness must sit where it can outlive the failure it arbitrates — a separate power feed, switch or rack — and it holds no data: a file share witness needs a few megabytes, a cloud witness a single blob, because its only job is to break a tie. Sizing the votes and placing the witness decide which side of a future partition keeps serving — a design decision, not a default.
Level and prerequisites
L4 — membership and the decision, not the recovery: how a cluster counts votes, what a witness adds, and why losing quorum stops the cluster from serving, without developing what happens to workloads afterwards. Prerequisites are the L3 sheet on hosts, virtual machines and their virtual devices, and the L3 storage sheets, on why shared storage lets two hosts run the same workload; the restart after a failure, node isolation and the timeouts are the sibling sheet on high availability and failure handling.
Where to go next
- Knowledge — the index.
- Server and virtualization — the area.
- Virtualization and networking — this sheet’s node.
- Companion sheets, still in draft: high availability and failure handling, which owns the restart and the node isolation deliberately left out; the storage sheets on shared and distributed storage, which own the storage-level split brain named only here; and affinity and scheduling, which owns placement once the cluster is quorate again.
References
- Proxmox Server Solutions — Proxmox VE Administration Guide (sections Cluster Manager, Quorum, Cluster Network and QDevice) — https://pve.proxmox.com/pve-docs/pve-admin-guide.html — the quorum definition, the majority requirement and read-only consequence, the single vote per node, the three-node recommendation, the corosync/pmxcfs stack, and the QDevice arbitrator.
- Proxmox Server Solutions — High Availability (Proxmox VE wiki) — https://pve.proxmox.com/wiki/High_Availability — the HA prerequisites list, including the recommendation of at least three cluster nodes for reliable quorum.
- Microsoft — Deploy a quorum witness (Windows Server failover clustering) — https://learn.microsoft.com/en-us/windows-server/failover-clustering/manage-cluster-quorum — the three witness types, Dynamic Quorum, the disk and file share witness requirements, the cloud witness arbitration blob, and the partition-in-space warning.
- LINBIT — DRBD User’s Guide 9.0 — https://linbit.com/drbd-user-guide/drbd-guide-9_0-en/ — read for the storage-level meaning of split brain; referenced by name only, not quoted in the body.
- Microsoft — Storage Spaces Direct overview — https://learn.microsoft.com/en-us/windows-server/storage/storage-spaces/storage-spaces-direct-overview — read for the failover-clustering context of a distributed storage pool; referenced by name only, not quoted in the body.