Articles

Block, file and object storage: three access models

Block, file and object storage: what each model addresses, and who owns the metadata.

Reading: 4 minServer & Virtualization

Article cover: Block, file and object storage: three access models

Storage is reached through one of three access models, and the difference is not the medium or the cable underneath: it is who names the data. In block storage the device offers numbered blocks and the host’s filesystem turns them into files. In file storage a server owns the files and hands them out by name. In object storage the store keeps the data and its metadata together as one addressed object. The model you are in decides where the index of names lives.

                        who names the data?
   BLOCK                    FILE                     OBJECT
   client sees              client sees              client sees
   a block device           a file tree              a key -> an object
        |                        |                        |
   host filesystem          file server (NAS)        object store
   owns the names           owns the names           owns the names
        |                        |                        |
   blocks on the disk       files on the disk        objects + metadata
   no file metadata         metadata is the          metadata is part
   at the device            server's index           of the object

Who owns the metadata

Every model has an owner for one thing: the namespace, the index of what exists and what it is called. The three definitions carry the difference.

  • Block storage is “A method of storing data in blocks”, and the file structure is not on the device at all: “The arrangement of those blocks into files occurs using a host file system”. The device knows blocks; the host knows names.
  • File storage is “A method of storing data as files”. A file server is “A computer that serves files to clients”, and a NAS (Network Attached Storage) is “A storage device that connects to a network and provides file access services to clients”. The server holds the tree.
  • Object storage is “A storage device that provides object services”, where an object is “The encapsulation of data and associated metadata”. The store holds the index and answers by key.

Block storage: the device hands over numbered blocks

Disk devices “store data in fixed size blocks (e.g., 512 byte blocks, or 4,096 byte blocks)”, and the block service “provides block level access to storage for other systems”. Nothing above the blocks exists until a host puts a filesystem there — which is why an administrative platform treats such storage as a place for “large raw images” and says that “It is usually not possible to store other files (ISO, backups, ..) on such storage types”.

A cloud block volume follows the same rule: Amazon EBS offers volume types that “differ in performance characteristics and price”, and the buyer picks a size and an I/O profile, not a file layout.

The metadata owner is the host. A block array can sit on a SAN (Storage Area Network) — “A network whose primary purpose is the transfer of data between computer systems and storage devices” — but its knowledge stops at blocks; a SAN is only “usually (but not necessarily) identified with block I/O services”. DAS (Direct Attached Storage) is the single-host case: “A storage device locally connected to a server and dedicated to that server”.

File storage: the server owns the names

In the file model the metadata is the service. The client asks for a path; the server resolves it against its own tables and returns a file. That is the trade: the server takes responsibility for the namespace, and can therefore offer the same namespace to several clients at once.

The file model normally carries a complete directory tree, and Proxmox VE calls this class file level — storage that gives “access to a fully featured (POSIX) file system”, so the client sees a filesystem, not a device. How that namespace is shared, and over which protocols, is a question about sharing rather than about the model.

Object storage: the metadata travels with the data

In the object model there is no tree to mount. The unit is the object, “The encapsulation of data and associated metadata”: metadata is not a layer the client rebuilds from blocks, it is part of the thing being stored, and the store is the index that answers whether a key exists.

Azure Blob Storage is the clearest large example: it is “Microsoft’s object storage solution for the cloud”, optimized for “massive amounts of unstructured data”. A storage account “provides a unique namespace”, and a container “organizes a set of blobs, similar to a directory in a file system” — similar, and only similar: it groups and names, but it is not a tree a filesystem walk can traverse.

The object model scales past a single server’s namespace: DSaaS (Data Storage as a Service) “hides limits to scalability”.

The three models side by side

Proxmox VE classifies a storage as file level or block level, and lists ZFS as “both”. The table fixes the same properties for each model:

Block File Object
Unit the client handles numbered block on a device file at a path object addressed by key
Who owns the namespace the host filesystem the file server the object store
Where the metadata is on the host on the server inside the object
What the client does not see names, directories which blocks hold the file the store’s index
Address form device offset path and name key inside a container

The model is defined by where the namespace sits, and everything an administrator can do with it follows from that placement.

Limits, the common error and a security note

The common error is to treat the model as a property of a box, when it is a property of the interface. One ZFS pool can be presented as either; cloud providers sell both shapes: block volumes with Amazon EBS, object storage with Azure Blob Storage. Identical media and network, and the rules still differ, because the namespace is in a different place.

A second limit is the object namespace: the phrase “similar to a directory” is a deliberate understatement, because an application written to walk a POSIX tree must be adapted, and the gap is serious enough that a hierarchical namespace can be switched on for a storage account.

Security note, labelled as interpretation: the point of enforcement follows the metadata owner. Where the store owns the index, access is decided against that index and the client does not reach the underlying blocks; where the host owns the index, the host is the enforcement point, because there is no server-side namespace to consult.

Level and prerequisites

L3 — conceptual: which layer owns the names.

This is the vocabulary sheet for the storage node: it names the three models and fixes the one property that separates them. It sits above L2 operational work and below the L4 design work of sizing shared pools and migration. Prerequisites are the L1 sheets on storage media and performance and on filesystems, users, groups and permissions, and the L2 sheets Local storage: partitions, filesystems and volume management and Mounts and filesystems.

Where to go next

How each is shared, and over which protocols, belongs to the sibling sheet NAS and SAN: file-level and block-level shared storage. Within this node, Virtual disks: thin and thick provisioning and Snapshots and their limits follow from the block model, and ZFS: pools, datasets, snapshots and scrub shows one system presenting both classes. The complementary sheets are RAID: what redundancy buys, and what it does not and Replication and the mistakes it replicates. Area routes: Knowledge, Server and virtualization, Storage.

References

  • SNIA — SNIA Dictionary (entries: block services, block storage, file server, file storage, object, object service, object storage, Network Attached Storage, Storage Area Network, Direct Attached Storage, Data Storage as a Service, OSD) — URL not recorded in this batch.
  • Amazon Web Services — Amazon EBS volume types — https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volume-types.html — block volume types and the performance profile of each.
  • Microsoft — Introduction to Blob (object) Storage — https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction — the object model: a unique namespace, containers and blobs, and the hierarchical-namespace option.
  • Proxmox Server Solutions — Storage — https://pve.proxmox.com/wiki/Storage — the file-level and block-level storage classes and the thin-provisioning note.
  • SNIA — Common RAID Disk Data Format (DDF) Specification, Version 2.0 rev 19 — URL not recorded in this batch — the shared-storage-model framing of a block aggregation.
Nodrius Field Kit — cover

Get the Nodrius Field Kit

Enter your email and we'll send you the complete Field Kit: eight technical resources in one download.

Your email address is sent to Nodrius and used to deliver the Field Kit to you by email. Marketing messages are separate and optional: the checkbox does not affect your download, and you are only added to our updates list if you tick it. Privacy Policy.

Privacy & cookies

This site does not use cookies, analytics or tracking, and it does not profile you. There is nothing technical to switch off, so Accept and Reject change nothing about how the site works: either choice lets you browse everything normally.

The only difference is what your browser remembers: two entries in local storage — your choice here, so this notice is not shown again, and a marker that the Field Kit offer has already been made — so the offer is presented once and not on every page. They contain no identifier, they are not shared with anyone, and the site sets no cookie for either.

What we do with your data
How your email address is used when you ask for a Resource, how long it is kept and which rights you have — in the Privacy Policy.
What the site stores in your browser
Two entries in local storage — your choice on this notice and whether the Field Kit offer has been shown — and no cookies, no session storage, no third-party content: in the Cookie Policy.