Chunking and metadata: the shape of what gets retrieved
A chunk is the unit of retrieval, of context and of citation; its size decides all three.

A chunk is not an implementation detail. It is the unit that gets embedded, the unit the retriever returns, the unit the generator reads, and — if the interface shows sources — the unit you are able to cite. Choosing its size and its boundaries decides three things at once, and the choice is made at index time, which means changing it later means re-indexing the corpus.
Two reasons to chunk, and one trade-off
Chunking is “the process of breaking down large text into smaller segments called chunks”, an “essential preprocessing technique that helps optimize the relevance of the content ultimately stored in a vector database”. The documentation gives two reasons, and they pull in opposite directions: “to ensure embedding models can fit the data into their context windows, and to ensure the chunks themselves contain the information necessary for search”.
The trade-off is stated just as plainly: “The trick lies in finding chunks that are big enough to contain meaningful information, while small enough to enable performant applications and low latency responses for workloads such as retrieval augmented generation and agentic workflows.” Size is therefore not a setting to copy from a blog post; it is a balance between what a chunk can mean on its own and what a query can afford to carry.
The two families
Fixed size. Split by a fixed number of characters or tokens, with an overlap. The vendor guidance offers a starting point rather than a rule: “Define a fixed size that’s sufficient for semantically meaningful paragraphs (for example, 200 words or 600 characters) and allows for some overlap (for example, 10-15% of the content)”, and, elsewhere on the same page, “We recommend starting with a chunk size of 512 tokens (approximately 2,000 characters) and an initial overlap of 25%, which equals 128 tokens.”
Overlap exists for a reason that is easy to forget: a boundary cut through a sentence leaves two chunks that are each missing half of an idea, and the overlap restores continuity — “overlapping a small amount of text between chunks can help maintain continuity and context”, while “the optimal overlap might vary depending on your content type and use case”.
Structure-aware. Split where the document itself changes subject: “Break content into meaningful units that preserve context and semantic relationships across sentences and paragraphs. This approach produces chunks that better maintain semantic coherence and can span page boundaries.” The same documentation notes a page-based variant (“splitting by pages (defined by character length)”) and a hard constraint: this family only works if preparation preserved the boundaries (see the previous sheet).
Neither family is right in general. Fixed size is predictable and blind; structure-aware is faithful and depends on the extractor.
The third option: add context to the chunk
A chunk that says “the maximum is 200 units” is unambiguous only next to the sentence that came before it, and retrieval sees the chunk alone. The response to this is a preprocessing technique that prepends a short description of the surrounding document to every chunk before embedding it, so that the fragment’s context travels with it — the technique Anthropic calls contextual retrieval, whose measured effect on retrieval failures is registered in the retrieval-failures sheet of this layer.
Two properties matter when it is proposed for a project: it is applied at indexing time to every chunk (a cost that is paid once per document, and re-paid on re-index), and the note in the source material is that it is a technique to be validated — “always run evals” — not a default.
Metadata is what makes a chunk addressable
A vector is not addressable. What makes a chunk usable by the rest of the system is the payload attached to it, and the stores treat that payload as a first-class, filterable object: “For performant filtering, create payload indexes for the fields you plan to filter on. For best results, create payload indexes before ingesting data.”
The fields a chunk needs, and who consumes each:
| Field | Consumer |
|---|---|
| source document identity and version | citation, deletion, de-duplication |
| position (section, page, offset) | citation, “show me the context around this” |
| permission or tenancy tags | the access filter applied at query time |
| document or chunk date | freshness rules, conflict resolution |
| structural type (heading, table, code, prose) | reranking and context assembly |
In a relational store the same thing is a column, which is why the vector extension’s own documentation shows the pattern of filtering and ordering in one statement — and notes that the ordering must be combined with a limit for the index to be used at all.
One failure to avoid by design: a chunk whose metadata cannot be filtered on is a chunk that will be returned to the wrong reader or never returned at all. Adding the field later means re-indexing, for the same reason as changing the size.
What size does to the rest of the pipeline
- Too small — precise matching, but each hit carries little context; the generator receives fragments, and the citation points at something that is meaningless out of context.
- Too large — good context per hit, fewer hits, more tokens per query, and a higher chance that the answer is diluted inside the chunk; the position effect described in the retrieval-failures sheet applies to whatever is put in front of the model.
- Overlap too generous — the same text arrives twice, consuming the top-k budget and producing the appearance of two sources where there is one.
- Inconsistent — a corpus chunked by three different rules at three different times is a corpus whose retrieval quality cannot be reasoned about.
Level and prerequisites. L3 — integration level: the reader must be able to choose a chunking strategy for a given corpus and to specify the metadata a chunk must carry. No chunker is configured and no index is measured here. Prerequisites: the previous sheet of this layer (what preparation can preserve).
Where to go next
- Automation & AI — the macro-area this sheet belongs to.
- Applications — the node this sheet sits in.
References
- Pinecone — Chunking Strategies for LLM Applications — the definition of chunking, the two reasons for it and the size trade-off.
- Microsoft — Chunk documents in Azure AI Search — the two families, the overlap guidance, the 512-token starting point and the structure-aware alternative.
- Qdrant — Filtering — payload indexes and when to create them.
- Anthropic — Introducing Contextual Retrieval — the technique of adding context to a chunk, and the instruction to evaluate rather than assume.
- pgvector — README — the relational form of the same design: metadata as columns, filter and ordering in one statement.