Articles

Context assembly and citations: what the model receives and what it can prove

Retrieval gives a list, assembly gives a context, citations give a verifiable pointer.

Reading: 6 minAutomation & AI

Article cover: Context assembly and citations: what the model receives and what it can prove

Three artefacts are routinely conflated: the candidate list the retriever returns, the context that is finally put in the request, and the citation that the interface shows the reader. They are produced by different steps, they fail differently, and a pipeline that treats them as one thing ends up unable to explain its answers.

Assembly is a budget decision with an ordering decision inside it

The budget belongs to the L2 material: the prompt and the reserved output share the context window, and every retrieved chunk spends tokens from it. Assembly is the step that decides how to spend them, and it has three levers.

How many candidates to include. More chunks raise the chance that the answer is present and lower the chance that it is used: the positional effect found in long-context studies is that material “at the beginning or end of the input context, and significantly” worse “in the middle of long contexts” is what the model actually relies on. Twenty chunks are not a superset of five; they are a different prompt.

In what order. The same study is the argument for putting the strongest evidence where it is read: at the edges of the block, not buried. Ordering is free and it is one of the few levers that costs nothing.

With what markers. Retrieved text is untrusted input that will be read alongside your instructions, so it has to be visibly separated from them — delimited, labelled with its source, and never presented in the form of an instruction. This is a security boundary as much as a formatting choice, and the retrieval-failures sheet treats it as one.

Citations: three ways to get one, and they are not equivalent

A provider feature. One API offers citations as a first-class option: “Citations return the exact passages that support each claim, so you can verify answers and surface sources to your users.” The mechanism matters more than the marketing: the API extracts the cited text itself, so — in its own words — “citations are guaranteed to contain valid pointers” to the supplied documents. The model does not get to invent a quote that is not in the document, because the document is what the citation is resolved against.

Annotations returned by the API. The other provider’s responses carry machine-readable annotations attached to spans of the output text: in the API description an output item carries an annotations array whose entries have "type": "file_citation" with an index into the text, a file_id and a filename. This is the same idea — a pointer into the source — expressed as structured metadata rather than as inline markers.

A convention you build. If the API offers neither, the pattern that works is: ask for a verbatim quote plus the identifier of the chunk it came from, then check the quote against the chunk yourself with a string comparison. It is a weaker guarantee than the first mechanism and it is enormously better than nothing: a fabricated quotation fails the comparison, and [FACT TO VERIFY] discipline in a pipeline is worth more than elegant prose in a compliance report.

In all three cases the pipeline must keep the mapping from position in the prompt back to (document, chunk, offset). Without that mapping, assembly and citation cannot be separated at all, which is why the chunking sheet insists that this metadata is carried from ingestion onward.

What a citation proves, and what it does not

A citation proves that a passage exists in a source and that the system claims it supports the sentence. It does not prove that the passage actually supports it. The study that audited generative search engines measured exactly this gap: on average “a mere 51.5%” of generated sentences were fully supported by their citations, and “only 74.5% of citations support their associated statement”. The vocabulary that comes out of that work is the one to use in evaluation: citation recall (are the statements supported at all?) and citation precision (does each cited passage support the statement it is attached to?).

Two practical consequences follow:

  • “The answer has citations” is not a quality metric. A response where every sentence carries a reference to the wrong paragraph scores perfectly on presence and badly on precision.
  • A citation is a claim about a text, not about the world. The document could be outdated, wrong or superseded; the citation only establishes what the system read.

What citations cost

The mechanism that guarantees valid pointers is not free, and the cost is honest and small: enabling citations “incurs a slight increase in input tokens because of system prompt additions and document chunking”, while the cited text itself “does not count toward your output tokens” — a design detail worth knowing, because it means the provider chose to make quoting cheaper than paraphrasing.

Building citations yourself inverts the cost: the verification step is yours, and the quoting you request spends output tokens.

The assembly checklist

  1. Is the strongest evidence at the edges of the retrieved block rather than in its middle?
  2. Is every retrieved passage delimited and labelled with its source, so it cannot be mistaken for an instruction?
  3. Does the context fit the window, with room reserved for the answer (the L2 sheet’s shared budget)?
  4. Is the position-to-source mapping kept for every chunk that entered the context?
  5. Does the answer path produce citations by a mechanism whose failure mode you know: a resolved pointer, an annotation, or a quote you verify yourself?
  6. Is citation quality measured — recall and precision — rather than assumed from the presence of markers?

Level and prerequisites. L3 — integration level: the reader must be able to assemble a context deliberately and to produce citations with a known guarantee. Prerequisites: the chunking and hybrid-search sheets of this layer for what is being assembled, and the L2 sheets on context and structured output for the budget and the response shape.

Where to go next

References

  • Anthropic — Citations — what the feature returns, the guarantee that cited text is resolved against the supplied documents, and the token-cost details.
  • OpenAI — API description — the file_citation annotation shape attached to output text.
  • Liu et al. — Lost in the Middle — where in the input relevant material is actually used.
  • Liu, Zhang and Liang — Evaluating Verifiability in Generative Search Engines — the measured gap between citations and support, and the recall/precision vocabulary.
Nodrius Field Kit — cover

Get the Nodrius Field Kit

Enter your email and we'll send you the complete Field Kit: eight technical resources in one download.

Your email address is sent to Nodrius and used to deliver the Field Kit to you by email. Marketing messages are separate and optional: the checkbox does not affect your download, and you are only added to our updates list if you tick it. Privacy Policy.

Privacy & cookies

This site does not use cookies, analytics or tracking, and it does not profile you. There is nothing technical to switch off, so Accept and Reject change nothing about how the site works: either choice lets you browse everything normally.

The only difference is what your browser remembers: two entries in local storage — your choice here, so this notice is not shown again, and a marker that the Field Kit offer has already been made — so the offer is presented once and not on every page. They contain no identifier, they are not shared with anyone, and the site sets no cookie for either.

What we do with your data
How your email address is used when you ask for a Resource, how long it is kept and which rights you have — in the Privacy Policy.
What the site stores in your browser
Two entries in local storage — your choice on this notice and whether the Field Kit offer has been shown — and no cookies, no session storage, no third-party content: in the Cookie Policy.