Accuracy, completeness and faithfulness: three different questions
Grounded in the sources, true, and complete are three separate things to measure.

“Is the answer right?” hides at least three questions, and a single quality score answers none of them cleanly:
- Faithfulness — is what the answer says supported by the material it was given?
- Accuracy — are the statements true of the world?
- Completeness — did the answer cover what the question required?
They are measured differently, they move in different directions when you change the system, and a pipeline that reports one number is hiding the trade-offs it is making.
Faithfulness: grounded in what was given
The definition used by the reference-free RAG metric suite is compact: “Faithfulness refers to the idea that the answer should be grounded in the given context. This is important to avoid hallucinations, and to ensure that the retrieved context can act as a justification for the generated answer.”
The way that suite computes it is worth knowing, because it is the shape most faithfulness metrics take: the answer is split into statements, each statement is checked against the context, and the score is the proportion supported — in its formal notation, the number of supported statements over the total. It is a proportion, not a verdict, and it is produced by a model being asked to verify — which is why the next sheet insists that a judge’s agreement with humans be measured rather than assumed.
The same idea appears at a different granularity in the long-form factual-precision work, which “breaks a generation into a series of atomic facts and computes the percentage of atomic facts supported by a reliable knowledge source”. Smaller units, same question: is each claim supported by something, and by what.
Two failure directions matter, and they are not opposites of each other:
- Unfaithful — the answer adds content the context does not support. This is the failure the metric is named for.
- Faithful and useless — every sentence is supported by the retrieved chunks, and the chunks were the wrong ones. The metric is satisfied, the user is not. That failure is a retrieval failure, and it is why faithfulness measured alone can improve while the product gets worse.
Accuracy: true of the world
Accuracy is the harder question, because “true” is not a property of the retrieved text. An answer can be perfectly faithful to an outdated document, or to a document that was always wrong, and it is then inaccurate while scoring full marks on faithfulness.
For retrieval systems, two practical proxies appear in the literature:
- Citation precision — “each provided citation should support its associated statement”, measured as the proportion of citations that do; the audit that defined it found that a large share of citations fail this test, which is the empirical reason not to treat citations as proof.
- Factual precision against a reliable source — the atomic-fact proportion described above, with “reliable knowledge source” as the second half of the definition, which is where the definition of reliable has to be decided explicitly rather than assumed.
Completeness: the question that reference-free metrics cannot answer alone
Completeness asks whether the answer contains what the question needed. Reference-free frameworks capture the adjacent property — the metric suite defines answer relevance as “the idea that the generated answer should address the actual question that was provided” — but relevance is not coverage: an answer can address the question and still omit half the required content.
Completeness therefore needs something to be complete against: a reference answer, or a checklist per case written when the case was created. That is an argument for the evaluation set carrying expectations (the previous sheet) rather than only inputs.
Why the three have to be reported separately
They trade against each other under the levers a team actually pulls:
| Change | Faithfulness | Completeness | Notes |
|---|---|---|---|
| Fewer, better chunks | up | down | less to be unfaithful to, less covered |
| More chunks | down | up | more noise, more material |
| Reranking | up | ~ | promotes the relevant chunk, keeps the same set |
| Stricter “cite or refuse” instruction | up | down | refusal is a completeness failure by design |
| Bigger model | ~ | ~ | moves accuracy more than faithfulness |
Those directions are reasoning about the mechanisms described in this layer, not measured results — and the point of the table is exactly that each lever has to be measured on your own set, in the three columns separately. A single score would let a team improve one and believe it had improved the product.
Level and prerequisites. L3 — integration level: the reader must be able to say which of the three questions a metric answers and to design an acceptance criterion that names the right one. The metrics themselves are measured in the evaluation sheet that follows. Prerequisites: the retrieval-failures and test-cases sheets of this layer.
Where to go next
- Automation & AI — the macro-area this sheet belongs to.
- Operations and risk — the node this sheet sits in.
References
- Es et al. — RAGAS — the definitions of faithfulness, answer relevance and context relevance, the reference-free framing, and the statement-level computation.
- Min et al. — FActScore — atomic facts and factual precision against a reliable source.
- Liu, Zhang and Liang — Evaluating Verifiability in Generative Search Engines — citation recall and citation precision, and the requirement that a citation support its statement.
- NIST — AI 600-1 — confabulation, the failure family that faithfulness attempts to bound.