Human evaluation and acceptance criteria: when is it good enough
Metrics measure change; humans set the threshold, and it must exist before the number.

Automated metrics describe change. Deciding that a system is good enough to ship is a human judgement, and it has to be written down before the measurement exists — otherwise the threshold is chosen by whichever number the last run produced.
The two are connected by validation rather than by trust: the metrics used throughout this layer are proxies, and each of them was checked against human judgement before anyone treated it as a measure.
Why humans stay in the loop
The metrics are calibrated against people. The RAG metric suite describes its own validation: annotators were asked to judge which of two answers was more faithful, and the metrics “agreed in around 90% of the cases” with them. The number is not a property of the metric in the abstract; it is what that metric achieved on that dataset, and it is the kind of figure to demand for any judge you adopt.
Model judges are a substitute, and their target is human preference. The LLM-as-a-judge work frames the motivation plainly — human preferences are “otherwise very expensive to obtain” — and then verifies its judges against “3K expert votes” and “30K conversations with human preferences” collected in MT-bench and Chatbot Arena. The whole point of the exercise is agreement with people.
And people are the scarce resource. FActScore’s motivation begins from a factual observation: human evaluation “is time-consuming and costly”. That is why the division of labour below matters more than either method on its own.
What humans are good at, and what they are not
Good at: defining what “good” means for this product; judging what no metric encodes — appropriateness, tone, usefulness, whether the answer understands the user’s situation; and noticing a failure mode the metrics were never designed to catch.
Not good at: repetitions. Human evaluation is expensive and variable, which makes it a poor regression detector and a good reference. The design that follows is not “pick one”: it is humans set and validate the criteria; automated metrics run the comparisons.
The bias the process has to know about
Human ratings are instruments with their own distortion. In the study that audited citations, the answers readers rated as more useful tended to be the ones with lower citation precision — fluency and confidence read as quality, and a well-written answer feels more supported than it is. An annotation process that does not know this will reward the answers that are hardest to check.
What an acceptance criterion contains
Four parts, and the sheet’s recommendation is that all four are written:
- The metric — named precisely, with the version of what computes it (a judge is a dependency, not a definition).
- The threshold — a number, with the direction and the population attached.
- The population — which evaluation set, which cases, and under which conditions: all cases, the labelled subset per failure mode, the cases that used to fail. A threshold without a population is unfalsifiable.
- The consequence — what happens when it is missed: block the release, escalate to a decision, or accept with a written reason and a date (the regressions sheet’s rule).
A criterion of that shape can be argued about, which is the point. “It feels better” cannot.
Multi-metric, because one number hides the trade
The benchmark that made multi-metric measurement a design principle measures “7 metrics (accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency) for each of 16 core scenarios” so that “metrics beyond accuracy don’t fall to the wayside” and trade-offs are exposed.
The application-level version is a smaller set with the same intent. For a retrieval pipeline, the minimum is:
| Criterion | What it protects |
|---|---|
| grounding / faithfulness on the set | that answers stay supported by the sources |
| citation precision, where citations are shown | that the pointers are right |
| a retrieval-side metric | that the right passage is found at all |
| latency | that the answer arrives while the user is waiting |
| cost per request | that the system is affordable at the rate it is used |
The three quality criteria are the previous sheets’ metrics; the last two are operational, and they belong in the same document because a prompt edit that improves the first three can move the last two without anyone noticing.
The process
- Humans define criteria and annotate a small, stable set. Small enough to maintain, stable enough to compare against.
- Metrics are validated against that set, with the agreement recorded rather than assumed.
- Metrics run on every change (offline), and on live traffic (online) as a second signal.
- Human review is reserved for cases the metrics are uncertain about, and for the periodic re-validation of the metrics themselves.
- Criteria are versioned. Changing a threshold is a dated decision with a reason, not a silent drift.
So when is it good enough?
When the criterion you wrote has been met on the population you defined, and the failures that remain are the ones you decided to accept in writing. That is the whole answer available at this level, and it is a better one than any score: a system with no written criterion cannot be good enough — it can only be liked.
Level and prerequisites. L3 — integration level, and the sheet that closes this layer: the reader must be able to state an acceptance criterion and to defend each of its four parts. Prerequisites: the evaluation sheets of this layer and the L2 material on reporting results with their conditions.
Where to go next
- Automation & AI — the macro-area this sheet belongs to.
- Operations and risk — the node this sheet sits in.
References
- Es et al. — RAGAS — the reported agreement between its metrics and human annotators.
- Zheng et al. — Judging LLM-as-a-Judge — human preferences as the target, the benchmarks used to verify judges, and the biases judges show.
- Min et al. — FActScore — human evaluation as the reference and as a cost.
- Liang et al. — HELM — multi-metric measurement as a design principle.
- Liu, Zhang and Liang — Evaluating Verifiability in Generative Search Engines — the relationship between perceived utility and citation precision.