Articles

Human evaluation and acceptance criteria: when is it good enough

Metrics measure change; humans set the threshold, and it must exist before the number.

Reading: 6 minAutomation & AI

Article cover: Human evaluation and acceptance criteria: when is it good enough

Automated metrics describe change. Deciding that a system is good enough to ship is a human judgement, and it has to be written down before the measurement exists — otherwise the threshold is chosen by whichever number the last run produced.

The two are connected by validation rather than by trust: the metrics used throughout this layer are proxies, and each of them was checked against human judgement before anyone treated it as a measure.

Why humans stay in the loop

The metrics are calibrated against people. The RAG metric suite describes its own validation: annotators were asked to judge which of two answers was more faithful, and the metrics “agreed in around 90% of the cases” with them. The number is not a property of the metric in the abstract; it is what that metric achieved on that dataset, and it is the kind of figure to demand for any judge you adopt.

Model judges are a substitute, and their target is human preference. The LLM-as-a-judge work frames the motivation plainly — human preferences are “otherwise very expensive to obtain” — and then verifies its judges against “3K expert votes” and “30K conversations with human preferences” collected in MT-bench and Chatbot Arena. The whole point of the exercise is agreement with people.

And people are the scarce resource. FActScore’s motivation begins from a factual observation: human evaluation “is time-consuming and costly”. That is why the division of labour below matters more than either method on its own.

What humans are good at, and what they are not

Good at: defining what “good” means for this product; judging what no metric encodes — appropriateness, tone, usefulness, whether the answer understands the user’s situation; and noticing a failure mode the metrics were never designed to catch.

Not good at: repetitions. Human evaluation is expensive and variable, which makes it a poor regression detector and a good reference. The design that follows is not “pick one”: it is humans set and validate the criteria; automated metrics run the comparisons.

The bias the process has to know about

Human ratings are instruments with their own distortion. In the study that audited citations, the answers readers rated as more useful tended to be the ones with lower citation precision — fluency and confidence read as quality, and a well-written answer feels more supported than it is. An annotation process that does not know this will reward the answers that are hardest to check.

What an acceptance criterion contains

Four parts, and the sheet’s recommendation is that all four are written:

  1. The metric — named precisely, with the version of what computes it (a judge is a dependency, not a definition).
  2. The threshold — a number, with the direction and the population attached.
  3. The population — which evaluation set, which cases, and under which conditions: all cases, the labelled subset per failure mode, the cases that used to fail. A threshold without a population is unfalsifiable.
  4. The consequence — what happens when it is missed: block the release, escalate to a decision, or accept with a written reason and a date (the regressions sheet’s rule).

A criterion of that shape can be argued about, which is the point. “It feels better” cannot.

Multi-metric, because one number hides the trade

The benchmark that made multi-metric measurement a design principle measures “7 metrics (accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency) for each of 16 core scenarios” so that “metrics beyond accuracy don’t fall to the wayside” and trade-offs are exposed.

The application-level version is a smaller set with the same intent. For a retrieval pipeline, the minimum is:

Criterion What it protects
grounding / faithfulness on the set that answers stay supported by the sources
citation precision, where citations are shown that the pointers are right
a retrieval-side metric that the right passage is found at all
latency that the answer arrives while the user is waiting
cost per request that the system is affordable at the rate it is used

The three quality criteria are the previous sheets’ metrics; the last two are operational, and they belong in the same document because a prompt edit that improves the first three can move the last two without anyone noticing.

The process

  1. Humans define criteria and annotate a small, stable set. Small enough to maintain, stable enough to compare against.
  2. Metrics are validated against that set, with the agreement recorded rather than assumed.
  3. Metrics run on every change (offline), and on live traffic (online) as a second signal.
  4. Human review is reserved for cases the metrics are uncertain about, and for the periodic re-validation of the metrics themselves.
  5. Criteria are versioned. Changing a threshold is a dated decision with a reason, not a silent drift.

So when is it good enough?

When the criterion you wrote has been met on the population you defined, and the failures that remain are the ones you decided to accept in writing. That is the whole answer available at this level, and it is a better one than any score: a system with no written criterion cannot be good enough — it can only be liked.

Level and prerequisites. L3 — integration level, and the sheet that closes this layer: the reader must be able to state an acceptance criterion and to defend each of its four parts. Prerequisites: the evaluation sheets of this layer and the L2 material on reporting results with their conditions.

Where to go next

References

  • Es et al. — RAGAS — the reported agreement between its metrics and human annotators.
  • Zheng et al. — Judging LLM-as-a-Judge — human preferences as the target, the benchmarks used to verify judges, and the biases judges show.
  • Min et al. — FActScore — human evaluation as the reference and as a cost.
  • Liang et al. — HELM — multi-metric measurement as a design principle.
  • Liu, Zhang and Liang — Evaluating Verifiability in Generative Search Engines — the relationship between perceived utility and citation precision.
Nodrius Field Kit — cover

Get the Nodrius Field Kit

Enter your email and we'll send you the complete Field Kit: eight technical resources in one download.

Your email address is sent to Nodrius and used to deliver the Field Kit to you by email. Marketing messages are separate and optional: the checkbox does not affect your download, and you are only added to our updates list if you tick it. Privacy Policy.

Privacy & cookies

This site does not use cookies, analytics or tracking, and it does not profile you. There is nothing technical to switch off, so Accept and Reject change nothing about how the site works: either choice lets you browse everything normally.

The only difference is what your browser remembers: two entries in local storage — your choice here, so this notice is not shown again, and a marker that the Field Kit offer has already been made — so the offer is presented once and not on every page. They contain no identifier, they are not shared with anyone, and the site sets no cookie for either.

What we do with your data
How your email address is used when you ask for a Resource, how long it is kept and which rights you have — in the Privacy Policy.
What the site stores in your browser
Two entries in local storage — your choice on this notice and whether the Field Kit offer has been shown — and no cookies, no session storage, no third-party content: in the Cookie Policy.