Articles

Evidence quality and freshness: when one piece of evidence can carry a claim

Sufficiency, provenance and currency decide when one piece of evidence can carry a claim.

Reading: 5 minCybersecurity Governance

Article cover: Evidence quality and freshness: when one piece of evidence can carry a claim

A governance claim — this control is implemented, this risk is treated — is only as strong as the evidence someone can check. SP 800-53A states the problem plainly: “the content and applicability of existing assessment evidence varies” (lines 1841–1842). The evidence has one job: to be the material “needed to determine whether the security and privacy controls are effective in their application” (lines 1844–1845).

Three properties decide whether one piece of evidence can carry a particular claim: sufficiency and applicability (how much was gathered, and whether it addresses this system), provenance (who produced it, when, over what, by what method) and currency (whether it still describes the system as it operates now). Reusing a report produced elsewhere, or a vendor’s certificate, is a judgement, not a box to tick.

Sufficiency and applicability

SP 800-53A gives an assessment two attributes that “help define the level of effort for the assessment” (lines 1082–1083): the depth attribute, “the rigor and level of detail in the examination, interview, and testing processes” (lines 1087–1088), and the coverage attribute, “the scope or breadth of the examination, interview, and testing processes, including the number and types of specifications, mechanisms, and activities to be examined or tested” (lines 1090–1092). Together they declare how much evidence was gathered — not what was looked at, but how hard and how widely.

Applicability makes sufficiency relative. The attribute values “are based on the assurance requirements specified by the organization” (lines 1095–1096), so the same artefact can be sufficient for one claim and insufficient for another; what changes is the claim.

Provenance: four things every artefact carries

When evidence is reused, SP 800-53A makes the assessor answer more than one question: “determine the credibility of the assessment evidence, the appropriateness of previous analysis, and the applicability of the assessment evidence to current system operating conditions” (lines 1963–1964). A reused result carries its context: “the date and type of the original assessment are documented in the assessment plan and assessment report” (lines 1965–1966). Date and type record when and how; credibility is about who produced it.

what every artefact has to answer before it can carry a claim

who produced it   → producer, and whether that competency fits this claim
when              → the date of the original assessment
over what         → scope, and applicability to current operating conditions
by what method    → the type of assessment, so the reader can judge sufficiency

The trap is a producer who is also the only verifier. SP 800-53A lets product evidence “produced during the product testing, evaluation, and validation processes” be used “to the extent that it is applicable” (lines 693–696), but only to that extent: where the independent evaluation “did not test a particular configuration setting that is employed by the organization in a system, then the assessor may need to supplement the original test results with additional testing” (lines 1970–1973). A vendor’s report whose only producer and only verifier is the vendor is an assertion about a product, not about the control the organisation implemented; it is one component, never the proof of the claim.

What the assessment expects to find

SP 800-53A tells organisations to draw on what already exists: “control assessment results and associated assessment documentation, artifacts, and evidence available for system components from previous assessments” (lines 672–674). Where the evidence comes from an outside provider, the procedure is adjusted to “obtain the requisite assessment evidence from external providers” (lines 1836–1837). Reusing another party’s evidence is a judgement with work attached, not an automatic saving. (The mapping of evidence to the control it supports is a neighbouring sheet’s subject.)

Currency is a property with mechanisms behind it

Sources name currency as a quality and then require instruments to maintain it. SP 800-53A Rev. 5 control CA-05(01) requires “automated mechanisms used to ensure the accuracy, currency, and availability of the plan of action and milestones for the system” (lines 10556–10557), a determination the assessor tests (lines 10559–10560). SP 800-137 puts the same duty on the monitoring programme: “An organization’s ISCM strategy and program are routinely reviewed for relevance and are revised as needed” (lines 242–243), and manual processes must be “repeatable and verifiable to enable consistent implementation” (lines 248–249) — repeatable so their output can be compared over time, verifiable so someone else can check it. SP 800-53 Rev. 5 CA-7 sets the cadence in the control itself: the organisation establishes “[Assignment: organization-defined frequencies] for monitoring and [Assignment: organization-defined frequencies] for assessment of control effectiveness” (lines 6769–6770). Currency is not a feeling about a document; it is a value the organisation instruments.

The limit: fresh relative to the claim

Freshness is measured against the claim, not the calendar. The reuse criteria state the trend — “as the time between current and previous assessments increases, the credibility and utility of the previous assessment results decrease” (lines 2003–2005) — but as a reason to look for change, not an expiry date. Controls “deemed effective during previous assessments may have become ineffective due to changing conditions within the system or its environment of operation” (lines 1987–1989), so reuse “necessitates the identification of any changes that have occurred since the previous assessment and the impact of these changes on the previous results” (lines 1992–1994). A configuration baseline from last week can be stale for a change-controlled system and perfectly current for an annual review; old evidence is not automatically bad, and new evidence is not automatically sufficient.

The same discipline applies to a screenshot: it is evidence of what a screen showed to whoever captured it, and it must carry a producer, a date, a scope and a method like any other artefact. No corpus source states this in these words; it is this sheet’s framing.

Level and prerequisites

L4 — assurance: this sheet does not collect evidence or run an assessment; it judges whether one artefact can support a claim, which is the reasoning an evidence programme rests on. Prerequisites: the L3 sheet that keeps one traceable chain from risk to control to owner to evidence, and the L2 sheet that defines control objectives and verification criteria. Collection and reuse at programme scale, and the mapping from requirement to control, belong to neighbouring sheets.

Where to go next

References

  • NIST SP 800-53A Rev. 5 — the depth and coverage attributes, the reuse criteria, evidence from external providers and product testing, and the currency of the plan of action and milestones.
  • NIST SP 800-137 — routine review of the ISCM strategy and programme, and repeatable and verifiable manual processes.
  • NIST SP 800-53 Rev. 5 — CA-7 and the organisation-set frequencies for monitoring and assessment of control effectiveness.
Nodrius Field Kit — cover

Get the Nodrius Field Kit

Enter your email and we'll send you the complete Field Kit: eight technical resources in one download.

Your email address is sent to Nodrius and used to deliver the Field Kit to you by email. Marketing messages are separate and optional: the checkbox does not affect your download, and you are only added to our updates list if you tick it. Privacy Policy.

Privacy & cookies

This site does not use cookies, analytics or tracking, and it does not profile you. There is nothing technical to switch off, so Accept and Reject change nothing about how the site works: either choice lets you browse everything normally.

The only difference is what your browser remembers: two entries in local storage — your choice here, so this notice is not shown again, and a marker that the Field Kit offer has already been made — so the offer is presented once and not on every page. They contain no identifier, they are not shared with anyone, and the site sets no cookie for either.

What we do with your data
How your email address is used when you ask for a Resource, how long it is kept and which rights you have — in the Privacy Policy.
What the site stores in your browser
Two entries in local storage — your choice on this notice and whether the Field Kit offer has been shown — and no cookies, no session storage, no third-party content: in the Cookie Policy.