Documentary conformity and operational effectiveness: a control written down and a control that works
A policy proves a control was specified; only an observed effect shows it works.

Every control exists twice: as a specification somebody wrote, and as a mechanism running in a real system. Documentary conformity is a statement about the first; operational effectiveness is a statement about the second. Governance mistakes begin when the two are presented as one claim.
The difference is not a matter of taste: the sources define what a control assessment is for and what it collects, and whether a control is effective is settled by evidence, not by producing the document that describes it. The sections below state that bar, give the management-system form of the same idea, and explain why the confusion survives.
A specification and an observation look alike
On paper the two are hard to tell apart: both are text, both are dated, both can be filed — and only one answers a question about effectiveness.
the claim what can answer it
"we have a policy" -> a specification: what should happen
"we have evidence" -> an observation: what did happen
only the observation is evidence of effectiveness;
a document proves the control was specified, not that it works
The bar: what an assessment is a statement about
SP 800-53A Rev. 5 keeps the purpose narrow: once in place, controls are assessed “to determine their overall effectiveness (i.e., the extent to which the controls are implemented correctly, operating as intended, and producing the desired outcome …)” (lines 477–480). The definition has three clauses; only the first is about anything written down: “implemented correctly” can be checked against a design document; “operating as intended” and “producing the desired outcome” cannot, because they describe behaviour in a running system, and documents do not run.
The same publication names what an assessment collects: “Evidence of the effectiveness of implemented controls” (line 345), not evidence of the documentation. A signed policy states an intention; it does not become evidence of effect by being signed.
SP 800-137 treats effectiveness as a variable rather than a settled state. Continuous monitoring “Ensures continued effectiveness of all security controls” (line 210), and the obligation is not limited to what tools can see — all implemented security controls, “including management and operational controls, must be regularly assessed for effectiveness, even if the monitoring of such controls cannot be automated or is not easily automated” (lines 258–260). A control confirmed effective at one moment is not thereby effective later, so assessment repeats rather than concludes.
The management-system form of the same idea
The ISO/IEC 27001 version arrives through the SC 27 Journal. Clause 9.1 does not ask whether the ISMS is documented; its “intention … is to consider the ISMS and its processes and determine the extent to which planned activities are realized and planned results are achieved” (lines 730–733), and the Journal adds that “This intention accords with the definition of effectiveness” (lines 736–738). The duty is tied to objectives — Clause 6.2 “is that information security objectives shall be monitored” (lines 716–717) — and the Journal closes the escape route that measurability must mean numbers: “the requirement to be measurable does not imply that the results must be quantifiable. Qualitative results (e.g., “yes/no” answers) can also be considered, when supported by appropriate evidence” (lines 757–760). A yes/no answer is admissible, but read the condition attached to it.
What a document set actually is
A policy, a standard, a procedure and a plan are specifications: each states what should happen. The ISO Store pages draw the certifiable line: ISO/IEC 27001 “defines requirements an ISMS must meet” (line 20) and is the document an organisation is certified against, while ISO/IEC 27002 “provides best practice recommendations and cannot be certified to” (line 39). Certification itself is a statement about a system, not a measurement of an outcome. Holding a certificate, the ISO page says, “may bring an additional layer of confidence, as an accreditation body has provided independent confirmation of the certification body’s competence” (line 61). A certificate or a document set is testimony that a system exists and is described — not a measurement that it works.
Why the confusion survives
A folder of signed procedures is cheap to produce and easy to present: it can be reviewed in an afternoon and yields a satisfying picture of control. And an implemented control with no record of its effect tells exactly the same story as a control that does not work — both are silent. The absence of evidence is indistinguishable from the absence of the thing until somebody observes the control operating. That symmetry, more than carelessness, is what lets documentary conformity pass for effectiveness.
The limit: necessary, not sufficient
An undocumented control is hard to manage and impossible to audit: without a specification there is nothing to assess against, and a record of effect with no stated intention has no threshold to clear. Documentary conformity is therefore a real and necessary condition. The error is treating it as sufficient — accepting the description of a control as the observation of it.
Level and prerequisites
L4 — assurance: this sheet sets what counts as proof for a programme, not for one record. It assumes the L2 sheet on control objectives and verification criteria, which owns the criterion, the assessment object and the method, and the L3 sheet that verifies a remediation and the residual risk it leaves; it re-derives neither. Metric design, the quality and freshness of one piece of evidence, and internal audit are neighbouring subjects owned by other sheets in this layer.
Where to go next
- Knowledge — the Knowledge register.
- Cybersecurity Governance — the macro-area.
- Assurance — the node.
References
- NIST SP 800-53A Rev. 5 — the three-clause effectiveness definition and the evidence an assessment collects.
- NIST SP 800-137 — continuous monitoring’s duty to ensure continued effectiveness, including controls whose monitoring cannot be automated.
- ISO/IEC JTC 1/SC 27, SC 27 Journal Vol. 3 Issue 4 — the intention of Clause 9.1, its accord with the definition of effectiveness, the Clause 6.2 monitoring requirement, and the qualitative-results point.
- ISO/IEC 27001:2022 and ISO/IEC 27002:2022 (ISO Store pages) — which document carries requirements and can be certified to, and what a certificate does and does not say.