Articles

Generated text versus executed actions: what actually happened

A sentence saying an action was taken and the action having happened are two facts.

Reading: 5 minAutomation & AI

Article cover: Generated text versus executed actions: what actually happened

The sentence “the record has been updated” and the record having been updated are two different facts, and a pipeline that does not keep them apart eventually reports the first as if it were the second. The distinction is not philosophical: it decides what a log entry means, what an audit can prove, and what the interface is allowed to tell the user.

The model proposes; something else performs

The mechanism is explicit in the tool-calling documentation on both sides. The function definition lets the model “pass data to your application, where your code can access data or take actions suggested by the model” — suggested by the model. The other provider’s description is equally direct: the response carries a tool_use block and then “your code runs the lookup”; for tools the caller defines, “you write the schema and your application executes each call”.

So the artefact the model produces is a proposal, written in a structured form so that a program can read it. Whether it becomes an action is decided after that point, by your code and your rules — which is why the same mechanism can be used to not act, and why “the model did something” is never an accurate description of what happened.

Why the confusion is a real failure, not a pedantic one

Text can describe actions that never occurred. The NIST profile gives the family a name: confabulation is “the production of confidently stated but erroneous or false content”. A model that has been told about a tool can produce a fluent account of having used it, and nothing about that text distinguishes it from an account of a call that really ran.

Logs turn the confusion into a false record. If a system logs the assistant’s message and an operator later reads “the file was written” as evidence about the filesystem, the log has become a source of error rather than a record. The evidence of an action is the executor’s result — never the sentence.

Readers act on the text. A user who is told that a change was applied will not check; a downstream agent that reads the same text will treat it as ground truth. In agentic architectures, the risk of acting on mistaken text is what OWASP catalogues as Excessive Agency: functionality and permissions that let an automated system take “unintended or risky actions”.

Keeping the two apart, mechanically

The disciplines are cheap and each one removes a specific failure:

  • Return results as results. A tool’s outcome — including its failures — travels back through the tool channel, as the previous sheet describes. An action whose result was never returned is an action the model cannot know about, and it will fill the gap.
  • Record four events, not one. The proposal (what was asked, verbatim, with its identifier), the decision (which rule or which human authorised it), the execution (what was actually run, by which identity), and the outcome (what changed where, or the error). Collapsing them into a single line is the origin of the whole problem.
  • Let the interface speak from the result. The phrase “done” belongs to the executor’s report, not to the model’s narration. If the pipeline has no result, the honest interface says that the action was requested and that its outcome is unknown.
  • Verify the effect, not the call. A call can succeed at the protocol level and produce the wrong effect — the wrong record, the wrong namespace. Verification belongs to whoever owns the outcome, and it is the same POSTCONDITION discipline the L2 material applies to any mutation.

The boundary with L4

Everything about whether an action should be allowed — permissions, sandboxing, deny-by-default, human approval for high-impact operations, and the audit trail as a security control — is L4 material. What belongs at this level is the invariant those controls depend on: the system must never treat generated text as evidence that an action happened. Approval rules can only be enforced on a pipeline that knows the difference between the request and the effect.

Two sentences worth putting in the interface

  • “Requested” means the model asked for it, and the pipeline received the request.
  • “Applied” means an executor reported success, and the effect was verified where verification is possible.

Everything else — “I have updated”, “I saved”, “I sent” — is narration, and narration is not evidence.

Level and prerequisites. L3 — integration level: the reader must be able to design a pipeline in which the proposal, the decision, the execution and the outcome are separately recorded. Prerequisites: the tool-calling sheet, and the L2 material on verification of mutations.

Where to go next

References

  • OpenAI — Function calling — the “take actions suggested by the model” wording and the client-side execution of the call.
  • Anthropic — Tool use — “your code runs the lookup” and “your application executes each call”.
  • NIST — AI 600-1 — the definition of confabulation.
  • OWASP — Top 10 for LLM Applications (2025), LLM06 Excessive Agency — the risk of automated systems acting on insufficiently controlled authority.
Nodrius Field Kit — cover

Get the Nodrius Field Kit

Enter your email and we'll send you the complete Field Kit: eight technical resources in one download.

Your email address is sent to Nodrius and used to deliver the Field Kit to you by email. Marketing messages are separate and optional: the checkbox does not affect your download, and you are only added to our updates list if you tick it. Privacy Policy.

Privacy & cookies

This site does not use cookies, analytics or tracking, and it does not profile you. There is nothing technical to switch off, so Accept and Reject change nothing about how the site works: either choice lets you browse everything normally.

The only difference is what your browser remembers: two entries in local storage — your choice here, so this notice is not shown again, and a marker that the Field Kit offer has already been made — so the offer is presented once and not on every page. They contain no identifier, they are not shared with anyone, and the site sets no cookie for either.

What we do with your data
How your email address is used when you ask for a Resource, how long it is kept and which rights you have — in the Privacy Policy.
What the site stores in your browser
Two entries in local storage — your choice on this notice and whether the Field Kit offer has been shown — and no cookies, no session storage, no third-party content: in the Cookie Policy.