Articles

Treating retrieved text as untrusted input

Retrieved documents, web pages and tool results are data the model reads, not commands.

Reading: 4 minAutomation & AI

Article cover: Treating retrieved text as untrusted input

An agent’s context is one flat stream. The operator’s instructions, the body of a retrieved PDF, the HTML of a fetched page and the JSON a tool returned all arrive as the same kind of thing: tokens the model conditions on and may act upon. Nothing marks which are commands and which are content. That is the whole difficulty of untrusted input — and the research that named it states the problem as a warning: “We argue that LLM-Integrated Applications blur the line between data and instructions,” so an adversary can “remotely (without a direct interface)” work through “data likely to be retrieved” (lines 26–27, 27–30).

This sheet takes the defender’s reading: not how the attack is built, but which text must be treated as hostile, and what posture follows.

What counts as untrusted input

Untrusted input is not “input from a hostile user”: a person typing an attack into a chat box is direct injection, owned by the earlier sheet in this batch. Untrusted input is any text the system did not author but the model will read — a retrieved document, a fetched page, an email body, a code comment or a tool’s returned payload. Provenance decides trust, not tone. OWASP draws the boundary at the source: “Indirect prompt injections occur when an LLM accepts input from external sources, such as websites or files” (lines 220–221).

Three channels: documents, the web, tool results

Three channels matter here, and they share one property — each is read on the system’s initiative, not typed by the user.

  • Documents. Google lists “emails, documents, or calendar invites that instruct AI to exfiltrate user data” (line 61).
  • The web. The OWASP scenario has exactly this shape: “A user employs an LLM to summarize a webpage containing hidden instructions that cause the LLM to insert an image linking to a URL” (lines 292–294).
  • Tools. A tool’s return value re-enters the context as text — in the benchmark’s words, “is often fed back to the LLM as part of the context” (lines 18–22).

The three are one channel from the model’s point of view, so a defence aimed at only one leaves the same door open.

Why the model cannot tell them apart

A language model conditions on a single token sequence. No privilege flag marks a substring as data rather than instruction, and the retrieval and tool layers hand their output straight back into that sequence. The benchmark’s definition names the mechanism: attackers place “malicious instructions … embedded within the content processed by LLMs, aiming to manipulate these agents into executing detrimental actions against users” (lines 14–19). NIST makes the same point: indirect injection is when “adversaries remotely (i.e., without a direct interface) exploit LLM-integrated applications” (lines 538–540). The attacker never touches this system; it writes text that this system’s own retrieval later decides to read.

What the text can be made to do

Once the output is wired to a tool, the consequences are ordinary security ones. INJECAGENT groups the intentions “into two primary types: direct harm to users and exfiltration of private data”, and reports agents “vulnerable to attacks 24% of the time” under a ReAct-prompted GPT-4 (lines 29–34). The catalogue of harms belongs to the sibling injection sheet; the assumption to carry here is that untrusted text reaching the context may be an instruction with access to any tool the agent holds.

The containment posture

No single fix exists. Google’s account of its production defences is layered, aiming at “meaningfully elevating the difficulty, expense, and complexity faced by an attacker” (line 64) rather than removing the risk: content classifiers that “filter out harmful data containing malicious instructions” (line 73), plus confirmation before consequential actions: “potentially risky operations … may trigger an explicit user confirmation request” (line 82).

INJECAGENT’s survey of defences records two families — black-box methods such as “placing special delimiters around external content”, and white-box methods such as “separating prompts and data into two channels using a secure front-end for formatting alongside a specially trained LLM” (reading-order extract line 261): one a convention, the other an attempt to give instructions and data separate inputs.

The original paper’s summary is honest: “effective mitigations of these emerging threats are currently lacking” (lines 40–41), and the defensive approach “seems to follow a ‘Whack-A-Mole’ style” (line 768). Treating retrieved text as untrusted is therefore a posture — assume it may be hostile, minimise what following it can trigger, require confirmation before high-impact actions — not a filter that makes it safe. The controls that implement it (input and result validation, least-privilege tool permissions, sandboxing, human approval) each have their own sheet in this batch.

Level and prerequisites. L4 — the reader must reason about trust boundaries across retrieval, the web and tool calls, and hold several incomplete controls at once. Prerequisites: the L1 sheet on how a model generates text and what its context window holds, the L3 sheet on retrieval, and the sibling L4 sheet on direct and indirect prompt injection.

Where to go next

References

  • Greshake et al. — Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — the data/instruction blur, remote delivery of injected prompts, and the state of mitigation.
  • Zhan et al. — INJECAGENT: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents — external content fed back as context, the harm classes, the 24% baseline and the defence families.
  • Google — Mitigating prompt injection attacks with a layered defense strategy — the document and web channel, and the layered controls.
  • NIST — AI 600-1, Generative AI Profile — indirect injection defined inside the information-security risk.
  • OWASP — LLM Top 10 (2025) — the definition of indirect injection and the webpage-summary scenario.
Nodrius Field Kit — cover

Get the Nodrius Field Kit

Enter your email and we'll send you the complete Field Kit: eight technical resources in one download.

Your email address is sent to Nodrius and used to deliver the Field Kit to you by email. Marketing messages are separate and optional: the checkbox does not affect your download, and you are only added to our updates list if you tick it. Privacy Policy.

Privacy & cookies

This site does not use cookies, analytics or tracking, and it does not profile you. There is nothing technical to switch off, so Accept and Reject change nothing about how the site works: either choice lets you browse everything normally.

The only difference is what your browser remembers: two entries in local storage — your choice here, so this notice is not shown again, and a marker that the Field Kit offer has already been made — so the offer is presented once and not on every page. They contain no identifier, they are not shared with anyone, and the site sets no cookie for either.

What we do with your data
How your email address is used when you ask for a Resource, how long it is kept and which rights you have — in the Privacy Policy.
What the site stores in your browser
Two entries in local storage — your choice on this notice and whether the Field Kit offer has been shown — and no cookies, no session storage, no third-party content: in the Cookie Policy.