Untrusted Text Stays Evidence

Prompt injection is the native attack on any system that mixes instructions with untrusted text. Every agent in this course mixes them. The model reads a system prompt, a user request, a retrieved page, and a tool result in one window. Nothing in that window says which of them can give orders.

The release-gate note asked for a measurement that can change a launch decision. The text that arrives after the launch comes from a page the agent opened by itself. Someone wrote that page to reach your tools.

Trust Is Several Properties That Interact

Trust is a set of properties: privacy, behavior under attack, fairness, transparency, accountability, staying on the assigned goal, and resistance to misuse. One score hides which of them broke. For an agent they break through each other.

Retrieval carries the privacy failure. A hostile webpage breaks the behavior you tested, and tools turn a harmful plan into an executed one. Trust is the operating envelope for autonomy, and the envelope sets how far the agent goes before a human signs.

Agents Leak Through Their Tools

The leaks stack. A model memorizes pieces of its training data, and a system leaks private context through the window it assembles. An agent adds its own path: a secret handed to a tool, or one user's context carried into another user's turn.

State is the safety feature, and a shared context window is shared state. Data minimization does the heavy work, because an agent cannot leak a record it never reached.

Access control per user, redaction on the way out, and isolation between sessions hold the rest. Logs name which tool touched what data. Tests that deliberately try to extract sensitive information tell you whether the posture holds.

Prompt Injection Is the Native Agent Attack

Hostile text arrives on a webpage, inside a retrieved document, or in a tool result. A user probing the policy is the mild case. The hard case is text written for the agent and never shown to the human.

The loop note named five steps: observe, decide, act, examine, remember. Injection enters at observe, and the damage lands at act. Between those two steps, nothing asks who wrote the sentence the agent is following.

System instructions, developer policy, user intent, retrieved documents, webpages, emails, and tool outputs are different kinds of text. An agent that flattens them into one stream obeys whichever sentence arrived last.

• Quote and summarize untrusted content as evidence, with its source label attached.
• Tools declare their permissions and side effects before the agent calls them.
• A sensitive action stops for confirmation at the moment it runs.
• Logs keep a record of which instruction source caused which decision.

A suite built from normal prompts passes without touching any of this. The risk appears when the agent reads hostile text while holding tools, memory, private context, and authority to act.

The Same Capability Serves Both Sides

A model that finds a bug for you finds one for an attacker. Defenders get bug discovery, alert triage, and automated response. Attackers get phishing, reconnaissance, exploit generation, and social engineering in volume.

The net effect depends on capability, access, monitoring, policy, and the speed of defensive adoption. The last term is the one your team sets.

A Trustworthy Agent Hands You Evidence

The agent earns trust with a record: what it read, what it did, what it refused, what it doubts, and what needs a human signature. Tone proves nothing.

The workflow note called this a tape: what the agent relied on, what it changed, and how you undo it. Injection makes the tape load-bearing, because the tape holds the sentence that redirected the run. A more capable agent raises the value of an inspectable trail.

Safety measured as refusal behavior measures the wrong surface. The dangerous failures come from allowed actions: weak evidence, stale context, poisoned instructions, or a stop that came too early.

Policy Needs Evidence That Maps to Harms

Policy has to be science-based and evidence-based, and the phrase carries work. Written from vibes, it lags reality. Written from benchmark theater, it overfits the wrong signal, and it does that with a chart.

The evaluation that counts maps to an actual harm and an actual mitigation. An injection evaluation clears that bar when a failed run costs the agent a tool permission.

The Builder Test

Take one agent you run and list every text source it reads.

• Mark which sources can instruct the loop, and which stay evidence for the whole run.
• List the tools that reach private data, and the ones that run without a permission check.
• Name the actions that stop for a human, and the uncertainty that forces escalation.
• Write down how the system fails, and where that failure becomes visible to a person.

Then poison a page yourself, point the agent at it, and read which label failed.

What Carries

Every note in this course repaired one joint of the loop. Reasoning sharpened decide and tools built act. Grounding and the tape held examine and remember, and the release gate decided whether the loop ships at all.

The boundary between instructions and evidence guards observe. An outsider can write to that step, and a poisoned observation spends your tools. Agent safety starts when you decide what text can control the loop. Decide it before a webpage decides it for you.