Provenance labelling stops prompt injection from causing harm by moving the question of trust out of the model. Every fact an AI agent receives is tagged by the system with its origin, and rules outside the model decide which origins may authorise which actions. Injected text can fool the model, but it cannot change the tag the system attached.
What do the standards ask for?
No single rule says "label provenance", but several point there.
The OWASP Top 10 for LLM Applications ranks prompt injection first and, in substance, recommends separating and clearly identifying external content, enforcing least privilege, and requiring human approval for high-risk actions. It accepts that no method reliably prevents injection.
The EU AI Act, in Article 15, requires high-risk AI systems to achieve appropriate accuracy, robustness and cybersecurity, including resilience against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting vulnerabilities.
The NIST Generative AI Profile names information integrity and content provenance among the concerns for generative systems.
And payments rules already embody the principle in a narrower form. The EU Instant Payments Regulation requires payment service providers to verify the payee's name, because a payee detail supplied by a requester is not the same as one confirmed by the account holder's bank.
For the OWASP view in full, see OWASP prompt injection and agents that act.
Who does this apply to?
Any firm running agents that read content they did not write and can take actions with effect: paying, changing records, sending messages. In regulated firms, cybersecurity and operational resilience duties make this a supervisory matter as well as a security one.
Why does provenance work where filtering fails?
Prompt injection exploits a basic fact about language models: they cannot reliably tell instructions from data, because both arrive as text. Our glossary entry on what prompt injection is explains the mechanism. Filters try to spot malicious text and lose to rephrasing.
Provenance takes a different route. It does not ask the model whether text is trustworthy. The system that delivers each piece of input already knows its channel: this came from the core ledger, this from a verified counterparty, this from the customer's typed message, this from an emailed attachment. The system attaches that label. The text cannot rewrite it, because the label is not part of the text.
Rules then work on labels, not words. "A change to payee bank details requires details from the vendor master file or a verified counterparty" is enforced by the control layer that executes the change. If the only source for the new details is an external document, the action is refused, however persuasive the document was to the model.
How should provenance be designed for agents?
Assign labels at the boundary. Labels are set by the connector that brings data in, never by the model.
Use a small, ordered set of classes. For example: issued by a system of record; signed by an identifiable party; declared by a customer or person; external and unverified.
Carry labels through derivation. A summary of an external email is still external. Anything the agent derives inherits the weakest label among its inputs, so laundering untrusted text through a summarisation step does not upgrade it.
Set minimum provenance per action. Reading needs nothing. Drafting a reply can rely on declared facts. Changing records or moving money needs facts from systems of record or signed parties.
Show labels to reviewers. A person approving a held action should see which facts came from where.
Take an illustrative case. A treasury operations agent at a corporate lender receives an email, apparently from a borrower, asking to redirect a loan drawdown to a new account, with hidden instructions telling the agent to treat the request as pre-approved. The agent drafts the change. The control layer sees that the new account details carry only an external label, while the borrower's registered details in the loan system are sealed. The drawdown change is refused and filed with that reason; the request goes to a person who calls the borrower on the number held on file.
The same pattern underlies indirect injection through documents an agent reads.
How do you meet the requirements with provenance controls?
| Requirement | Control | Evidence it leaves |
|---|---|---|
| Identify external content (OWASP) | Labels assigned by connectors at ingestion | Provenance label on every input |
| Resilience to manipulation (EU AI Act Article 15) | Minimum provenance enforced per action outside the model | Refused verdicts citing provenance |
| Least privilege (OWASP) | Actions limited by both tool scope and fact strength | Tool and rule configuration |
| Human approval for high risk (OWASP) | Held actions show reviewers each fact's label | Review records with labels |
| Information integrity (NIST profile) | Derived facts inherit the weakest input label | Lineage of derived facts |
| Payee verification (payments rules) | Payee changes need system-of-record or signed details | Payee change records with sources |
| Adversarial testing | Injection tests through every input channel | Test reports |
For a broader set of defences, see how to prevent prompt injection in AI agents, and to check them, how to test an agent against prompt injection.
What will a security reviewer, auditor or supervisor ask?
- Who assigns provenance labels, and can the model alter them?
- Which actions require which minimum provenance?
- What happens to the label when the agent summarises an external document?
- Show us an action refused because of provenance.
- How do reviewers see where a fact came from?
What does provenance not cover?
It does not stop the model being confused, so injected text can still distort what an agent says or drafts; output review and limits on outbound messages handle that. It does not judge whether a sealed source is correct, only that it came from the system of record. And it is not a legal requirement in itself; it is a way to meet several. For the binding rules, see our guide to AI agent regulation mapped to controls.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
Where Pryme fits
The Pryme Context Protocol stamps every fact reaching an agent with its provenance class: sealed, signed, declared or external, strongest first. The stamp is applied by the platform, not the model. An external claim cannot authorise a payment, and the verdict records which bound decided.
Questions people ask
Can a model be trained to ignore injected instructions?
Models can be made more resistant, but not immune. Provenance assumes some injection will succeed and makes sure it cannot authorise actions that matter.
Is a customer's own message external or declared?
It is declared: stated by an identified person but not verified. It can support a reply or a draft, but should not alone justify a high-impact action such as changing payment details.
Does provenance slow agents down?
Very little. Labels travel with data, and rules are checked when the agent acts. The main cost is held actions where facts are too weak, which is the point.