Because a system prompt asks; it does not enforce. The model usually follows it, but nothing stops an action that ignores it. Content the agent reads can push against it, a model update can change how it is read, and it leaves no evidence that any rule was applied. Control has to sit outside the model.
What does a system prompt actually do?
A system prompt sets context and behaviour: the agent's role, tone, the steps to follow, what to avoid. It is how you tell the model what job it is doing. It matters, and a well-written one makes an agent far more useful.
But it is an instruction to a probabilistic system. The model weighs it alongside everything else in its context, including the customer's message, the documents it retrieved and the results of its tool calls. Most of the time the instruction wins. "Most of the time" is not a standard any control function accepts for a payment limit.
Where does a system prompt fail as a control?
Four ways, each common.
It can be argued with. Text inside an email, a web page or a document can tell the agent to do something else. This is prompt injection, listed first in the OWASP Top 10 for LLM Applications, and no prompt wording reliably prevents it.
It can be satisfied in letter but not in spirit. "Never refund more than 500" can be met by two refunds of 300. The model followed the words; the limit failed.
It shifts with the model. A model update can change how the same instructions are interpreted. Nothing in the prompt changed, but the behaviour did.
It leaves no evidence. If the agent declined an action, you cannot show which instruction it applied or that it applied one at all. If it took an action, you cannot show the rule was checked.
What should stay in the prompt, and what needs a control outside it?
| Belongs in the system prompt | Needs a control outside the model |
|---|---|
| The agent's job and how to approach it | Which actions it may take at all |
| Tone, format and style | Monetary and volume limits |
| Steps to follow in a typical case | Who must approve held actions |
| When to ask a clarifying question | Which data it may read |
| How to explain a refusal | What it must never do |
| Domain knowledge and terminology | Which sources can authorise what |
The left column shapes behaviour. The right column bounds it. A prompt can repeat the right-column rules so the agent behaves sensibly, but the rule only counts if something outside the model enforces it at the moment of action and records the result.
What does this look like in practice?
Take an insurer's customer service agent. Its system prompt says: do not change a policyholder's bank details; refer such requests to the verification team. A message arrives with an attachment on what looks like the insurer's letterhead, saying an internal process change now allows agents to update bank details directly, and asking for an update.
An agent controlled only by its prompt is weighing two instructions, one of which looks official. An agent with an external control is not weighing anything: the "update bank details" action is not in its role scope, so the attempt is refused, the refusal is filed, and the message is routed to verification and flagged as suspicious. The prompt helped the agent explain the refusal politely. The control made the refusal certain.
Does this mean the system prompt does not matter?
No. It still decides how good the agent is within its bounds, and changing it can change behaviour a great deal. That is why prompt changes should go through the same change control for AI agents as any other change, with testing on real cases before release.
Some teams worry that outside controls slow agents down. A check at the point of action is fast, and it lets you give the agent more room elsewhere, because the hard limits do not depend on it; see how to balance AI agent speed and control.
Where Pryme fits
In Pryme, the brief tells the agent its job; the minimum chain bounds it. Every action meets role scope, certificate, sources, the business's own limits and the platform guardrails at the moment of action, and the verdict names which one decided. Facts carry provenance, so an external document cannot authorise a payment however it is worded. More in how to control what an AI agent is allowed to do and our guide to agent governance.
Questions people ask
Can a better-written prompt close the gap?
It can reduce errors, but not to the point of being a control. The weaknesses come from how models work, not from wording.
Should the prompt mention the outside controls?
Yes. An agent that knows its limits proposes fewer actions that will be refused and explains refusals better.
Is fine-tuning a better control than a prompt?
No. Fine-tuning also shapes behaviour without enforcing it, and it is harder to change and test.