A model checking its own work brings the same blind spots to the check that it brought to the work. If it misread a policy, it will misread it again when asked whether it followed the policy. Supervision has to sit outside the model, in rules and people that do not share its errors, or it is not supervision.
Why is self-checking so tempting?
Self-checking is cheap and easy to build. Ask the model to produce an answer, then ask it, or a second call to the same model, to review the answer against the rules. Reasoning models do a version of this internally, working through steps and revisiting them before they respond, and the results are often better than a single pass. A reasoning model can catch arithmetic slips and obvious contradictions this way.
It is natural to extend the idea: if the model can catch some of its own mistakes, let it be the control. Ask it to confirm the refund is within policy before issuing it. Ask it to check that the customer is not vulnerable before chasing a debt. The pattern looks like a control and produces a log line that looks like evidence.
The problem is not that self-review never helps. It is that it fails in exactly the cases a control exists for.
Where does self-supervision fail?
Self-review fails when the error comes from something the model believes, rather than something it overlooked. Those are the errors that matter most.
| Source of error | Example | Can self-review catch it? |
|---|---|---|
| A slip in execution | Adds two figures wrongly | Often, if asked to recheck |
| A misread rule | Treats a fee waiver limit of 50 as 500 | Rarely: it rereads the rule the same way |
| A wrong fact in its sources | Relies on a superseded policy document | No: the same document says the same thing on review |
| A manipulated input | Follows instructions hidden in a customer's email | No: the manipulation also shapes the review |
| A gap in its training | Does not recognise a sign of financial difficulty | No: it does not know what it does not know |
The middle three rows are the dangerous ones. The model is consistently, confidently wrong, and a second pass by the same model is consistently, confidently wrong in the same direction. Prompt injection is the clearest case. If a customer email contains hidden text telling the agent to approve the refund, a self-check reading the same context can be steered by the same text.
Does using a different model solve it?
Partly. A second model trained differently may catch errors the first misses, and using one to review another is a reasonable quality technique. But two foundation models often share training data, share common failure patterns and can be influenced by the same manipulated input. Diversity reduces correlated error; it does not remove it.
More importantly, a model reviewing a model is still a judgement, not a rule. It produces an opinion about whether the action is allowed. A supervisor in a regulated business needs something firmer: a decision that follows from stated bounds, can be explained by pointing to them, and is the same every time for the same facts.
What does supervision outside the model look like?
Supervision outside the model has two parts, and neither depends on the model's own view of its work.
The first is rules. Whether an action is allowed should be decided by a layer that reads the agent's role, its certificate, the provenance of the facts it relies on, the customer's limits and the platform guardrails, and applies them mechanically. The model proposes an action. The rules decide. A model that has been manipulated into proposing a large refund still meets a limit it cannot argue with.
The second is people. Where the rules say an action needs a person, a named approver decides. That person brings knowledge the model lacks, and a different set of blind spots, which is the point. The question of who should own an AI agent in a business is partly a question of who supplies that outside view.
Certification belongs here too. It measures the model's behaviour on real cases, scored against a floor someone else set. The model does not mark its own exam.
Does it matter where this layer runs?
The supervisory layer has to be independent of the model in function, not necessarily in location. Some firms run it in their own environment for data or resilience reasons, which raises questions covered in whether you can run an agentic operating system on your own infrastructure. What matters is that the model cannot change the rules, cannot approve its own held actions, and cannot edit the record of what it did.
Where Pryme fits
In Pryme, the model proposes and the platform decides. Every action meets the minimum chain at the moment of action, outside the model. Attest scores the agent on your own cases against floors the model never sees, and held actions wait for a named person. We do not ask the model whether it should be allowed to act. See the governed agentic operating system, explained.
So where should self-review sit?
Keep self-review for what it does well: catching slips and improving the quality of drafts before a person sees them. Do not count it as a control. Any check that decides whether an action may proceed should run outside the model, against rules or people that do not share its view of the world.
Questions people ask
Is asking the model to explain its reasoning a form of supervision?
It helps a person understand the proposal, which supports supervision. But an explanation is the model's account of itself and can be plausible and wrong. Treat it as input to a decision, not as the decision.
Can a model ever be trusted to act without outside checks?
Not for actions that matter. Even at the highest depth, actions should meet bounds the model does not control. Depth changes what is allowed, not who checks it.
Does this apply to small, low-risk actions too?
The check can be light for low-risk actions, but it should still sit outside the model. A light external rule is more reliable than a careful self-review.