Reviewing an AI agent's draft is neither doing the work yourself nor checking a colleague's. Untaught approvers fall into one of two habits: they approve almost everything because the drafts look polished, or they redo each case from scratch because they do not trust it. Both waste the agent and weaken the control.
Why is reviewing an agent's work a different skill?
A colleague's errors tend to be visible. Rushed work looks rushed, a missing document is obvious, uncertainty shows in the wording. Experienced reviewers learn to spot those signs over years.
An agent's errors look different. Its drafts are fluent, well structured and confident whether they are right or wrong. It can cite a policy clause that does not quite say what it claims, apply a rule correctly to the wrong product, or rely on a fact from a source that was out of date. None of that shows in the tone. A reviewer who judges by how a draft reads will be misled exactly when it matters.
There is also a difference in what the reviewer is responsible for. With a colleague, the reviewer checks the work. With an agent, the reviewer is the control that decides whether a held action goes ahead. That is closer to an authorisation than a proofread.
What are the two failure modes?
Rubber-stamping. The approver sees a clean draft, the agent has been right the last fifty times, the queue is long, and they approve. Over weeks, approval becomes reflex. The record shows a named person reviewed each item; in practice, nobody did.
Redoing. The approver does not trust the agent, so they open every source, rebuild the case and write their own outcome, then compare. Each review takes as long as doing the work. Queues back up, the agent delivers no value, and the approver concludes, wrongly, that agents do not work.
Both come from the same cause: the approver does not know what to check. Training fixes that by giving them a method that is faster than redoing and more reliable than trusting.
What should approver training cover?
The content is specific to how agents are governed and how they fail. Six areas cover most of it.
- Reading the reason for the hold. Every held item should name the bound that stopped the agent: a customer limit, a weak source, a capability above its certified depth. The approver learns to start there, because it tells them what to check.
- Checking provenance. Facts reaching the agent carry where they came from, whether from the system of record, a named party, the customer or an outside email. Approvers learn which sources can support which decisions, and that an external claim cannot authorise a payment.
- Recognising agent-typical errors. A clause quoted out of context, a rule applied to the wrong case type, a figure carried over from an old document, an instruction followed from inside an attachment.
- Edit, refuse or escalate. When to fix a draft, when to reject it, when to send it to someone more senior. And how to record the reason in each case.
- What not to check. Which parts of the draft the agent's bounds already guarantee, so the reviewer does not waste time on them.
- Their own authority. That refusing is part of the job, that they will not be penalised for it and what happens to refused work.
How should the training be delivered?
On real cases, not slides. The most effective format is calibration: a set of held items from the business's own work, some correct and some with planted errors of the kinds above, reviewed by approvers individually and then discussed as a group. People see what they missed, learn what others caught and converge on a shared standard.
New approvers should start paired with experienced ones, reviewing together before reviewing alone. Calibration should repeat on a schedule and after any change to the agent's capability, sources or bounds, because the errors to look for change with them. Keeping track of who has completed which session is routine compliance administration; our piece on whether an AI agent can track compliance training looks at automating it.
Is this different from training the agent?
Very. We do not train agents in the sense of changing a model's weights for a customer. We qualify them: we change their brief, sources and bounds and certify them on the customer's cases, as set out in qualification, not training. People are the opposite. Their judgement does improve with practice and feedback, and that improvement is what makes them a reliable control.
It is also different from general AI awareness courses. Approvers do not need to know how models work. They need to know how to review this agent's work on this process. Our argument that AI literacy is a supervision requirement, not a training course makes the same point from the regulatory side. Nor do they need technical specialists alongside them; as our piece on whether you need data scientists to deploy AI agents argues, the skills that matter most here are operational.
How do you know the training is working?
Watch the approvers' decisions. Approval rates that hover near total with almost no edits suggest rubber-stamping. Review times close to the time it takes to do the work from scratch suggest redoing. Second reviews of a sample of approved items show whether errors are getting through. Planted cases in the live queue, used sparingly, show whether approvers catch what they were trained to catch.
The trend matters more than any single figure. Good training should move approvers towards consistent, reasonably quick reviews that catch the errors that matter and leave a clear reason for every edit or refusal.
What is the trade-off?
Training approvers takes time away from the queue, and calibration sessions need preparation: real cases, planted errors, someone to lead the discussion. Organisations that deployed agents to save time can resent the new overhead.
Without it, the hold-and-approve step either does nothing or undoes the agent's value. Before your next agent goes live with held items, set out what its approvers will be trained on, run a calibration session on real cases and make sure each approver knows they may refuse. Our guide on setting up named approvers for an AI agent covers selecting them; the deployment approach is in deploying an AI agent: train, shadow, certify, widen.
Questions people ask
How long should approver training take?
Enough to cover the six areas above on real cases, often a few hours for a single capability, followed by a period of paired review. Calibration sessions then recur, typically each quarter and after significant changes.
Should approvers be tested before they can approve?
Yes, on a set of real held items with known answers. An approver who cannot reliably catch planted errors in a test will not catch real ones under queue pressure.
Do senior managers who approve high-value items need the same training?
They need the same core method, adapted to the decisions they make. Seniority does not protect against fluent, confident drafts; if anything, senior approvers have less time per item and benefit most from knowing where to look.