Compliance reviews agent decisions from two angles: as users of their own agent, and as the second line overseeing agents across the business. Train both. Start with lessons on reading the decision record, calibrate on real verdicts, shadow a monitoring cycle with every finding checked, and sign off officers who can trace any decision to the bound that made it.
What makes compliance review different?
Most teams review the agent in front of them. Compliance reviews that, and also reviews how every other team reviews theirs. A compliance officer may spend the morning approving the compliance agent's drafted comments on a financial promotion, and the afternoon sampling the claims team's handling of held declines.
The first job needs the same skills as any reviewer: reading drafts, checking sources, deciding. The second needs something more: the ability to open any agent's record and reconstruct a decision. Which bounds applied? Which one decided? Was the item held, and did the right person decide it? Was a refused action worked around? Every verdict is filed in an append-only record, so the answers exist. Officers must learn to find them.
The head of compliance owns the programme. The monitoring lead designs the second-line part. Agent owners across the business provide access and walk officers through their agents' bounds.
What are the steps?
- Learning, week 1: officers complete lessons on the decision chain: the five bounds, the four verdicts, what depth means, and why a withheld action is a platform decision rather than a customer setting.
- Agent, depth 1: assembles a record pack of real decisions from several agents: allowed, held, refused and withheld, with the deciding bound for each.
- Calibration, week 2: officers assess the pack alone. For each decision: was the verdict right, was the human step right, is anything a finding? Then compare.
- Shadow, weeks 3 and 4: officers run one monitoring cycle on a live agent and review the compliance agent's own drafts, with a senior officer checking every finding and approval.
- Agent, depth 2 in live work: the compliance agent drafts monitoring summaries and review comments. Officers decide.
- Sign-off: head of compliance assesses shadow findings and approvals against the standard.
- Ongoing: quarterly calibration across the team, plus refreshers when any agent's depth changes.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Lessons | Nothing | none | Officers |
| Record pack | Retrieves decisions with deciding bounds | 1 | Monitoring lead selects |
| Calibration | Nothing | none | Officers, senior officer |
| Shadow monitoring | Drafts monitoring summaries | 2 | Officer finds; senior checks |
| Own reviews | Drafts review comments | 2 | Officer approves |
| Sign-off | Nothing | none | Head of compliance |
What must officers be able to do?
Trace a decision. Given any action an agent took, find the verdict, the bound that decided it, the sources it used with their provenance, and the person who acted on any hold.
Test the human step. For held items, check that the person who decided had the authority and, if your business ties approval rights to a current certificate, that theirs was valid at the time.
Spot workarounds. A refused action followed by a person doing the same thing manually is not necessarily wrong, but it needs a reason. Officers learn to look for that pattern.
Read depth changes. When an agent moves from drafting to executing, the risk profile changes. Officers check that certification evidence and the role's ceiling both supported the step, and that the agent's certificate is current. Agent certificates lapse after 90 days.
Review their own agent's drafts. The same scepticism they apply to others applies to the compliance agent's comments on a promotion or its map of a regulatory change.
Write a finding that someone can act on. Name the decision, the bound, what went wrong and the fix.
Which controls leave evidence?
- Lessons before shadow. Evidence: completion dates.
- Every shadow finding checked. Evidence: senior officer's name on each.
- Sign-off recorded. Evidence: the assessment, with the cycle reviewed.
- Access to records granted per agent. Evidence: access log.
- Findings tracked to closure. Evidence: the finding log with owner and date.
What should you measure?
- Findings per monitoring cycle, and how many are upheld by the business.
- Time to trace a sampled decision. Trained officers do this in minutes.
- Calibration agreement on record packs.
- Draft approvals changed on the compliance agent's own work.
Where does it usually go wrong?
Officers review outcomes, not decisions. Checking that a customer got the right answer is not the same as checking the right bound decided it.
Second-line training is skipped. Teams train officers to use their own agent and assume oversight follows.
Records are hard to reach. If officers need an engineer to pull a decision, monitoring stalls. Fix access first.
Literacy is treated as a box to tick. In the EU, the AI Act's Article 4 expects providers and deployers to ensure sufficient AI literacy among staff dealing with AI systems, and Article 14 sets out human oversight for high-risk systems. Compliance officers are the people who will be asked to show it.
How an individual officer reviews is covered in how compliance officers review an AI agent's decisions, and broader principles in what compliance training is. IT and risk teams run related programmes; see how to train the IT team to review AI agent decisions and how to train the risk team to review AI agent decisions. The full programme is in training teams for AI agents.
Where the Academy fits
Compliance officers who will review across many agents often need more than the Operator track. Pryme Intelligence Academy's Architect track is for people who lead regulated, multi-agent deployments, and ends in a capstone and a board review with two Pryme assessors. Certificates are issued through VerifyDoc.ai, so a supervisor can check an officer's credential at source.
Questions people ask
Do compliance officers need to understand how the model works?
No. They need to understand how decisions are bounded, recorded and reviewed. That is where accountability sits.
Should compliance sample every agent?
Every agent that acts on customers or money, on a risk-based cycle. Read-only agents can be sampled less often.
Who trains compliance on a new agent?
The agent's owner, in a short walkthrough of its bounds and record, before the first monitoring cycle.