The risk team reviews agent decisions at two levels: its own work, such as limit breaches, incident triage and risk register updates the agent drafts, and second-line sampling of front-line verdicts. Train both through learning, calibration on past breaches and sampled records, two weeks of shadow review beside a certified risk manager, and sign-off with no missed red line.
What does the risk team actually review?
Risk functions use agents in two ways. Directly, an agent can monitor key risk indicators, flag limit breaches, draft incident classifications, summarise control test results and propose risk register updates. Those drafts and holds reach risk managers for decision.
Indirectly, risk oversees agents run by other teams. It samples their decision records, checks that verdicts named the right bound, and challenges certification evidence. That second job is less obvious and more often untrained. A risk analyst who can classify an incident but cannot read a front-line agent's record will miss the systemic problem behind ten small ones.
The chief risk officer owns the programme. A senior risk manager with a current certificate runs calibration and shadowing.
What should risk reviewers learn first?
- The minimum chain. Every action meets five bounds: role scope, certificate, sources, the business's own limits, the platform guardrails. The verdict names which bound decided. Risk reviewers use that field to spot when a business limit, rather than the agent's evidence, is doing all the work.
- Provenance. Sealed, signed, declared, external. A decision resting on declared or external facts deserves more scrutiny.
- Depth and its expiry. Agent certificates lapse after 90 days, and a run that clears no depth drops the agent to depth 1 the same day. Risk should know which agents are near expiry and what a drop would do to operations.
- Refused versus withheld. A refusal reflects the agent's bounds or the business's limits; a withheld action is a platform decision for every agent. Neither is a fault, and both belong in the risk picture.
The steps
- Person (chief risk officer): lists decision types and red lines. Decision types: confirm or reclassify a breach, classify an incident, approve a register change, record a sampling finding. Red lines might include: closing a breach without root cause; classifying an incident without reading the decision record; signing off a sample without checking provenance.
- Person (risk analyst): completes the learning above.
- Person (senior risk manager): runs calibration on 40 cases: past breaches, past incidents and sampled front-line records with seeded errors.
- Agent, depth 2: drafts breach notes, classifications and register updates in shadow mode; the trainee decides each in a shadow queue while the senior decides for real.
- Agent, depth 1: selects the second-line sample of front-line records each week; trainee and senior review it independently.
- Agent, depth 1: logs both sets of decisions and mismatches.
- Person (senior risk manager): debriefs twice a week.
- Person (chief risk officer): signs off after two weeks at the bar.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Key risk indicator monitoring | Tracks and reports | 3 | Risk samples the reports |
| Limit breach | Flags and drafts a note with the source data | held | Risk manager confirms, escalates or closes |
| Incident classification | Drafts a classification from the record | 2 | Risk manager decides severity |
| Risk register update | Proposes wording and rating | 2 | Owner and risk manager agree |
| Second-line sample | Selects and presents front-line records | 1 | Risk analyst reviews and records findings |
How should second-line sampling be calibrated?
Seed errors into anonymised front-line records and ask each reviewer to find them. Typical seeded issues: a reviewer approved a held item where the deciding fact was external; a refusal was logged as a fault; an agent continued acting at depth 3 the day after its certificate failed; a reason that says only "OK".
Discuss what each finding means. One error is a coaching point for the front-line lead. Several of the same kind across reviewers suggest a training gap. The same error across teams points to a design issue in the agent's brief or the business's limits. Risk reviewers should leave calibration able to say which of the three they are looking at.
Which controls matter, and what evidence do they leave?
- Independent shadow decisions. Evidence: trainee and senior decisions filed before comparison.
- Second-line sample records. Evidence: each sampled case, the finding and its classification.
- Incident classification from the record. Evidence: a link to the decision record in each classification.
- Sign-off against a written bar. Evidence: correct decisions out of the total and the red-line count, signed.
- Certificate tracking. Evidence: a register of agent certificates and expiry dates, reviewed monthly.
What should you measure?
Agreement with the senior on breaches and classifications. Seeded errors found in sampling. Findings that led to a change in training, limits or the agent's brief. Red-line misses, which must be zero. And time from a breach flag to a decision.
Where does it usually go wrong?
Risk is trained on its own queue and never on sampling. The second line then reviews front-line agents through dashboards alone.
Classifications are copied from the agent's draft. The draft is a starting point; severity is a risk judgement.
The team treats withheld verdicts as anomalies to investigate. They are platform decisions and should be recorded, not chased.
Operations and onboarding teams are often the front line risk samples; their programmes are in training the operations team and training the onboarding team. For daily practice, see how risk managers review an agent's decisions and how underwriters review an agent's decisions. The reviewer onboarding checklist for the risk team records readiness, and the guide to training teams for AI agents sets the order.
Where the Academy fits
Pryme Intelligence Academy's Operator track covers how a governed agent decides, reading the workspace and when something goes wrong. Risk leads who oversee regulated, multi-agent deployments can continue to the Architect track, which ends with a capstone and a board review with two Pryme assessors and earns Certified Pryme Architect, valid 24 months.
Questions people ask
Does the risk team need the same certificate as front-line reviewers?
At least the same knowledge, because they judge front-line decisions. Senior risk staff overseeing several agents usually need deeper training on how agents are designed and certified.
How large should the second-line sample be?
Large enough to see patterns by team and decision type, small enough to review properly each week. Start modest and increase where findings appear.
Can risk approve a depth increase?
Risk challenges the evidence and advises. The decision follows the governance the business has set, and the depth itself still depends on certification evidence.