Risk managers review an agent's decisions to test the controls, not only the outcomes. For a single decision they follow first-line steps: read the record, find the deciding bound, check the evidence, then approve, edit or decline with a reason. Their distinct job is to ask whether the bounds themselves sit where the firm's risk appetite says.
How is a risk manager's review different from the first line?
First-line reviewers ask "is this decision right?" Risk managers ask "is the system deciding the way we agreed it would?" That means looking across many decisions at once, at the edges where the agent nearly stopped, and at the reviewers whose approvals the firm relies on.
It also means reviewing what never reached a person. A decision allowed at depth 3 was never seen before it happened. If the bounds are too loose, the first line will never find out, because nothing arrives in their queue.
What does a risk review cover?
- A sample of decisions, read from the record. Each one shows the verdict, the sources and their provenance, the brief version, the agent's certificate at the time and the deciding bound.
- The deciding bound across the sample. Of role scope, certificate, sources, the firm's own limits and platform guardrails, which decides most often? A firm whose agents are mostly stopped by the platform guardrails has set its own limits too loosely to matter.
- Near-misses. Allowed decisions that came close to a limit tell you more than decisions well inside it.
- Provenance mix. How much of the agent's reasoning rests on declared or external facts? An agent approving refunds mainly on customers' own statements carries more risk than one reading the firm's ledger.
- Reviewer behaviour. Approval rates, edit rates and time on item by person.
- Certificate status. Certificates lapse after 90 days; a run that clears no depth drops the agent to depth 1 the same day. Lapses near busy periods are an operational risk.
What findings should a risk manager act on?
| Finding | What it suggests | Action |
|---|---|---|
| Many near-misses on one limit | The limit may be too high, or the business is testing it | Review the limit with its owner |
| Allowed decisions resting on external facts | Sources bound is too permissive | Require sealed or signed sources for that action |
| One reviewer approves almost all holds quickly | Rubber-stamping | Sample their approvals; retrain or reassign |
| Refusals rising | A source or limit has changed | Find the change; confirm it was intended |
| No edits for weeks | Reviewers are not reviewing, or the agent is excellent | Test with a seeded case |
A worked example
Take an insurer with an agent that settles small motor claims at depth 3 below a limit set in the firm's own rules. In a quarterly review, the risk manager notices that a cluster of settlements sit just under the limit, all from claims involving the same repair garage. Each decision was within bounds and each was correct on its own evidence, which was the garage's estimate, signed by a named party.
The pattern is the finding. The risk manager records it, asks the claims fraud team to look at the garage, and recommends that the firm's limits hold any claim within a set band below the limit where the estimate comes from a garage with more than a set number of claims in a quarter. The limit owner agrees, the change is recorded, and the next quarter's review checks that it worked.
What does the risk manager approve or decline?
When reviewing an individual decision, the same three outcomes apply. More often, the output of a risk review is a recommendation to the limit owner: tighten this, loosen that, add a source requirement. The record of the review, and of what changed because of it, is itself evidence for internal audit and supervisors.
Operations and property managers review with a process lens; compare how operations managers review an agent's decisions and how property managers review them. Building this skill in a team is covered in how to train the risk team to review agent decisions, and a market view in risk manager training in insurance in Brazil. For every role, see AI agent skills by role.
Where the Academy fits
Pryme Intelligence Academy's Operator track teaches how a governed agent decides and how to read the workspace, which is the ground a risk review stands on. Risk managers who design limits across several agents may go on to the Architect track, built for regulated, multi-agent deployments. A firm can check any certificate at source through VerifyDoc.ai, and a lapsed one shows as lapsed.
Questions people ask
How often should risk review agent decisions?
Monthly for patterns, with a deeper review each quarter and after any change to an agent's brief, sources or limits.
Should risk use the same sample as the first line?
No. Draw an independent sample, weighted towards near-misses and weak provenance, so the review tests what the first line might miss.
What is a seeded case?
A test item with a known correct answer, placed in the queue to check whether reviewers catch it. It must be recorded as a test.