How to train the financial crime team to review AI agent decisions

Financial crime reviewers need a programme that treats every closure as a decision someone may later have to defend. Teach how the agent triages alerts and screening hits, calibrate on real discounted and escalated cases, shadow live review with every closure checked, then sign off. The bar: reviewers can explain, from the record, why each alert was closed or escalated.

What will the financial crime team review?

A financial crime agent typically triages transaction monitoring alerts and sanctions or watchlist screening hits, gathers the customer's profile and history, drafts a case narrative and a recommendation, and requests information for periodic due diligence reviews. In tightly bounded cases, such as a screening hit where the name, date of birth and nationality all clearly differ, it may discount the hit itself at depth 3. Everything else is drafted at depth 2. Decisions to report suspicion stay with the nominated officer.

The FATF Recommendations set the international standard here: a risk-based approach, customer due diligence, and reporting of suspicious transactions. Supervisors in most markets expect firms to show how alerts are handled and that the people handling them are competent. Reviewer training is part of that evidence.

The team reviews three kinds of output: drafted narratives and recommendations; held items where the agent stopped; and a sample of hits it discounted alone.

The head of financial crime operations owns the programme. The nominated officer approves the sign-off standard. Quality assurance runs sampling.

How does the programme run, step by step?

1. Learning, week 1: reviewers complete lessons on the agent's decision chain: which bounds govern discounting, how sources are labelled, and when it must escalate.
2. Agent, depth 1: assembles a practice set of past alerts and hits with the drafted narrative and the final human decision, anonymised.
3. Calibration, week 2: reviewers decide alone, then compare. The nominated officer attends at least the first session.
4. Shadow, weeks 3 to 4: reviewers handle live alerts; a senior investigator checks every closure and escalation before it is final.
5. Agent, depth 2 and 3 in live work: drafts narratives; discounts clear screening false positives within bounds; holds the rest.
6. Sign-off: head of financial crime operations and the nominated officer assess shadow decisions on accuracy and narrative quality.
7. Ongoing: quality assurance sampling, monthly calibration, and a refresher whenever typologies or screening lists change materially.

What must reviewers learn to check?

The discounting rationale. A closed alert needs a reason a stranger could follow: what was checked, what it showed, and why that rules out concern. Reviewers check the agent's rationale is specific to the customer, not boilerplate.

Evidence weight. The customer's explanation is declared. Your transaction data is sealed. A counterparty's verified documents may be signed. Closure resting only on the customer's explanation needs particular care.

Expected activity. Does the activity fit what the customer told you at onboarding? The agent compares; reviewers judge whether a mismatch is innocent.

Linked parties. The agent may miss connections across accounts. Reviewers learn to ask what else is linked.

Auto-discounted screening hits. Sample them. A discount on a partial name match with a missing date of birth should not have happened at depth 3.

Tipping off. Requests for information must not reveal suspicion. Reviewers check the agent's drafted customer messages for anything that could.

A worked calibration case shows the difference between a weak and a strong closure. An alert fires on three incoming transfers of 9,000 each in a week to a personal account. The agent drafts a closure: the customer says the payments are from selling a car. A weak reviewer accepts it. A strong one notes the explanation is declared, checks whether the sender is a known party and whether the account history shows similar sales, and either finds supporting evidence or escalates.

Which controls leave evidence?

• Lessons before shadow. Evidence: completion dates.
• Every shadow closure checked. Evidence: senior investigator's name.
• Sign-off includes the nominated officer. Evidence: both names on the assessment.
• Auto-discounts sampled. Evidence: QA sample log.
• Narratives meet a written standard. Evidence: QA scores against it.

What should you measure?

• Shadow accuracy and narrative scores per reviewer.
• QA findings on closures, by reason.
• Auto-discount errors in samples.
• Alert ageing, since backlogs are themselves a risk.
• Escalations accepted by the nominated officer.

Where does it usually go wrong?

Narratives become templates. Reviewers accept boilerplate rationales because they read well. Calibrate on rationale quality, not just outcomes.

Auto-discounting is left unsampled. It feels safe because the bounds are tight. Sample anyway.

Backlog pressure. Teams under backlog pressure close faster and check less. Track quality alongside throughput.

Incidents untested. Practise what happens if the agent discounts something it should not have; see how to run an AI agent incident drill for the financial crime team.

New starters follow the reviewer onboarding checklist for the financial crime team, and skills by level are set out in the AI agent competency matrix for the financial crime team. Credit teams run the same four stages on lending decisions; see how to train the credit team to review AI agent decisions. The full programme is in training teams for AI agents.

Where the Academy fits

Pryme Intelligence Academy's Operator track covers how a governed agent decides, human review and when something goes wrong, with module tests set in the learner's own industry and role. Its practicum, two weeks of real review decisions with no missed red line, signed off by a Pryme assessor, gives a supervisor independent evidence of reviewer competence that anyone can check at source.