Fraud analysts new to an agent workspace should release nothing alone in their first two weeks. Week one teaches how the agent scores alerts, which signals and sources it uses, and what it holds. Week two brings live alerts, with a senior analyst confirming every release. Sign-off depends on catching what the agent missed, not clearing what it flagged.
What is the agent doing in the fraud team?
In a payments, card or e-commerce fraud team that runs agents, the agent typically scores transactions and account events as they happen, holds those above a threshold, gathers device, behaviour and history signals, groups related alerts, and drafts a recommendation to release, block or escalate (depth 2). Where certified, it may release low-risk false positives within the business's limits (depth 3). Blocking a customer, closing an account or reporting fraud stays with people.
New analysts therefore inherit a queue that is already triaged, with a recommendation on every alert. That helps, and it is also the main risk: an analyst who learns to agree with the agent misses what it missed. How the day shifts is set out in what changes in a fraud analyst's day when AI agents arrive. The fraud operations lead owns onboarding; a certified senior analyst coaches.
What does the fortnight contain?
- Day 1. Person (fraud operations lead): explains the agent's bounds: scoring threshold, what it holds, what it may release alone, and what it must escalate.
- Day 1 to 2. Person (new analyst): learns signal provenance. Transaction data from the core system is sealed. A chargeback confirmed by the card scheme is signed. A customer's claim that a payment was theirs is declared. A merchant's emailed tip is external, a reason to look and nothing more.
- Day 2 to 3. Agent, depth 2: scores and recommends on live alerts while the coach decides them aloud.
- Day 4 to 5. Person (new analyst): works past alerts in a sandbox, including several confirmed frauds that the agent scored low, with outcomes hidden.
- Week 2. Person (new analyst): decides live alerts, with the coach confirming every release before it takes effect.
- Week 2. Person (new analyst): records a reason for every disagreement, marking whether the agent was too strict or too lenient.
- End of week 2. Person (fraud operations lead): signs off release rights for standard alerts if no fraud was released and missed-signal catches are sound.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Score | Scores events, holds above threshold | 2 | Threshold changes |
| Gather | Device, behaviour, history signals | 1 | Further enquiry |
| Recommend | Release, block or escalate | 2 | The decision |
| Low-risk release | Releases within limits | 3 | Anything outside limits |
| Block and report | Nothing | none | Analyst and lead |
Why must new analysts practise on the agent's misses?
Because clearing false positives teaches agreement. Most alerts the agent holds are not fraud, so a new analyst who agrees with every "release" recommendation will be right most of the time and still miss the frauds that matter. The sandbox set should include confirmed frauds the agent scored low or recommended releasing. An analyst who finds them by reading the signals, not the score, is learning the job.
The same principle applies in the second week. Coaches should ask, for every release, "what would make you wrong?" The analyst should be able to answer from the signals.
What red lines must they hold?
No release on an external claim. A customer email saying "this payment is genuine" is declared at best. Release decisions rest on verified signals.
No release outside their authority. Values or customer types above the analyst's limit go to the senior analyst.
No tipping off. Where fraud links to suspected money laundering, customer contact must not reveal the suspicion.
No contact outside approved scripts. When an analyst calls a customer to confirm a transaction, they use the approved verification steps. They never ask for full card details or passwords, because those are exactly the requests fraudsters imitate.
How should corrections be recorded?
Every overturn gets a reason and a direction: missed signal (agent too lenient), false signal (agent too strict), stale list, unclear rule, or analyst judgement. The direction matters because a team that records only false signals will push the agent towards leniency over time. New analysts who learn to record missed signals keep the agent sharp.
Which controls and evidence should onboarding leave?
- Sandbox with hidden frauds. Evidence: which frauds the analyst found.
- Confirmed releases in week two. Evidence: both analysts named on each.
- Direction on every overturn. Evidence: reason codes with direction.
- Staged release rights. Evidence: dated authority changes.
What should the lead measure?
Frauds the new analyst caught that the agent scored low. Releases later confirmed as fraud, which must be zero before sign-off. Ratio of missed-signal to false-signal corrections. Time per alert. And a sample of releases each month for the first quarter.
Where does it usually go wrong?
Speed over judgement. Queues are long, and a new analyst learns that agreeing is fast. The hidden-fraud sandbox exists to break that.
Training on screens, not typologies. Analysts who know the tools but not how account takeover or authorised push payment scams look cannot see what the agent missed.
Release rights granted in one step. Raise authority by value and customer type gradually.
Related plans cover a new credit analyst and a new payments operations analyst, who often work beside fraud. The AI agent training plan template for fraud analysts and the reviewer onboarding checklist for the fraud team turn this into documents, and the wider approach is in training teams for AI agents.
Questions people ask
Can a new analyst change the agent's scoring threshold?
No. Thresholds are part of the business's limits, set by the fraud lead and tested through re-certification.
What if the new analyst thinks the agent is too strict?
Record false-signal reasons. If a pattern holds across analysts, the agent owner can act on it, with a second approver because it loosens the agent.
How long before a new analyst handles high-value alerts?
After a period of accurate standard work, often several weeks or more, set by the lead.