Fraud analysts should drill the case where the agent works perfectly and still loses money: fraudsters learn its release limit and keep every payment just below it. In a tabletop of about 90 minutes, the fraud team practises pausing the release agent to draft-only, escalating to the fraud manager, tightening limits, recording decisions and reporting to customers, partners and leadership.
What does a fraud drill rehearse?
Fraud teams review transactions and account activity flagged as suspicious. Agents typically score and summarise held transactions, release low-risk ones at depth 3 within the business's limits, and draft recommendations on the rest for an analyst. Analysts decide on blocks, customer contact and account restrictions.
Unlike most incidents, a fraud incident has an adversary. The scenario should include someone adapting to the bounds. The fraud manager owns the drill; a senior analyst facilitates. Analysts, someone from customer operations and someone from payments take part. For the format, see what an incident drill is.
The scenario
Saturday, 07:00. Overnight, about 300 card-not-present transactions of just under 200 each, the agent's release limit, were flagged as suspicious, scored as low risk because each looked like a customer's normal spend, and released at depth 3. Customers begin reporting unrecognised payments. An analyst on the weekend shift notices that the amounts cluster just under the limit and share a merchant category.
The drill step by step
- Person (facilitator): reads the scenario and starts the clock. The weekend timing is deliberate: who is on call?
- Person (analyst on shift): escalates to the on-call fraud manager.
- Person (fraud manager): pauses the release agent to draft-only. Releases stop; every flagged transaction is held for an analyst. The verdicts now name the business's own limits.
- Person (fraud manager): tightens the business's own limits for the pattern once the pause is in place, such as holding all transactions in the merchant category, so analysts can work through the backlog.
- Person (analyst): lists every release in the window from the record and matches them to customer reports.
- Person (payments contact): starts recovery through the card scheme's processes for the confirmed fraudulent payments.
- Person (customer operations): contacts affected customers, blocks and reissues cards where needed.
- Person (fraud manager): records decisions and decides on reporting to leadership, the card scheme and any authorities required.
- Person (facilitator): runs the debrief, including how the agent's bounds should change.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Alert | Has released transactions | 3 | Spotting the pattern |
| Pause | Holds every release | Dropped to 2 | Tightening limits |
| New limit | Applies the tightened limit | 2 | Defining the pattern |
| Scope | Nothing new | 2 | Listing releases from the record |
| Recovery | Nothing | 2 | Card scheme processes |
| Customers | Drafts contact messages | 2 | Calls, blocks, reissues |
| Report | Nothing | 2 | Reporting decisions |
What is distinctive about fraud incidents?
Every release was within bounds. The agent applied the limit correctly. The bounds themselves were predictable. Fraud teams should learn to treat any fixed limit as information an adversary can discover, and to watch for clustering just below it.
The debrief often leads to changes in the business's own limits: a lower limit for new merchants, a cap on releases per merchant category per hour, or holds when many transactions approach the limit at once. These are changes to bounds, not to a model, and they are filed in the record like any other change.
The drill should also cover the customers caught in the pause. While every flagged transaction is held, genuine customers see payments declined or delayed on a Saturday morning. Customer operations needs a short, honest script that explains a security check is in progress without describing the fraud pattern, and analysts should clear held legitimate payments in order of customer impact, not in the order they arrived.
Which controls does the drill test, and what evidence do they leave?
The pause. Evidence: the limit change in the record, with name and time.
Targeted limits. Evidence: the new limit and who defined it.
Recovery. Evidence: each recovery request, with time.
Customer contact. Evidence: each block and reissue with the analyst's name.
What should the fraud manager measure?
- Time from the first customer report to the pause.
- Value released after the pattern began and before the pause.
- Recovery achieved through scheme processes.
- Whether on-call arrangements worked at a weekend hour.
Where do fraud drills go wrong?
Weekend cover untested. Many fraud teams discover in a real event that the on-call manager cannot change limits from home. Test it.
Pausing without a new limit. A pause holds everything, including legitimate transactions. Customers start complaining that cards are declined. Move quickly to a targeted limit.
No change after the drill. If the bounds stay predictable, the pattern will return.
Neighbouring drills cover the other angles: the operations team drill covers capacity when held queues swell, and the compliance team drill covers reporting decisions. Structure your drill with the AI agent incident drill script template, and see how to train a team to handle an AI agent incident and training teams for AI agents.
Where the Academy fits
Pryme Intelligence Academy's Operator track includes a module on teaching the agent, which covers changing its bounds and sources after an event like this, and a module on when something goes wrong. Fraud teams that design the release agent itself may need someone on the Builder track, whose capstone is an agent certified in Attest to depth 2 or higher.
Questions people ask
Should the release limit be secret?
Keep it internal, but assume it can be discovered by testing. Design bounds that do not rely on secrecy, such as caps on clustering.
Can the agent block a card?
Many businesses let an agent place a temporary block at depth 3 within bounds, with a person deciding on anything permanent. Decide this before the drill.
How often should fraud teams drill?
Quarterly is common because patterns change fast. Rotate scenarios so the team does not rehearse only the last attack.