Gather the agent's owner, approvers, operations, risk, IT, legal and communications for two hours around a realistic failure, then release injects on a timeline and ask at each point: who notices, who lowers the agent's depth, how do we find what it did, who do we tell? A facilitator keeps time, a scribe records decisions, and gaps become owned actions.
Why rehearse an agent failure on paper?
Agent failures do not look like system outages. Nothing goes down. The agent keeps working, quickly and confidently, and the problem is in what it is doing. By the time anyone notices, it may have taken hundreds of actions. The response needs people who have rarely worked together, such as the agent owner, the communications team and the risk function, to make fast decisions about something new to most of them.
A tabletop exercise lets them make those decisions once, without customers on the line. It finds the gaps in your plans: the step no one owns, the record no one can retrieve, the approval that takes a day when it needs an hour.
The exercise owner is usually the head of agent operations or the operational resilience lead. A facilitator who did not design the agent runs the session. A scribe records everything.
What are the steps?
- Exercise owner: chooses the scenario and objectives. One failure, described in enough detail to be believable, with three or four things the exercise should test.
- Facilitator: writes the injects: timed updates that move the scenario on. "09:40: a customer emails asking why their statement shows a fee they were told was waived." "10:15: a second team reports the same."
- Agent, depth 1: produces realistic material for the injects from historic, anonymised verdict records: what a list of affected actions would look like, what a refused action record contains. Nothing from live customers.
- Exercise owner: invites participants with decision rights, not deputies who cannot decide.
- Facilitator: runs the session, releasing injects and asking each role what they would do, who they would call and what they would need.
- Scribe: records each decision, who made it, how long it took, and every point where someone said "I don't know" or "we'd have to check".
- Facilitator: runs a short debrief at the end while memories are fresh.
- Exercise owner: writes the report: what went well, gaps found, actions with owners and dates.
- Exercise owner: tracks actions to closure and schedules the next exercise.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Scenario | Nothing | none | Designing it |
| Injects | Nothing | none | Writing them |
| Material | Produces samples from historic records | 1 | Approving use |
| Invitations | Nothing | none | Choosing decision-makers |
| Session | Nothing | none | Facilitating |
| Record | Nothing | none | Scribing |
| Debrief | Nothing | none | Facilitator |
| Report | Drafts from the scribe's notes | 2 | Writing conclusions |
| Actions | Nothing | none | Owning them |
What scenarios are worth exercising?
Choose failures that would test the plan, not ones the plan obviously handles.
Quiet, wide error. A source document changes and the agent applies an old fee rule to every case for three days. It passed its checks because the average quality looked fine; the problem is concentrated in one case type, the trap described in an average score hides the failure that matters.
Manipulated input. An email to a shared mailbox contains instructions that lead the agent to draft payments to a new account. Did the bounds hold? How would you know?
Lapsed certificate. A capability's certificate expired and no one noticed. Should its actions in the meantime be reviewed?
Provider outage. The model provider is unavailable at month-end. Can the team take the work back, and how fast?
Widened bound. A limit was raised by mistake and the agent acted on cases above the intended ceiling.
What questions should every exercise answer?
- Who notices first, and through what signal?
- Who has the authority to lower the agent's depth, and how quickly does it take effect?
- How do we list every action the agent took in the affected period, and who reviews them?
- Who takes the work back, and are they ready? The weekly figures from the weekly AI agent performance review should show whether the fallback team is still practised.
- Who tells customers, what do we say, and who approves it?
- Does this need reporting to a regulator, a partner or the board, and who decides?
- How do we restore depth, and what evidence would we need?
What evidence does the exercise leave?
The scenario and objectives, dated before the session.
Attendance, with each person's role and decision rights.
The decision log: every decision, who made it, and the time from inject to decision.
Gaps found: each point where the plan was silent, wrong, or depended on someone absent.
Actions, each with an owner and a date, tracked to closure.
Supervisors who ask about operational resilience will want to see that agent failures are part of the testing programme. This evidence shows it.
Where does it usually go wrong?
The scenario is too easy. The agent fails loudly, someone switches it off, everyone agrees. Nothing is learned. Make the failure quiet and the evidence ambiguous.
Deputies attend. Deputies say "I'd have to check with my manager", which is realistic but tests nothing. Get the decision-makers.
Records are assumed. Participants say "we'd pull the logs". Ask for a real example in the room: who would do it, from which system, and how long it would take.
Actions are not tracked. The report is filed and forgotten. Review open actions at the start of the next exercise.
Shadow mode is forgotten as an option. During recovery, the agent can continue in shadow, drafting without acting, while people work; that option is explained in how to run an AI agent in shadow mode. Make sure the exercise considers it, along with the steps in train, shadow, certify, widen for restoring depth.
Questions people ask
How often should we run these exercises?
At least once a year for each important agent, and after any significant change in what it does or the systems it touches.
Should the agent's builders take part?
Yes, but not as facilitators. They know how the agent works and can answer technical questions; they should not steer the scenario.
Can we run the exercise with a live agent?
A tabletop is a discussion, not a live test. If you want to test live controls, such as lowering depth, do that separately in a test environment.