An incident drill is a timed rehearsal in which a team responds to a simulated failure as if it were real, carrying out each step rather than discussing it. For AI agents, a drill tests whether people can spot an agent acting wrongly, stop it, find the affected actions in the record and report, within a set time.
Why do incident drills matter for agents?
Agents fail differently from people. A person who misreads a rule makes one bad decision; an agent working from a wrong source can make the same bad decision many times in an hour. The window between first error and containment decides how much damage is done.
Teams that have only read the incident procedure lose that window. They hunt for the stop control, argue about who is allowed to use it, and only then start to look at the record. A drill finds those delays when they cost nothing.
How does an incident drill work in practice?
A drill has a script, a clock and an observer:
- Inject. The facilitator plants a fault in a lab workspace, such as a source replaced with a wrong policy, so the agent starts drafting wrong decisions.
- Detect. The team must notice from what reaches them: unusual held items, a reviewer's doubt, a customer query.
- Contain. A named person pauses the agent or drops its depth.
- Scope. The team uses the record to list every action affected since the fault.
- Report. The incident lead sends the first internal report.
- Debrief. The observer reads back the times and what slowed each step.
An incident drill script template gives the injects and the observer's sheet.
What does an incident drill look like in a real team?
Take a compliance team at a payments firm. The facilitator replaces the sanctions list source with an out-of-date copy in a lab workspace. The screening agent starts allowing transfers it should hold. A reviewer notices after four items, the team lead pauses the agent in six minutes, and scoping takes 25 minutes because nobody had filtered the record by source before.
That last gap becomes the next training session. The full method is in running an incident drill for the compliance team, and the technical side in the IT team's drill.
How is an incident drill different from a tabletop exercise?
A tabletop exercise is a discussion: the team sits around a scenario and talks through what they would do. An incident drill is done: people press the buttons, query the record and send the report, against a clock. Tabletops are good for finding gaps in the plan. Drills are good for finding gaps in the people's ability to carry it out.
Run a tabletop first when the procedure is new, then drill it.
Which related terms should you know?
- Incident commander: the person who leads the response and decides.
- Near miss: a real event caught before harm, which makes good material for the next drill.
- Containment: stopping the harm from spreading, usually by pausing the agent or reducing its depth.
For the broader skill, see how to train a team to handle an agent incident and the topic guide on training teams for AI agents.
Where the Academy fits
Pryme Intelligence Academy's Operator track has a module on when something goes wrong, and its labs run in a free Build workspace where a fault can be planted safely. Drills built there use the same product and record the team will use in production.
Questions people ask
How often should a team run an incident drill?
Before go-live, then at least twice a year, and after any real incident or any change in the agent's depth.
Should drills be announced?
The first should be. Once the team is comfortable, unannounced drills test detection as well as response.