PrymeIntelligence

Training teams for AI agents·Definition

What is a tabletop exercise?

A tabletop exercise is a guided discussion of a realistic scenario to test who does what. How to run one on an AI agent failure, and what it teaches.

A tabletop exercise is a structured discussion in which a group walks through a realistic scenario, such as an incident, step by step, saying what they would do and who would decide, without touching any live system. For AI agents, it rehearses the human response when an agent misbehaves, a certificate lapses or a source turns out to be wrong.

Why does a tabletop exercise matter?

Agent failures are rare, which is exactly why teams are unprepared for them. When one happens, the first hour decides how much damage is done. Who can pause the agent? Who tells customers? Who checks the record to see how many cases were affected? A tabletop surfaces the gaps in those answers while nothing is at stake.

It also trains the people who are not on the queue. Executives, legal, communications and the agent's owner rarely see an agent's day-to-day work. In a tabletop, they discover what they would need to know and how fast they could get it.

How does a tabletop exercise run?

A typical session takes one to two hours:

  1. Set the scene: one agent, its capabilities and depth, the customer's limits, the day and time.
  2. Inject the first event: "Customers report receiving letters quoting the wrong fee."
  3. Discuss: each role says what they do next and what they need to know.
  4. Inject more: "The record shows 400 letters sent overnight. The source document was replaced two days ago."
  5. Close and debrief: what worked, what was missing, who owns each fix.

A facilitator keeps time and plays the events. A note-taker captures every gap. The guide to running a tabletop exercise for an agent failure gives full scenarios.

An example

A utility runs a tabletop on its billing agent, which drafts and sends routine payment reminders at depth 3. The scenario: an Attest run overnight clears no depth and the agent drops to depth 1, so reminders stop. Nobody in the room knew that the failed run would stop sends the same day, or who would be told. The team agrees that the agent's owner receives an alert, operations switches to manual reminders, and the owner reports on the cause within a working day. They also realise no one had read access to the record outside office hours, and fix it.

How is it different from a red team exercise?

A tabletop exercise is talk: people discuss a scenario around a table. A red team exercise is action: a group deliberately tries to make the agent misbehave by sending it real inputs. The tabletop tests the people and the plan. The red team tests the agent and its bounds.

It also differs from a lab exercise, where individuals practise a task hands-on in a safe workspace with a known correct outcome. A tabletop has no single right answer; it tests coordination.

Which terms are related?

  • Red team exercise: an active attempt to break the agent.
  • Lab exercise: hands-on individual practice.
  • Post-incident review: the real-world version, after something has happened.

Tabletops expose what untrained teams do not know, a point made in why agent pilots fail when nobody is trained. Whether such exercises should be required is covered in should agent training be mandatory. For the full approach, read training teams for AI agents.

Questions people ask

Who should attend a tabletop exercise?

Everyone with a role in the response: the agent's owner, a team lead, a reviewer, someone from risk, and a decision-maker who can pause the agent or speak to customers.

How often should you run one?

Before go-live, after any significant change to the agent, and at least once a year. Vary the scenario each time.

Train the people who run your agents.

Pryme Intelligence Academy: free courses on the real product, audio lessons, labs in a free workspace, and certificates anyone can check on VerifyDoc.ai. Teams get seats, an admin view and a free readiness assessment.