PrymeIntelligence

Training teams for AI agents·Playbook

How to run an AI agent incident drill for the operations team

Run an AI agent incident drill for the operations team: an agent drops to depth 1 after a run that clears no depth; pause, escalate, re-plan, record and report.

The operations team's drill should rehearse a different kind of incident: nothing wrong has been sent, but an agent's overnight certification run has cleared no depth and dropped it to depth 1. Over about two hours, operations managers practise pausing dependent agents to draft-only, escalating, re-planning the day around a larger held queue, recording decisions and reporting.

Why is a depth drop an incident?

Certificates lapse after 90 days; a lapse, or a run that clears no depth, drops the agent to depth 1 the same day. That is the platform working as designed: an agent that cannot show it still meets the floor loses its authority to act. But for the operations team, the effect is immediate. Work the agent did alone at depth 3, and drafted at depth 2, now arrives as items for people to do by hand. Service levels slip within hours.

The operations manager owns the drill and runs it. Team leaders, workforce planners, the person who manages certification, a customer communications lead and representatives of dependent teams take part. For the general format, see what an incident drill is.

The scenario

Monday, 06:30. The overnight Attest run for the order processing agent failed the stable gate at every depth. The agent is at depth 1: it can read but not draft or act. An orchestrator at depth 5 that routes work between agents now sends orders to the processing agent, which can do nothing with them. A downstream invoicing agent at depth 3 waits for processed orders that do not arrive. By 08:00, 3,000 orders are waiting.

The drill step by step

  1. Person (facilitator): reads the scenario and starts the clock.
  2. Person (operations manager): confirms the depth drop in the workspace and reads the Attest result: which gate failed and on what cases.
  3. Person (operations manager): pauses dependent agents to draft-only. The orchestrator and invoicing agent are held through the business's own limits, so they do not act on partial or stale inputs.
  4. Person (operations manager): escalates to the certification owner, customer communications and leadership.
  5. Person (workforce planner): re-plans the day. How many trained people are needed to process orders by hand? Who comes off other work?
  6. Person (team leaders): prioritise the queue, for example orders with same-day delivery commitments first.
  7. Person (certification owner): investigates the failed gate. Drift usually means the agent's outputs have moved away from expected answers on real cases; a source change is a common cause.
  8. Person (customer communications lead): decides on customer messages about delays.
  9. Person (operations manager): records each decision with time; the facilitator runs the debrief, including the conditions for recertification.
Step The agent does Depth Stays with a person
Depth drop Reads only 1 Confirming and reading the result
Pause dependents Orchestrator and invoicing held Dropped to 2 Tightening limits
Escalate Nothing new 1 and 2 Certification, comms, leadership
Re-plan Nothing 1 Workforce decisions
Prioritise Nothing 1 Team leaders
Investigate Is replayed against cases 1 Certification owner
Communicate and record Nothing 1 and 2 Customer messages, decisions

What does this drill teach that others do not?

Three things. First, dependencies. Operations teams often know their agents but not how they connect. The orchestrator and the invoicing agent were affected by a failure in a third agent. A dependency map is a common outcome of the drill.

Second, capacity. A depth drop turns automation into manual work instantly. The drill reveals how many trained people the team really has, and whether they can do the work without drafts.

Third, recertification discipline. The temptation is to push the agent through Attest again quickly. Depth is earned, not configured: the agent returns only when evidence shows the drift has been fixed.

Customer communication is part of the operations response. Orders will be late, and customers would rather hear that from the business than discover it. The drill should produce a short, honest message about delays, approved by the customer communications lead, and a rule for when it goes out.

Which controls does the drill test, and what evidence do they leave?

Depth drop. Evidence: the Attest run that cleared no depth and the same-day drop to depth 1, both in the record.

Pausing dependents. Evidence: limit changes for each dependent agent, with names.

Re-plan. Evidence: the revised rota and priority list.

Recertification. Evidence: the fix, the new Attest run and the gate results.

What should the operations manager measure?

  • Time from the depth drop to dependent agents being paused.
  • Backlog at its peak and time to clear it.
  • Trained people available against the number needed.
  • Time to recertification, and whether the fix addressed the failed gate.

Where do operations drills go wrong?

Treating a depth drop as an IT outage. It is an operations event: work must still be done.

Dependents left running. An orchestrator or downstream agent acting on incomplete inputs can create a second incident.

Rushing recertification. A second failed run costs more time than a careful fix.

Operations links every other team's drill; compare the payments operations team drill for money-moving work and the legal operations team drill for a misdirected message. Build your drill with the AI agent incident drill script template, and see how to train a team to handle an AI agent incident and training teams for AI agents.

Where the Academy fits

Pryme Intelligence Academy's Operator track is for operations managers and team leads. Its eight modules include reading the workspace, when something goes wrong and running the rhythm, which together cover spotting a depth drop, responding to it and keeping work moving. Certified Pryme Operator is valid for 12 months.

Questions people ask

Can we stop an agent from dropping to depth 1?

No. A lapse, or a run that clears no depth, drops the agent the same day by design. What you can do is plan for it, with trained people and a known process.

Should the orchestrator be paused too?

If it would route work to an agent that cannot act, or downstream agents would act on incomplete inputs, yes.

How often should operations drill a depth drop?

Twice a year, and ideally timed near a real recertification date so the plan is fresh.

Train the people who run your agents.

Pryme Intelligence Academy: free courses on the real product, audio lessons, labs in a free workspace, and certificates anyone can check on VerifyDoc.ai. Teams get seats, an admin view and a free readiness assessment.