How to run an AI agent incident drill for the practice team

A clinic practice manager should drill an incident that mixes money, patients and insurers: the billing agent sends payment reminders to patients whose treatment is covered by their insurer, because the pre-authorisation feed stopped updating. In about an hour, the practice team practises pausing the agent to draft-only, escalating, correcting accounts, recording decisions and apologising to patients.

What does a practice team drill rehearse?

In a private clinic or medical practice, the practice team runs bookings, billing, insurer claims and patient communications. A billing agent typically drafts invoices, tracks insurer pre-authorisations and sends routine payment reminders at depth 3 within bounds. Write-offs, fee changes and refunds above the practice's limit are held for a person.

The practice team is small, so the drill must fit around clinics. The practice manager owns it and usually facilitates. Receptionists, the billing lead, a clinician and, if available, someone from the practice's software supplier take part. For the general format, see what an incident drill is.

The scenario

Tuesday, 08:30. A patient calls, upset: she received a reminder for 400 for physiotherapy her insurer pre-authorised. The billing lead checks: the insurer portal connection stopped updating authorisation status last Wednesday. Since then, the agent sent reminders to 35 insured patients, and two were passed to the practice's debt recovery process.

The drill step by step

1. Person (facilitator): reads the scenario and starts the clock.
2. Person (receptionist): apologises, takes details and escalates to the practice manager.
3. Person (practice manager): pauses the billing agent to draft-only. No more reminders without a person. The verdicts name the practice's own limits.
4. Person (billing lead): restores the insurer connection with the supplier and confirms the window.
5. Person (billing lead): lists every reminder sent to an insured patient from the record.
6. Person (practice manager): recalls the two accounts from debt recovery at once.
7. Person (receptionists): call each affected patient to apologise and confirm their account is correct.
8. Person (practice manager): records decisions and decides on reporting, internally to the owner and, if any personal data went to the wrong place, to the data protection lead.
9. Person (facilitator): runs the debrief.

What makes this drill specific to a practice team?

Size. In a team of five, the person who spots the problem may also be the one who fixes it, calls patients and decides on reporting. The drill shows whether one person can do all that, or whether the practice needs a named deputy.

The insurer status was signed: it came from an identifiable party through a connection. When the connection stopped, the agent used the last known status, which for new treatments was "not authorised". A useful new bound is to hold any reminder where the insurer status is older than a set period, so a person checks before chasing a patient.

Practice teams also learn from each other. A community of practice, where practice managers from different clinics share drill scenarios and lessons, spreads good bounds quickly.

How do you fit the drill around clinics?

A practice team cannot close reception for an afternoon. Run the drill in a protected hour before clinics start or after they end, with one receptionist covering the phones and swapping in halfway through. Keep the scenario on paper or in a practice workspace, so nothing reaches real patients.

Give each person two roles, because that is what a real event will demand. The billing lead plays both the person restoring the connection and the person listing affected patients; the practice manager both pauses the agent and decides on reporting. Where one person cannot do both in time, the drill has found the gap that a deputy should fill.

Invite one clinician for the last twenty minutes. Clinicians are often the first person a patient complains to in the treatment room, and they need to know what to say and whom to tell.

Which controls does the drill test, and what evidence do they leave?

The pause. Evidence: the limit change in the record.

Debt recovery recall. Evidence: the recall request with time.

Patient calls. Evidence: a note of each call on the patient's account.

Freshness bound. Evidence: the new rule and who added it.

What should the practice manager measure?

• Time from the patient's call to the pause.
• Time to recall the accounts from debt recovery.
• Time until every affected patient had a call.
• Whether the freshness bound was in place before reminders resumed at depth 3.

Where do practice team drills go wrong?

Fixing the connection and moving on. The 35 patients still believe they owe money. Call them.

Forgetting debt recovery. Accounts already passed on must be recalled first; that is where harm grows fastest.

No deputy. If the practice manager is away, someone else must be able to pause the agent.

Practice teams can learn from larger teams' drills: the collections team drill covers chasing the wrong people at scale, and the claims team drill covers insurance figures that go wrong. New practice staff can learn the steps through the reviewer onboarding checklist for a practice team. The overall approach is in training teams for AI agents.