PrymeIntelligence

Training teams for AI agents·Playbook

How to run an AI agent incident drill for the legal operations team

Run an AI agent incident drill for legal operations: a matter update sent to the other side, pause to draft-only, escalate, record and report.

Legal operations teams should rehearse a misdirected communication, because it touches privilege, confidentiality and the client relationship at once. The scenario: the update agent emails a client's case summary to the opposing lawyer. In about 90 minutes, paralegals practise pausing the agent to draft-only, escalating to the supervising lawyer and risk partner, recording decisions and handling the fallout.

What does a legal operations drill rehearse?

Legal operations teams in law firms and in-house legal departments use agents to draft matter updates, summarise documents, track deadlines and assemble bundles. A matter update agent may send routine status emails at depth 3 within bounds, for example to a client's named contacts; anything substantive is drafted at depth 2 for a lawyer to approve. Contact details come from the matter management system.

A misdirected email in legal work can waive privilege, breach confidentiality and damage the client's case. The risk partner or general counsel owns the drill; a senior paralegal or legal operations manager facilitates. Paralegals, a supervising lawyer, the firm's risk or compliance lead and IT take part. For the general format, see what an incident drill is.

The scenario

Wednesday, 17:30. On a commercial dispute matter, someone added the opposing firm's lawyer to the matter's contacts list to track correspondence, but tagged them as a client contact by mistake. At 17:00 the matter update agent sent its weekly status email at depth 3 to all client contacts, including the opposing lawyer. The email summarised the client's settlement position. The opposing lawyer replies, noting receipt.

The drill step by step

  1. Person (facilitator): reads the scenario and starts the clock.
  2. Person (paralegal who sees the reply): escalates to the supervising lawyer immediately. The test: does she act before the end of the day?
  3. Person (legal operations manager): pauses the agent to draft-only across the practice group. Every update is now held for a person. The verdicts name the firm's own limits.
  4. Person (supervising lawyer): contacts the opposing lawyer, asking them to delete the email unread and confirm, following the professional rules that apply to inadvertent disclosure in that jurisdiction.
  5. Person (paralegal): checks the record for other matters where a non-client contact was tagged as a client and received updates.
  6. Person (risk partner): decides on client notification and whether to notify the firm's insurer.
  7. Person (legal operations manager): corrects the contact record and agrees a new rule, such as holding any update whose recipient's email domain belongs to a firm on the other side.
  8. Person (paralegal): records each decision with owner and time.
  9. Person (facilitator): runs the debrief.
Step The agent does Depth Stays with a person
Alert Has sent the update 3 Spotting the reply
Pause Holds every update Dropped to 2 Tightening limits
Contact the other side Nothing 2 Supervising lawyer
Scope Nothing new 2 Listing other affected matters
Client and insurer Nothing 2 Risk partner
Fix Applies the new hold rule 2 Correcting contacts
Record Files verdicts 2 People's decisions

What is specific about legal operations incidents?

The agent did nothing outside its bounds. It sent an update to contacts tagged as client contacts, which it was certified to do. The contact record, a declared fact entered by a person, was wrong.

That is a lesson for paralegals who maintain matter data: the agent will act on what they enter. A new bound, holding any update to an address on the opposing side's domain, turns a data-entry error into a held draft rather than a disclosure.

Rehearse what the firm tells the client. The client's settlement position may now be known to the other side, and the supervising lawyer must explain that promptly and plainly, along with what the firm has done and what it means for the case. Drafting that call in the drill helps the lawyer separate facts from reassurance.

Which controls does the drill test, and what evidence do they leave?

The pause. Evidence: the limit change in the record, name and time.

Contact with the other side. Evidence: the supervising lawyer's email and the reply confirming deletion.

Scope across matters. Evidence: the list of matters checked and the result.

New bound. Evidence: the domain hold rule and who added it.

Professional conduct rules on inadvertent disclosure differ by jurisdiction. This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.

What should the risk partner measure?

  • Time from the reply to the pause.
  • Time to the supervising lawyer contacting the other side.
  • Matters found with the same contact error.
  • Whether the domain hold rule was added before depth 3 was restored.

Where do legal operations drills go wrong?

Waiting until morning. Late-afternoon incidents get left overnight. The drill should run at 17:30 for this reason.

Fixing one matter. The same tagging error may exist elsewhere. Check the record across the practice group.

Blaming the paralegal who tagged the contact. The fix is a better bound and better data checks, not blame.

Compare neighbouring drills: the operations team drill covers capacity when an agent drops depth, and the payments operations team drill covers money that cannot be recalled. New legal operations reviewers can learn the steps through the reviewer onboarding checklist for a legal operations team, and expected skills by level are in the AI agent competency matrix for a legal operations team. The overall method is in training teams for AI agents.

Questions people ask

Should matter updates go out at depth 3 at all?

Routine status updates to verified client contacts can, once certified. Anything describing strategy, settlement or advice should be drafted at depth 2 and approved by a lawyer.

Who decides whether privilege has been waived?

A lawyer, usually the supervising partner with the risk partner, on the facts and the law that applies. The drill rehearses getting them the facts quickly.

How often should legal operations drill?

Twice a year, and whenever matter management data or agent capabilities change.

Train the people who run your agents.

Pryme Intelligence Academy: free courses on the real product, audio lessons, labs in a free workspace, and certificates anyone can check on VerifyDoc.ai. Teams get seats, an admin view and a free readiness assessment.