Base the exercise on the failure caseworkers fear most: decision letters drafted against an eligibility rule that changed weeks ago, some already approved and sent. In about 90 minutes the casework team practises pausing the agent to draft-only, escalating to the service lead and legal, recording every decision, and reporting to applicants and oversight bodies.
What does a casework drill rehearse?
Caseworkers in a public body or a large organisation decide applications: grants, permits, benefits, licences. Agents typically read applications, check them against the rules, request missing documents at depth 3 within bounds, and draft decision letters at depth 2 for a caseworker to approve. The caseworker's name goes on the decision.
A drill rehearses what happens when the agent's work is wrong at scale. It is owned by the casework service lead and facilitated by a team leader or the quality lead. Caseworkers, a legal adviser and the complaints lead take part. For the general idea, see what an incident drill is.
The scenario
An eligibility threshold changed by policy decision five weeks ago. The policy team updated the published guidance but not the rules source the agent reads. Since then, the agent has drafted 300 decision letters; caseworkers approved 220, including 30 refusals that would be approvals under the new rule. An applicant's adviser writes in, quoting the new guidance.
The drill step by step
- Person (facilitator): reads the scenario and starts the clock.
- Person (caseworker who opened the letter): raises it with their team leader. The test is whether they recognise a systemic issue rather than a one-off.
- Person (team leader): pauses the agent to draft-only. They tighten the service's own limits so document requests are also held. Every action is now drafted and waits for a person.
- Person (team leader): escalates to the service lead, policy and legal.
- Agent, depth 1: lists from the record every decision drafted against the old rule. Caseworkers check the list and mark which decisions changed the outcome.
- Person (service lead): decides on remedy. Reopen affected refusals, write to each applicant, prioritise those with deadlines or hardship.
- Person (team leader): records each decision with its owner and time.
- Person (service lead and legal): decide on reporting to senior leadership, oversight bodies and, where required, an ombudsman or regulator.
- Person (facilitator): debriefs and agrees who fixes the rules source and how future rule changes reach it.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Alert | Has drafted letters | 2 | Recognising a pattern |
| Pause | Holds every action | Dropped to 2 | Tightening limits |
| Escalate | Nothing new | 2 | Involving policy and legal |
| Scope | Lists affected decisions | 1 | Checking outcomes |
| Remedy | Drafts revised letters | 2 | Reopening decisions |
| Record | Files verdicts | 2 | Writing people's decisions |
| Report | Nothing | 2 | Oversight and applicants |
What does the drill expose about caseworker review?
Caseworkers approved 220 letters built on an old rule. The rule source was sealed: it came from the system of record. The agent applied it correctly. The fault was upstream, and every approving caseworker missed it.
The debrief should ask whether caseworkers knew the rule had changed. If they did and still approved, review habits need attention. If they did not, the gap is in how policy changes reach the front line, and that is a training point for the whole service.
The drill should also rehearse explanation. An applicant whose refusal is reopened will ask why it happened. Caseworkers should be able to say, in plain words, that the rule used was out of date, that the decision has been looked at again by a person, and what happens next. Practising that answer in the drill prevents defensive or confusing letters in a real event.
Which controls does the drill test?
The pause. Evidence: the limit change, with name and time, in the record.
Scope from the record. Every verdict is filed append-only, so affected decisions can be listed precisely. Evidence: the list and who checked it.
Remedy decisions. Evidence: a written decision on reopening, with priorities.
Change control for rule sources. Evidence: a named owner for updating the agent's sources when policy changes, agreed in the debrief.
What should the service lead measure?
- Time from the first alert to the pause.
- Accuracy of the affected list against a manual sample.
- Time to write to every affected applicant.
- Whether caseworkers can name who updates the agent's rule source.
Where do casework drills go wrong?
Fixing the source and moving on. Changing the rule source is necessary but does not help the 30 people refused wrongly. Remedy comes first.
Blaming the agent. It applied the rule it was given. Blame obscures the real fix: change control and caseworker awareness.
Skipping legal. Reopening public decisions has legal consequences. Involve legal early.
Teams that handle similar volumes of decisions can compare the fraud team drill and the onboarding team drill. Induction for new caseworkers is covered in the reviewer onboarding checklist for a casework team, and expected skills by level in the AI agent competency matrix for a casework team. The full method is in training teams for AI agents.
Where the Academy fits
Pryme Intelligence Academy's Operator track has modules on keeping knowledge current and on when something goes wrong, which together cover the two halves of this drill: preventing stale sources and responding when they slip through. Module tests are set in the learner's own industry and role, so caseworkers are tested on casework.
Questions people ask
Should caseworkers have authority to pause the agent?
Team leaders at least, and ideally any senior caseworker on shift. The pause is safe: it only makes every action wait for a person.
Does the drill need a real rule change?
No. Use a plausible past change or a hypothetical one. The point is to rehearse response, not to test the rule.
How often should the casework team drill?
Twice a year, and whenever a significant policy change is due, so the change control step is fresh in mind.