How to train the casework team to review AI agent decisions

Caseworkers learn to review an agent's drafted assessments in four stages: lessons on what the agent reads and why it holds, calibration on real anonymised cases, two weeks of shadowed review with every decision checked, and sign-off against a written standard. The standard must include meaningful review: a caseworker who approves drafts without reading the evidence has not passed.

What is the casework team actually reviewing?

A casework agent, in a council, a housing association, a benefits service or a charity, typically reads applications and supporting evidence, checks them against eligibility rules, drafts an assessment and a decision letter, books routine appointments, and requests missing documents. Its assessments sit at depth 2: a caseworker decides and signs. Requests for missing evidence and appointment booking may run at depth 3 within set bounds.

Decisions here change people's lives: whether someone gets housing support, a grant, a service. Where a decision has legal or similarly significant effects, data protection law restricts decisions based solely on automated processing. The caseworker's review is what makes the decision a human one, and only if the review is real. Public bodies also carry transparency and fairness duties beyond those of private firms. Training has to make meaningful review the default, under caseload pressure.

The casework manager owns the programme. The safeguarding lead and the information governance lead help set the sign-off standard.

What does the programme look like?

1. Learning, week 1: caseworkers complete lessons on how the agent assesses: what sources it uses, how each is labelled, the bounds it works within, and the difference between a held case and a refused action.
2. Agent, depth 1: builds a practice set from past cases with the drafted assessment and the caseworker's final decision, anonymised.
3. Calibration, week 2: each caseworker decides the set alone, then compares in a session. Disputed cases go to the policy owner, not to a vote.
4. Shadow, weeks 3 and 4: caseworkers review live drafts, with a senior caseworker checking every decision before it is issued.
5. Agent, depth 2 in live work: drafts assessments and letters; requests missing evidence at depth 3 within bounds.
6. Sign-off: the casework manager assesses shadow decisions for accuracy, reasons and safeguarding handling.
7. Ongoing: sampling and monthly calibration, plus refresher lessons when policy changes.

What should caseworkers learn to check?

The evidence behind each finding. A drafted assessment says "income below threshold". The caseworker checks which document showed that, and whether it was a payslip, a bank statement or the applicant's own statement. Declared figures may be enough for some decisions and not for others; the rules say which.

What is missing. The agent may not notice what an applicant did not say. Caseworkers learn to ask: is there anything about this household the assessment should have considered?

Discretion. Many schemes allow discretion in exceptional circumstances. The agent applies the rules; discretion stays with the caseworker, who must record the reason.

Safeguarding and vulnerability. If anything in the file suggests risk to a person, that comes first, before the eligibility decision.

The letter. Can the applicant understand why the decision was made and how to challenge it? Reasons must be specific, not generic.

Third-party contact. Advocates and relatives contact caseworkers constantly. Checking consent before sharing is a skill to practise.

Which controls leave evidence?

• Lessons before shadowing. Evidence: completion dates.
• Meaningful review. Each approval records the evidence the caseworker checked. Evidence: the checked-sources field on each case.
• Discretion documented. Evidence: a reason on every case where discretion was used.
• Safeguarding first. Evidence: the referral or contact record precedes the decision.
• Sign-off recorded. Evidence: the assessment with sampled cases and outcome.

What should you measure?

• Shadow accuracy and reasons quality per caseworker.
• Reviews and appeals on agent-drafted decisions, and how many succeed.
• Approval time per case. Very short times can signal rubber-stamping.
• Safeguarding flags actioned within your local standard.

Where does it usually go wrong?

Caseload pressure turns review into a click. Managers must accept that meaningful review takes time, and plan caseloads for it.

Discretion disappears. Caseworkers start deferring to the draft and stop using discretion where it applies. Calibration should include cases where discretion was right.

Policy changes outrun training. A rule changes and the agent is updated, but caseworkers are not briefed. Tie every policy change to a short refresher.

Incident handling untested. Practise what happens when the agent drafts something badly wrong; see how to run an AI agent incident drill for the casework team.

Fraud teams use the same stages with sharper adversarial checks; see how to train the fraud team to review AI agent decisions. The reviewer onboarding checklist for the casework team and the AI agent competency matrix for the casework team give the detail, and the full programme is in training teams for AI agents.

Where the Academy fits

Pryme Intelligence Academy's Operator track teaches how a governed agent decides and how to review held work, with module tests set in the learner's own industry and role, so caseworkers are tested on casework. Lessons can be read or listened to, which helps caseworkers who spend much of the day on visits or calls. The practicum requires two weeks of real review decisions with no missed red line.