Caseworkers review an agent's decisions case by case and in patterns. Case by case, they read the record, find the deciding bound, check the evidence and its source, and decide with a reason the person could understand. In patterns, they look for unfairness no single case would reveal, a duty public bodies carry.
Why does casework review need two levels?
Single cases fail in visible ways: a wrong fact, a missing document, a vulnerable person treated like anyone else. A careful caseworker catches those in the queue.
Pattern failures are quieter. An agent may apply a rule correctly every time and still produce outcomes that fall harder on one group, for example people whose income records come from a source the agent reads less reliably. No single decision looks wrong. Public bodies face additional transparency and fairness duties, and under the EU AI Act some public uses are high-risk, so pattern review is part of the job.
What are the steps for a single case?
- Read the verdict. Held items wait for you; allowed items are reviewed by sample; refused items need a person to pick up the request another way.
- Find the deciding bound. Was the action held because the authority's own limits always hold refusals, or because a source was missing? The answer tells you whether you are deciding the case or completing the evidence.
- Check each fact's provenance. Records from other public systems are sealed or signed. The applicant's statements are declared. A third party's letter may be signed or external.
- Check vulnerability and context. Notes, prior contact, language needs, and anything outside the agent's sources.
- Check the reason. Would it make sense to the person? Does it name the rule and the fact?
- Decide. Approve, edit or decline within your delegated authority.
- Record why. A sentence a colleague, a complaints officer or an ombudsman could follow.
What should be reviewed in patterns?
| Pattern | How to see it | What it might mean |
|---|---|---|
| Refusal rates by area or group | Compare across the caseload weekly | A source or rule bearing unevenly |
| Holds overturned by caseworkers | Count edits and declines by reason | The agent's drafts drift from policy |
| Cases closed for non-response | Check contact attempts and channels | People missing messages the agent sent |
| Caseworker approval speed | Time per decision | Skimming under workload |
| Differences between caseworkers | Approval rates on similar cases | Inconsistent judgement, or unclear policy |
A team that meets weekly to go through these, as described in how to run a weekly review of AI agent decisions with a team, catches drift early.
What may the agent do, and what stays with the caseworker?
The agent may gather evidence, check completeness, calculate amounts under clear rules, and draft letters and assessments. The caseworker decides entitlements, refusals, reductions, closures and referrals; handles any safeguarding concern; and speaks to people when something is unclear. Some authorities let agents complete routine changes, such as recording a new address from a verified source, at depth 3, with sampling.
An example: a pattern in the refusals
Take a regional benefits team running a weekly review. Individual refusals all look correct. But the pattern table shows refusals for one district running well above the others. The caseworkers pull ten cases and find that applicants from that district often submit income evidence as photographs, which the agent treats as declared rather than signed, and refuses when the declared figure is unclear.
Nothing in any one case was wrong by the rule. The team edits the pending refusals to request better evidence instead, and asks the agent's owner to route photographed income evidence to a person. The record shows the decisions changed and why.
Which controls make review dependable?
- Caseworkers have authority to decline, not just access to the queue.
- Samples of allowed decisions are random and recorded.
- Pattern review happens on a fixed rhythm, with minutes.
- Approval rights depend on current training.
Other roles review in their own ways; compare how IT administrators review an AI agent's decisions and how underwriters review an AI agent's decisions. Market detail is in AI agent training for caseworkers in public sector in Japan. The full role map is in AI agent skills by role.
Where the Academy fits
The Operator track in Pryme Intelligence Academy includes modules on human review, reading the workspace and running the rhythm, which cover both single-case review and the weekly pattern work. Tests are set in the learner's own industry and role. Certified Pryme Operator needs a practicum of real review decisions, signed off by a Pryme assessor, and is valid for 12 months.
Questions people ask
Who should lead the pattern review?
A senior caseworker or team manager who did not make the decisions under review, with access to the full record.
What if a pattern suggests the policy is unfair?
Decide individual cases fairly within your authority and escalate the policy, with the pattern as evidence. Changing the agent cannot fix a rule that is itself the problem.
How often should allowed decisions be sampled?
Weekly at first, and after any change to the agent, its sources or its limits. Reduce the rate only when samples have been clean for a sustained period.