Human oversight of an AI agent needs more than someone watching a dashboard. A human oversight procedure names the people who oversee each capability, what they review and when, the powers they hold (to approve, reject, override, hold and stop), the training they need, and the record their oversight leaves. The template and a worked sample follow.
What does oversight mean for an agent that acts?
For a model that produces predictions, oversight often means a person reviewing outputs. For an agent that acts, it means people can understand what the agent is doing, intervene before an action takes effect where it matters, override or reverse what it did, and stop it. The general idea is set out in what human oversight is.
Article 14 of the EU AI Act requires high-risk systems to be designed so that people can oversee them effectively: understand their capabilities and limits, stay alert to over-reliance, interpret outputs, decide not to use or to override an output, and interrupt the system. Article 26 then requires deployers to assign that oversight to people with the competence, training and authority to do it. This procedure is how a deployer shows both. More in EU AI Act human oversight, as a control.
When should you use it?
Before any agent with depth above read goes live, and for every high-risk use regardless of depth. Review it when an agent's depth changes, because oversight should change with it: an agent at depth 2 is overseen through every draft; an agent at depth 3 is overseen through sampling, alerts and the power to stop.
Not every action needs a person in the loop. Human-in-the-loop vs human oversight under the EU AI Act explains the difference: a person approving each action is one form of oversight, not the only one.
How do you fill in each section?
- Oversight mode per capability is one of: approve each item, sample after action, monitor by alert.
- Overseer is a named role with deputies, never a mailbox.
- Powers list what the overseer can do without asking anyone. Stopping the agent must be one of them; the steps sit in the AI agent suspension procedure. Reversing an action the agent took sits in the undo procedure for agent actions.
- What they see is the information the overseer has: the agent's inputs and sources, not only its conclusion.
- Automation bias measures are steps that keep overseers thinking: recording a reason when agreeing with a high-impact output, periodic blind reviews.
- Training covers the agent's limits and known failure modes.
- Evidence is the record that oversight happened.
The template
| No. | Section | What to write |
|---|---|---|
| 1 | Agent and capabilities | From the inventory, with depth |
| 2 | Oversight mode per capability | Approve each, sample, monitor |
| 3 | Overseers | Named roles and deputies |
| 4 | Powers | Approve, reject, override, hold, reverse, stop |
| 5 | What overseers see | Inputs, sources, provenance, reasoning |
| 6 | Review frequency and volume | Per mode |
| 7 | Automation bias measures | Specific steps |
| 8 | Training and competence | Content, frequency, records |
| 9 | Escalation | When overseers escalate and to whom |
| 10 | Evidence | Records left by each activity |
| 11 | Review of the procedure | Frequency and triggers |
A worked sample
A hypothetical health insurer's claims agent, three capabilities.
| Capability | Depth | Mode | Overseer | Powers | Evidence |
|---|---|---|---|---|---|
| Check claim against policy terms | 3 | Sample 25 a day; alert on unusual patterns | Claims quality lead | Override check result; stop capability | Sample records; alert log |
| Draft decline letter | 2 | Approve each | Senior claims handler | Reject, edit, approve | Approval record per letter |
| Pay claims under 200 | 3 | Sample 10 a day; all first claims held | Claims payments supervisor | Hold, reverse before settlement, stop capability | Sample records; hold records |
Automation bias measure: handlers approving a decline must select which policy term applies, rather than accepting the agent's choice, and a monthly blind sample compares their decisions with an independent reviewer's.
What overseers see: the policy terms the agent relied on, the claim documents, and where each fact came from.
Health data is special-category data under GDPR, which is why decline letters stay at depth 2 here.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
Where Pryme fits
Pryme makes several powers structural: held work goes to a named person, any capability can be dropped to read-only at once, and every verdict is filed with the bound that decided it and the provenance of its facts. See AI agent regulation, mapped to controls.
Questions people ask
Does every agent action need a person to approve it?
No. Approval of each item suits high-impact actions. For routine work at depth 3, oversight through sampling, alerts and the power to stop is usually more effective than approving hundreds of items a day.
How do we show oversight is effective, not just present?
Measure it: how often overseers reject or edit, how quickly they act on alerts, and whether blind reviews agree with them. An overseer who never rejects anything may not be overseeing.
Who should approve the procedure?
The agent's accountable owner, with second-line review. For high-risk uses, compliance should confirm it meets the deployer's duties.