PrymeIntelligence

Regulation and standards·Regulation

Human-in-the-loop vs human oversight under the EU AI Act

Human-in-the-loop vs human oversight under the EU AI Act: why approving every action is not what Article 14 asks, and what effective oversight is.

Article 14 asks for effective human oversight, not a person approving every action. Oversight means people who understand the agent's limits, watch how it performs, can spot automation bias, can override or reverse its output, and can stop it. Approving everything often fails that test, because rubber-stamping is the opposite of oversight.

What does Article 14 require?

Article 14 applies to high-risk AI systems. In substance it requires that they be designed so natural persons can oversee them effectively while in use, with oversight aimed at preventing or minimising risks to health, safety and fundamental rights. Oversight measures must be proportionate to the risks, the level of autonomy and the context.

The people assigned to oversight must be enabled, as appropriate, to:

  • understand the system's capacities and limitations and monitor its operation, including detecting anomalies and unexpected performance;
  • remain aware of automation bias, the tendency to over-rely on the output;
  • correctly interpret the output;
  • decide not to use the system in a particular case, or disregard, override or reverse its output;
  • intervene in its operation or interrupt it through a stop procedure that brings it to a safe state.

Providers build the means of oversight into the system. Deployers, under Article 26, must assign oversight to people with the competence, training, authority and support to do it.

Who does it apply to?

Providers and deployers of high-risk AI systems under the EU AI Act. Agents that are not high-risk are outside Article 14, but GDPR's rules on solely automated decisions and sector conduct rules often demand similar safeguards. Our article on EU AI Act human oversight as a control sets out the control design in full.

Why is human-in-the-loop not the same as oversight?

Human-in-the-loop means a person makes or approves each decision, as defined in what is human-in-the-loop. Human-on-the-loop means a person monitors and can intervene, as in what is human-on-the-loop. Article 14 does not prescribe either. It prescribes outcomes: understanding, monitoring, interpretation, the power to override and the power to stop.

For agents, approval of every action has three problems.

Volume defeats attention. A reviewer asked to approve hundreds of agent drafts a day will approve almost all of them. That is automation bias in its purest form, and Article 14 names it as something oversight must counter.

Approval without context is not understanding. A reviewer who sees only the agent's final draft, not the facts it used or the limits it ran into, cannot interpret the output correctly.

It hides the real controls. If every action is approved, there is little pressure to define the agent's bounds, test its performance or build a stop procedure. Oversight becomes a person, not a system.

Approval of individual actions still has a place. It is right where an action is hard to undo or the agent has not yet earned the authority to act alone. The point is to use it deliberately, for specific actions, rather than as a blanket answer.

What does effective oversight of an agent look like?

Requirement Control Evidence it leaves
Understand capacities and limits Each overseer briefed on what the agent may do, its certified capabilities and known failure modes Training records; capability sheets
Monitor operation and anomalies Dashboards of actions, refusals, holds and outcomes, with thresholds that alert Monitoring reports; alerts
Counter automation bias Sample reviews where the overseer works the case before seeing the agent's answer; approval rates tracked Blind review records; approval statistics
Interpret output correctly Overseer sees the facts used, their sources and the limits applied, not only the draft Decision records with context
Override or reverse Overseer can change any agent output and reverse actions where possible Override logs
Not use the system Overseer can route a case away from the agent Routing records
Interrupt and stop Stop procedure per capability, tested Stop test results
Competent, authorised people Named overseers with authority and time to act Assignment records; training

Tracking approval rates is the single most useful measure. If reviewers approve nearly everything, either the agent is very reliable, and the review is wasted effort better spent on sampling, or the reviewers are not really reviewing. Both need action. Our human oversight procedure template for AI agents gives a ready structure.

What will a supervisor or auditor ask?

  • Who oversees each high-risk agent, and what training and authority do they have?
  • What do overseers see when they review a case?
  • How do you detect and counter automation bias?
  • How often do overseers override the agent, and what happens next?
  • Show us the stop procedure working.
  • Why did you choose per-action approval, or not, for each capability?

What does Article 14 not cover?

It does not require a human decision on every case. It does not dictate a staffing ratio. It does not replace GDPR's specific safeguards for solely automated decisions. The NIST AI RMF reaches similar conclusions from a different direction; see NIST AI RMF and human oversight of AI agents. For the full map, see AI agent regulation, mapped to controls.

This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.

Where Pryme fits

In Pryme, per-action approval is a verdict, not a default. An action is held, drafted and waiting for a named person, when the agent's depth or your limits require it. Depth is earned on evidence from Attest, so a capability moves from held to allowed only when it has proven itself on your own cases. Any capability can be stopped, and a run that clears no depth drops the agent to depth 1 the same day.

Questions people ask

Can oversight be done by sampling rather than reviewing each case?

Yes, where the risk and the agent's proven performance support it. Sampling should be designed, with enough cases and blind review to detect problems, and combined with thresholds that pull more cases into review when outcomes shift.

Does a stop button have to stop the whole agent?

It must allow intervention that brings the system to a safe state. For agents, stopping a single capability is often more useful than stopping everything, but you need both.

Who should oversee an agent: IT, compliance or the business?

The business team that owns the process, with support from risk and IT. Overseers need to understand the work well enough to know when the agent is wrong.

Decide what your agents may do. Prove it. Enforce it.

Pryme Intelligence certifies each agent on your own cases, checks every action against five bounds at the moment it happens, and files every verdict.