Hospitals, clinics and health authorities that use high-risk AI systems are deployers under the EU AI Act. Article 26 requires them to assign oversight to people with the necessary competence, training and authority. In healthcare, that means clinicians and administrators who understand the agent's limits, can override it at once, and are supported in doing so.
What does Article 26 require of a healthcare deployer?
In substance, a deployer of a high-risk system must use it according to the provider's instructions; assign human oversight to natural persons with competence, training and authority, and give them support; ensure input data under its control is relevant and sufficiently representative; monitor operation, inform the provider of risks and suspend use if a serious risk appears; report serious incidents; keep logs under its control for an appropriate period; inform workers' representatives before using the system at work; and inform people subject to its decisions where required. Public bodies, and private bodies providing public services, may also need a fundamental rights impact assessment under Article 27.
The distinction between provider and deployer matters in healthcare, because many systems arrive inside products from manufacturers. That split is covered in who is the provider and who is the deployer of an AI agent.
Which healthcare agents are high-risk?
Healthcare meets the high-risk list in several ways:
- AI in regulated medical products. AI that is itself a regulated medical device, or a safety component of one, and must undergo third-party conformity assessment, is high-risk through the Act's link to product safety law.
- Eligibility for public healthcare. Public authorities evaluating whether people are eligible for healthcare services fall within the high-risk uses.
- Emergency triage. Systems that evaluate and classify emergency calls or prioritise patients in emergency care are high-risk.
- Workforce. Recruitment and management of clinical and support staff by AI fall within the high-risk workforce uses.
Many administrative agents, such as one drafting discharge letters for a clinician to sign or scheduling outpatient appointments, are not high-risk. They still process health data, which is special-category data under the GDPR, and good practice extends the same discipline to them.
What do competence, training and authority mean in a hospital?
Competence depends on the use. A triage nurse overseeing an agent that prioritises emergency calls must know what inputs the agent relies on, where it is weak, such as atypical presentations or language barriers, and when to override it. An eligibility officer must read the facts behind a recommendation, separating sealed records from the authority's own systems, signed confirmations from other bodies, declared information from the patient, and external documents that need checking. A ward manager overseeing a rostering agent must understand what it optimises and what it misses.
Training must be specific and current. An agent's certificate lapses after 90 days and is renewed against real cases; a run that clears no depth drops it to depth 1 the same day. Changes at renewal, such as new sources or a changed depth, must reach the people overseeing it before they rely on it. Health services already require clinical staff to keep their skills current; agent oversight belongs inside that habit, not beside it.
Authority is the most sensitive point in healthcare. A nurse who overrides the agent's triage priority, or a clinician who rejects its draft, must be able to do so immediately, without needing a senior's approval, and must never be blamed for following clinical judgement over the agent. Where the culture makes override feel risky, oversight exists only on paper.
Support means time, access to the record and clear escalation. A triage queue sized so that overseers can only accept the agent's order is not supported oversight.
How does a healthcare deployer meet Article 26's competence duty?
| Requirement | Control | Evidence it leaves |
|---|---|---|
| Oversight assigned to named people | Named overseers per high-risk capability and shift | Rotas linked to assignment register |
| Necessary competence | Scenario assessment on the service's own cases, including atypical ones | Assessment records per overseer |
| Training kept current | Briefing at each agent recertification or change | Briefing records linked to agent change log |
| Necessary authority | Written policy that overseers may override without approval | Policy; overrides in the record |
| Support | Queue sizes and staffing that allow real review | Queue metrics and review times |
| Monitor and suspend | Clinical safety lead can suspend the agent | Suspension events, incident reports |
| Inform workers' representatives | Consultation before go-live of workforce agents | Consultation records |
What counts as proof is set out in what evidence of competence is, and the deployer definition in what a deployer is under the EU AI Act.
What will a regulator or inspector ask?
Healthcare is inspected from several directions: market surveillance authorities under the Act, medical device authorities, health service inspectorates and data protection authorities. Expect: which agents are high-risk, and who oversees each? How were overseers trained and assessed? Show a case where staff overrode the agent and what happened next. Can a nurse override without approval? When did you last suspend the agent, and how did care continue? How are patients told an AI system was involved in a decision about them?
What does Article 26 not cover?
It does not regulate clinical practice, which professional and health law govern. It does not replace medical device rules for manufacturers. It does not apply to non-high-risk agents. It does not prescribe training content. And it does not replace GDPR duties on health data, including impact assessments under Article 35. Related deployer duties in other sectors are covered in Article 26 duties in payments and in insurance.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
Where the Academy fits
Pryme Intelligence Academy teaches people to run governed agents on the real product, with lessons that can be read or listened to as audio, which suits shift workers. Module tests are set in the learner's own industry and role, so an eligibility officer is tested on eligibility cases. The levels are explained in AI literacy and certification for people who work with agents.
Questions people ask
Do clinicians need Article 26 training for agents that draft letters?
Drafting agents are usually not high-risk, so Article 26 may not apply. Clinicians signing agent drafts still need to check them, and a short, practical module on what the agent does and where it errs is good practice.
Who in a hospital should be able to suspend an agent?
A named clinical safety lead, and for triage agents the senior clinician on shift. Suspension should be quick and backed by a manual process everyone has practised.
Does the hospital or the manufacturer train staff?
The manufacturer, as provider, must supply instructions for use. The hospital, as deployer, must assign competent, trained people. It can use the manufacturer's training but remains responsible for competence.