Health, biometric, genetic, racial or ethnic, political, religious, trade union, sex life and sexual orientation data form GDPR's special categories. An AI agent may process them only with a lawful basis plus one of a narrow set of extra conditions, such as explicit consent. Most agents should never see such data at all; the rest need tightly scoped access.
What does GDPR require for special category data?
Processing special category data is prohibited unless two things are true. You need a lawful basis under Article 6, as for any personal data. And you need one of the specific conditions GDPR provides for these categories. The conditions most relevant to businesses, in substance, are:
- the person's explicit consent;
- obligations in employment and social security law;
- protecting someone's vital interests where they cannot consent;
- data the person has manifestly made public;
- establishing, exercising or defending legal claims;
- substantial public interest on a legal basis, with safeguards;
- health or social care purposes under professional confidentiality.
Member state law can add detail, and in the UK the Data Protection Act 2018 sets out further conditions and a requirement for an appropriate policy document for some of them.
Special category data also raises the stakes elsewhere. Large-scale processing of it is a standard trigger for an Article 35 impact assessment. Automated decisions under Article 22 may only use it with explicit consent or substantial public interest, plus safeguards.
Who does it apply to?
Any controller whose agents receive, infer or store this data, and its processors. That reaches well beyond healthcare. An insurer's claims agent reads medical reports. A bank's support agent hears that a customer has cancer and needs a payment holiday. An HR agent sees sickness absence. In each case the agent is processing health data.
Why are agents a particular risk for special category data?
Three features of agents make this harder than for conventional systems.
Agents receive it unasked. Customers volunteer health and life details in free text. An agent built to handle billing will still hear about a diagnosis. You cannot stop people telling you; you can control what the agent does next.
Agents infer it. An agent that concludes a customer "may have a cognitive impairment" from how they write has created health data. Inferences about health or ethnicity are special category data even if no one stated them.
Agents spread it. If the agent writes that detail into a case note, a shared memory or a summary sent to another team, the data now sits in places it was never meant to be, often without the extra condition that justified its first use.
How do you restrict and control access?
The principle is that special category data should flow only to capabilities that have a documented condition for it, and nowhere else. Our guide to GDPR data minimisation for AI agents covers the general method; for these categories it needs to be stricter.
| Requirement | Control | Evidence it leaves |
|---|---|---|
| Condition for each use | Each capability that may touch special category data has its basis and condition recorded | Records of processing with conditions |
| Keep it from agents that lack a condition | Sources containing such data are not connected to those agents; fields are masked | Source permissions; masking rules |
| Handle volunteered data | Agent recognises disclosures, does not store them in general memory, and routes to a trained person | Routing records; memory exclusion rules |
| Control inferences | Agent is not permitted to record inferences about health, ethnicity or similar unless the capability requires it | Brief restrictions; output checks |
| Explicit consent where relied on | Consent captured in a separate, specific statement and recorded | Consent records |
| Decisions with significant effects | Decisions using such data held for a named person unless a valid exception applies | Hold records |
| Security | Encryption; access logging; narrower staff access | Access logs; security review |
| Impact assessment | DPIA before go-live, revisited on change | Signed DPIA |
The routing control matters most in practice. When a customer says "I've just come out of hospital", the agent's job is to acknowledge it, avoid repeating it into systems that do not need it, and pass the case to someone trained to support them. It should not decide what the disclosure means.
What will a supervisor or auditor ask?
- Which agents can see special category data, and what condition covers each?
- What happens when a customer discloses health information to an agent that has no business holding it?
- Can any agent infer sensitive characteristics, and where would those inferences be stored?
- Show us a recent disclosure and trace where the data went.
- If special category data leaked from an agent's memory or output, how would you know?
The last question links to breach handling; see GDPR data breaches caused by AI agents for what to have ready.
What do these rules not cover?
They do not cover criminal offence data, which has its own separate restrictions. They do not prevent an agent recognising that a disclosure has happened, which is necessary to route it properly. They do not settle sector rules: health data in the United States falls under HIPAA, and financial data under GLBA, covered in whether GLBA applies to AI agents handling customer data. For the full set, see AI agent regulation, mapped to controls.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
Where Pryme fits
In Pryme, an agent can only reach the sources named in its brief, and the minimum chain checks role scope and sources on every action. An agent without a documented reason to see health data never has it connected. Attest's scope and escalation gates test, on your own past cases, that the agent routes sensitive disclosures to a person rather than acting on them.
Questions people ask
Is a customer's mention of illness in a chat health data?
Yes. Information about a person's physical or mental health is health data however it arrives. The question is not whether you hold it but whether you have a condition to use it and whether you keep it only where needed.
Can an agent use explicit consent captured in a chat?
It can, if the consent is specific, informed, clearly given and recorded, and the person can refuse without detriment. A buried tick box or a vague "yes" in conversation is weak. For anything significant, capture consent through a dedicated step.
Are photographs biometric data?
Only when processed with specific technical means that allow unique identification, such as facial recognition. An agent reading a scanned photo ID to check details is not processing biometric data in that sense unless it matches faces.