Continuous is the key word in Article 9 of the EU AI Act: every high-risk AI system needs a risk management system run as an iterative process across its whole lifecycle. For an AI agent that acts, that means identifying risks per capability, testing against set thresholds, and feeding what happens in production back into the agent's bounds.
What does Article 9 require?
The provider of a high-risk system must establish, implement, document and maintain a risk management system. It is not a one-off assessment. The Act describes a cycle, regularly reviewed and updated, with these steps in substance:
- Identify and analyse the known and reasonably foreseeable risks the system can pose to health, safety or fundamental rights when used for its intended purpose.
- Estimate and evaluate the risks that may arise when it is used as intended and under reasonably foreseeable misuse.
- Evaluate other risks that emerge from data gathered in post-market monitoring.
- Adopt targeted measures to address the risks found.
The measures must bring the remaining risk to an acceptable level, judged per hazard and overall. The order of preference is set: eliminate or reduce risk through design first, then mitigate and control what cannot be eliminated, then inform and, where appropriate, train deployers.
Testing is part of the system. High-risk systems must be tested against metrics and probabilistic thresholds defined in advance and suited to the intended purpose, before release and as needed throughout development. The provider must also consider whether the system is likely to have an adverse impact on people under 18 and other vulnerable groups.
Who must run it?
Article 9 is a provider duty for high-risk systems. You are the provider if you develop the agent, or have it developed, and put it into service under your own name, or if you modify a bought system enough to become its provider. Whether an agent is high-risk in the first place is the earlier question, set out in EU AI Act risk tiers explained for AI agents and worked through in how to classify an AI agent under the EU AI Act.
Deployers feed the loop. What they see in production is the main source of the post-market data Article 9 expects the provider to use, and their own limits and oversight are part of the controls that keep residual risk acceptable.
What does continuous risk management look like for an agent that acts?
A credit model's risk sits in one output. An agent's risk sits in its capabilities, and the same agent may hold several with very different stakes: read a balance, draft a letter, freeze a card, send a payment. Risk management that rates the agent as a whole will either over-control the harmless capabilities or under-control the dangerous one.
We think an agent's risk system should be organised around four questions per capability:
- What is the worst plausible action? Not the worst answer. For a payment capability, it is the wrong payee at the maximum amount.
- How reversible is it? An internal note can be corrected. A settled payment or a sent adverse decision often cannot.
- What could make it happen? A wrong source, a manipulated document, an ambiguous instruction, a failed tool returning stale data.
- What stops it? A limit enforced outside the model, a person who must approve, a refusal rule, or nothing.
Foreseeable misuse also looks different. For an agent, it includes staff asking it to do jobs outside its brief, customers trying to talk it into exceptions, and outsiders planting instructions in content it reads. Each is foreseeable, so each belongs in the analysis.
Continuous means the loop runs on evidence. Every refused, held and overridden action is data about risk. An agent whose actions are increasingly held, or whose approvers increasingly overrule it, is telling you its risk estimate is out of date.
How do you meet Article 9 for an agent?
| Requirement | Control | Evidence it leaves |
|---|---|---|
| Identify foreseeable risks | Risk register per capability, with worst plausible action and reversibility | Register, versioned and signed by an owner |
| Consider foreseeable misuse | Misuse scenarios tested: out-of-brief requests, social pressure, injected instructions | Test cases and results |
| Design out risk first | Irreversible actions removed from capabilities where not needed | Capability list showing what was removed and why |
| Mitigate what remains | Limits by amount and counterparty; human approval before hard-to-undo actions | Bound settings and approval records |
| Test against preset thresholds | Each capability must clear a defined floor on cases from your own operation | Test runs with pass and fail per threshold |
| Use post-market data | Monthly review of refusals, holds, overrides and incidents | Review minutes and changes made |
| Consider vulnerable groups | Tests with cases involving vulnerable customers and minors where relevant | Results and resulting controls |
| Inform deployers | Residual risks stated in the instructions for use | Instructions section on residual risk |
Data quality is a frequent root cause, which is why Article 9 and Article 10 on data governance work best as one review. Firms already using the NIST AI Risk Management Framework will find its Map, Measure and Manage functions line up well with this cycle.
What will a supervisor or auditor ask?
- Show the risk register for this agent. Which capability carries the most risk, and why?
- What misuse did you test for?
- What thresholds must each capability meet, and who set them?
- What did production data change in the risk assessment last quarter?
- What residual risk did you accept, and who signed it off?
- What would trigger you to remove a capability?
What does Article 9 not cover?
It does not set the thresholds for you; it requires that you set them in advance and justify them. It does not replace sector risk frameworks such as model risk management in banking, which supervisors will expect to see integrated, not duplicated. It is not a fundamental rights impact assessment, which some deployers must do separately. And it does not apply as a legal duty to agents outside the high-risk tier, though any agent that can move money deserves the same discipline. For the wider map, see AI agent regulation, mapped to controls.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
Where Pryme fits
In Pryme, each capability has a depth, from read to moving money, and depth is earned through Attest rather than configured. Every action meets five bounds at once and the lowest wins; the verdict names which bound decided. That record of refusals, holds and overrides is the production evidence Article 9's loop needs.
Questions people ask
Is a risk register enough to meet Article 9?
No. A register is one artefact. Article 9 requires a process that runs across the lifecycle, including preset testing thresholds, use of production data and documented decisions about residual risk.
How often should an agent's risk assessment be reviewed?
On every material change to model, brief, sources or capabilities, and on a regular cycle in between. The Act requires regular systematic review without fixing a frequency.
Can deployers rely on the provider's Article 9 work?
For the provider's design, yes. But your configuration, sources and limits create risks the provider never assessed. Your own risk process under sector rules should cover them.