DORA requires a risk-based programme of digital operational resilience testing, and AI agents that support important services belong in it. Test them against outage of their model provider, manipulated inputs, attempts to exceed their authority, runaway actions, silent behaviour change and recovery after a stop, and include them in threat-led testing where it applies.
What does DORA require for resilience testing, in plain words?
DORA (Regulation (EU) 2022/2554), applying from 17 January 2025, requires financial entities to keep a sound and comprehensive testing programme as part of their ICT risk management. In substance:
- the programme is risk-based and proportionate to the entity's size and risk profile;
- it uses a range of tests, such as vulnerability assessments, network security assessments, scenario-based tests, performance tests, end-to-end tests, source code reviews where feasible and penetration tests;
- ICT systems and applications supporting critical or important functions are tested regularly;
- tests are carried out by independent parties, internal or external;
- findings are prioritised, remediated and tracked to closure;
- entities identified by their competent authorities also carry out advanced threat-led penetration testing on live production systems supporting critical functions, which can extend to ICT third-party providers.
For the regulation overall, see what DORA is, and for whether it reaches agents, does DORA apply to AI agents in financial services.
Who has to test agents?
Every financial entity within DORA's scope, applying the programme to its agents in proportion to their role. Agents supporting critical or important functions need the most attention. Entities selected for threat-led testing should make sure the scope covers agents in those functions and, where relevant, the platforms they run on.
Why does testing an agent need new scenarios?
Conventional resilience testing asks whether a system stays up, resists intrusion and recovers. Those questions still apply to agents. But an agent can also stay up, resist intrusion and still do the wrong thing, because its behaviour depends on what it reads and on a model that can change. Testing has to cover integrity and authority as well as availability.
Testing for resilience is also different from comparing variants. A/B testing tells you which version performs better on average. Resilience testing asks what happens under stress, attack and failure. Do not let one stand in for the other, and do not let a vendor's general test reports stand in for tests on your own configuration.
Which scenarios should you test agents against?
| Requirement | Control | Evidence it leaves |
|---|---|---|
| Availability under provider failure | Simulate model provider outage and slowdown; confirm fallback to staff within tolerance | Test report with time to fallback |
| Integrity under manipulated input | Inject instructions in documents, emails and web content the agent reads | Records showing refusals or held actions |
| Authority limits hold | Attempt to push the agent beyond its scope, depth and the entity's limits | Verdict records naming the bound that refused |
| Delegated access bounded | Have the agent act for a low-privilege user and attempt high-privilege actions | Refusal records |
| Scale and runaway behaviour | Trigger loops or bulk instructions; confirm rate limits and stop | Logs of throttling and stop events |
| Silent behaviour change | Re-run certification cases after a model update before reliance | Comparison of results across versions |
| Recovery and reconstruction | Stop the agent mid-batch; identify and resolve every affected case | Case list and resolution times |
| Human fallback capacity | Run a period with the agent at read-only and measure throughput | Staffing and service level results |
| Threat-led testing | Include agent manipulation in red-team scope where threat-led testing applies | Red-team findings and remediation |
| Independence and remediation | Testers independent of the agent's builders; findings tracked | Finding register and closure evidence |
The manipulated-input scenario matters most for agents that read external content. An emailed invoice with new bank details, or a web page with hidden instructions, should never be able to authorise a payment. The test proves the bound holds in practice.
The reconstruction scenario is the one most entities skip. Restarting an agent is easy; finding every case it touched while wrong is not, unless every action was recorded with its inputs and verdict.
What will a supervisor ask?
- Which agents are in your testing programme, and on what basis did you scope them?
- Show us the results of an injected-instruction test.
- What happened when you simulated a model provider outage?
- How long did it take to identify every case affected by a stopped agent?
- Were agents in scope for threat-led testing?
- Who tested them, and were they independent?
What does DORA testing not cover?
It does not certify an agent's accuracy for its business purpose; that is model validation and certification. It does not set pass marks for agent behaviour. It does not test fairness or legal compliance of outcomes. And it is an EU requirement: UK firms test against their own resilience rules, set out in UK operational resilience rules and AI agents, and the international baseline is in Basel Committee operational resilience principles and AI agents. For the full control map, see AI agent regulation mapped to controls.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
Where Pryme fits
Pryme checks every action against five bounds and files the verdict, so an injected-instruction or authority test leaves a record naming the bound that refused. Attest replays agents against the entity's own cases after any model change, and a run that clears no depth drops the agent to depth 1 the same day, which turns silent behaviour change into a tested, visible event.
Questions people ask
Is prompt injection testing part of DORA resilience testing?
DORA does not name it, but its testing duties cover threats to ICT systems supporting the entity's functions. For agents that read external content, injected instructions are one of the most relevant threats, so include them.
How often should agents be tested?
Systems supporting critical or important functions should be tested regularly, and agents should also be re-tested after any material change, such as a new model or new sources.
Can certification replace resilience testing?
No. Certification shows the agent performs its job on real cases. Resilience testing shows what happens under failure and attack. Both are needed.