Validating an AI agent under SS1/23 means an independent team testing whether each capability is sound and fit for use, on the bank's own cases, with the exact versions of model, brief, sources and tools recorded. The validation file should end in a decision: what the agent may do, at what authority, with which conditions, until when.
What does SS1/23 require of validation?
SS1/23's fourth principle requires independent model validation. In substance:
- validation is carried out by people independent of model development, with the competence and authority to challenge;
- its depth is proportionate to the model's risk tier;
- it covers the soundness of the method, the quality of the data, the model's performance against its intended use, and its limitations;
- findings are recorded, rated and acted on, with escalation where they are not;
- validation is repeated periodically and when models change materially, with performance in use reviewed between validations.
The fifth principle then requires mitigants where validation finds weaknesses. For the framework as a whole, see PRA SS1/23 model risk management for agents, and for the general idea, our glossary entry on model validation.
Who does it apply to?
Primarily UK banks and building societies with permission to use internal models for regulatory capital, with other firms encouraged to apply the principles. For whether agents fall within scope at all, see does PRA SS1/23 cover AI agents.
What changes when the model being validated is an agent?
Traditional validation reviews a method the bank can read, data the bank can trace and outputs the bank can back-test. An agent's core method, the language model, cannot be fully inspected, and its outputs include actions. Validation shifts its weight accordingly.
From method review to behaviour testing. You cannot review the language model's internals, so you test its behaviour in the agent's actual configuration, on cases that reflect real use. Method review still applies to what the bank controls: the brief's criteria, the choice of sources and the design of limits.
From one output to several properties. Validate whether answers are grounded in sources, whether the agent refuses and escalates correctly, whether it stays inside its scope, and whether its actions are correct. A good average across all of these can hide a weak one.
From the model to its controls. An agent is only as safe as the limits that stop it. Validation should verify that limits work: try to make the agent exceed an amount, act outside its tools, or act on an untrusted document, and confirm the attempts are blocked.
From a date to an expiry. Agents change through model updates, brief edits and new sources. Validation should name what it covered and say when its conclusion stops holding. A validation report without an expiry quietly becomes a statement about an agent that no longer exists.
From validators who know models to validators who know the work. Judging whether an agent handled a mandate change or a hardship disclosure correctly needs people who understand the business process as well as the technology. Independence matters, but so does competence in the activity being automated.
What should the validation file hold?
Take an illustrative case. A UK bank validates an agent that handles business account address and signatory change requests. The file records the language model version, brief version, sources and tools. Conceptual review confirms the brief's criteria match the bank's mandate policy. Testing on several hundred past requests shows strong results on address changes and a weakness on signatory changes where the mandate is ambiguous. Control verification confirms signatory changes cannot execute without a person. The conclusion: address changes may execute within limits; signatory changes are approved for drafting only; validation expires in a set period or on any component change.
| Requirement | Control | Evidence it leaves |
|---|---|---|
| Independence | Validators outside the build team with authority to challenge | Independence statement; reporting line |
| Scope | Versions of model, brief, sources and tools validated | Scope section of the file |
| Conceptual soundness | Review of brief criteria against policy and of source selection | Conceptual review findings |
| Data quality | Sources checked for accuracy, currency and origin | Source review |
| Performance | Testing per capability on the bank's own cases, with floors per metric | Test results by capability and metric |
| Limitations | Known weaknesses stated plainly | Limitations section |
| Control verification | Attempts to exceed limits, act out of scope, or act on untrusted inputs | Verification results |
| Findings | Rated findings with owners and deadlines | Findings log |
| Conclusion | Authority level per capability, conditions and expiry | Signed validation conclusion |
| Re-validation triggers | Component changes and expiry date | Trigger list; inventory flags |
The conclusion and status flow into the agent's entry in the SS1/23 model inventory.
What will a PRA supervisor or internal audit ask?
- Who validated this agent, and how are they independent of its builders?
- What exactly was validated: which model version, brief and sources?
- Where did the test cases come from?
- How did you verify that the agent's limits work?
- What did validation conclude about each capability, and what conditions apply?
- What happened after the last model provider update?
What does validation under SS1/23 not cover?
It does not replace monitoring between validations, governance or mitigants; it informs them. It does not assess conduct outcomes under the FCA's rules or data protection under UK GDPR, though validation evidence can support both. For a comparable approach outside the UK, see how OSFI E-23 applies to AI agents. For the full map, see our guide to AI agent regulation mapped to controls.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
Questions people ask
Can validators use the vendor's test results?
As input to their review, yes. But the vendor's tests used the vendor's cases and configuration. Independent validation should test the agent as the bank runs it, on the bank's own cases.
How large should an agent validation test set be?
Large enough to cover each capability's common cases and its rare, high-impact ones with confidence. For capabilities that move money or affect vulnerable customers, emphasise depth on hard cases over breadth on easy ones.
Does every brief change require full re-validation?
Not every one. Changes to criteria, scope, sources or tools should trigger re-validation of the affected capabilities. Cosmetic wording changes can follow a lighter review, recorded as such.