OSFI Guideline E-23 expects every model to move through a governed lifecycle: design, independent review, approval, deployment, monitoring and retirement. An AI agent fits that cycle if you validate what makes it behave as it does, its brief, sources, tools and limits as well as its model, on your own cases, and approve the agent with conditions that expire.
What does E-23 require across the lifecycle?
OSFI's Guideline E-23 sets expectations for model risk management at federally regulated financial institutions. In substance, it asks institutions to run an enterprise-wide framework that covers every model through its life, with the intensity of controls proportionate to each model's risk rating. The lifecycle runs:
- design and development: a clear purpose, suitable data, a documented method and its limitations;
- review: independent assessment of whether the model is sound and fit for its intended use;
- approval and deployment: a decision by an accountable person, with any conditions, and controlled implementation;
- monitoring: ongoing checks that performance stays within expectations, with defined triggers for action;
- decommissioning: orderly retirement, with records kept.
The guideline covers models built with AI and machine learning, and expects institutions to keep a model inventory and assign risk ratings. For whether agents are in scope at all, see how OSFI E-23 applies to AI agents.
Who does it apply to?
Federally regulated financial institutions in Canada: banks, federally regulated insurers, trust and loan companies and similar firms supervised by OSFI. Provincially regulated institutions answer to provincial regulators, who may take a similar approach. Our piece on E-23 model risk and agents in Canadian banks covers the framework side; this one covers validation.
What does validation mean when the model is an agent?
Validation of a traditional model checks its method, data and outputs. An agent has more parts that shape behaviour, and each needs a place in the lifecycle.
What is being validated. The language model is a vendor component. The agent's behaviour also depends on its brief, the sources it reads, the tools it can call and the limits on each. Change any one and behaviour changes. Validation should name the version of each that it covers.
On whose data. Vendor benchmarks show general capability. E-23's question is whether the model is fit for its intended use at your institution. That can only be answered on your own historical cases: your products, your customers' language, your edge cases.
At what grain. An agent that reads statements, drafts letters and updates contact details has three capabilities with different risk. Validate and rate each separately, so a weakness in one cannot hide behind the others.
With what conditions. Approval should state what the agent may do, at what level of authority, with what limits, and for how long. An approval without expiry assumes nothing will change, which is rarely true of agents.
How should each lifecycle stage handle an agent?
Take an illustrative case. A federally regulated insurer builds an agent to handle policy change requests. At design, the brief, sources and three capabilities are documented, with the risk of each. At review, the validation team replays several hundred past change requests and finds the agent handles address and payment-date changes well but misreads some beneficiary changes. At approval, the head of the business approves address and payment-date changes for execution within limits, and beneficiary changes for drafting only, both for a fixed period. Monitoring tracks overrides and complaints and re-runs a fixed case set weekly. When the model provider updates the model, the agent drops to drafting until it is re-validated.
| Requirement | Control | Evidence it leaves |
|---|---|---|
| Design and development | Agent brief, sources, tools and limits documented with intended use and known limitations | Versioned design record |
| Risk rating | Each capability rated by impact and reversibility | Rating in the model inventory |
| Independent review | Validation by a team outside the build, on the institution's own cases | Validation report with findings per capability |
| Approval | Named approver; conditions on scope, authority and duration | Signed approval with expiry |
| Deployment | Limits enforced before every action; change control | Configuration record; verdict log |
| Monitoring | Fixed case replay, override and complaint tracking, defined triggers | Monitoring reports; trigger log |
| Re-validation | Model, brief, source or tool change triggers review | Change log linked to re-validation |
| Decommissioning | Agent retired, access removed, records retained | Retirement record |
What will OSFI or internal audit ask?
- Which agents are in the model inventory, and how is each capability rated?
- What exactly was validated: which model version, brief, sources and tools?
- Were the test cases drawn from your own operation?
- What conditions did approval set, and when do they expire?
- What triggered the last re-validation, and what changed?
- How was the last agent retired, and where are its records?
What does E-23 not cover?
E-23 does not tell you how to test agents specifically or what thresholds to use. It does not address privacy, which falls under PIPEDA, or conduct towards customers, which falls to the FCAC and others. It does not replace operational and technology risk expectations. Comparing approaches helps: the UK's equivalent validation expectations are covered in PRA SS1/23 model validation for agents, and the basics in our glossary entry on model validation. For the full map, see our guide to AI agent regulation mapped to controls.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
Where Pryme fits
Attest is built for this lifecycle stage. It replays an agent on the institution's own cases across seven gates and sets the agent's depth, capped by the role's ceiling. Certificates lapse after 90 days, and a lapse, or a run that clears no depth, drops the agent to depth 1 the same day, which gives approval conditions an expiry and monitoring a consequence.
Questions people ask
Does every agent need full independent validation under E-23?
Intensity should match risk. A low-risk agent that only reads and summarises may need a lighter review than one that acts on customer accounts. Rate each capability and scale validation to the rating.
Can a vendor's validation report be relied on?
As input. It describes the vendor's testing, not fitness for your intended use. Your validation should still test the agent on your own cases with your configuration.
How often should an agent be re-validated?
On every material change and at a set interval. Model updates, new sources and new tools are material for agents, so re-validation is often event-driven.