PrymeIntelligence

Regulation and standards·Article

Model risk rules already cover AI agents

Model risk rules already cover AI agents. Supervisors' model risk expectations extend to agents that rely on models to produce outputs used in decisions.

Banks need not wait for agent-specific rules. Model risk guidance already reaches any system that uses a quantitative or AI method to produce outputs the business relies on, and an AI agent built on a model and used in decisions falls inside it. The work is adapting inventory, validation, monitoring and governance to a system that acts.

What do model risk frameworks require?

Model risk is supervised in most banking markets. In the United States, the interagency guidance known as SR 11-7 sets expectations for model development, validation and governance. In the UK, the PRA's SS1/23 sets model risk management principles for banks. In Canada, OSFI's Guideline E-23 covers model risk for federally regulated institutions. The wording differs; the core elements are much the same.

In substance, firms are expected to keep an inventory of their models, validate them independently before use and periodically after, monitor their performance, control changes, define the limits of their intended use, and govern all of this with clear roles and senior oversight. Our pages on SR 11-7 for AI agents and PRA SS1/23 for AI agents go into each framework.

Is an AI agent a model?

It contains one, and in the sense that matters to supervisors it behaves like one. Model risk frameworks generally define models broadly, by what they do: they apply statistical, mathematical or AI methods to input data to produce outputs that inform decisions. An agent reads inputs, applies an AI model and produces outputs, often decisions or actions themselves. Our explainer on whether an AI agent is a model under bank model risk rules works through the definitions.

The more useful framing is that an agent is a composite. It combines a model, which the bank usually does not build, with instructions, sources, tool connections and bounds, which the bank does control. Model risk arises from all of them together. A strong model given stale sources produces wrong outputs; a weak bound lets a reasonable output become a harmful action.

How do the core elements apply to agents?

Each element of model risk management still applies, but each needs adapting.

Element Traditional model AI agent
Inventory Model name, purpose, owner, tier Also: the agent's depth, each capability and the depth it needs, its sources, the underlying model and version
Validation Conceptual soundness, outcomes analysis, benchmarking Replay on the bank's own cases per capability; refusal, scope and grounding tests
Ongoing monitoring Performance against expectations, drift in inputs Also: verdict patterns, reviewer edits, attempts beyond bounds
Change control Re-validation on material changes Also: model version changes by the provider, source updates, new capabilities
Use limits Documented intended use Enforced bounds checked at the moment of action
Governance Owner, validator, committee, senior management Same, plus named approvers for held actions

Two adaptations matter most. Validation must be per capability, because one agent can be strong at one task and weak at another. And use limits must be enforced, not just documented, because an agent will act on whatever authority it holds.

What is hardest about validating an agent?

Three things. First, the bank cannot inspect the underlying model in the way it could inspect a regression. Validation shifts towards testing behaviour on representative cases, rather than reviewing the model's construction.

Second, behaviour depends on the sources the agent reads. A validation that passed against last quarter's policy library may not hold against this quarter's. Source changes need to be treated as potential material changes.

Third, the model can change under the bank's feet. Providers release new versions, and an agent's behaviour can shift as a result. Contracts and monitoring should give the bank notice, and the bank should re-test affected capabilities before relying on them. Treating certification as something that expires and must be renewed on fresh evidence fits these conditions better than a validation that stands until someone notices a problem.

How should an agent appear in the model inventory?

As one entry with its capabilities listed beneath it, or as one entry per material capability, depending on the bank's approach. Either way, the entry should record the underlying model and version, the sources the agent relies on, the agent's depth and the depth each capability needs, the owner, the validation status per capability, and the date the evidence expires. A model risk inventory entry template for an AI agent gives a format.

Tiering should reflect what the agent can do, not only what model it uses. An agent that only reads and drafts carries less model risk than the same agent executing payments.

Do AI governance frameworks outside banking say the same?

Broadly. Singapore's IMDA has published a Model AI Governance Framework, which, despite its name, is about governing AI generally rather than model risk in the banking sense. It still points towards similar practices: accountability, human involvement proportionate to risk, and ongoing monitoring. Our explainer on whether Singapore's Model AI Governance Framework covers AI agents looks at how it applies.

This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.

What is the trade-off?

Bringing agents into model risk management adds them to a process that is already stretched in many banks. Validators must learn to test behaviour rather than construction, monitoring needs new signals, and change control must cope with provider updates the bank does not control.

Leaving agents outside model risk management is not an option a supervisor is likely to accept. Add each agent to the inventory with its capabilities, validate per capability on your own cases, treat model and source changes as triggers for re-testing, and enforce use limits at the point of action. The wider mapping of rules to controls is in our guide to AI agent regulation, mapped to controls.

Questions people ask

Who should validate an AI agent?

An independent validation function, as for any model, with the skills to test behaviour on cases. The team that built or configured the agent should not validate it. Validators may need new tools and training for agent-specific tests.

Does a provider's model card count as validation?

No. It is useful documentation about the underlying model. It does not show how the agent behaves on the bank's cases, with the bank's sources and bounds, which is what validation must establish.

Are agents that only draft in scope of model risk management?

If their outputs inform decisions the bank relies on, generally yes, though the tier may be lower. Depth affects the level of risk and therefore the intensity of validation and monitoring.

Decide what your agents may do. Prove it. Enforce it.

Pryme Intelligence certifies each agent on your own cases, checks every action against five bounds at the moment it happens, and files every verdict.