Validate an AI agent under SR 11-7 on the guidance's three legs. Conceptual soundness reviews the brief, sources, tools and limits against policy. Ongoing monitoring verifies the running agent matches what was validated and benchmarks it against people. Outcomes analysis checks its actions against what later proved correct: disputes upheld, refunds reversed, complaints made.
What does SR 11-7 require of validation?
The interagency model risk guidance SR 11-7 defines validation as the set of processes and activities that verify models are performing as expected, in line with their design objectives and business uses. It names three core elements:
- Evaluation of conceptual soundness: assessing the quality of the model's design and construction, its documentation, assumptions, data and methods, and the developmental evidence behind them.
- Ongoing monitoring: confirming the model is appropriately implemented and used, and performing as intended, through process verification and benchmarking against alternatives.
- Outcomes analysis: comparing model outputs with actual outcomes, for example through back-testing.
Validation should be independent of development, carried out by people with the right competence and influence, scaled to the model's risk, and repeated periodically. Where issues are found, the bank should fix them or restrict use, and may use a model with known limits only with compensating controls. For scope, see is an AI agent a model under SR 11-7; for the general term, what model validation is.
Who does it apply to?
Banking organisations supervised by the Federal Reserve and the OCC, and those supervised by the FDIC under the same approach. For a short answer on scope, see does SR 11-7 apply to AI agents.
What does each leg mean for an agent?
Conceptual soundness
The language model cannot be opened up, so conceptual review focuses on what the bank controls. Does the brief describe the task as the bank's policy does? Are the criteria it applies the right ones? Are the sources authoritative and current? Are the tools the minimum the task needs, and are the limits set where the risk says they should be? Is the chosen model suitable for this kind of task, based on the bank's own testing?
Developmental evidence for an agent is its test record: how it performed on the bank's own past cases, per capability, on grounding, refusals, escalation, scope and action accuracy.
Ongoing monitoring
Process verification for agents asks whether the agent running in production is the agent that was validated. Is the model version the same? The brief? The sources? Are the limits implemented as designed, so that an action above the threshold is actually held? Are the inputs reaching the agent the ones it should see?
Benchmarking compares the agent with an alternative. The natural benchmark is the bank's own staff: run the agent in drafting mode on cases people are handling, and compare.
Outcomes analysis
Agents act, and many actions have later outcomes that show whether they were right. A dispute the agent resolved may be reopened. A fee waiver may draw a complaint. A document request may turn out unnecessary. Tracking these against the agent's actions is back-testing for agents.
Take an illustrative case. A US bank validates an agent that resolves ATM cash dispute claims under a set amount. Conceptual review finds the brief omits a policy rule on claims filed after a set number of days; the brief is fixed. Process verification finds the production agent connected to an extra tool not in the validated design; it is removed. Benchmarking against analysts shows comparable accuracy. Six months of outcomes analysis shows a cluster of reopened claims at one merchant type, which leads to a restriction on those claims until the agent is retested.
How do you meet SR 11-7 validation for agents?
| Requirement | Control | Evidence it leaves |
|---|---|---|
| Conceptual soundness | Review of brief, criteria, sources, tools and limits against policy | Conceptual review findings |
| Developmental evidence | Per-capability testing on the bank's own cases | Test reports |
| Process verification | Production components and limits checked against validated design | Verification checklist and results |
| Benchmarking | Agent compared with staff on the same cases | Benchmark report |
| Outcomes analysis | Agent actions tracked against later outcomes | Outcomes reports |
| Independence and competence | Validators separate from builders, with AI and business skill | Validator profiles; reporting lines |
| Issues and restrictions | Findings fixed or use restricted with compensating controls | Findings log; restriction records |
| Periodic revalidation | Scheduled and change-triggered revalidation | Revalidation plan; inventory status |
Validation results should flow into the agent's record in the SR 11-7 model inventory.
What will examiners ask?
- How did you assess conceptual soundness when the core model is a vendor's?
- How do you know the agent in production is the one you validated?
- What did you benchmark the agent against?
- What outcomes analysis do you run on agent actions?
- What restrictions came out of validation, and are they enforced?
What does SR 11-7 validation not cover?
It does not set specific tests or pass marks for agents; the bank decides and justifies them. It does not replace monitoring by the business or compliance testing of consumer outcomes. It does not approve an agent's authority; governance does that on the strength of validation. For the binding rules beyond model risk, see our guide to AI agent regulation mapped to controls.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
Where Pryme fits
Attest supplies the developmental evidence: replays on the bank's own cases scored against floors across seven gates. Its stable gate checks that the agent gives the same answer to the same question across runs, certificates lapse after 90 days so the evidence is renewed on recent cases, and the platform enforces the validated limits on every action, so process verification and outcomes analysis start from a record of what the agent was allowed to do and did.
Questions people ask
Can a bank validate an agent without seeing inside the language model?
Yes. SR 11-7 already accepts that vendor models may be partly opaque and asks banks to compensate with testing, monitoring and outcomes analysis. For agents, testing on the bank's own cases does most of that work.
What is a sensible benchmark for an agent?
The bank's own staff doing the same task. Comparing the agent's drafts with decisions people actually made gives a direct, relevant benchmark.
How soon after deployment should outcomes analysis start?
As soon as outcomes accumulate. For processes with quick feedback, such as disputes or refunds, weeks; for slower ones, set interim checks and extend the analysis as data arrives.