Ongoing monitoring under SR 11-7 checks that a model is implemented as designed, used as intended and still performing. For an AI agent, that means continuous checks on its components, its scope and its outcomes, with thresholds set in advance. A crossed threshold must trigger action: restricted authority, revalidation or retirement, not a note in a report.
What does SR 11-7 require of ongoing monitoring?
SR 11-7 treats ongoing monitoring as one of the three core elements of validation. In substance, it should:
- confirm that the model is appropriately implemented and is being used and performing as intended;
- evaluate whether changes in products, exposures, activities, clients or market conditions mean the model needs adjustment, redevelopment or replacement;
- verify that any extension of the model beyond its original scope is valid;
- include process verification, benchmarking and analysis of overrides, since frequent overrides may signal that a model is not performing well.
Monitoring continues after implementation, at a frequency suited to the model's nature, risk and the availability of new data. Results should lead to action where problems appear. For the scope question, see is an AI agent a model under SR 11-7.
Who does it apply to?
Banking organisations supervised by the Federal Reserve and the OCC, and banks supervised by the FDIC under the same approach. For the short answer on whether it reaches agents, see does SR 11-7 apply to AI agents.
What should be monitored for an agent?
Agents need monitoring at a pace traditional models rarely do, because their components change on other people's schedules and their use can widen without anyone deciding it should.
Implementation. Is the running agent the validated one? Check model version, brief version, sources and connected tools daily against the validated design. A model provider's update or an added tool is a change in implementation.
Use within scope. Is the agent doing only what it was approved to do? Track attempted actions outside scope and how they were handled. A rise in refused actions suggests the agent is being asked for things beyond its design, or is reaching for them.
Extension beyond original scope. Has the agent been given new channels, products or customer groups? SR 11-7 asks that extensions be shown valid. For agents, that means testing before the agent handles the new segment.
Performance. Replay a fixed set of known cases on a schedule. Track grounding failures, escalation rates and action accuracy where outcomes are known.
Overrides. Track how often reviewers change what the agent drafted, and in which direction. A high rate signals poor performance. A rate that falls to zero may signal that reviewers have stopped reviewing.
Changes in the environment. New products, fee changes, policy changes and shifts in customer behaviour all change what the agent meets.
When must monitoring trigger action?
Decide in advance, per capability, what each signal does when it crosses its threshold. An illustrative action set:
- Implementation mismatch: authority drops to drafting until the change is reviewed and, if material, revalidated.
- Fixed-case replay below floor: authority drops to drafting; revalidation required.
- Override rate above threshold: owner investigates within a set time; authority reduced if cause not found.
- Override rate near zero over a period: review of reviewer behaviour; sample audit of approvals.
- Unapproved extension of scope: affected cases moved to people; extension tested before resuming.
- Outcome errors above threshold: restriction plus revalidation; retire if the cause cannot be fixed.
Take an illustrative case. A US bank's agent handles stop-payment requests on cheques. Daily implementation checks flag that the model provider has released a new version. The policy drops the agent from executing to drafting automatically. Fixed-case replay on the new version shows no change in performance, so the model risk team approves restoration the next morning. A week later, override rates climb for requests involving business accounts; investigation finds a new product with different terms, and the agent is restricted for that product until its sources are updated and retested.
How do you meet SR 11-7 monitoring for agents?
| Requirement | Control | Evidence it leaves |
|---|---|---|
| Implemented as designed | Daily check of production components against validated design | Implementation check log |
| Used as intended | Out-of-scope attempts tracked; limits enforced | Refused-action reports |
| Performing as intended | Scheduled fixed-case replay; action accuracy where outcomes known | Replay results; accuracy reports |
| Override analysis | Override rates tracked by capability and direction | Override reports |
| Environmental change | Product, policy and customer changes reviewed for impact | Change impact assessments |
| Extension of scope | New segments tested before the agent handles them | Extension test records |
| Pre-set thresholds and actions | Thresholds and responses defined per capability | Monitoring plan |
| Action on results | Restrictions, revalidation and retirement executed and recorded | Action log linked to signals |
Monitoring status and restrictions should be visible in the agent's SR 11-7 model inventory entry.
What will examiners ask?
- What do you monitor for each agent, and how often?
- How would you know if the model provider changed the model?
- What thresholds trigger action, and who set them?
- Show us a time monitoring led to a restriction.
- How do you detect reviewers who no longer challenge the agent?
What does SR 11-7 monitoring not cover?
It does not tell you what thresholds to set; the bank must justify its own. It does not replace business-line oversight of customer outcomes or compliance monitoring under consumer protection rules, though the signals overlap. For the full set of binding rules, see our guide to AI agent regulation mapped to controls.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
Where Pryme fits
Pryme ties monitoring to authority. Attest's stable gate checks that the agent gives the same answer to the same question across runs, certificates lapse after 90 days so behaviour is re-checked on recent cases, and a run that clears no depth drops the agent to depth 1 the same day, so the restriction is not left to a meeting. Held items and overrides are filed in the append-only record, giving the bank its override analysis without a separate log.
Questions people ask
How often should AI agents be monitored?
More often than most traditional models. Component checks can run daily at low cost; fixed-case replays weekly; outcomes analysis as outcomes arrive. The right cadence depends on how often the agent acts and what its actions can do.
Should authority fall automatically when a threshold is crossed?
For capabilities that change records or move money, yes. Automatic restriction removes the delay between detection and action. A person restores authority after review.
What is the most overlooked monitoring signal?
Reviewer behaviour. Firms track whether the agent is wrong, but not whether people have stopped checking it. Falling override rates with rising volumes deserve a look.