Through evidence, not assurances. No regulator has a single audit standard for agents, but supervisors in banking, insurance and data protection converge on the same asks: know which agents you run, document what each may do, test them independently, keep records that reconstruct every decision, and show that named people oversee them. An auditor tests whether that evidence holds up.
What do regulators look for when they audit an agent?
The expectations come from several rulebooks, but they overlap:
- An inventory. Model risk guidance such as SR 11-7 and PRA SS1/23 expects a complete inventory. An agent that is missing from it is the first finding.
- Documented purpose and limits. What the agent is for, what it may touch, and what it may not do.
- Independent testing. Model risk rules expect validation by people independent of the build. The EU AI Act requires risk management and testing for high-risk systems under Article 9.
- Records that reconstruct decisions. The EU AI Act's Article 12 requires automatic logs for high-risk systems, and Article 26 asks deployers to keep the logs under their control. Data protection rules require records of processing under GDPR Article 30.
- Human oversight that works. Article 14 for high-risk systems; elsewhere, governance and named accountability, such as under SM&CR in the UK.
- Third-party oversight of the platform and model provider, under DORA in the EU and outsourcing rules elsewhere.
Insurance supervisors take a similar line; see NAIC and state insurance regulators on AI agents.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
What is harder to audit about an agent than a model?
A model's audit samples outputs and checks them. An agent's audit has to follow actions: what the agent read, which tool it called, what it changed, and whether that was within its permissions at that moment. Three gaps come up repeatedly.
Logs that record outputs but not reasons. A record that the agent "closed alert 4471" says nothing about why. Auditors want the evidence the agent relied on, linked to the action.
Permissions that changed over time. If an agent's limits were widened last month, the auditor must be able to see which limits applied to each past action.
Oversight that is a click. A record that a person approved means little if the approval took two seconds. Auditors increasingly test whether reviewers actually changed outcomes.
What may the agent do, and what stays with a person?
Agents can help prepare for audit: assembling evidence packs, matching log entries to cases, and flagging gaps. They must not audit themselves.
- Agent may: gather records, draft control descriptions, produce samples on request.
- Stays with a person: selecting the audit sample, assessing whether controls work, forming the audit opinion, and agreeing remediation.
Independence is the reason. An auditor relying on the agent's own account of its behaviour has no independent evidence.
An illustrative example
Take a regional bank whose internal audit team reviews an agent that handles card disputes. The auditors request the agent's inventory entry, its brief, its approved limits and its test results. Then they pick 30 disputes at random and ask the bank to show, for each, what the agent read, what it decided, which limit applied, and who approved any refund above that limit.
For all but three, the bank produces the full trail in minutes. For those three, the trail ends at "refund issued", because those disputes went through an older integration that did not log the agent's reasoning. The finding is not that the agent was wrong; it is that the bank could not show whether it was right. The bank routes all disputes through the logged path and the auditors retest.
Which controls make an agent auditable?
| What the auditor asks | Control | Evidence |
|---|---|---|
| Which agents exist? | Agent inventory with owners | Inventory record |
| What may each one do? | Versioned limits per capability | Limit history |
| Was it tested? | Independent testing on real cases | Test reports |
| Why did it do this? | Decision record with sources and the limit that applied | Per-action log |
| Did a person check? | Holds with named approvers | Approval records and override rates |
Certification audits under ISO/IEC 42001 follow a similar logic; see what to expect from an ISO/IEC 42001 certification audit for AI agents. Sector-specific views are in whether regulators accept AI agents in AML work and whether banking regulators allow AI agents to act for customers. The full map is in AI agent regulation, mapped to controls.
Where Pryme fits
Every verdict in Pryme, whether allowed, held, refused or withheld, is filed in an append-only record that names the bound that decided it and the certificate in force at the time. An auditor can pick any action and see what applied.
Questions people ask
Do regulators require an external audit of AI agents?
Generally not as a separate requirement. Agents fall under existing internal audit, model validation and supervisory review. Some firms choose external certification, such as ISO/IEC 42001, for assurance.
How long should agent decision logs be kept?
As long as the longest retention period that applies to the underlying activity, such as credit, AML or complaint records, and at least as long as any specific AI rule requires.
Can internal audit use agents to audit other agents?
To gather and organise evidence, yes. The judgement on whether controls work must stay with the auditors, so their conclusions rest on independent review.