Give the agent the evidence work and keep the judgement. It gathers incidents, breaches, complaints, audit findings and regulatory changes for each business area, drafts inherent risk ratings with the reasons shown, and maps existing controls at depths 1 and 2. The head of compliance sets every rating, and the risk committee approves the assessment.
What is a compliance risk assessment, and who owns it?
A compliance risk assessment asks, area by area, how exposed the firm is to breaking the rules that apply to it, and how well its controls reduce that exposure. It produces an inherent rating, a view of control strength, a residual rating, and from those a compliance monitoring plan for the next period. It is the document that decides where second-line attention goes.
The head of compliance owns it. Business heads own the risks in their areas and answer for the controls. The risk committee or board approves the result. The assessment is the practical form of the risk-based approach: effort follows exposure, and the reasoning has to be written down.
The agent belongs on the preparation side of that line. Our position on AI compliance agents as second line, never the last word applies directly: it can assemble the case for a rating, but a rating is a judgement someone must stand behind.
What are the steps, and who acts at each one?
- Person, head of compliance: sets the scope and method. Business areas, rule sets, the rating scale and the definitions of each level. The agent works inside these; it does not invent a scale.
- Agent, depth 1: gathers the evidence per area. Breaches, incidents, complaints themes, audit and monitoring findings, regulatory correspondence, open remediation actions, and regulatory changes since the last cycle. Each item is stamped with its source.
- Agent, depth 1: compares with last cycle. Last period's ratings, what was expected to change, and what did. A rating that moves without new evidence, or evidence that arrives without a rating move, is flagged.
- Agent, depth 2: drafts inherent risk ratings. For each area, a proposed rating against the defined scale, with the evidence that supports it and the evidence that cuts the other way.
- Agent, depth 2: maps controls. Which controls address which risks, when each was last tested and with what result. Controls with no test evidence are marked "untested", not "effective".
- Person, business area heads: challenge the draft. They correct facts and add context the records miss. Their changes are tracked.
- Person, head of compliance: sets inherent, control and residual ratings. The agent's drafts are inputs. Where the final rating differs from the draft, the reason is written.
- Agent, depth 2: drafts the monitoring plan and committee paper from the approved ratings.
- Person, risk committee: approves.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Scope | Nothing | none | Method, scale, areas |
| Evidence | Gathers and stamps sources | 1 | Confirming completeness |
| Compare | Flags unexplained movement | 1 | Explaining it |
| Inherent draft | Proposes ratings with reasons | 2 | Setting the rating |
| Controls | Maps controls and test status | 2 | Judging effectiveness |
| Challenge | Records changes | 1 | Business heads' corrections |
| Residual | Nothing | none | Head of compliance decides |
| Paper and plan | Drafts from approved ratings | 2 | Committee approval |
Which controls matter, and what evidence does each leave?
No rating without its evidence. Every draft rating links to the items behind it. A rating with no evidence is shown as unsupported, not hidden. Evidence: the linked item list for each area.
Control effectiveness needs a test, not a description. A control described in a procedure is not proven to work. The agent may only mark a control effective where a dated test result exists. Evidence: test references on every "effective" mark.
Final ratings are a person's act. The agent's role has no capability to set a final rating. Evidence: the approver's name and any written reason for departing from the draft.
Sources are stamped. Audit findings and breach logs come from the systems that hold them; a business head's comment is recorded as declared, not as fact. Evidence: provenance on each input.
Every draft and change is filed. You can reconstruct what the agent proposed, what was changed and by whom. Evidence: the append-only record.
What should you measure?
- Ratings changed by the reviewer, by area and direction. Many upward changes suggest the agent misses context; none at all suggests nobody is reading.
- Evidence coverage: areas where one or more source types returned nothing. Empty can mean clean, or a feed that stopped.
- Controls marked untested, and whether the monitoring plan covers them.
- Unexplained movement flagged at step 3 and how each was resolved.
- Time from evidence cut-off to committee approval. The agent should shorten preparation, not the thinking.
Where does it usually go wrong?
Anchoring. The reviewer sees a neatly argued draft rating and accepts it. Ask the head of compliance to note their own view of a sample of areas before opening the draft, and compare.
Quiet areas look safe. An area with no complaints and no breaches may simply have no monitoring. The agent should report absence of evidence as its own finding.
The previous cycle's text is recycled. Drafting from the previous assessment carries old reasoning forward. Require each area's narrative to cite evidence from the current period.
Specialist assessments are left out. Product reviews and affordability work hold risk information. Link the outputs of product fair value assessments with an AI agent and income and expenditure assessments with an AI agent into the evidence set.
AI risk is forgotten. The firm's own agents are a compliance risk area too. Where they make or support consequential decisions, a separate assessment may be required; Colorado AI act impact assessments for AI agents shows one example.
Where Pryme fits
In Pryme, the assessment agent reads incident, audit and complaints records through the Pryme Context Protocol, so each item carries its provenance class. Its role scope allows depth 1 and 2 only: it can propose ratings but has no capability to set them. Every draft, change and approval is filed, so the committee can see how each rating was reached.
Questions people ask
Can the agent run the whole assessment unattended between cycles?
It can keep the evidence current, flagging new breaches, findings or rule changes against each area as they arrive. Whether a rating should move mid-cycle is still a person's call. The broader answer is in whether an AI agent can do a compliance risk assessment.
How do you test the agent before it drafts ratings?
Replay it against a past cycle where the final ratings are known. Score whether it gathered the same evidence, flagged the same gaps and proposed defensible ratings. Disagreements with the final ratings are worth reading one by one.
Does the agent make the assessment less independent?
Not if business heads cannot edit the evidence set. They add comments, which are recorded as their statements, but the source items come from the systems of record.