A monitoring test asks whether a control worked, on real cases, during a defined period. The agent does the mechanical part on far more cases than a person: draw the sample, apply the test script, record the evidence and draft exceptions. The monitoring reviewer confirms every exception, the monitoring lead signs off results, and business owners agree the actions.
What is compliance monitoring testing, and who owns it?
Compliance monitoring is the second line's testing of whether the business meets its regulatory obligations and whether its controls operate as intended. It runs on a risk-based plan, usually documented using something like a compliance monitoring plan template, with each test defined by a control, a population, a sample, a test script and pass criteria. It is distinct from transaction monitoring, which watches customer activity for financial crime.
The compliance monitoring team owns the tests and findings; the head of compliance owns the plan; business owners own remediation. Testing by hand limits sample sizes and makes results slow. An agent widens samples and speeds results, but the second line's independence and judgement remain with people.
What does the agent prepare, and what does a person approve?
- Person, monitoring lead: approves the test. Control, population, period, sample method and size, test script and pass criteria.
- Agent, depth 1: extracts the population from the system of record and records the extract.
- Agent, depth 1: draws the sample using the approved method, or tests the whole population where the script allows.
- Agent, depth 1: applies the test script to each case: was the approval given by an authorised person, within the deadline, with the required evidence attached, and so on. Each step result is recorded with the evidence relied on.
- Agent, depth 2: drafts exceptions. Each failing case with the step failed, the evidence and a note of possible causes.
- Person, monitoring reviewer: confirms each exception and re-tests a sample of passes.
- Agent, depth 2: drafts the test report. Results, exceptions, themes, root causes and proposed rating. The structure follows what is covered in whether an agent can prepare a compliance monitoring report.
- Person, monitoring lead: signs off the result and rating.
- Person, business owner: agrees actions; the agent tracks them to closure.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Test design | Nothing | None | Monitoring lead approves |
| Population | Extracts and records | 1 | None |
| Sample | Draws by approved method | 1 | None |
| Test | Applies script, records evidence | 1 | None |
| Exceptions | Drafts with evidence | 2 | Reviewer confirms |
| Pass re-test | Nothing | None | Reviewer samples passes |
| Report | Drafts results and rating | 2 | Lead signs off |
| Actions | Tracks to closure | 1 | Owner agrees and delivers |
Which controls apply, and what evidence does each leave?
Population completeness. The extract is reconciled to a control total from the system. Evidence: the reconciliation, because a test on an incomplete population proves nothing.
Script applied as approved. The agent uses the approved script version and cannot change pass criteria. Evidence: script version on each result.
Passes re-tested by a person. Agents tend to make the same mistake consistently. A reviewer re-testing a sample of passes catches systematic errors that confirming exceptions would never show. Evidence: re-test results.
Exceptions confirmed. Evidence: reviewer name and decision on each.
Independence. The agent running tests is not one that performs the controls being tested. Evidence: separation recorded in the agent inventory.
What should you measure?
- Tests completed against plan.
- Sample sizes compared with manual testing.
- Exceptions confirmed versus rejected by reviewers.
- Errors found in re-tested passes, which should be rare and each investigated.
- Actions closed on time by business owners.
Where does it usually go wrong?
Scripts written for people. "Check the file is adequate" cannot be applied consistently. Scripts must define each step and its pass criteria precisely before an agent can apply them. Writing them that way improves manual testing too.
Testing what is easy to test. Agents make data-rich tests cheap, and the plan drifts towards them. The plan should follow risk, not convenience.
Exceptions without root causes. Twenty exceptions with the same cause are one finding. Group them.
Testing agents with agents only. Where business agents perform controls, testing them with a monitoring agent alone means two systems may share blind spots. Include people in the sample review.
Evidence the agent cannot see. Some controls leave their evidence outside the system of record: a signed paper form, a phone call, a meeting. The agent will fail these cases for missing evidence when the control may have worked. Mark such steps in the script as needing a person, and do not report them as exceptions until a reviewer has looked.
Period boundaries blurred. Cases opened in the test period but completed after it, or the reverse, are easy to include or exclude by accident. The script should state which date defines the population, and the agent should apply it exactly.
Results not linked to the obligations register. A failed test should show which obligations are at risk. Without that link, findings are hard to prioritise.
The overall stance on compliance agents is in AI compliance agents.
Where Pryme fits
In Pryme, the testing agent reads populations from systems of record as sealed facts and holds depth 1 for testing and depth 2 for drafting exceptions and reports. Changing a test script or pass criteria is outside its role scope. Where the controls under test are performed by Pryme agents, every verdict they reached is already in the append-only record, so the population is complete by construction.
Questions people ask
Can the agent test the whole population instead of a sample?
Often, yes, where the test is data-based. Full-population testing removes sampling risk. Tests that need judgement on each case may still use samples.
Who rates the test result?
The agent proposes a rating against your criteria; the monitoring lead decides. The rating affects what goes to committee, so it needs a person's judgement.
Should business owners see results before sign-off?
Yes, to confirm facts and agree actions. The rating and findings remain the second line's to decide.