Compliance monitoring of agents: test the bounds, not just the outputs

Compliance monitoring of AI agents should test whether each agent stayed inside its bounds, not only whether its answers looked right. A sample of good-looking outputs says little about the action an agent attempted and was refused, the depth it used or a limit it came close to. Bounds are the control; outputs are what the control produces.

Why isn't output sampling enough?

Traditional compliance monitoring of a customer-facing process often samples the work: pull a set of calls, letters or files and check whether each met the standard. Applied to agents, the natural version is to sample replies and check whether they were accurate, clear and fair.

That is worth doing, but it misses the agent-specific risks. An agent's output can look fine while the agent was operating on an expired certificate, reading a source it should not have had, or acting at a depth the customer never approved. Equally, an agent can produce a poor-looking output while every control worked as designed. Output quality and control effectiveness are different questions, and monitoring an agent means answering both.

There is a deeper reason too. With people, the bounds are mostly in training and supervision, so outputs are the main evidence of whether they were followed. With agents, the bounds are written down and enforced at the moment of action, so they can be tested directly. Not testing them wastes the most checkable control the firm has.

What should a bounds test check?

Every action an agent takes should meet five bounds at once: its role scope, its certificate, its sources, the customer's own limits and the platform guardrails. A bounds test asks, for a sample of actions or for all of them, whether each held.

1. Role scope. Did the agent only attempt actions within its defined role?
2. Certificate. Was the agent's certificate valid, at the depth that action needed, at the moment of each action?
3. Sources. Did the agent read only the sources it was permitted, and were they the versions in force?
4. Customer limits. Were amounts, counts and approval requirements respected?
5. Platform guardrails. Were any withheld actions attempted, and were they stopped?

The test should also look at the verdicts: how many actions were allowed, held, refused and withheld, and which bound decided each one. A shift in that mix is often the first sign of a problem.

How do you test bounds in practice?

There are three useful methods, and a good monitoring plan uses all of them.

Query the verdict record. If every verdict is filed with the bound that decided it, monitoring can test the population rather than a sample: every action above a threshold, every action on an expired certificate, every refused attempt to read an unpermitted source. This is where agents make monitoring easier than it is for people.

Reconcile configuration. Compare the agent's live bounds with what was approved: the agent's depth, the customer limits, the source list. Unapproved drift in configuration is a finding even if no harmful action resulted.

Probe. Send test requests designed to approach or exceed a bound, in a controlled environment or with clearly marked test cases, and check that the agent was refused or held. This is the agent equivalent of a mystery shop. The approach is set out in how to run compliance monitoring tests with an AI agent.

What should the monitoring plan contain?

A compliance monitoring plan for agents can follow the usual structure, with tests aimed at bounds as well as outcomes.

Who should run the tests?

The second line, independently of the team that runs the agents. The tests are only meaningful if the people performing them are not the people whose configuration is being tested. An agent can help: it can run the queries, assemble results and draft the report. Whether an AI agent can prepare a compliance monitoring report is a fair question, and the answer is yes for the preparation. The conclusions belong to the monitoring team.

Agents can also help test the firm's wider controls, as discussed in whether an AI agent can test compliance controls. The same principle applies there: the agent prepares and executes defined tests; a person judges the results.

What do bounds findings tell you that output findings cannot?

Output findings tell you about one answer. Bounds findings tell you about the system. A single poor reply is a coaching point. An agent that acted on an expired certificate for a week, or whose approved depth was quietly raised without a certification run, is a control failure that affects every action in that period.

Bounds findings are also easier to fix at the root. A configuration error can be corrected and reconciled. A pattern of poor outputs needs diagnosis before anyone knows what to fix.

Where Pryme fits

In Pryme, every verdict is filed with the bound that decided it, so monitoring teams can query the full population rather than relying on samples. Certificate status, depth and customer limits at the moment of each action are part of the record. This fits the view of AI compliance agents as second line, never the last word.