Someone who knows the work, had no part in building or buying the agent, and can say no. For read and draft capabilities, one such person is usually enough. For capabilities that act alone or move money, add a second reviewer from risk. Each is named on the certificate, with what they accepted and any conditions set.
What does the reviewer need?
Four qualities, and all four matter.
| Quality | Why it matters | Who tends to have it |
|---|---|---|
| Knowledge of the process | To judge whether an answer would be right in practice | Senior practitioners, team leaders, quality leads |
| Independence from the build | To read failures without wanting the agent to pass | Anyone outside the project and procurement team |
| Authority to refuse | So a "no" sticks | Someone senior enough that the project cannot overrule them informally |
| Time to read | Because the work is in the failures, not the summary | Someone whose diary allows it |
A reviewer missing any one tends to wave results through. A senior executive with authority but no time will read the summary. A practitioner with knowledge but no authority will raise concerns that get overruled.
Who should not review?
- Anyone who built the agent or wrote its brief.
- Anyone who selected the vendor or signed the contract.
- Anyone whose targets depend on the agent going live.
- The vendor, for a certificate that decides depth in your operation.
- A scoring model on its own, however well calibrated.
Some of these people should be in the room to answer questions. None should make the decision.
What should the reviewer look at?
Failures first, then everything else. A useful order:
- Every failed case, read in full, with the agent's output and the expected outcome.
- Results per gate against the floors. A clear pass on every gate is required; see what a certification gate means and what a certification floor means.
- When the floors were set. They should predate the run.
- Coverage of the case set. Whether hard, rare, refusal and escalation cases are present in enough number.
- The configuration tested. Model, brief, sources and tools match what will run in production.
- Changes to expected outcomes. Any case whose expected answer was altered, and why.
- Proposed conditions. Which cases should go to a person regardless of the pass.
These materials should come together in one place; the certification evidence pack is the usual form.
When should there be two reviewers?
When the capability will act without a person checking each case, or will move money or take actions that are hard to undo. The practitioner judges whether the answers are right. The risk reviewer judges whether the evidence and controls are enough for the risk. Both sign. Disagreement between them is resolved upward, not by one deferring to the other.
What happens when the reviewer rejects?
The capability stays where it was, and the reviewer records why. The team fixes the cause and reruns. If the reviewer rejected because the case set was weak, the team adds cases. The process is the same as for a failed run, set out in how to handle a failed certification run.
Rejections should be normal and unremarkable. A reviewer who never rejects is either working with an unusually good team or not reading closely.
An illustrative example
Take a general insurer certifying an agent to issue routine policy documents at depth 3. The project lead proposes that she review the results, since she knows the agent best. The risk team points out she is not independent.
The reviewers chosen are the policy servicing team leader, who has run the process for years, and a second-line risk manager. The team leader reads every failed case and notices that two "passes" involved policies with a pending cancellation, where the agent issued documents a person would have held. The cases had been marked as passes because the expected outcome was wrong. She sends them back. The expected outcomes are corrected, the agent is fixed and rerun, and both reviewers sign the certificate with a condition that policies with a pending cancellation are always held.
How does this compare with ISO-style review?
In management system certification, an external auditor reviews the organisation's processes. Agent certification review is internal, closer to the work, and decides whether a specific agent may act. Both matter; the difference is set out in whether AI agent certification is the same as ISO certification.
Where Pryme fits
Attest produces the results per gate, the failed cases and the configuration tested in one evidence pack, and records the named reviewer's acceptance with the certificate. The wider reasoning is in what it means to certify an agent.
Questions people ask
Can the same reviewer handle renewals?
Yes, and continuity helps. Rotate occasionally so familiarity does not turn into assumption.
How long should a review take?
As long as reading the failures takes. If the reviewer cannot read every failed case, the review is too rushed or the failure count is too high.
Should reviewers be trained?
Briefly, yes: in reading gate results, spotting weak case sets and recording decisions. Their process knowledge is what makes them valuable, and that cannot be taught quickly.