A failed certification run should drop the agent to the highest depth it cleared that same day, or to depth 1 if it cleared none, before anyone has worked out why it failed. Depth rests on evidence. The moment the latest evidence says the agent no longer meets its floor, keeping its authority until a review meeting means letting an unproven agent act on real customers in the meantime.
Why not wait for the investigation?
The instinct after a failed run is to understand it first. Perhaps the certification set had a labelling error. Perhaps a source was briefly unavailable. Perhaps the failure is in one narrow case type that rarely occurs live. All of these are possible, and all of them take time to establish.
That time is the problem. While the investigation runs, the agent is still drafting, executing or moving money at whatever depth it held before. If the failure turns out to be real, every action taken in the gap was taken by an agent you had reason to doubt. If the failure turns out to be a false alarm, the cost of acting was a few days of work routed to people. Those two costs are not symmetrical, and the decision should follow the larger one.
The whole point of choosing to certify an agent is that its authority rests on current evidence. The principle that depth is earned, not configured only means something if it works in both directions. Evidence raises depth; evidence must also lower it, on the same terms and at the same speed.
What does "the same day" mean in practice?
It means the drop is automatic, triggered by the run result, not by a person reading a report. If the downgrade depends on someone opening a ticket, it will happen when that person has time, which on a busy week may be the following week.
The mechanics are simple. The failed result updates the agent's certificate status. The certificate is one of the bounds every action must clear, so from that moment the agent's actions are checked against the depth the run cleared, or depth 1 if it cleared none. Anything above that is either refused or, where the customer has set it up that way, held for a person. Nobody has to remember to do anything.
| Before a run that clears no depth | After it |
|---|---|
| Agent sends routine replies itself (depth 3) | Agent reads and summarises; replies are refused or routed to people |
| Agent drafts refunds for approval (depth 2) | Drafting stops; refund requests go to the team directly |
| Verdicts name the certificate as satisfied | Verdicts name the certificate as the deciding bound |
Isn't dropping the whole agent too blunt?
It can feel that way, especially when the failure was in one gate and the agent's other results were strong. The trade-off deserves a straight answer.
Dropping the whole agent is blunt on purpose. A finer response, such as removing only the capability that failed, requires someone to judge which capabilities the failure affects, and that judgement is the investigation you have not done yet. A drift failure, for example, may show up first in one capability and later in all of them, because the cause is a model change that touches everything.
Even depth 1 is less disruptive than it sounds. The agent can still read, search and summarise. People handling the work can still use it to find information and prepare their own replies. What stops is the agent acting on its own account. That is the right thing to stop until you know why the evidence failed.
What happens to work already in flight?
This is where same-day downgrades most often go wrong. An agent may have scheduled actions queued, drafts waiting in the held queue, or multi-step tasks part-completed. Each needs a defined treatment.
- Scheduled actions that the agent would execute later should be re-checked at the moment of action against the new depth. If enforcement happens at the moment of action, this is automatic.
- Held drafts waiting for approval can stay in the queue, but the approver should see that they were prepared by an agent whose certificate has since failed, and review them with that in mind.
- Part-completed tasks should stop at the next step that needs more than the new depth, and be handed to a person with the steps so far recorded.
The procedure for each of these is set out in how to handle a failed certification run.
How does the agent earn depth back?
Through the same route it earned depth originally: evidence. The usual sequence is to find the cause, fix the brief, sources or bounds, rerun certification on a set that includes the cases that failed, and have the process owner accept the result. The depth gate then sets the depth the new evidence supports, which may be lower than before.
What should not happen is a restoration of the old depth on the strength of an explanation. "We found a labelling error" may be true, but the fix is to correct the label and rerun, not to declare the failure void. If the labels were wrong, that is worth knowing too, because labelling is the hidden cost of certification and errors there undermine every run, not only the failed one.
What should the record show?
A failed run is a governance event and should leave the same quality of evidence as a pass. That means the run itself, the gates that failed and by how much, the time the depth changed, the actions refused or held as a result, the cause once found, the fix, and the rerun that restored authority. A failed certification run report template gives a consistent shape for this.
Auditors and supervisors are likely to be more interested in how you handled failures than in your passes. A record showing that authority fell within hours, and returned only on fresh evidence, is the strongest proof that certification is a control and not a formality.
What should you change?
Check whether a failed run in your setup lowers depth automatically or waits for a person. If it waits, change that first. Then decide, in advance, how in-flight work is treated, so that nobody has to improvise on the day. A same-day rule only protects you if it is mechanical.
Questions people ask
Does a failed run in one capability affect all the agent's capabilities?
In our model the agent keeps only the highest depth the run cleared, or drops to depth 1 if it cleared none, because the cause of a failure is not known until it is investigated. Its depth can then be restored when a rerun clears it.
Can a person override the downgrade to keep a critical process running?
The process should continue through people, not through the agent at its old depth. If an agent is critical enough that a downgrade stops the process, that dependency is itself a risk worth recording.
Should customers be told when an agent's depth drops?
Usually there is nothing to tell them, since their requests are still handled, just by people. Where response times change noticeably, say so plainly without detail.