Widening depth is an evidence decision, not a confidence decision

Teams usually widen an agent's authority when they feel comfortable: no complaints for a while, few reviewer edits, a sponsor wanting savings. Comfort is not evidence. Depth should rise only when scored results on the business's own cases show the agent clears the standard at the new depth, and the decision should be recorded with that evidence attached.

What is the difference between confidence and evidence?

Confidence is a feeling built from impressions. Evidence is a record built from scored cases. They often point the same way, which is why they get confused. They diverge in exactly the situations that matter.

Every item in the left column can be true while the agent is unready for more authority. No complaints may mean errors have not reached customers yet, because a person was catching them. Good reviewer impressions may reflect fluency rather than accuracy. Months of running prove nothing about the next depth if all of it was at the current one.

Why is widening the moment that matters most?

Because it changes who catches the agent's mistakes. At draft, every error passes in front of a person before it reaches anyone. Moving to execution within bounds removes that person for routine work. Errors that a reviewer used to catch now reach customers and records directly.

The evidence needed to make that move must therefore answer a specific question: if no person checks this work, how often will it be wrong, how wrong, and is that acceptable? Confidence cannot answer it. Scored cases can, because they show what the agent would have done on each one and whether it was right.

What evidence should a widening decision rest on?

Evidence specific to the capability and the depth proposed.

• Scored cases from the business's own operation, enough to cover the normal mix and a meaningful number of the rare cases.
• Results against each test that matters: whether answers were grounded in sources, whether the agent refused when it should, stayed in scope, escalated what it could not handle and avoided harmful actions.
• Results at the proposed depth. If the agent will execute, test what it would execute, not what it drafts.
• Live signals that agree: reviewer edit rates falling to cosmetic changes, refusal patterns stable, no recent incidents on the capability.
• A written standard set before the results were seen, so the bar was not adjusted to fit them.

The more consequential the capability, the higher the bar. A capability that affects a customer's access to credit, insurance or services, the kind discussed in our explainer on what a consequential decision is, needs stronger evidence and closer monitoring after widening than one that updates a contact preference.

Why should widening and narrowing be asymmetric?

Because the risks are asymmetric. Widening too early exposes customers to errors nobody is catching. Narrowing too early costs some efficiency for a while. So widening should need evidence, and narrowing should need only a credible signal.

If reviewer edits suddenly shift to substantive changes, if a source changes, if an incident occurs, the capability owner should be able to lower depth at once, and restore it only when fresh evidence supports it. Our guides on when an agent's depth should be widened and when it should be reduced set out the triggers for each direction.

How should the decision be recorded?

As a decision, with a name and the evidence attached. The record should show the capability, the old and new depth, the cases the evidence was drawn from, the results against each test, the standard they were measured against, who approved the change and when the evidence expires. An autonomy decision matrix template gives a format.

Recording it this way does two things. It makes the decision defensible later: if something goes wrong after widening, the record shows what was known and decided. And it makes widening repeatable: the next capability can be judged against the same kind of evidence, rather than against whoever feels most strongly.

What does a confidence decision look like in practice?

Consider an agent drafting replies to billing disputes for a utility. After two months, reviewers say they barely change anything, complaints have not risen, and the operations manager proposes letting the agent send replies on its own. Nobody has scored the drafts against a standard. Nobody has checked whether the "barely change anything" includes the occasional changed refund amount. Nobody has looked at how the agent handled the small number of disputes involving vulnerable customers.

The evidence-based version of the same decision would replay the agent against a few hundred recent disputes, score each against the written standard, separate cosmetic from substantive edits in the live record, examine every vulnerable-customer case individually, and widen only the dispute types that cleared. Some would move to execution. Some would stay at draft. The decision would take a few more weeks and be far easier to stand behind.

What is the trade-off?

Evidence takes longer to gather than confidence takes to form. Teams will feel held back, and sponsors will see savings arrive later. Some capabilities that would have been fine will wait longer than they needed to.

That delay is the price of knowing. Widening on confidence works until the first time it does not, and then nobody can show why the decision was made. For every depth increase you are considering, write down the standard, gather scored cases at the new depth and attach the results to the decision before it is approved. The full deployment sequence is set out in our guide to deploying an AI agent: train, shadow, certify, widen.

Where Pryme fits

In Pryme, depth is earned, not configured. Attest replays the agent against real cases from the customer's own operation and scores every answer against a floor on seven gates, each of which must clear on its own. The agent's role sets a ceiling; the evidence sets the depth; the lower wins.