PrymeIntelligence

Agent governance·Answer

What should an AI agent do when it is unsure?

When an AI agent is unsure, it should stop short of acting, say exactly what it does not know, and hand the case to a named person or ask the customer.

Stop short of acting and say what is missing. An unsure agent may keep reading and drafting, but it should not execute. It either asks the customer a precise question, when the gap is information only they hold, or holds the case for a named person, stating what it knows, what it does not, and why that matters.

Why not let the agent make its best guess?

Because a guess that turns into an action is a decision nobody made. If the agent picks the more likely of two accounts and pays the wrong one, there is no rule, no person and no evidence behind the choice. It cannot be explained afterwards, and it may not be reversible.

Guessing is fine in a draft. A draft that says "I have assumed the account ending 1234; the customer has two linked accounts" gives a person something to check. The line is between drafting under uncertainty, which helps, and acting under it, which does not.

How should "unsure" be defined?

Not by the model's own sense of confidence. A model can be fluent and wrong, or hesitant and right. Its stated certainty is not a reliable trigger.

Define uncertainty in observable terms instead, so that it can be tested:

Trigger Example What the agent does
A required fact is missing No delivery address on the order Asks the customer, if the fact is theirs to give
Two sources disagree Contract says one fee, billing system another Holds for a person; shows both
A fact is weak Bank details arrived only in an email Holds; external claims cannot authorise a payment
The request is ambiguous "Send it to my usual account" with two on file Asks the customer which
The case is outside what it was tested on A product it has not handled before Holds; routes to the specialist team
A rule does not cover the situation Policy silent on a joint account where one holder has died Holds; routes to the process owner

Each row names a condition someone can check after the fact. That is what lets you test, before go-live, whether the agent escalates when it should.

What should the hand-over contain?

A held case should be quick to pick up. It should state the action the agent would take if the gap were filled, the specific gap, the sources it did check, and which bound caused the hold. A person should not have to redo the agent's reading to understand the question.

The hold should also name a person or a team queue with a named owner, not "someone". And it should be filed, so you can later see how often each trigger fires. A rising rate on one trigger usually points to a source or brief problem worth fixing. Being able to reconstruct why an agent took an action depends on this record as much as on the record of actions taken.

What does this look like in practice?

Take a wealth manager's client service desk. A client writes: "Please move 20,000 to my usual account today." The client has two external accounts on file, one used last month and one used for several years.

The agent does not choose. It checks that the amount is within the client's limits and that both accounts are verified, then replies: "You have two accounts on file, ending 1234 and 5678. Which should we use?" The client answers 5678. The agent drafts the transfer and, because moving money is held for this client segment, sends it to the named relationship manager with the client's reply attached.

Had the client written back with a new account number, the agent would have held the case for the verification process, not asked again.

Which controls make this work?

  • Triggers written as conditions, tested before go-live on real cases where the right answer was to escalate.
  • Execution blocked under uncertainty; drafting allowed.
  • Questions to customers only about their own facts. The agent does not ask a customer to resolve a conflict in your own records.
  • Named owners for every hold queue.
  • The same discipline for outbound messages. A message drafted under uncertainty should not go out unreviewed; see whether an AI agent should send emails without review.
  • Trigger rates reviewed monthly.

Each hold should name the trigger or bound that caused it, for the reasons in why every agent decision needs a named bound. The wider model is in our guide to agent governance.

Questions people ask

Will this make the agent escalate too much?

At first, perhaps. Measure which triggers fire most, then fix the cause: add the missing source, clarify the rule, or tighten the trigger. Do not fix it by telling the agent to be more confident.

Should the agent tell the customer it is unsure?

It should tell them what it needs or that a person will look at the request, and roughly when. It need not describe its internal state.

Can an unsure agent still answer simple questions?

Yes, if the answer is grounded in a source. Uncertainty about one action does not stop it reading, explaining or drafting.

Decide what your agents may do. Prove it. Enforce it.

Pryme Intelligence certifies each agent on your own cases, checks every action against five bounds at the moment it happens, and files every verdict.