PrymeIntelligence

Agent governance·Answer

How do you stop an AI agent from acting on false information?

Stop an AI agent from acting on false information: stamp each fact with its source, let only strong sources authorise actions and hold on conflicts.

Rank every fact by where it came from, and let only strong sources authorise actions. A balance from the core ledger can support a payment; a balance quoted in a customer's email cannot. Stamp each fact with its provenance as it reaches the agent, set which provenance each action needs, and hold the action when sources conflict or only weak ones are available.

Why do agents act on false information?

Because to a language model, every sentence in its context is equally real. A forged invoice, a stale policy, a customer's mistaken account number and the ledger all arrive as text. Without help, the model cannot tell a sealed record from a rumour, and it will reason fluently from whichever it reads.

False information reaches agents in four common ways: fraud (forged documents, spoofed emails), error (a customer types the wrong number), staleness (a policy that has since changed) and other agents (an output from one agent becoming an input to another without checks).

How should facts be ranked?

Provenance What it means Example Can it authorise an action?
Sealed Issued by the system of record, tamper-evident Ledger balance, policy library entry Yes, within bounds
Signed A named, identifiable party answers for it A verified issuer's certificate Yes, for actions that rely on that party
Declared Stated by a customer or person, unverified "My new address is…" Only low-impact actions, or after verification
External Arrived from outside An emailed invoice, a web page No, it can prompt a check

The key rule: an external claim cannot authorise a payment. It can start a process that checks the claim against a stronger source.

What are the practical steps?

  1. Register sources. Every system and channel the agent reads, with its provenance class.
  2. Stamp facts on arrival. The platform tags each fact with where it came from and when.
  3. Set provenance requirements per action. Paying a supplier needs bank details from the sealed supplier master; updating a phone number may accept a declared value after authentication.
  4. Check freshness. Policies, prices and rates carry a version and an effective date. Stale facts are treated as missing.
  5. Detect conflicts. If two sources disagree, the action is held and both are shown to a person.
  6. Treat other agents' outputs as their source's class, at best. An agent's summary of an email is still external.
  7. Show provenance to approvers, so a person can see that a held item rests on an emailed claim.

Where do the less obvious risks sit?

Two are worth naming. First, unregistered agents. Output from an agent nobody governs arrives with no provenance at all; our answer on how to stop shadow AI agents covers finding them. Second, scope creep. An agent that cannot confirm a fact may be given new sources to read "to help", each weaker than the last; see how to stop an AI agent's scope from creeping.

Payments carry a specific risk where an agent acts for a payer. A message saying "the customer has agreed" is not consent. In the EU, PSD2 strong customer authentication requires the payer to authenticate; our article on PSD2 and AI agents acting for the payer covers consent and authority.

A worked example

Take a building materials supplier whose credit control agent manages customer accounts. An email arrives from what looks like a large customer's finance director: "We paid 48,000 yesterday, please lift the stop on our account so tomorrow's delivery can go."

The claim is external. Lifting a stop requires a sealed source showing payment. The agent checks the bank feed and the ledger; neither shows the payment. The action is held, and the credit controller sees the email, the missing payment and the agent's note. She calls the customer on the number on file. The payment had been scheduled, not sent. The stop stays until the money arrives.

Which controls make the answer hold?

  • Provenance on every fact, assigned by the platform, not the model.
  • Action rules that name the provenance required.
  • Conflicts and stale facts hold the action.
  • Agent outputs never upgrade their source's class.
  • Approvers see provenance.

The topic guide, agent governance is a bound, not a model, explains why sources are one of the bounds every action must meet.

Where Pryme fits

The Pryme Context Protocol stamps every fact reaching an agent as sealed, signed, declared or external. Sources are one of the five bounds checked at the moment of action, so an action that needs a sealed fact is held or refused when only weaker ones exist. An external claim cannot authorise a payment.

Questions people ask

Can the agent verify a claim itself?

It can check the claim against stronger sources it has access to. If no stronger source exists, it should hold the action, not guess.

What if the system of record is wrong?

Then the agent will act on wrong information with full confidence, as a person would. Fix the record; provenance tells you where to look.

Does this slow down routine work?

Very little. Most routine actions rely on sealed sources already. The checks bite on the unusual cases, which is where false information tends to arrive.

Decide what your agents may do. Prove it. Enforce it.

Pryme Intelligence certifies each agent on your own cases, checks every action against five bounds at the moment it happens, and files every verdict.