Hallucination is when an AI model produces output that sounds confident and plausible but is false or not supported by any source: an invented policy clause, a wrong figure, a citation to a document that does not exist. Language models generate likely text, not checked facts, so hallucination is a property of how they work, not an occasional fault.
Why does hallucination matter more once agents act?
In a chat, a hallucination is a wrong sentence. A person might catch it. In an agent, the invented detail can become an argument to an action: a made-up account number in a payment, a fabricated deadline in a letter to a customer, a non-existent exception applied to a refund. The error leaves the conversation and lands in your records.
The most dangerous hallucinations in agents are small and specific. An invented reference number looks exactly like a real one. A slightly wrong amount passes a glance.
What forms does hallucination take in agents?
- Invented facts: a policy term, rate or date the sources do not contain.
- Invented arguments: values passed to a tool that do not come from any record.
- False citations: a real document cited for a claim it does not make.
- Invented steps: the agent reports doing something, such as "I have checked the sanctions list", that it never did.
The last one is easy to miss. An agent's account of its own work is generated text too. The record of which tools it actually called is what counts. How agents choose those steps is covered in how an AI agent decides what to do next.
How do you contain hallucination in practice?
You cannot remove it from the model, so you contain it around the model. Ground answers in retrieved sources and check citations. Make the agent return structured output for actions, so every argument is a named field that can be compared against a record. Check key values, such as amounts, account numbers and dates, against source systems before any action runs. And log what the agent did from the tool calls, not from its narrative.
For example, a payroll bureau's agent prepares a correction for an employee's overtime. Its draft states 12 hours at a rate of 25. The check compares both values with the signed timesheet and the employee's contract. The hours match; the rate does not, because the contract says 22. The action is held with the mismatch shown.
How is hallucination different from a reasoning error?
A hallucination is an invented fact. A reasoning error is a wrong conclusion drawn from correct facts, such as applying the right fee to the wrong tier. Both lead to bad actions, but they are caught differently. Hallucinations are caught by checking values against sources. Reasoning errors are caught by testing the agent on real cases where the correct outcome is known. Asking the model to show its steps, as in chain-of-thought reasoning, can help with the second, not the first.
Where Pryme fits
Attest scores every answer in a certification run against a floor, including whether it is grounded in the sources given. Agents that hallucinate on the customer's own cases do not earn depth. A run that clears no depth drops the agent to depth 1 the same day. See the governed agentic operating system, explained.
Questions people ask
Will better models stop hallucinating?
Newer models tend to hallucinate less on common tasks, but none can be relied on not to. Design every agent on the assumption that some outputs will be invented.
Is hallucination the same as lying?
No. A model has no intent. It produces likely text, and sometimes likely text is false. The fix is checking, not persuasion.