Over-reliance on AI is the habit of accepting an AI system's output without the scrutiny the situation needs, usually because the system is right most of the time. It is not trust; trust can be earned and checked. Over-reliance is trust that has stopped being checked, so the occasional error passes as easily as the routine correct answer.
Why does over-reliance matter with agents?
An agent that is usually right is the setting where over-reliance grows fastest. Reviewers see dozens of correct drafts and start approving on sight. The control that was meant to catch the rare bad action stops catching it. Because the record still shows a person approved, the failure is invisible until something goes wrong.
It also erodes skill. A loan officer who has not assessed a case from scratch in months may no longer be able to when the agent is paused. That is part of the case for certifying loan officers before agents go live.
How does it show up in practice?
The signs are visible in the record and the room:
- approval times falling steadily while case complexity does not;
- edit and rejection rates drifting towards zero;
- reviewers unable to say why an item was held;
- staff struggling when an agent is paused and the work returns to them;
- customers hearing "the system decided" as an explanation.
A specific risk sits at handover points. When an agent passes a live chat to a person, the person may adopt the agent's framing of the problem rather than listen afresh; see how to hand a live chat over to a person. When an agent's owner leaves, the successor may inherit its settings without understanding them, which is covered in how to hand over an AI agent when its owner leaves.
What is an example?
A benefits office's agent drafts eligibility letters and holds those that change a payment. Over three months, the reviewer's approval time drops from minutes to seconds. A letter based on a declared change of address, never verified, stops a family's payment in error. The draft looked like every other. Public bodies carry particular duties of fairness here, set out in what AI literacy means for public bodies.
How is over-reliance different from automation bias?
Automation bias is the cognitive tendency to favour suggestions from automated systems over other information, even contradicting evidence. Over-reliance is the pattern of behaviour, and often the organisational condition, that results from it: queues sized too large, metrics that reward speed, reviewers who are never shown the agent's errors. You cannot train bias out of people entirely. You can design work and training so that over-reliance does not set in.
Rubber-stamping is the visible act; over-reliance is the condition that produces it.
Which terms are related?
- Rubber-stamping: approval without examination.
- Meaningful human review: what over-reliance undermines.
- Skill fade: loss of the ability to do the work without the agent.
For the wider topic, read AI literacy and certification for people who work with agents.
Where the Academy fits
Pryme Intelligence Academy tests against over-reliance directly. The Operator practicum is two weeks of real review decisions, and a single missed red line fails it regardless of overall accuracy. Module tests are set in the learner's own industry and role, so the cases resemble the ones that will reach their queue.
Questions people ask
How do you measure over-reliance?
Watch approval times, edit rates and rejection rates per reviewer over time, and seed occasional known-bad items in practice queues. A reviewer who misses them is relying on the agent.
Should reviewers be rotated to prevent it?
Rotation helps, as do breaks and varied work. The stronger fix is training that shows reviewers the agent's real errors, so they keep looking for them.