Most fail because they answer the wrong question. A pilot shows that an agent can do a task on clean cases while its builders watch. Production needs proof it can be allowed to, daily, under an owner who did not build it. Without that proof, risk cannot sign and the pilot stalls.
What does a stalled pilot look like?
The demonstration goes well. Leadership is enthusiastic. Then the pilot enters a long middle period: security wants a review, compliance asks how decisions will be recorded, the operations team says it has not been consulted, and the data protection officer asks what lawful basis covers sending customer records to a model provider. Months pass. The pilot is not cancelled; it simply never goes live. The pattern is described in more detail in why agent pilots stall after the demo.
None of those questions is unreasonable. They are the questions any regulated business asks before letting something act on its customers and books. The pilot failed because it was not designed to answer them.
What are the common causes?
| Cause | How it shows up | What fixes it |
|---|---|---|
| No operational owner | Nobody in the business will accept the agent | Name the owner before the pilot starts |
| Success defined afterwards | Results are argued over rather than read | Write criteria in advance |
| Clean test cases | Fails on live data, or never meets it | Shadow on live work within the pilot |
| No way to bound the agent | Risk cannot approve what it cannot limit | Bounds enforced outside the model |
| No record of decisions | Audit and compliance cannot sign | Every verdict filed, naming the bound |
| Overstated return | First review shows far less than the slide | Count reviewer time and rework |
| The wrong process | Ambitious work with heavy judgement | Start with frequent, checkable work |
| Tied to one vendor's tooling | Cannot be governed or moved | Keep briefs, sources and bounds yours |
The fourth and fifth rows cause the most stalls in regulated firms. A compliance team that cannot see what an agent is permitted to do, or reconstruct what it did, is right to refuse.
What does a failing pilot look like next to one that succeeds?
Say two insurers pilot agents to help with motor claims.
The first builds a capable agent that reads claim notes and photos and recommends settlement amounts. Its builders run it on 50 tidy past claims and the recommendations look sensible. When the pilot ends, the claims director asks who will own it, the risk team asks how settlement recommendations will be limited, and the data team asks where photos are processed. There are no answers ready. Eight months later, the agent is still a demonstration.
The second starts smaller: an agent that prepares claim files, chases missing documents and drafts acknowledgements. The claims operations manager owns it from the start. Success criteria are written before work begins. The agent runs in shadow on live claims, then is certified on 300 past claims at depth 2. Bounds are set outside the model and every verdict is filed. Risk signs because it can see the limits and the record. The agent is in production within three months, and settlement recommendations become the next project, with a working method behind them.
The first pilot was more impressive. The second one changed how the company works.
What should you do differently?
- Design the pilot to produce the evidence production needs: ownership, bounds, certification results and a record.
- Involve risk, security and data protection in week one, not after the demo.
- Measure return with reviewer time included, as in how to measure the ROI of an AI agent.
- Choose suppliers on governance, not demos, using the questions in how to choose an AI agent vendor.
- Plan the route to production before the pilot starts, including the depth the agent will hold on day one.
The full sequence is in our guide to deploying an AI agent.
Where Pryme fits
Pryme is built around the question pilots usually skip. Authority is granted per capability and per customer, proven on the customer's own cases through Attest, enforced at the moment of action, and it expires. That gives risk and compliance something concrete to approve.
Questions people ask
Is a failed pilot a waste of money?
Not if it teaches you what production requires. It becomes a waste if the next pilot repeats the same design and stalls in the same place.
How long should a pilot run?
Long enough to include a full business cycle on live work in shadow, usually two to three months. Shorter pilots tend to show only the demonstration.
Should a pilot be run by an innovation team?
It can be supported by one, but the operational owner should run it. Pilots owned outside operations struggle to find a home when they end.