Complaints about an AI agent are the most precise feedback a support team gets. Each one points at a specific place where the agent's bounds, sources or brief fell short: a refusal that was wrong, a promise it should not have made, a fact it did not have. Treat them as test cases, not noise.
What do complaints about the agent reveal?
Ordinary complaints tell you about products, prices and processes. Complaints about the agent tell you about the agent, and because the agent's behaviour is defined in a small number of places, each complaint can usually be traced to one of them.
| What the customer says | What it usually points to | Where to fix it |
|---|---|---|
| "It told me something that turned out to be wrong" | A stale, missing or misapplied source | Sources and their versions |
| "It wouldn't help with something simple" | Over-refusal or bounds set too narrow | Bounds or the brief's refusal guidance |
| "It promised me something and nobody followed through" | A commitment the agent had no authority to make | Bounds on promises; certification cases |
| "I had to explain everything again to a person" | A poor handover | The handover note and its required fields |
| "It kept going round in circles" | No clear escalation trigger | Escalation rules in the brief and bounds |
| "I didn't realise it wasn't a person" | Inadequate disclosure | How the agent introduces itself |
Each row maps to something you can change, test and verify. That is unusual for customer feedback, which is usually diffuse. Agent complaints are diagnostic.
Why are they better than satisfaction scores?
Satisfaction scores average everything. A customer who rates a conversation two out of five might be unhappy with the agent, the answer, the product, the price or their day. You learn that something was wrong, not what.
A complaint about the agent is specific by nature. The customer has taken the trouble to say what went wrong, usually with an example. Combined with the verdict record of the conversation, which shows what the agent did, which bounds were checked and which sources it relied on, the complaint becomes a reproducible case. You can see exactly where the agent went off course.
There is a selection effect worth acknowledging. Most customers who have a poor experience do not complain. So complaints undercount problems, and the ones you receive are a sample of a larger set. That makes each one more valuable, not less.
Should the agent handle complaints about itself?
No. It can recognise and log them, acknowledge them in approved wording and gather the conversation record. It should not investigate or decide them. The question of whether an AI agent can handle complaints about itself comes down to conflict: an agent judging whether its own behaviour was wrong is marking its own homework.
A person should review the conversation and decide whether the complaint is upheld. The method is set out in how to handle complaints about an AI agent's own decisions. The wider question of whether an AI agent can handle customer complaints at all has a more nuanced answer, but complaints about the agent itself are the clearest case for a person.
In regulated markets the usual complaint-handling obligations apply whatever the complaint is about. A complaint about an agent's conduct is still a complaint, with the same deadlines and the same expectation of a fair outcome. The limits on what an agent may do in that process are covered in FCA complaints handling: what an AI agent may do.
How do you turn a complaint into evidence?
In four steps, ideally within days of the complaint arriving.
- Pull the record. Retrieve the conversation and the verdicts filed for it: actions attempted, bounds checked, sources relied on, the deciding bound for each.
- Find the cause. Decide which of the categories above the complaint falls into, and confirm it against the record.
- Make it a case. Add a version of the conversation, with personal data handled properly, to the certification set, labelled with the outcome that should have happened.
- Fix and rerun. Change the source, bound or brief, then run at least the affected slice of certification to confirm the fix holds.
Step three is what makes complaints compound. Each complaint becomes a permanent test. Over time, the certification set fills with exactly the cases that once went wrong, and the agent is checked against all of them every time it is recertified.
Do complaints about the agent always mean the agent was wrong?
No, and it is important to say so. Some complaints are about a correct refusal: the customer wanted something outside policy and the agent declined, properly. Those complaints are still useful. They show where customers' expectations differ from the firm's rules, which may point to a communication problem, a policy worth revisiting, or simply a refusal that could be explained better.
A refusal that is correct but poorly explained is a fixable problem. The agent can be taught to give a reason and a next step. A correct refusal that customers keep complaining about may be a signal for the policy owner, not the agent owner.
What should be reported upwards?
Agent complaints deserve their own line in complaint reporting, separate from complaints about products or staff. Report the count, the categories, how many were upheld, and what was changed as a result. The last item is the most useful to a board or risk committee, because it shows the loop working: complaint, cause, fix, test.
Firms that bury agent complaints inside general categories lose this signal, and often discover a pattern only when it has become large.
Where Pryme fits
In Pryme, every conversation's verdicts are filed in an append-only record, so a complaint can be traced to the exact bound, source or brief instruction involved. Upheld complaints can be added to the customer's certification set in Attest, so each one becomes a permanent test. This follows the principle behind the AI support agent that knows when to say no.
Questions people ask
How quickly should a complaint about the agent lead to a fix?
The investigation should follow normal complaint timelines. If the cause is a wrong source or a bound that let through a harmful action, fix it at once rather than waiting for the next recertification.
Should the customer be told what was changed?
They should be told the outcome of their complaint and, where appropriate, that the issue has been corrected. Detailed configuration changes are internal.
What if complaints about the agent are rare?
That is good news, but check that customers can complain easily and that complaints about the agent are being recognised as such. Rare can mean well-behaved or hard to report.