Resolution rate rewards an agent for closing conversations, whether or not they stayed closed. To measure a support agent properly, add what it hides: correct refusals, holds a person agreed with, promises kept, repeat contacts and the quality of handovers. Together they show whether the agent did the right thing, not merely whether it finished something.
What does resolution rate miss?
Resolution rate counts conversations marked as resolved. It is a sensible starting point and a poor finishing one. Its blind spots are specific:
- It trusts the close. A conversation the agent marks resolved counts, even if the customer comes back the next day with the same problem.
- It penalises correct refusals. A request the agent rightly declined and handed over looks like a failure to resolve.
- It ignores what was promised. An agent that resolves conversations by committing to things the firm then fails to deliver scores well.
- It counts speed, not quality. Closing quickly with a wrong answer and closing slowly with the right one look the same.
An agent tuned to maximise resolution rate will close conversations eagerly, decline to hand over and promise whatever ends the chat. Each of those makes the number better and the service worse. The same trap applies to its cousins, containment rate and deflection rate; the case that deflection is not resolution is part of the same argument.
Which measures should sit beside it?
Five additions cover most of what resolution rate misses.
| Measure | What it tells you | How to collect it |
|---|---|---|
| Repeat contact | Whether "resolved" stayed resolved | Same customer, same issue, any channel, within a set window |
| Correct refusals | Whether the agent declined the right requests, and only those | Reviewer sample of refused and handed-over conversations |
| Holds agreed with | Whether drafts the agent held were good enough to send | Share of held drafts approved without edits, by request type |
| Promises kept | Whether commitments the agent made were delivered | Match commitments in replies against later events |
| Handover quality | Whether people could continue without asking the customer to repeat | Reviewer sample of handovers |
Two familiar measures stay on the scorecard as context. Time to resolution shows whether customers wait too long, though it should be read alongside repeat contact, since a fast close that fails is not fast. And escalation rate shows how often the agent hands over; the question of how high an AI support agent's escalation rate should be has no single answer, but it should be interpreted, not minimised.
How do you measure a correct refusal?
By review, because only a person can say whether a refusal was right. Take a regular sample of conversations where the agent declined, held or handed over. For each one, a reviewer marks whether the decision was correct, too cautious or not cautious enough.
This produces two numbers worth watching. Over-refusal, where the agent declined something it should have handled, costs customers service and staff time. Under-refusal, found by sampling conversations the agent resolved, is where it handled something it should have declined. The second is rarer and more serious, and resolution rate actively hides it, because those conversations count as successes.
Why is repeat contact such a useful check?
Because it is the customer's own verdict. A customer who comes back about the same issue is telling you it was not resolved, whatever the agent recorded. Repeat contact is also hard to game: it depends on what customers do after the conversation ends, not on how the agent labels the conversation.
Two design choices matter. First, count across channels. A customer who chatted, then phoned, has made a repeat contact even though the chat channel never sees it. Second, fix the window. Too short and you miss customers who wait a few days; too long and you catch unrelated new issues. Choose a window that suits the product, and keep it stable so trends mean something.
How should the measures be read together?
As a set, because they trade off against each other. Push correct refusals up by making the agent more cautious and resolution may fall. Push resolution up and repeat contact may rise. The point of a scorecard is to make those trade-offs visible so that nobody optimises one number at the others' expense.
A useful habit is to read changes in pairs:
- Resolution up, repeat contact flat: the agent is improving.
- Resolution up, repeat contact up: the agent is closing conversations it should not.
- Escalation down, under-refusals up: the agent is holding on to cases it should release.
- Holds agreed with rising, by request type: those types may be ready for more depth.
The last pairing is where measurement connects to authority. Evidence that held drafts are consistently accepted is exactly what certification needs before a capability can move from drafting to sending.
What should you report, and to whom?
Support leaders need the full scorecard weekly. Executives need a short version that cannot be misread: resolution next to repeat contact, and escalation next to correct refusals. Risk and compliance need under-refusals and promises kept, because those are where customer harm and firm liability arise.
Whatever the audience, avoid reporting resolution rate alone. A single number invites a single target, and a single target invites the agent's designers to optimise for it.
What should you do differently?
Add repeat contact and correct refusals to your support agent's scorecard first; they are the two that most change how the agent looks. Then add holds agreed with, promises kept and handover quality as your review process matures. Remove resolution rate from any target that stands alone. A support agent measured this way is pushed towards the behaviour described in the AI support agent that knows when to say no.
Questions people ask
How large should the review sample be?
Large enough to see each request type and each kind of refusal several times a week. Increase sampling for request types with the most authority or the most complaints.
Can an agent do the reviewing?
A separate agent can pre-screen and flag likely errors to speed review. Final judgements on whether a refusal or resolution was correct should be made by people.
What about customer satisfaction scores?
Keep them as context. They tell you how customers felt, which matters, but they are too broad to tell you what to fix.