Build agent review into the contact centre's existing rhythm rather than beside it. Teach it in induction, calibrate advisers alongside the quality team using real held contacts, use the nesting period as the shadow stage with every decision checked, and sign off advisers before they handle held items alone. Repeat calibration monthly, because turnover means new reviewers arrive constantly.
What will advisers review, and why is the contact centre different?
A contact centre agent typically answers routine chats and messages, authenticates customers, handles simple requests at depth 3, drafts replies on sensitive matters, and holds refunds above limit, account changes, complaints and anything suggesting vulnerability. Advisers decide the held items and take escalated contacts. The quality team samples both the agent's handled contacts and advisers' decisions.
Three things set the contact centre apart from most other teams that review agents. Scale: many advisers, many shifts, sometimes several sites. Turnover: new advisers join every month. And an existing training machine: induction, nesting, quality scoring and coaching already run. A review programme that ignores that machine will be bypassed. One that uses it will stick.
The head of customer operations owns the programme. The quality lead co-owns calibration and sign-off. The agent's owner briefs trainers on every change to the agent's bounds.
What are the steps?
- Induction: new advisers complete lessons on how the agent decides, what each verdict means, and what to do with each type of hold.
- Agent, depth 1: assembles a practice set of held contacts with their outcomes, anonymised, refreshed monthly.
- Calibration, end of induction: advisers decide the set alone, then compare in a session run jointly by a team leader and a quality analyst. The quality scorecard and the gold standards must agree.
- Nesting, two weeks: advisers handle live held items with a nesting coach checking every decision before it takes effect.
- Agent, depth 3 in live work: handles routine contacts, holds the rest.
- Sign-off: team leader and quality analyst assess nesting decisions against the standard. Advisers who pass handle held items alone.
- Ongoing: quality sampling of each adviser's held-item decisions, and monthly calibration across sites and shifts.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Induction lessons | Nothing | none | New advisers |
| Practice set | Lists held contacts and outcomes | 1 | Trainer anonymises |
| Calibration | Nothing | none | Advisers, team leader, quality |
| Nesting | Handles routine; holds sensitive | 3 | Adviser decides; coach checks |
| Sign-off | Nothing | none | Team leader, quality analyst |
| Quality sampling | Lists decisions to score | 1 | Quality team |
What must advisers learn to check?
Refunds and credits above limit. Is the request within policy? Is there a pattern of repeat claims on the account? The agent has held it; the adviser decides with the full history.
Account changes. Changes to contact details, especially after failed authentication, are a common account takeover route. Advisers learn the verification standard and never skip it.
Complaints. A customer's dissatisfaction may be buried in a routine message. Advisers check that the agent recognised it and route it correctly.
Vulnerability. Signs the agent flagged, and signs it may have missed. Advisers handle the customer first.
What the agent already told the customer. Before deciding a held item, read the conversation. If the agent set an expectation, the adviser needs to know.
Routine contacts, through quality sampling. Did the agent give the right information? Was it clear the customer was talking to an AI system? In the EU, the AI Act requires people to be told when they interact with one.
Which controls leave evidence?
- Lessons in induction records. Evidence: completion before nesting.
- Nesting decisions checked. Evidence: coach's name on each.
- Sign-off by team leader and quality. Evidence: both names.
- Scorecard aligned with gold standards. Evidence: the scorecard version references the calibration library.
- Held-item rights tied to sign-off. Evidence: the access list matches the sign-off register.
What should you measure?
Measure cohorts as well as individuals. If one month's intake scores well below the last, the cause is usually in training, not in the people.
- Nesting accuracy per adviser and per cohort.
- Quality scores on held-item decisions after sign-off.
- Variance across sites and shifts on calibration cases.
- Account takeover incidents traced to adviser decisions.
- Repeat contacts after a held item was decided, a sign the decision did not resolve the issue.
Where does it usually go wrong?
Quality and training disagree. The quality scorecard marks an adviser down for a decision calibration said was right. Run calibration jointly so they cannot drift.
Nesting coaches are not trained on the agent. Coaches who learned the job before the agent arrived coach the old way. Train coaches first.
Turnover outruns calibration. Monthly sessions with mixed cohorts keep new and experienced advisers aligned, and stop each cohort developing its own habits.
Night and weekend shifts drift. Smaller off-peak teams calibrate less and decide held items with fewer people to ask. Rotate calibration sessions so every shift hosts one, and collect independent decisions from absent advisers in advance.
Held items are treated as failures. Dashboards that count holds against advisers push them to clear holds without checking.
How managers review is covered in how contact centre managers review an AI agent's decisions, and their own training in the AI agent training plan template for contact centre managers. The reviewer onboarding checklist for the contact centre is the adviser-level version. Fraud teams run the same stages with sharper adversarial checks; see how to train the fraud team to review AI agent decisions. The full programme is in training teams for AI agents. Pryme Intelligence Academy offers seats with paid Pryme Intelligence plans and an admin view of progress and certificate expiry, which helps where cohorts change monthly.
Questions people ask
Should every adviser be signed off for held items?
Not necessarily. Some contact centres keep held items with a smaller, more experienced group. Either way, only signed-off advisers should decide them.
How do we handle outsourced sites?
The same stages, the same calibration library and joint sessions with your own team. Outsourced advisers deciding your customers' held items need your standard.
Does sign-off expire?
It should be refreshed when the agent's bounds change, and reviewed at least yearly through quality sampling.