How to train the collections team to review AI agent decisions

Collections reviewers need four stages: lessons on how the agent sets up arrangements and why it holds, calibration on real cases where affordability or vulnerability was in doubt, two weeks of shadowed review with every decision checked, and sign-off. The standard should weigh customer outcomes as heavily as recovery: a reviewer who approves unaffordable plans has not passed.

What does the collections team review?

A collections agent typically reads account and payment history, drafts reminders within contact rules, sets up payment arrangements inside an approved grid at depth 3, records promises to pay, and holds anything outside its bounds: plans below the grid, write-offs, disputes, complaints and any case showing signs of vulnerability. It does not decide on forbearance beyond the grid, and it never decides how to treat a vulnerable customer.

So reviewers face three kinds of work. Held cases, where they decide. A sample of arrangements the agent set up itself, where they check the plan was affordable and the customer understood it. And the agent's messages, sampled for tone and accuracy. The second is where training usually falls short, because an arrangement the agent set up looks finished.

The head of collections owns the programme. Conduct risk co-signs the sign-off standard. The agent's owner attends calibration.

What are the steps?

1. Learning, week 1: collectors complete lessons on how the agent decides: the grid as a bound, the customer's own limits, the platform floor, and the four verdicts.
2. Agent, depth 1: assembles a practice set of past held cases and self-arranged plans with outcomes, including some plans that later broke.
3. Calibration, week 2: collectors decide alone, then compare. Affordability and vulnerability cases come first.
4. Shadow, weeks 3 and 4: collectors review live held items and samples; a team leader checks every decision before it takes effect.
5. Agent, depth 3 in live work: arranges within the grid, drafts contact, holds the rest.
6. Sign-off: head of collections and conduct risk assess shadow decisions on accuracy and customer outcome.
7. Ongoing: weekly sampling of each collector and monthly calibration.

What should reviewers check?

Affordability. The plan must fit the customer's income and essential spending. Reviewers check which figures the agent used and where they came from: a verified bank feed, or the customer's own statement on a call. A plan resting entirely on declared figures may still be right, but the reviewer should know that.

Vulnerability signals. A mention of illness, bereavement, mental health, job loss or caring duties. The agent should hold; reviewers check it did, and handle the customer appropriately before any collection decision.

Contact rules. Time of day, frequency, channel, and any request to stop or change contact. Reviewers sample messages against the rules.

Tone. Accurate messages can still read as threats. Reviewers flag wording that pressures rather than informs.

Disputes and complaints. A customer who says "I don't owe this" has raised a dispute; collection should pause until it is resolved. Reviewers check the agent recognised it.

Promises to pay recorded correctly. Wrong dates or amounts lead to wrong follow-up contact.

Take one illustrative case through calibration. The agent set up a plan of 150 a month for a customer whose stated income, given on a call, left 200 a month after essentials. The plan sits inside the grid, so the agent acted at depth 3 without holding it. Half the collectors approve it in review; half query it, because the customer also mentioned a second debt the agent did not count. The gold standard the team agrees: where a customer mentions other debts, the agent holds the plan for a collector, and the brief is changed to say so.

Which controls leave evidence?

• Lessons before shadow. Evidence: completion dates.
• Every shadow decision checked. Evidence: team leader's name.
• Sign-off includes conduct risk. Evidence: both names on the assessment.
• Self-arranged plans sampled weekly. Evidence: sample log with findings.
• Vulnerability handled before collection. Evidence: the vulnerability record precedes the decision.

What should you measure?

• Shadow accuracy per collector.
• Arrangements broken within three months, split by agent-arranged and collector-arranged.
• Vulnerability flags missed in sampling.
• Contact rule breaches found in samples.
• Complaints about contact or tone.

Where does it usually go wrong?

Recovery targets distort sign-off. If the standard rewards plans agreed, collectors approve unaffordable ones. Weight outcomes.

Samples are skipped. Self-arranged plans look finished. Protect sampling time.

Vulnerability training is separate. Collectors learn vulnerability once at induction and agent review later. Teach them together, on the same cases.

Tone is left to taste. Build an approved library of good and bad wording, owned by conduct risk.

How an individual collector reviews is covered in how collections agents review an AI agent's decisions. The reviewer onboarding checklist for the collections team and the AI agent training plan template for collections agents set out the schedule. Risk teams follow the same four stages from the second line; see how to train the risk team to review AI agent decisions. The full programme is in training teams for AI agents.

Where the Academy fits

Pryme Intelligence Academy's Operator track mirrors these stages: lessons, labs in a free Build workspace, then a practicum of two weeks of real review decisions signed off by a Pryme assessor, with no missed red line allowed. For collections, missing a vulnerability signal is exactly that kind of red line. Module tests are set in the learner's own industry and role.