What is coaching?

Coaching is one-to-one help to improve how a person does their work, built on their own real cases rather than general material. A coach asks questions, reviews decisions with the person and agrees what to change. For people who review AI agents, coaching starts from the record of the held actions they decided and the reasons they gave.

Why does coaching matter for agent oversight?

Because the skills that matter most are judgement skills, and judgement improves through feedback on your own decisions. A course can explain why an emailed invoice cannot support a payment. Coaching sits with the reviewer, opens the three held payments they approved last week on emailed evidence, and talks through what they saw and what they missed. That conversation changes behaviour in a way the course alone rarely does.

Agents make coaching more exact. Every verdict and every human decision is filed, so a coach does not have to rely on memory or impressions. They can see how long a reviewer takes, how often they correct the agent, whether their declines carry reasons, and how their decisions compare with colleagues on similar cases.

How does coaching work in practice?

A coaching cycle for a reviewer usually runs like this:

1. Pick a focus from the record, such as approving too fast or not escalating disputed facts.
2. Review real cases together. Five to ten decisions, discussed one by one.
3. Ask before telling. What did you see? What made you decide? What else could it have been?
4. Agree one or two changes. Specific and checkable.
5. Check again after two to four weeks, using the record.

The reviewer coaching plan template gives a format for writing this down.

An example

Take a hypothetical outsourcing provider running a review queue for a client's refunds. A team lead notices that one reviewer has declined twice as many held refunds as colleagues, with brief reasons. In a coaching session, they open eight of the declines together. The reviewer had been treating every refund to a card different from the original payment as suspicious. The house rule only requires extra checks when the card was added in the past week. They agree that he will check the card's added date before declining. Two weeks later his decline rate is in line with the team's. Whether outsourced staff should be on the queue before training is covered in can outsourcing providers use AI agents before staff are trained.

How is coaching different from training evaluation?

Coaching improves an individual's performance. Training evaluation judges whether a training programme worked, across everyone who took it. They use some of the same evidence, such as decisions in the record, but for different purposes. Evaluation might show that a programme left reviewers weak on provenance; coaching fixes it for each person. Frameworks such as the Kirkpatrick model belong to evaluation.

Coaching also differs from calibration. Calibration brings a group together to agree a standard on shared cases. Coaching works with one person on their own cases. A team usually needs both: calibration sets the standard, coaching helps individuals meet it.

Who should coach?

Team leads, senior reviewers and champions with current standing on the queue. Coaching takes time, so leads need it built into their week, not added on top. The guide to training teams for AI agents covers how to plan for it.

Where the Academy fits

Coaching picks up where a course ends. Pryme Intelligence Academy's Operator track includes a two-week practicum of real review decisions signed off by a Pryme assessor; teams that coach well during that fortnight find the practicum becomes the start of a habit rather than a one-off test.