Less supervising output, more managing judgement. A team lead with agents on the team plans a reviewer rota by decision type, runs calibration so reviewers judge alike, coaches people on when to approve, edit or escalate, and reads a new set of numbers. The team does fewer tasks and makes more decisions, and the lead manages that shift.
Why does the rota change?
Because held items need named people with the authority to decide them. A held payment above a limit cannot wait for whoever is free; it needs a reviewer cleared to approve that type and value. So the rota is built by decision type, not just by hours or channel.
A simple rota covers four questions for each shift: who reviews routine held items, who decides high-value or sensitive ones, who handles escalations from reviewers, and who covers if an agent drops depth. That last one matters because certificates lapse after 90 days and a run that clears no depth drops the agent to depth 1 the same day. A lead who knows a re-certification is due on Thursday schedules more people on Thursday.
What is calibration and how often should it run?
Calibration is getting reviewers to reach the same decision on the same item. Without it, the outcome for a customer depends on who happened to review their case.
Run it weekly at first. Take five to ten real held items from the record, have reviewers decide them independently, then compare. Where decisions differ, settle the right answer and the reason, and write it down. Once reviewers agree consistently, move to fortnightly or monthly, and run an extra session after any incident or change to limits.
How does coaching change?
The lead used to coach people to do the work better: write clearer replies, process faster, make fewer errors. Now the agent prepares most of the work, so coaching targets the decision.
| Pattern in the record | What it suggests | Coaching focus |
|---|---|---|
| Approves nearly everything, very quickly | Not reading the facts behind drafts | Provenance, red lines, slowing down on external facts |
| Edits nearly every draft | Doing the agent’s job, or the agent needs fixing | Check with the agent owner first, then coach on accepting good drafts |
| Escalates often | Unsure of authority | Clarify the approver rule; pair with a senior reviewer |
| Long waits on held items | Overloaded or avoiding hard calls | Rota and confidence |
The record makes these patterns visible per person, which is fairer than the old practice of sampling a few tickets.
Which numbers does a lead read now?
A short set, daily:
- Held items waiting, by type and age.
- Refusals by bound: role scope, certificate, sources, the business’s limits, platform guardrails. A sudden cluster usually means a broken feed or a misset limit.
- Reviewer overturn rate per person and per task.
- Certificate dates for each agent on the team.
- Red line escalations.
Output volume still matters, but it no longer says much about the team’s quality on its own.
What stays the same?
People still need one-to-ones, development and recognition. And the lead still owns the team’s outcomes. What is different is that some of those outcomes are produced by agents working within limits the lead does not set. When limits look wrong, the lead brings evidence from the record to the owner of the limit and to the governance lead; see what an AI governance lead does and how the role is defined.
HR will be redesigning roles at the same time; what an HR team needs to know when agents join the workforce covers their side. A competency matrix, such as the AI agent competency matrix for an IT team, shows the structure a lead can use for any team. New leads in support teams can follow onboarding a new support team lead. The overview is in AI agent skills by role.
Where the Academy fits
Team leads are a named audience for the Operator track in Pryme Intelligence Academy, whose modules include human review, reading the workspace and running the rhythm. The admin view shows each person’s progress, results and certificate expiry dates, and a business can choose to make approval rights depend on a current certificate.
Questions people ask
Should the team lead review items too?
Usually yes, for the hardest cases and for calibration. A lead who never reviews loses touch with what the team sees.
How many reviewers does a team need?
Size from held volumes by type and time per decision, measured over the first weeks, plus cover for certificate lapses.
Who decides if an agent’s limit is wrong?
The owner of that limit, on evidence. The lead’s job is to spot it and bring the record.