Split the training into three parts that fit shift work. Reviewers learn the concepts from audio or reading lessons in quiet hours. Short live sessions at shift handover cover what changed and practise on real held items. Calibration is recorded, so night and weekend staff decide the same cases and hear the same discussion. Completion is tracked by shift.
What is blended training for agent reviewers, and who owns it?
Blended training mixes self-paced material with live sessions. For teams that review an agent's held actions around the clock, it is the only format that works: nobody can gather a 24-hour team in one room, and a single annual course is out of date the first time the agent's depth or bounds change. The general idea is explained in what blended learning is.
The training covers three things. The concepts: depth, the decision chain, held, refused and withheld verdicts, provenance, certificates that lapse, and the record. The practice: deciding real held items correctly and fast. And the shared standard: deciding the same case the same way whatever the shift.
A training lead owns the programme. Each shift has a shift lead who runs the live sessions and a champion who facilitates calibration. The operations manager signs off that each shift is covered.
The steps
- Person (training lead): maps the shifts and finds the quiet windows in each: the hour after the night peak, the mid-afternoon dip, the weekend early morning.
- Person (each reviewer): works through the concept lessons alone, reading or listening as audio during quiet windows or on the way to work. One module a week is a realistic pace.
- Person (each reviewer): completes the practice labs on a free practice workspace where the agent runs without touching live work.
- Person (shift lead): runs a 15-minute live session at each handover once a week. It covers what changed in the agent's bounds or sources and works through three real held items from the previous shift.
- Agent, at its current depth: keeps working the live queue. At depth 2 every reply or action is drafted and held; at depth 3 routine work proceeds within bounds and exceptions are held. The live sessions use those held items as material.
- Person (champion): runs calibration every two to four weeks. Each reviewer decides the same ten held cases alone, on their own shift. The champion leads a discussion with whichever shift is on, records it, and the other shifts listen to the recording and add written comments within a week.
- Person (training lead): publishes the agreed standard from each calibration and sends it to whoever owns the agent's brief.
- Person (operations manager): checks completion by shift, not just overall, before any increase in the agent's depth.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Map shifts | Nothing | Current | Finding quiet windows |
| Concept lessons | Nothing | Current | Learning at own pace |
| Practice labs | Runs in practice workspace | Practice | Deciding practice cases |
| Handover sessions | Supplies held items as material | 2 or 3 | Discussing real decisions |
| Live queue | Works within bounds; holds exceptions | 2 or 3 | Reviewing held items |
| Recorded calibration | Nothing | Current | Agreeing the standard |
| Completion check | Depth change waits on it | Current | Sign-off by shift |
Why record calibration rather than repeat it?
Repeating a calibration session on every shift produces three slightly different standards. One recorded discussion, with written comments from the other shifts, produces one. The night shift may raise a point the day shift missed; the comment goes to the champion, who updates the standard for everyone.
Recording also helps with a common problem: the night shift's decisions differ from the day shift's because night reviewers have fewer people to ask. A shared recording gives them the reasoning, not just the outcome.
What controls does blended training need, and what evidence do they leave?
Completion by shift. Evidence: each reviewer's lessons, labs and module tests, grouped by rota.
Live session attendance. Evidence: the shift lead's note of who attended and which held items were discussed.
Calibration. Evidence: each reviewer's independent decisions, the recording, written comments and the agreed standard.
Depth changes. Evidence: the operations manager's sign-off that every shift was trained before the agent's depth rose. Reviewers without current training should not hold approval rights at the new depth.
What should the training lead measure?
- Completion of concept lessons and labs, by shift.
- Agreement on calibration cases, by shift, over successive sessions.
- Sampled review accuracy by shift: if nights are consistently lower, the blend is not reaching them.
- Time from a change in the agent's bounds to every shift having had a handover session on it.
Where does blended training go wrong?
Live sessions only on days. The training lead works days, so the live sessions happen on days. Nights get an email. Rotate the lead or give each shift lead the material and the authority.
Audio without practice. Listening to a lesson on held actions is not the same as deciding one. Every module needs a lab or a real held item.
Calibration that only one shift hears. If the recording is not shared and commented on, the night shift's standard drifts.
Ignoring rota changes. People move between shifts. Completion should follow the person, and the record should show who is trained on whichever rota they are on.
Teams spread across countries face the same problem with time zones and languages added; see how to train a team across several countries to oversee AI agents. Shift-heavy organisations put this into practice in their rollouts: compare rolling out training across a telecoms operator and across a hospital. For the shorter answer, see how to train reviewers who work different shifts, and for the full method, training teams for AI agents.
Where the Academy fits
Pryme Intelligence Academy's lessons can be read or listened to as audio, with highlights and notes, and every lab runs in a free Build workspace, which suits reviewers who learn in fragments across shifts. Module tests are set in the learner's own industry and role. For teams on paid plans, an admin view shows progress, results and certificate expiry for each person.
Questions people ask
How long should each live session be?
Fifteen to twenty minutes at handover. Longer sessions eat into the shift and people stop attending.
Can calibration be done fully asynchronously?
The independent decisions can. The discussion works better live for at least one shift, recorded for the others, so there is real argument to hear.
Should night staff be paid for training time?
They should train in paid time, as day staff do. Expecting nights to train on their own time is a common reason blended programmes fail.