A practicum is a period of supervised real work that serves as the final proof of a reviewer's skill. The reviewer decides real held actions on the live queue, an experienced person checks each decision, and the run is assessed for accuracy and, above all, no missed red line. Courses teach concepts; the practicum shows they can be applied.
Why is a practicum needed?
Because the job is judgement on real cases, and only real cases test it. A written exam can show that a reviewer knows what provenance means. It cannot show that they will notice, at the end of a long shift, that the only support for a held payment is an emailed invoice. Labs with sample data come closer, but sample cases are tidier than real ones and carry no consequence.
The general idea of a practicum comes from professions such as teaching and clinical work, where people must show competence on the job before being trusted with it alone. Agent review fits the same pattern. The reviewer's decision is the control on a held action; the business should see that control work before relying on it.
How does it work in practice?
The reviewer works the live approval queue for a set period, usually two weeks, making real decisions. For each one, they record what they decided and why. A supervisor checks every decision, or a defined sample of routine ones and every non-routine one. At the end, the run is assessed:
- Accuracy. What share of decisions were correct, judged by the supervisor and, where needed, by the process owner.
- Red lines. Whether any decision crossed a line that must never be crossed, such as approving a payment on external evidence alone or releasing data to an unverified person. One miss is a fail, however accurate the rest.
- Reasons. Whether declines and edits carry reasons that would teach the builder and survive an audit.
- Escalation. Whether the reviewer passed on what they should not decide.
The reviewer is productive throughout. The cost is the supervisor's time, not lost queue capacity.
What does a practicum record look like?
A log, one line per decision, with the case reference, the agent's proposed action, the facts and their provenance, the reviewer's decision and reason, and the supervisor's check. Logs differ by role. An operations manager's log includes calibration sessions they ran and cover decisions; an IT administrator's includes access and connection changes they approved. Formats are in the practicum log template for operations managers and the practicum log template for IT administrators.
How is it different from the other assessments?
| Assessment | What it shows | Where it happens |
|---|---|---|
| Module tests | Understanding of each concept | Online |
| Written exam | Knowledge across the whole track | Online, timed |
| Labs | Skill on sample cases | A safe workspace |
| Practicum | Reliable judgement on real cases, with no missed red line | The live queue |
Only the last one involves real consequences, which is why it comes last and why it is the hardest to pass.
When should it run?
Ideally during the agent's shadow period, when it runs at depth 1 or 2 and every action passes through a person anyway. Real volume, no extra risk. Running it before go-live also means the reviewer is certified by the time the agent acts at depth 3. For queues with low volume, extend the period until there are enough decisions to judge; two weeks with a handful of cases proves little.
An example
Take a hypothetical collections team. A collections agent drafts arrangements, reminders and hardship referrals, and holds anything involving vulnerability or a dispute. A new reviewer spends two weeks deciding those held drafts. The supervisor checks every hardship case and a sample of routine reminders. On day six, the reviewer approves a reminder to a customer who had mentioned job loss in an earlier message; the agent had held it for that reason. That is a missed red line, so the practicum is failed and restarted after coaching, however good the other decisions were. The full method for that team is in how to run a two-week practicum for collections agents, and for a technical role in how to run a two-week practicum for IT administrators.
Where the Academy fits
Pryme Intelligence Academy builds the practicum into the Certified Pryme Operator credential: two weeks of real review decisions, at least 95 correct in every 100 and no missed red line, signed off by a Pryme assessor. It sits alongside a written exam of 40 questions in 60 minutes, with 32 correct needed to pass.
Questions people ask
Can a practicum be done on sample data?
Not if it is to prove anything. Sample data belongs in labs. The point of a practicum is real cases with real consequences, under supervision.
Who should supervise?
Someone certified and experienced on that queue, with the process owner available for disputed decisions. The supervisor should not be the builder of the agent being reviewed.
What happens after a pass?
The reviewer may approve held actions alone, and calibration keeps their judgement aligned. The guide to training teams for AI agents covers what follows.