Route by risk, not by rank. Junior reviewers take drafts and routine held actions backed by sealed or signed sources. Senior reviewers take depth 4 actions, anything resting on declared or external evidence, vulnerable customers and patterns the agent has not seen before. Seniors sample juniors' decisions each week, and juniors move up as their certificates and record allow.
What is the split, and who owns it?
An agent's review queue mixes easy and hard items. A draft reply about delivery times sits next to a held refund for a customer who mentioned a bereavement. If everything goes to seniors, they become a bottleneck and juniors never learn. If everything goes to whoever is free, a junior ends up deciding the hardest case of the week. The split decides which held action goes to whom.
The team lead owns the routing, agreed with the process owner. The agent owner configures the agent to tag held actions with the signals the routing needs. Our explainer on the difference between an agent reviewer and an agent approver helps here: reviewers correct and send drafts; approvers take responsibility for actions. Our guide to AI agent skills by role covers what each needs to know.
What are the steps?
- Team lead: lists the risk signals. Depth of the action; amount; provenance of the key facts; any sign of vulnerability; whether the case type is new to the agent; whether the agent's draft sits close to a refusal.
- Agent owner: has the agent tag each held action with those signals, using what it already knows from its bounds and sources.
- Team lead: writes the routing. Juniors: depth 2 drafts and depth 3 holds where every key fact is sealed or signed. Seniors: depth 4, declared or external evidence, vulnerability, novelty.
- Team lead: writes the escalation route. A junior who is unsure passes the item up with a one-line reason. Passing up is never counted against them.
- Senior reviewers: sample juniors' decisions each week, looking for patterns rather than single errors.
- Team lead: sets the path up. Juniors take on a wider band when their certificate covers it and their sampled decisions show they are ready.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Tag risk signals | Marks depth, amount, provenance, flags | 1 | Nothing |
| Route draft | Sends to junior queue | 2 | Junior corrects and sends |
| Route routine hold | Sends sealed-source holds to juniors | 3 | Junior approves |
| Route high-risk hold | Sends to senior queue | 4 | Senior approves |
| Escalate | Moves item up with junior's reason | any | Senior decides |
| Sample | Pulls a sample of junior decisions | 1 | Senior reviews |
Why is provenance the best routing signal?
Amount is the obvious signal, and it matters. But the most useful single signal is often where the key facts came from. A refund backed by a sealed order record is routine at almost any size within limits. A small refund backed only by a customer's declared statement that the item never arrived needs judgement. A payee change backed by an external email needs scepticism. Juniors can handle the first well; the second and third are where experience pays.
Routing on provenance also teaches. Juniors learn that the strength of the evidence, not just the size of the action, is what makes a decision hard. When they ask to move up, the question becomes whether they handle declared and external evidence well, which seniors can see in the sample.
Avoid sending everything to the most senior person. As don't let the most senior person approve everything argues, that removes the second look and makes the team fragile.
Which controls make the split work, and what evidence do they leave?
Routing written down. Evidence: the routing table, versioned, with the signals used.
Signals visible on each item. Evidence: the tags on each held action in the record.
Escalation recorded with a reason. Evidence: escalated items with the junior's note.
Weekly sampling. Evidence: the sample, findings and any changes to routing.
Bands tied to certificates. A business can make each routing band depend on a current certificate for that depth. Evidence: reviewer certificate status on each decision date.
What should you measure?
- Senior queue length and wait, against your targets. A long senior queue means routing is too cautious or seniors too few.
- Escalations by junior, which should fall as they gain confidence, without dropping to none.
- Errors found in sampling, by signal. If juniors struggle with one kind of item, route it up or train for it.
- Juniors moved up a band, as a measure of whether the team is growing its own seniors.
How to staff an AI agent review queue and what a review queue is cover the capacity side.
Where does it usually go wrong?
Seniors take the easy items. When their queue is quiet, seniors pick from the junior queue and juniors get less practice. Keep queues separate.
Escalation feels like failure. Juniors stop passing items up because it looks bad. Say plainly that a good escalation is a good decision.
Sampling stops. Once juniors seem fine, seniors stop looking. Errors creep in quietly, and nobody notices until a customer complains or an auditor asks how juniors' decisions were checked.
Middle managers are left out. Team leads decide routing but managers above them change targets without asking. As middle managers decide whether AI agents work argues, the people running the queue need a say.
Questions people ask
Should juniors ever approve depth 4 actions?
Once certified for that depth and with a clean sampled record, yes, within a band. Depth 4 is not reserved for seniority; it is reserved for demonstrated competence.
How big should the weekly sample be?
Large enough to see patterns, small enough to be done properly. For most teams, a few dozen items per junior reviewer per week is workable.
What if the agent mis-tags an item?
Reviewers should flag it and pass the item to the right queue. Mis-tags are corrections to bank with the agent owner, because routing depends on them.