First, the outcome is put right for whoever it affects. Second, the correction is filed with a reason beside the agent's original output. Third, corrections are reviewed together for patterns, which lead to a tested change in the agent's brief, sources or bounds. What should not happen is the agent quietly changing its own behaviour from each correction.
Why not let the agent learn from every correction?
It sounds efficient: a person fixes a draft, the agent notices and does better next time. In practice it means the agent's behaviour changes continuously, without anyone testing the change or deciding to make it. One reviewer's preference becomes the agent's rule. A correction made in a hurry becomes a pattern. And the certificate describes an agent that no longer exists.
Corrections are valuable evidence. They should change the agent through the same route as any other change: reviewed, written into a new version, tested on the full case set, then released.
What happens to a single correction?
- Reviewer: fixes the work and puts the outcome right. If a wrong draft was caught before sending, that is enough. If an action had already happened, the reviewer reverses or amends it. Evidence: the corrected output beside the original.
- Reviewer: picks a reason category: wrong fact, wrong source, missed escalation, out of scope, tone, format, or reviewer preference. Evidence: the category on the filed record.
- Agent, depth 1: logs any downstream records it touched, so related systems can be checked. Evidence: a list of affected records.
- Team lead: checks the category on a sample of corrections each week, since categories are only useful if they are applied consistently. Evidence: the sample and any recategorisations.
What happens to corrections as a group?
| Pattern in corrections | Likely cause | What changes | Who decides |
|---|---|---|---|
| Wrong facts from one source | Stale or low-provenance source | Source removed or reprioritised | Platform owner and process owner |
| Missed escalations of one kind | Escalation rule unclear | Brief updated | Process owner |
| Tone or format edits | Brief style guidance weak | New approved examples | Process owner |
| Actions outside intended scope | Bounds too wide | Bounds narrowed | Process owner |
| Edits that change nothing material | Reviewer preference | Reviewer calibration | Team lead |
The last row matters as much as the others. Some corrections say more about reviewers than about the agent. If one reviewer changes three times as much as colleagues, that is a calibration conversation, not a brief change. The guide on collecting feedback on an AI agent's work covers how to structure this.
What does that look like in a team?
Say a lender's operations team uses an agent to draft responses to customers asking for settlement figures, which a person sends. Over a fortnight, reviewers correct 60 drafts.
The categories show 35 tone edits, 15 corrections where the figure included a fee that had been waived, and 10 escalations the agent should have made because the customer mentioned hardship. The fee corrections trace to a source: the waiver sits in a notes field the agent was not permitted to read. The process owner approves adding that field as a source, and the hardship rule in the brief is rewritten. Both go into a new version, which is tested on the full case set, including the 25 corrected cases. Tone edits are reviewed separately; most turn out to be two reviewers' personal style.
Every customer who received a wrong figure had already been sent a corrected one before any of this, because putting the outcome right comes first.
Which controls keep corrections useful?
- Corrections never change the agent directly. Patterns lead to a new version, tested before release.
- Original output always kept, alongside the correction and its reason.
- Corrected cases added to the case set, so the next certification tests what went wrong.
- Clear queues: corrections on held work stay in the held queue, while work the agent could not process belongs in the exceptions route described in working an operations exceptions queue. General task flow sits in the team's work queue.
The wider sequence is in our guide to deploying an AI agent.
Where Pryme fits
Every verdict in Pryme is filed in an append-only record, and a correction sits beside the original rather than replacing it. Because Pryme qualifies agents rather than training them, corrections change the brief, sources or bounds through a new version, which then goes through Attest on your cases.
Questions people ask
Should reviewers explain every correction in writing?
A category is enough for most, with a short note where the category does not tell the story. Long free-text explanations tend to be skipped when queues are busy.
How quickly should a pattern lead to a change?
Review patterns weekly. A pattern that affects customers should lead to a change, or a lower depth, within days. Cosmetic patterns can wait for the next scheduled version.
Do corrections count against the agent's certification?
They do not void it directly, but they are evidence. A rising correction rate should trigger a retest before the next scheduled recertification.