How to teach operations managers to bank corrections that improve an AI agent

Operations managers rarely make the corrections themselves. Their job is to run the system that banks them: a short reason-code list, a weekly review of correction patterns, a queue of proposed changes ranked by cost, and time protected for re-certification. Teach them to run that rhythm, and corrections from every reviewer become lasting improvements to the agent.

What is the operations manager's part in banking corrections?

Reviewers fix individual cases. Agent owners change the brief, sources and bounds. Between them sits a gap that, left alone, swallows most corrections: nobody reads the pattern, nobody decides which fixes matter most, and nobody makes room in the week for the change and the re-certification. That gap is the operations manager's.

In practice this means four things. Setting the reason codes reviewers use, so corrections can be grouped. Holding a weekly correction review, where patterns are read and decisions made. Keeping a ranked queue of changes for the agent owner. And protecting capacity, because every banked change needs a replay in Attest, and a replay that clears no depth drops the agent to depth 1, which pushes all its work back onto people the same day. A manager who has not planned for that will see the queue spike at the worst moment. The role map behind this is in AI agent skills by role, and the argument that managers are the real control is made in why operations managers are the real control in an AI agent deployment.

How should the manager run it, step by step?

1. Person (operations manager): sets five to seven reason codes with the agent owner and team leads, plus a "one-off" code. More than that and reviewers stop choosing carefully.
2. Agent, depth 2: drafts the work, and reviewers correct it with a code and a short note.
3. Agent, depth 1: produces a weekly summary of corrections by code, queue, product and reviewer.
4. Person (operations manager): chairs a 30-minute correction review with a team lead and the agent owner. The question for each pattern: bank it, watch it, or close it as a one-off.
5. Person (operations manager): ranks banked changes by repeat volume and by harm if left alone. A rare error that releases money outranks a common formatting fix.
6. Person (agent owner): makes the top changes, and Attest replays the agent against the team's own cases.
7. Person (operations manager): schedules the go-live for a time the team can absorb a fall to depth 1 if the replay clears no depth, and checks the following week that the pattern dropped.

What does the manager need to learn?

How to read a correction summary without being misled by volume. A high count can mean one reviewer applying a personal standard. Before banking, the manager should check whether the pattern holds across reviewers. If only one person corrects a particular thing, it may be a calibration issue among people rather than a fault in the agent.

How to tell agent fixes from process fixes. Some patterns persist because the process is ambiguous to people too. The manager is the right person to send those to the process owner instead of asking the agent owner for a brief change that only hides the ambiguity.

How to plan for re-certification. Every agent certificate lapses after 90 days in any case, so the manager should know when each agent's certificate is due and bundle banked changes into planned replays where possible, rather than triggering a replay for every small fix.

Which controls and evidence should exist?

• Mandatory reason codes. Evidence: the share of corrections with a code, visible in the weekly summary.
• Minutes of the correction review. Evidence: each pattern with its decision and owner.
• Ranked change queue. Evidence: the queue with dates in and out.
• Planned go-live with fallback. Evidence: the replay result and, if failed, the depth change in the record.

What should the manager measure?

Time from first appearance of a pattern to a live banked change. Repeat corrections per code in the two weeks after a change. The share of corrections closed as one-offs, which should be steady; a rising share often means reviewers have stopped diagnosing. Reviewer agreement, measured by having two reviewers look at a small sample of the same cases. And the number of unplanned falls to depth 1, which should be close to none.

Where does it usually go wrong?

The correction review gets cancelled. It is short and never urgent, so it is the first meeting to go when the queue is busy. Within a month the backlog of unread corrections is too large to read, and banking stops.

Everything gets banked. A manager eager to show progress sends every pattern to the agent owner, the brief fills with special cases, and re-certification becomes slow and fragile. Banking fewer, more important changes works better.

Nothing gets measured afterwards, so nobody knows whether banking helped. Risk managers face the same temptation to collect without acting, which teaching risk managers to bank corrections takes up. In payments, where cut-offs leave little slack, the cadence needs to be daily, as teaching payments operations staff to bank corrections describes.

Where the Academy fits

Pryme Intelligence Academy's Operator track was written for people who run a live workspace every day, operations managers included. Its modules on reading the workspace, keeping knowledge current and running the rhythm cover the cadence described here. The admin view for teams shows progress, results, certificates and expiry dates, so a manager can see who on the team is certified and when renewals fall due.