PrymeIntelligence

AI agent skills by role·Article

Why operations managers are the real control in an AI agent deployment

Operations managers run the queue and answer for outcomes, so they decide whether AI agent controls hold. Here is why, and what their training must cover.

Whoever owns the queue owns the control. In most AI agent deployments that is the operations manager, who sets limits, staffs review, runs the daily rhythm and answers for outcomes either way. An agent can be perfectly certified and still cause harm if the operation around it is run badly.

What does an operations manager actually control?

More than the org chart suggests. In a governed deployment, every agent action meets five bounds at once: role scope, certificate, sources, the organisation's own limits and the platform guardrails. The lowest wins. Operations managers usually write or approve the organisation's own limits for their area, and they decide how held actions are staffed and how fast they are decided.

They also own the parts no setting captures: who is on the review rota on a bank holiday, what happens when the queue doubles, which issues reach the manager before they reach the customer. A held action is drafted and waiting for a named person. If that person does not exist on the rota at the time, the control does not exist either.

Why does the rhythm of the operation matter more than the agent?

An agent's behaviour is mostly fixed between certifications. The operation around it moves every day. Volumes change, people go on leave, policies update, a new product launches. Each of those can quietly break a control that worked last week.

Operations managers who run a daily, weekly and monthly rhythm catch these early. Daily: held items by age, refusals by reason, anything withheld. Weekly: overrides by reviewer, calibration on disputed cases, sources due for refresh. Monthly: certificates approaching their 90-day lapse, limits still matching appetite, lessons from incidents. Without that rhythm, the first sign of trouble is a complaint.

What happens when something goes wrong?

Operations managers carry the incident. The common ones are predictable and worth rehearsing:

Event What the manager sees First decision
Certificate lapses Executed work turns into drafts Who covers the extra review volume
Failed certification run Agent drops to depth 1 the same day Pause dependent processes or staff them manually
Source goes stale Refusals rise on one topic Who owns the source and when it will be fixed
Reviewer error found Wrong approval in the record Correct the outcome, then sample that reviewer's other decisions
Withheld action blocks a process Work cannot proceed through the agent Route it to a person; no setting will change it

Every verdict and every human decision is filed in an append-only record, so the manager can reconstruct what happened. The skill is knowing which questions to ask of the record, quickly, while the customer is still waiting.

How should an operations manager think about depth?

As something earned, not requested. Depth runs from 1 (read) to 2 (draft for a person), 3 (execute routine work within bounds), 4 (move money or take hard-to-undo actions) and 5 (coordinate other agents). The role sets a ceiling; evidence from certification sets the depth; the lower wins.

Operations managers are often the first to ask for more depth because they feel the review load. That is legitimate, but the right request is "what evidence would earn depth 3 for this capability?", not "turn it up". A manager who understands this saves months of frustration and makes better cases for change.

What should an operations manager measure?

The measures an operations manager chooses become the behaviour of the team. Throughput and handle time still matter, but on their own they reward approving held items quickly, which is the opposite of what a held action is for. A manager running agents needs a second set of measures that sit beside the familiar ones.

Useful ones are simple. The age of the oldest held item at the end of each shift. The share of sampled review decisions later found correct, by reviewer. The number of overrides of refusals, with their reasons. The number of days each agent capability spent below its usual depth because of a lapse or a failed run. And the time between a source changing and the agent's sources catching up. None of these need new tools; all of them can be read from the record. Together they tell the manager whether the control is working, not just whether the work is moving.

The trade-off is attention. Every new measure competes with the old ones for the team's focus. Pick three, explain why, and review them in the weekly rhythm rather than adding them to a dashboard nobody reads.

What should an operations manager learn?

The decision chain and verdicts, well enough to explain them to their team. Workforce planning for review: held volume, decision time and cover for every hour the agent runs. Measures that reward correct decisions rather than fast ones. Incident handling using the record. And how to run calibration so reviewers across shifts decide alike.

They do not need to build agents. Design and certification belong to builders, and multi-agent deployments in regulated settings are designed by people such as a deployment architect. The operations manager's skill is running the system day to day. Pryme Intelligence Academy's Operator track covers running the rhythm as a module in its own right.

How does this compare across sectors?

The pattern holds, the detail differs. In e-commerce, peak trading periods multiply refunds and returns overnight; the briefs on operations managers in e-commerce in India and in Australia cover that. In property, the queue includes urgent repairs; see property managers as the real control. Risk managers set appetite that operations managers turn into limits, covered in risk managers as the real control. The full map is in AI agent skills by role.

Questions people ask

Should operations managers own the agent's limits?

They should own or co-own the limits for their area, with risk setting the appetite they sit within. Every limit change should carry a reason and an approver in the record.

How do you size the review team for an agent?

From held volume, the time each decision takes and the hours the agent runs, plus cover for leave and certificate lapses. Start from the queue the agent creates, not the volume it handles.

What is the biggest mistake operations managers make with agents?

Measuring reviewers on speed. It teaches people to approve, which turns a held action into an unreviewed one.

Train the people who run your agents.

Pryme Intelligence Academy: free courses on the real product, audio lessons, labs in a free workspace, and certificates anyone can check on VerifyDoc.ai. Teams get seats, an admin view and a free readiness assessment.