How to set service levels for work done by an AI agent

Set service levels per capability, and measure the work, not the agent. Each needs a quality level first, then turnaround split between work the agent completes and work held for a person, an approver target for held items, and a fallback level that applies whenever the agent's depth drops. Measure from the verdict records; the service owner signs.

Why do service levels need rethinking when an agent does the work?

Traditional service levels describe a team: how fast it answers, how many cases it closes in a day, how accurate its work is. When an agent joins, the work splits. Some items are completed by the agent in seconds. Some are drafted and held for a person, and wait for an approver. Some are refused and handled by people from scratch. One average turnaround hides all of this, and a fast average can conceal a held queue that is days old.

The service owner is the head of the team that provides the work. The service customer is whoever depends on it: another team, or the firm's customers through a product promise. The agent owner, often the same person as the service owner, answers for the agent's part. The general questions are covered in how to set service levels for an AI agent; this playbook is the step-by-step method.

What are the steps, and who acts at each?

1. Service owner and service customer: agree what the service is. The output, its quality, when it is needed, and what happens if it is late. Evidence: the agreed service description.
2. Agent owner: maps the service to capabilities and depths. Which parts the agent completes, which it drafts, which people do. Evidence: the capability map.
3. Analyst: takes a baseline from the team's work before the agent: turnaround, quality, volume. Evidence: the baseline report.
4. Service owner: sets the quality level first. The error rate, from sampling, that the service must not exceed, whoever does the work. Evidence: the quality level and sampling method.
5. Service owner: sets turnaround levels separately for agent-completed items and held items. Evidence: both targets.
6. Service owner and approvers' managers: set approver levels. How quickly held items must be reviewed, with cover arrangements. Evidence: the approver target and cover rota.
7. Service owner: sets the fallback level. What the service commits to while a capability is at depth 1 and people do the work. Evidence: the fallback target and who decides when it applies.
8. Agent, depth 1: measures every level from its verdict records and the sampling log. Evidence: the measurement method and queries.
9. Agent, depth 2: drafts the service report each period. The service owner reviews and sends it. Evidence: the issued report.

Why does quality come first?

Because an agent can meet any speed target by doing the work badly. A reply sent in ten seconds that gets the customer's balance wrong is not good service. Setting quality first, and making it a condition for counting turnaround, prevents the faster-but-worse trade from creeping in. A service period where quality falls below its level is a missed service, whatever the turnaround figures say.

Why do held items need their own clock?

Held items wait for a person, and that wait is often the longest part of the process. If the service level only measures the agent's own time, the held queue can grow for days and the report still looks green. A separate turnaround for held items, measured from the moment the agent holds to the moment a person decides, makes approver capacity visible.

That, in turn, needs an approver level. Approvers usually have other jobs, and held items compete with them. Agree with their managers how quickly items will be reviewed and who covers absences, and write it into the service description. A ready format is the internal service level agreement template for AI agents.

What is a fallback level, and why set one?

Depth can drop at any time: a certificate lapses, a drift check fails, an incident occurs. When that happens, people do the work the agent was doing, and turnaround slows. If the service customer was promised agent speed, the service is suddenly in breach through no one's fault but the plan's.

A fallback level says, in advance: while capability X is at depth 1, the service commits to the following, usually close to the baseline. The service customer knows what to expect, and no one is tempted to keep a failing agent running to protect a target.

What does an SLA look like in practice?

A service level agreement for agent-supported work typically lists, per capability: the quality level and how it is sampled; turnaround for agent-completed and held items; the approver review level; the fallback level and when it applies; how each is measured; and how often it is reported. Keep it to a page. If it runs longer, the service is probably several services.

What should you measure?

• Quality from random samples, per capability, each period.
• Turnaround for agent-completed items and for held items, as distributions, not only averages. Report the slowest items too.
• Approver response time against its level.
• Time spent at fallback, and turnaround during it.
• Refused items and how long people took to handle them.

Where does it usually go wrong?

One number for the whole service. It hides the held queue. Split it.

Targets set from the agent's demo speed. The demo ran on clean cases with no approvals. Set targets from shadow and early live runs.

Approvers are not part of the agreement. Held items wait, and the service team cannot fix it because approvers report elsewhere.

No fallback level. The first depth drop becomes a service breach and a political problem. Agree fallback before go-live, as part of train, shadow, certify, widen.