How do you stop an AI agent's performance from drifting?

You cannot stop the world around an agent from changing, so stop the parts you control from changing silently and watch the rest. Pin the model version, put sources and the brief under change control, compare live behaviour each week with the certification baseline, and set thresholds that send the agent back for recertification before customers notice.

Where does drift come from?

Drift is rarely the agent "getting worse" on its own. It comes from one of five places:

• The inputs change. A new product launches, a season shifts the case mix, a partner changes the format of its files.
• The sources change. A policy is rewritten, a field in a system of record is renamed, a reference table goes stale.
• The model changes. A provider updates the model behind the same name.
• The brief changes. Someone edits the instructions to fix one problem and creates another.
• The people change. Reviewers grow used to the agent and approve faster, so errors that used to be caught now pass.

The last one is easy to miss. If reviewers stop correcting drafts, the edit rate falls and the dashboard looks better while quality gets worse.

Which signals show drift early?

No single signal proves drift. A rise in refusals during a new product launch may be exactly right. What matters is that someone looks, decides and files the reason.

What is the routine, and who runs it?

1. Agent, depth 1: reports its own signals each day from the filed record. Evidence: a daily signal report.
2. Reviewer: scores a fresh weekly sample of live work against a written answer key. Evidence: scored cases.
3. Process owner: runs a weekly review of signals and scores, using a set format such as the weekly AI agent performance review. Evidence: minutes with decisions.
4. Platform owner: logs every change to model, sources and brief with its date. Evidence: a change log tied to the agent's versioned bundle.
5. Process owner: triggers recertification when a threshold is crossed or any controlled element changes. Evidence: the trigger and the new result.

What does catching drift look like?

Say a bank's financial crime team runs an agent that drafts summaries of alerts for analysts at depth 2. For months, analysts change little in the summaries. Then the edit rate climbs slowly over three weeks.

The weekly review finds two causes. A new payment product produces alerts with a reference field the agent was reading as the counterparty name. Separately, a reviewer who had joined the team was correcting more thoroughly than colleagues, which pushed the edit rate up for a good reason. The first is a source problem: the brief is updated and certification is rerun on alerts from the new product. The second is a reviewer problem in the other direction, and it prompts a calibration session so all analysts review to the same standard. The detail of measuring a compliance agent's performance matters here, because compliance outputs are judged on what they omit as much as on what they say.

Which controls hold the line?

• A certification baseline that every weekly score is compared against.
• Pinned model versions where the provider allows, with the version filed on every verdict.
• Change control on sources and brief, so nothing changes without a log entry.
• Thresholds written in advance that trigger recertification, not discussion.
• Automatic consequences. A run that clears no depth drops the agent to depth 1 the same day, and certificates lapse after 90 days even if nothing seems wrong.

The rest of the sequence is in our guide to deploying an AI agent.

Where Pryme fits

Attest's seven gates include stable, which checks that the agent gives the same answer to the same question across runs and across a conversation, and it must clear on its own. Each recertification re-runs the agent on your current cases. Because certificates lapse after 90 days, an agent cannot hold its depth on old evidence.