Concept drift is a change over time in the relationship between the inputs an AI system sees and the correct output for them. The cases look the same, but a rule, a policy or customer behaviour has changed. An AI agent that was correct on the old rules is now confidently wrong on the new ones.
Why does concept drift matter for agents?
It is the most dangerous kind of drift because nothing looks different. The questions customers ask are the same. The documents the agent reads are the same. The agent's answers are as fluent as ever. Only the world has changed, and the agent has no way to know unless its sources tell it.
For agents in regulated work, this happens constantly. A fee schedule changes, a complaint-handling deadline is revised, a sanctions list adds names, a lender tightens its affordability policy. An agent certified before the change is acting on yesterday's rules.
How does it show up in practice?
Concept drift rarely announces itself in the agent's inputs. It appears in outcomes and reviews:
- reviewers start correcting the same kind of answer;
- complaints rise on one topic;
- held items get refused more often for one reason;
- the agent's answers diverge from what a newly trained member of staff would say.
Detection works best when every answer is tied to its sources, so a reviewer can see which rule or document the agent relied on. The full method is in how to detect drift in a certified AI agent.
What does an example look like?
A lender's agent drafts responses to customers asking whether they can take a payment holiday. It was certified when the lender's policy allowed holidays for up to three months. The credit committee shortens that to two months for new requests. The policy document in the agent's sources is updated a fortnight later. For those two weeks, the agent drafts letters offering three months. Arrears officers catch some; others are approved by reviewers who also have not caught up. The drift was in the concept, not the data.
How is concept drift different from data drift and model drift?
| Drift | What changes | Example |
|---|---|---|
| Data drift | The inputs: their mix, format or distribution | More requests arrive in a new language |
| Concept drift | What the right answer is for a given input | A policy limit changes |
| Model drift | The system's behaviour or performance | A provider's model update changes how it answers |
They can occur together. The fix differs: data drift may need wider testing, model drift needs recertification on the new model, concept drift needs the source of truth updated and the agent recertified against cases decided under the new rule.
How do you manage it?
Tie the agent to sealed sources, such as the policy library, so a rule change reaches it the moment the record changes. Give someone the job of telling the agent's owner when policies change. Recertify on recent cases, not only historic ones, since old cases encode old answers. And treat certificates as perishable: the reason they lapse after 90 days is that the world keeps moving, which is the argument in drift is why certification is never finished. The NIST AI RMF approach to monitoring and drift for AI agents places this under its Measure and Manage functions.
See our guide to deploying AI agents for where drift monitoring fits in running agents.
Where Pryme fits
Through the Pryme Context Protocol, facts from systems of record such as the policy library arrive sealed, so the agent works from the record itself rather than a copy that can fall behind. Attest recertifies on recent cases before each 90-day lapse, and a run that clears no depth drops the agent to depth 1 the same day.
Questions people ask
Can an agent detect concept drift by itself?
Not reliably, because it judges new cases by the rules it was given. Detection needs signals from outside the agent: reviewer corrections, outcomes, and changes logged in the systems of record.
Does retraining fix concept drift?
For agents, the fix is usually not retraining a model but updating the sources and brief, then recertifying on cases decided under the new rule.