Stable gate: what it means for AI agents

The stable gate is the certification test that checks whether an AI agent gives the same answer to the same question across runs and across a conversation, without being told twice. It asks the same case more than once and fails the agent when the answers disagree, whichever answer looks better. Earlier writing on this blog called it the drift gate.

Why does the stable gate matter?

An answer that depends on the run cannot be relied on. If the same refund request is approved on one run and declined on the next, at least one answer was wrong, and nobody knows which the next customer will get. Language models can vary their output between runs, so consistency has to be tested, not assumed.

The other six gates ask whether an answer is acceptable. The stable gate asks whether the agent would give it again. It also covers the conversation: an agent corrected once should not need correcting again a few messages later. Change over time is a separate problem, handled by fresh cases and a certificate that lapses; that is the argument of drift is why certification is never finished.

How does the stable gate work in practice?

1. Repeat runs: the same cases are run more than once, and the answers compared case by case.
2. Within the conversation: cases over several turns check that the agent keeps to what it was told earlier.
3. Comparison by case type: disagreements are counted within each type of case, not hidden in an average.
4. Same answer, not same words: phrasing can vary; the decision, the facts and the action should not.
5. Its own floor: Attest publishes a floor of 86 for this gate.

Between runs, the question is different: whether the agent's work has moved since it was certified. That is drift, watched on live work as described in how to detect drift in a certified AI agent, and the kind of ongoing monitoring the NIST AI RMF expects, covered in NIST AI RMF monitoring and drift for AI agents.

What does that look like in an example?

A bank's complaints agent drafts responses for handlers. Each draft reads well. But when the certification cases are run twice, complaints that mention ill health are flagged as possible vulnerability on one run and not on the other. No single draft fails another gate badly enough to notice. The stable gate fails on that case type. The team adds explicit guidance to the brief, the cases are run again, and the answers agree.

How is the stable gate different from the adversarial gate?

The adversarial gate asks whether the agent holds a bound when someone pushes it. The stable gate asks whether it gives the same answer when nobody pushes, only asks again. An agent that holds firm on one run and gives way on the next fails both. The stable gate also differs from the grounded gate, which checks each answer against its sources; the stable gate checks answers against each other.

Which related terms should you know?

• Run: one pass of the agent through its certification cases.
• Model drift: a change in a model's performance over time, caught by re-running the agent's cases on recent work.

The full method is in what it means to certify an agent.

Where Pryme fits

Attest runs the agent's own cases more than once and compares the answers case type by case type. The stable gate must clear on its own; readiness is the lowest of the seven gates, never an average. A run that falls short certifies the highest depth it cleared, and certificates lapse after 90 days, so the check repeats on recent cases.