The place to control an AI agent is the moment it tries to do something: send, pay, change, close. Models will be swapped, upgraded and retrained by others, and their internals cannot be inspected in any useful way. The action can be. Check each proposed action against the agent's authority before it runs, whichever model proposed it.
Why is the model the wrong control point?
Most early AI governance was written for models. It asked how the model was trained, what data went in, how accurate it was on a benchmark and whether its outputs were biased. Those questions still matter for systems that score or classify. They fit poorly with agents, for three reasons.
First, you rarely own the model. Most agents run on a general model built by a provider, which you can choose but not inspect. Governing something you cannot see or change is governance by assurance letter.
Second, the model changes underneath you. Providers release new versions, deprecate old ones and alter behaviour in ways that are documented loosely if at all. A control that depends on the model behaving the way it did when you tested it is only as good as your last test.
Third, the model is not where the harm happens. A model that writes a poor sentence has done nothing. An agent that sends that sentence to a customer, refunds the wrong amount or closes an account has acted, and the action is what you answer for.
What does governing the action look like in practice?
Every time an agent proposes an action, something outside the model checks it before it runs. The check asks a small number of questions. Is this action within the agent's role? Has the agent been certified for this capability at this depth? Are the facts it relies on from sources strong enough for this action? Is it within the customer's own limits? Does it break any rule the platform applies to everyone?
The answer is a verdict: allowed, held for a named person, refused, or withheld at the platform floor. The verdict names the bound that decided it and is written to a record nobody can edit. None of this depends on which model proposed the action or how it reasoned. A refund of 900 against a limit of 500 is held whether the proposal came from an old model or a new one.
How is this different from relying on the model's own guardrails?
Model providers build safety behaviour into their models, and it does useful work: refusing clearly harmful requests, avoiding some kinds of offensive output. But that behaviour is designed for the provider's general idea of harm, not for your business's idea of authority. A model has no way to know that your agent may issue refunds up to 500 but not 501, or that this customer's limit is lower than the default.
Built-in guardrails are also probabilistic. They usually refuse what they should, and sometimes do not. Our piece on whether a model's own guardrails can govern an AI agent works through the gap in detail. The short version is that guardrails inside the model are a useful layer, but they cannot be your control, because you cannot set them, test them on your cases or see their decision.
| Question | Governing the model | Governing the action |
|---|---|---|
| What is checked? | Training data, benchmarks, output quality | Each proposed action against named bounds |
| When? | Before deployment and at review | Every time the agent acts |
| Who sets the limit? | Largely the provider | You and your customer |
| What happens on a model change? | Re-validate from scratch | Bounds still apply; re-certify behaviour |
| What evidence is left? | Test reports | A verdict per action, naming the deciding bound |
Does the model stop mattering?
No. Model quality decides how often the agent proposes the right action, and that drives the value of the agent. A weak model will be held and refused more often, which is costly. A model change can shift behaviour enough that an agent certified for a capability should be retested before keeping it.
The point is narrower. Model quality is a reason to choose one model over another and to re-certify after a change. It is not the control that stops a wrong action reaching a customer. Keep evaluating models. Do not mistake that evaluation for governance.
Why does this make vendor and multi-model estates manageable?
Most organisations will run agents built on several models, some built in-house and some supplied by vendors. Governing each model on its own terms produces a separate framework for every provider and a review backlog that never clears.
Governing the action gives one control that works across all of them. Every agent's actions pass through the same checks, against the same kinds of bounds, leaving the same kind of record. A vendor-supplied agent is governed by the same rules as your own, which is the argument in our guide to governing AI agents supplied by vendors. It also gives a board something stable to oversee, as we argue in boards should govern agent authority, not agent technology.
How does this fit the frameworks regulators recognise?
Frameworks such as the NIST AI Risk Management Framework are organised around functions rather than specific controls, and its Govern function asks for clear roles, policies and accountability across the system's life. Checking actions at the moment they happen is one practical way to give that function teeth for agents, because it turns policy into a check that runs every time. Our page on the NIST AI RMF Govern function for AI agents maps this in more detail.
Regulators generally care about outcomes for customers and about who is accountable. Neither depends on which model you used. Both depend on what the agent did and whether it was allowed to do it.
What is the trade-off?
Governing the action means building or buying a checking layer that sits between every agent and every system it touches. That is real engineering. It adds a small delay to each action and requires you to express your bounds precisely enough for software to check them, which many organisations have never had to do.
It also means accepting that some actions a capable model could handle well will be held or refused, because they fall outside a bound that was set conservatively. That is a cost in efficiency, paid for predictability.
The alternative is to trust that each model will behave as tested, on every case, after every update. When an agent does try something outside its limits, you want the check to catch it and the record to show it, which is the starting point for responding when an agent attempts an action outside its bounds. If your governance today is a folder of model evaluations, keep the folder and add a check at the point of action. The design behind that check is set out in our guide to agent governance as a bound, not a model.
Questions people ask
Do you still need to evaluate models if you govern the action?
Yes. Evaluation tells you which model is likely to propose the right actions most often and whether a new version behaves differently. It informs which model you choose and when you re-certify, but it is not the control that stops a wrong action.
What happens to an agent's authority when its model is upgraded?
The bounds stay the same, because they belong to the agent's role and the customer, not the model. The agent's behaviour should be retested on real cases before it keeps the depth it had, since a new model can change how often it proposes the right action.
Can the checking layer itself make mistakes?
It can if the bounds it checks are wrong or incomplete, which is why bounds need owners and review. But the check itself is deterministic: the same action against the same bounds gets the same verdict, which makes it testable in a way a model is not.