An AI governance framework is the structured set of principles, roles, policies, controls and review routines through which an organisation decides how AI is chosen, used, overseen and retired. It answers who may approve an AI use, which risks are acceptable, what evidence is kept, and who is trained to do what.
Why does a framework matter once agents act?
A chatbot that drafts text and an agent that moves money need different governance. Once an agent can execute work, the framework has to say how far it may act, who approves what it cannot do alone, and how authority is withdrawn when evidence weakens. A framework written for advisory AI tends to stop at "a human reviews outputs", which says nothing about which human, with what competence.
Frameworks also fail quietly through people. A role named in the framework is only a control if the person in it can do the job. That is the gap governance literacy is meant to close.
What does a framework contain in practice?
Most frameworks for agents end up with the same working parts:
| Part | What it settles for agents |
|---|---|
| Principles | What the organisation will and will not let agents do |
| Roles | Agent owner, builder, reviewer, approver, governance lead |
| Inventory | Every agent, its capabilities and its current depth |
| Risk and assurance | How each capability is tested before and after go-live |
| Approval rules | Which actions are held and who may release them |
| Records | What is filed for every decision, and for how long |
| Competence | The training and current credential each role needs |
Published frameworks help with the shape. Singapore's IMDA Model AI Governance Framework is one well-known example; how it reads for agents is set out in IMDA's framework and AI agents. The NIST AI Risk Management Framework and ISO/IEC 42001 are others, the first voluntary, the second certifiable.
What is an example?
Take a mid-sized payments firm adopting an agent for refund handling. Its framework names an agent owner in operations, sets the agent's refund capability at draft only until it is certified on the firm's own cases, requires refunds over the customer's limit to be held for a named approver, and requires every approver to hold a current certificate in reviewing agent decisions. The framework also says what happens when a certificate lapses: approval rights pause until renewal.
How is it different from an agent framework?
The names collide. An agent framework is software: a library developers use to build agents, connect tools and chain steps. An AI governance framework is organisational: it decides what those agents are allowed to do and who answers for them. The first is explained in what an agent framework is, and the difference between building agents and governing them in agent framework or agentic operating system.
A useful test: if it is installed, it is an agent framework. If it is approved by a board, it is a governance framework.
Which terms are related?
- AI policy: a document inside the framework that sets the rules staff follow.
- AI risk assessment: the exercise that feeds the framework's controls.
- Named accountability: the principle that each control has a person who answers for it.
For the training side of the topic, start with AI literacy and certification for people who work with agents.
Questions people ask
Do we need a framework if we only run one agent?
Yes, though it can be short. One agent still needs an owner, approval rules, a record and trained reviewers. A one-page framework that names those beats a long one written for a future you have not reached.
Who should own the framework?
Usually an AI governance lead reporting to an executive committee or the board, with risk and compliance reviewing it. Operations owns the parts that run daily, such as the review rota.