AI risk management is the ongoing process of identifying, assessing, treating and monitoring the risks that AI systems create, from design through use to retirement. It asks what could go wrong, how likely and harmful it is, what controls reduce it, and whether those controls work. For agents, the central risk is an action taken that should not have been.
Why does AI risk management matter once agents act?
The risks of a model that produces text are mostly about content: inaccuracy, bias, leaked data, harmful outputs. Agents add the risks of action. An agent can pay the wrong party, close the wrong account or send a message that creates a legal obligation. Those harms are faster and harder to undo than a bad answer.
That changes which controls matter most. Content risks are often treated after the fact, by reviewing outputs. Action risks need controls at the moment of action, because a review the next morning cannot recover a payment that has already settled. The approach to controlling action is set out in agent governance is a bound, not a model.
How does AI risk management work in practice?
| Stage | Question | For an agent, typically |
|---|---|---|
| Identify | What could go wrong? | Wrong action, wrong party, out-of-role action, manipulated input, data exposure |
| Assess | How likely and how harmful? | Classify each capability by what it can do and how reversible that is |
| Treat | What reduces it? | Bounds on role, depth, sources and limits; human gates; platform floor |
| Monitor | Is it working? | Verdict records, refusal and hold rates, certification results over time |
Classification is where most programmes start. Agents that read and agents that move money should not sit in the same risk tier, as covered in how to classify AI agents by risk.
Frameworks give the process shape. The NIST AI Risk Management Framework organises it into four functions, Govern, Map, Measure and Manage. ISO/IEC 23894 gives guidance on applying risk management to AI, discussed in ISO/IEC 23894 AI risk management for AI agents. For high-risk systems, Article 9 of the EU AI Act requires a risk management system across the lifecycle, covered in EU AI Act Article 9 risk management for AI agents.
What does that look like in an example?
A credit union plans an agent to help members with loan repayments. Risk identification lists: an incorrect payment date, a repayment taken from the wrong account, a payment plan offered to a member in difficulty without proper assessment, and exposure of one member's details to another. The team rates the wrong-account repayment as most harmful, because money moves. Treatment: the agent drafts repayment changes at depth 2 and a member services officer approves them. Monitoring: the officer's edits and rejections are reviewed monthly, and the agent is recertified every 90 days.
How is AI risk management different from model risk management?
Model risk management is an established discipline in banking, supervised under guidance such as SR 11-7 in the US and PRA SS1/23 in the UK. It focuses on the risk that a model is wrong or misused: poor design, bad data, flawed validation, use outside its intended purpose. AI risk management is wider. It includes model risk, but also the risks of how an AI system is deployed, what it is connected to, what it may do, and how people interact with it. An agent can run on a well-validated model and still create risk if its authority is too broad.
Questions people ask
Is AI risk management a legal requirement?
It depends on the system and the market. Under the EU AI Act, providers of high-risk systems must operate a risk management system. Elsewhere, sector regulators generally expect firms to manage the risks of the technology they use. This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
How often should AI risks be reassessed?
Whenever something material changes, such as a new model, a new connection or a new capability, and on a regular cycle in any case. For agents, a certification cycle that lapses every 90 days gives a natural rhythm for reviewing whether the risk assessment still holds.