Keep an AI agent change log as the append-only record of every change made to an agent after go-live: its brief, sources, connections, bounds, depth, model version and certification. Each entry says what changed, why, who approved it, what testing followed and how to undo it. Without one, you cannot explain why an agent behaved differently last Tuesday.
What counts as a change?
Anything that could change what the agent does. Teams often log model upgrades and forget the rest, yet most behaviour changes come from elsewhere:
- Brief: wording of instructions, examples, refusal wording.
- Sources: a new policy document, a retired one, a changed data feed.
- Connections: a new system it can read or act in.
- Bounds: depth for a capability, customer limits, role scope.
- Model: version or provider change.
- Certification: a new certificate, a lapse, a failed run.
A change to a source document counts even if nobody touched the agent. If the refund policy changes, the agent's answers change. Governing an agent means governing all of these, as agent governance is a bound, not a model argues.
When should you log a change?
Before it is made, with status "proposed", and again when it is approved, deployed and verified. The log is not a diary written afterwards. It is the record your change control process runs on; how to run change control for AI agents sets out the process.
Log external triggers too. When a regulation changes and an agent has to follow, the agent's change log entry should point to the entry in your regulatory change log, so the chain from rule to behaviour is visible.
How do you fill in each field?
- Change type comes from the list above. One entry per type; split combined changes.
- Reason is the problem the change solves, with a reference: an incident, a QA finding, a policy update, a request.
- Before and after must be specific enough to reproduce. Quote the old and new wording; give old and new depth.
- Risk rating decides how much testing is needed. Raising depth or adding a connection is high; fixing a typo in a refusal is low.
- Test evidence names the cases re-run and the result.
- Rollback says exactly how to reverse it.
- Approved by must not be the person who requested it for medium and high-risk changes. An agent must never approve its own change, a point made in whether an AI agent should be able to change its own permissions.
The AI agent change log template
| Field | Guidance |
|---|---|
| Change ID | Sequential, never reused |
| Agent | Name and inventory ID |
| Change type | Brief, source, connection, bound, model, certification |
| Status | Proposed, approved, deployed, verified, rolled back |
| Requested by | Named person |
| Reason and reference | Problem and link to incident, finding or request |
| Before | Exact prior state |
| After | Exact new state |
| Risk rating | Low, medium, high |
| Test evidence | Cases run, result, reference |
| Approved by | Named person, independent for medium and high |
| Deployed at | Date and time |
| Rollback | Exact reversal steps |
| Verified by | Named person and date |
A worked sample
A hypothetical lender's customer service agent, three entries.
| ID | Type | Reason | Before | After | Risk | Test evidence | Approved by |
|---|---|---|---|---|---|---|---|
| CHG-201 | Source | Fees policy revised | Fees policy v7 | Fees policy v8 | Medium | 120 fee questions re-run, all match v8 | Head of customer operations |
| CHG-202 | Brief | QA found agent offering payment holidays | No refusal for payment holiday requests | Refusal R-MO-06 added with route to financial support | Medium | 80 hardship conversations re-run; no offers made | Customer outcomes lead |
| CHG-203 | Bound | Pilot success | Address change: depth 2 | Address change: depth 3, limit one change per customer per 90 days | High | 400 address change cases replayed, certified | Head of operations and second-line risk |
CHG-202 is the one an auditor will ask about. It shows that a QA finding led to a specific change, which was tested before release. The log should make that story readable in one line.
The log keeps the record of change; the agent's own decisions are recorded elsewhere. Both are needed, because the audit trail should explain the decision, not just log it. Exception logs for specific processes, such as the three-way match exception log, sit alongside it.
Questions people ask
Should we log changes made during testing?
Not in the production log. Keep a separate test log, then summarise the final tested state in the production entry when the change is deployed.
How long do we keep the change log?
At least as long as the records of the decisions the agent made under each version. Otherwise you cannot reconstruct why a past decision was made.
Can the change log be a ticketing system?
Yes, if entries cannot be edited after approval and the fields above are mandatory. Many ticketing systems allow edits, so check before relying on one.