Tracking an AI agent's cost means recording everything it takes to run each month: model usage, platform and integration fees, the people who review its held work, certification, incidents and rework. Divide the total by completed work to get a cost per outcome you can compare with the manual process. The fields and an example follow.
Why is the model bill not the cost?
Because most of the cost of an agent in a regulated business is people. Someone approves its held items, someone reviews its refusals, someone re-tests it after every change, someone fixes what it got wrong. A team that tracks only the model bill will think a cheap agent is cheap and a careful one is expensive, when the opposite may be true. The argument is made in the cost of an agent is not the model bill. Depth, earned in stages as described in train, shadow, certify, widen, is the main lever on that cost.
The template therefore puts human time on the same sheet as compute, in the same units.
When should you use it?
Monthly, per agent, from the first day of shadowing. Shadowing costs count: they are what you paid to earn the depth the agent now holds. Keep the sheet running after go-live and review it whenever depth changes, because depth is the main lever on cost. More holds mean more review time; fewer holds mean less.
The method behind it is set out in how to track the cost of each AI agent. For typical cost ranges, see how much it costs to run an AI agent.
How do you fill in each line?
- Model usage comes from your provider's billing, split by agent if possible. If not, allocate by request volume.
- Platform and integration covers licences, hosting and connectors, allocated across agents.
- Review time is the minutes people spend on held items and refusals, times a loaded hourly rate. Sample it if you cannot measure it.
- Certification and testing includes the time to prepare cases and review results, spread over the certificate's life.
- Incidents and rework covers time spent investigating and correcting the agent's errors, plus any customer redress.
- Outcomes counts completed units of work: claims opened, invoices matched, queries resolved. Not conversations, not calls to the model.
- Manual comparison is the cost of the same outcome done by people, from your existing unit costs.
The template
| Line | Item | Basis | Amount |
|---|---|---|---|
| 1 | Model usage | Provider billing, by agent | |
| 2 | Platform licence share | Allocated | |
| 3 | Integration and hosting share | Allocated | |
| 4 | Review of held items | Minutes x rate | |
| 5 | Review of refusals and handoffs | Minutes x rate | |
| 6 | Certification and testing | Hours x rate, over certificate life | |
| 7 | Monitoring and daily review | Hours x rate | |
| 8 | Changes | Hours x rate | |
| 9 | Incidents and rework | Hours x rate, plus redress | |
| 10 | Total cost | Sum of 1 to 9 | |
| 11 | Outcomes completed | Count | |
| 12 | Cost per outcome | 10 / 11 | |
| 13 | Manual cost per outcome | Existing unit cost | |
| 14 | Share handled without a person | Outcomes with no hold / 11 |
A filled example
A hypothetical insurer's claims intake agent, one month.
| Line | Item | Basis | Amount |
|---|---|---|---|
| 1 | Model usage | Provider billing | 1,800 |
| 2 | Platform licence share | One of four agents | 2,500 |
| 3 | Integration and hosting share | One of four agents | 600 |
| 4 | Review of held items | 900 items x 3 min at 40 per hour | 1,800 |
| 5 | Review of refusals and handoffs | 400 x 5 min at 40 per hour | 1,333 |
| 6 | Certification and testing | 30 hours per quarter at 60 per hour, monthly share | 600 |
| 7 | Monitoring and daily review | 20 hours at 50 per hour | 1,000 |
| 8 | Changes | 8 hours at 60 per hour | 480 |
| 9 | Incidents and rework | One incident, 10 hours at 60 per hour | 600 |
| 10 | Total | 10,713 | |
| 11 | Claims opened | 4,000 | |
| 12 | Cost per claim opened | 2.68 | |
| 13 | Manual cost per claim opened | Existing unit cost | 6.50 |
Lines 4 and 5 together are bigger than the model bill. That is common, and it shows where to look for savings: if one type of held item is always approved unchanged, it may be a candidate for evidence and a higher depth. Practical options are in how to reduce the running cost of AI agents.
Questions people ask
Should we include the cost of the people who would have done the work?
Not as a cost of the agent. Put it on line 13 as the comparison. Mixing the two hides whether the agent is cheaper per outcome.
How do we measure review time if nobody logs it?
Sample. Time twenty held items across a week and use the average. Repeat the sample after any change to the agent or its depth.
Where should cost tracking sit operationally?
In the agent's runbook, as a monthly task with a named owner, alongside the other routine checks.