Internal auditors need a competency matrix that turns AI agents into something they can test. The essential skills are reconstructing any agent decision from the record, checking that each agent's depth was earned and is still current, and testing whether people really decide held actions. Levels rise by audit grade, from associate to head of audit.
What should an internal audit matrix measure?
Two things that most first-line matrices ignore: the ability to test agents, and the ability to stay independent of them.
Testing comes first. An auditor examining an accounts payable process used to sample invoices and trace approvals. Now some of those approvals were made by an agent at depth 3, others were held and decided by a named person, and a few were refused. The auditor must know what each verdict means, which bound decided it, and whether the agent's certificate was valid on the day. A certificate lapses after 90 days, and a run that clears no depth drops the agent to depth 1 the same day, so "was the agent allowed to do this?" has a date-specific answer.
Independence comes second. If auditors use an agent to select samples or draft workpapers, they must be able to show that the agent did not shape the conclusion. The guide to what internal audit needs to know about AI agents covers the subject in full; this matrix turns it into levels.
How do you fill in the matrix?
Each row describes a competency at three levels. Assign a target level to each audit grade, record actual levels after testing, and plan training for the gaps. Keep the descriptions concrete enough that two audit managers would rate the same auditor the same way.
The template
| Competency | Level 1: understands | Level 2: performs | Level 3: leads | Target by grade |
|---|---|---|---|---|
| Agent vocabulary | Explains depth, the four verdicts, provenance classes | Uses them correctly in workpapers | Teaches them to the audit committee | |
| Reconstruction | Knows the record is append-only | Rebuilds a sampled decision end to end | Designs reconstruction tests across agents | |
| Certificate and depth testing | Knows depth is earned and lapses | Checks the certificate was valid on the action date | Tests the lapse and downgrade process itself | |
| Testing human review | Knows what a held item is | Re-performs a sample of reviewer decisions | Assesses whether review is real or rubber-stamped | |
| Provenance testing | Names the four classes | Checks an action relied on the right class | Tests whether external data could authorise anything | |
| Using agents in audit work | Knows the independence risk | Documents the agent's role in each workpaper | Sets the audit function's own rules for agent use | |
| Reporting findings | Describes an agent finding accurately | Rates its severity | Presents agent themes to the board |
A worked example
Consider a hypothetical mid-sized insurer whose audit team has four associates, three seniors, two managers and a head of audit.
| Competency | Associate | Senior | Manager | Head of audit |
|---|---|---|---|---|
| Agent vocabulary | 2 | 2 | 3 | 3 |
| Reconstruction | 1 | 2 | 3 | 2 |
| Certificate and depth testing | 1 | 2 | 3 | 2 |
| Testing human review | 1 | 2 | 3 | 3 |
| Provenance testing | 1 | 2 | 3 | 2 |
| Using agents in audit work | 2 | 2 | 3 | 3 |
| Reporting findings | 1 | 2 | 3 | 3 |
Associates reach level 2 on vocabulary and agent use straight away, because they will meet both on their first engagement. Reconstruction stays at level 1 until they have shadowed a senior through at least one full rebuild.
The row that separates a good audit team from a compliant one is "testing human review". At level 3 a manager looks past whether a person approved each held claim and asks whether they could have refused it. Signs of rubber-stamping include approvals with no written reasons, a reviewer approving every item in seconds, or no refusals at all across a quarter. Those are findings in their own right.
The head of audit sits at level 2 on reconstruction by design. They need to read a rebuilt decision and challenge it, not perform the rebuild.
How does the matrix connect to audit's own approvals?
Audit approves its own work: sample selections, workpapers, findings and the final report. If an agent drafts any of these, the approver authority matrix for the internal audit team should name who signs each one, and the approver rule for the internal audit team should require a level 2 or higher in the matching competency. Many audit functions also bar auditors from reviewing held actions in any process they audit, so their reviewer skills are tested in practice exercises rather than live queues. Pryme Intelligence Academy's Foundations track is one way to bring associates to level 1 quickly. The wider picture of who needs which skill is in AI agent skills by role.
Questions people ask
Do auditors need to understand how the model works?
No. They need to understand how the agent is governed: its bounds, its certificate, its sources and its record. Audit tests whether the controls around the agent work, not the model's internals.
What evidence should an auditor ask for first?
The record of verdicts for a sample period, the agent's certification history, and the list of people who approved held actions with their own certificate status on each date. Those three show most of what matters.
Should audit sit in the reviewer rota?
Generally not for live work in processes they audit. The reviewer rota for the internal audit team should cover audit's own agent-drafted workpapers and nothing in the first line.