Practicum log template for internal auditors

An internal auditor's practicum log records audit judgements, not approvals. For each sampled agent decision, the auditor notes what was tested, which evidence they reconstructed from the record, their conclusion and reason, and a senior auditor's independent conclusion. The template below adds three red lines specific to auditing agents, and a worked example from a payments deployment.

What does an auditor "decide" in a practicum?

Most practicum logs record operational decisions: release, hold, return. Internal auditors do not make those. Their decisions are conclusions: this verdict was supported by the evidence, this approval was made by someone entitled to make it, this control operated or it did not. A practicum tests whether the auditor reaches the right conclusion on real items from a live deployment.

The raw material is the append-only record. Every verdict an agent produces is filed there with the bounds it met, the deciding bound, the sources and their provenance, and who approved any held action. A capable auditor can reconstruct any decision from it. A practicum checks that they actually do, rather than relying on the agent's own reports. The two weeks are set up as described in how to run a two-week practicum for internal auditors.

What should the auditor test?

A useful sample mixes four kinds of item:

• Allowed actions: did the action sit within all five bounds on that day?
• Held actions that were approved: did the approver hold a current credential for that depth on that date, and was the approval within their limit?
• Refusals and withheld actions: was the verdict correct, and was any override properly authorised?
• Certificate events: when the agent's certificate lapsed after 90 days, or a run cleared no depth and the agent dropped to depth 1, did the depth actually change that day?

The practicum log template

Header: auditor; senior auditor with a current certificate; deployment and agents in scope; period tested; sample size; dates; bar of at least 95 correct conclusions in every 100 and no missed red line.

Red lines

1. Concluding "no exception" on an item without reconstructing it from the record.
2. Missing an approval made by someone whose credential had lapsed on that date.
3. Accepting the agent's own summary or dashboard as the evidence.

A worked example

Take a sample payments business auditing a refunds agent over one quarter. Values are illustrative.

Test 3.1. Control: held refunds above 1,000 are approved by a certified person within their limit. Item: verdict R-5521, a refund of 1,400 held and approved. Evidence: record shows approver J. Lim; the credential register and the issuer's page show J. Lim's certificate lapsed nine days before the approval. Auditor: exception. Senior: exception. Red line 2: caught.

Test 3.2. Control: the agent's depth drops to 1 the day a certification run clears no depth. Item: the run failure on a Tuesday. Evidence: the auditor took the date from the monthly dashboard and confirmed no refunds were allowed that day. Senior: reconstructed from the record; two refunds were allowed at depth 4 in the hour between the failed run and the drop. Match: no. Red line 3: missed.

Test 3.3. Control: withheld actions cannot be overridden. Item: a request to reverse a settled refund, filed as withheld. Evidence: record shows the withheld verdict and no subsequent action. Auditor: supported. Senior: supported. Match: yes.

Test 3.2 fails the bar. The dashboard summarised by day, the record by minute, and only the record showed what happened.

What should the senior auditor look for at the end?

Beyond the score, look at the evidence column. An auditor who repeatedly lists the same kind of evidence, such as the dashboard, is not reconstructing. Count "unable to conclude" too: it is a legitimate answer, and an auditor who never uses it may be forcing conclusions.

The structure is close to the logs kept by the people auditors test. The practicum log for risk managers records the risk team's own judgements, and the log for underwriters shows the authority-limit checks auditors will later test. Whether auditors themselves need a certificate depends on whether they approve anything; see do internal auditors need a certificate to approve AI agent actions. The Builder and Architect tracks at Pryme Intelligence Academy go deeper into how agents are certified, which helps auditors who test certification itself. For the wider picture, see AI literacy and certification for people who work with agents.