How to run calibration sessions for financial controllers

Financial controllers calibrate on the journals and reconciling items an agent drafted and held during the close. Each controller reviews the same set alone: post, amend or reject, with the reason and the evidence. Compare across controllers and entities, agree a gold standard with the group finance director, and change the agent's brief before the next month end.

Why do financial controllers need calibration?

During the month-end close, an agent can draft accruals from purchase orders and goods receipts, propose prepayment releases, match intercompany balances, and explain reconciliation differences. It drafts at depth 2; posting to the general ledger stays with a controller or with the agent at depth 3 for routine, certified journal types within the company's own limits.

Controllers in different entities, or even in the same team, apply judgement differently. One accrues for a service once the supplier's email confirms delivery; another waits for a goods receipt. One clears a small intercompany difference to a suspense account; another investigates every one. Group results then contain inconsistencies that auditors find before management does. And the agent, corrected differently by each controller, drafts differently by entity. The general method is in how to run calibration sessions for AI agent reviewers.

The group financial controller owns the sessions. One entity controller facilitates. The external audit relationship lead may observe once a year.

Which items belong in the set?

Eight to ten items from the last close, drawn from several entities:

• An accrual based on a supplier's email confirming work done, with no goods receipt. The email is external.
• A prepayment release for a contract whose term changed mid-year.
• An intercompany difference of a small amount that has persisted for three months.
• A bank reconciliation difference the agent could not explain.
• A journal just above the agent's posting limit for a certified journal type.
• A journal the agent refused because the account code was outside its scope.

The session, step by step

1. Person (group financial controller): selects items from several entities and removes preparer names.
2. Person (each controller): reviews each item alone: post, amend or reject, with the reason and the evidence relied on.
3. Agent, depth 1: shows its drafted journal, the sources and their provenance, for example a purchase order and receipt from the purchasing system (sealed), a supplier statement (signed, if from a verified supplier portal), or an email (external).
4. Person (facilitator): reveals the spread, starting with the widest.
5. Person (controllers): explain their judgements, including materiality.
6. Person (group financial controller): agrees the gold standard, referring accounting policy points to the group finance director.
7. Person (group financial controller): sends the standard to the brief owner, including changes to which journal types the agent may post.
8. Person (group financial controller): reviews a sample of journals at the next close.

What does a gold standard look like for the close?

It sets the evidence threshold and the action. For example: "An accrual for services may be based on a supplier's emailed confirmation only below a set amount, and is flagged as resting on external evidence. Above that amount, a goods receipt or the budget holder's confirmation in the purchasing system is required."

On intercompany: "Differences below a set amount are cleared to a named account after one month and investigated at quarter end. Above it, both entity controllers agree the cause before the close."

On posting: "The agent posts certified recurring journals at depth 3 within its limit. Any journal touching revenue, provisions or tax is drafted only."

What does a session look like?

Take a hypothetical group with four entities. On the emailed accrual, two controllers posted it, one rejected it, and one posted half. Discussion showed that two entities had a local habit of accruing on email because their purchasing system was used inconsistently. The standard above was agreed, the brief owner changed the agent to flag email-based accruals above the threshold, and the group controller asked both entities to raise receipts properly. At the next close, the agent held fewer email-based accruals, because more receipts existed.

Which controls does calibration support, and what evidence does it leave?

Consistent judgements across entities. Evidence: spreads by entity over successive sessions.

Evidence thresholds. Evidence: the standard, approved by the group controller.

Agent posting scope. Evidence: the recorded decision on which journal types the agent may post.

Audit readiness. Evidence: the standards themselves, which auditors can test against.

What should the group financial controller measure?

• Agreement on shared items, by entity.
• Journals posted on external evidence alone above the threshold. The target is none.
• Audit adjustments related to accruals and intercompany, year on year.
• How often controllers amend the agent's drafted journals.

Where does it go wrong with financial controllers?

Calibrating during the close. Nobody has time. Use the week after.

Each entity calibrating alone. Group consistency is the point.

Not linking to the agent's posting scope. If calibration shows controllers disagree on a journal type, the agent should not be posting it at depth 3.

Credit controllers and underwriters calibrate on very different decisions; see calibration for credit controllers and for underwriters. For planning, see the AI agent training plan template for financial controllers, and the topic guide on training teams for AI agents.

Where the Academy fits

Pryme Intelligence Academy's Operator track covers human review, teaching the agent and keeping knowledge current, which maps onto turning a calibrated close standard into a change to the agent's brief. Controllers who design and certify the close agent itself may need someone on the Builder track, whose capstone is an agent certified in Attest to depth 2 or higher.