How to run calibration sessions for HR advisers

HR advisers calibrate on the employee cases an agent drafted and held: absence triggers, flexible working requests, policy questions and anything about health or complaints. Each adviser decides the same anonymised cases alone and edits the draft reply. Compare, agree a gold standard with the head of HR, and update the agent's brief so every team gets the same answer.

Why do HR advisers need calibration?

An HR agent typically answers routine policy questions from the handbook at depth 3, drafts letters for absence reviews and flexible working decisions at depth 2, and holds anything involving health, a grievance, a disciplinary matter or a possible discrimination issue for an adviser.

Consistency matters more in HR than in most functions. Two employees with the same absence record who receive different letters have grounds to complain, and inconsistent treatment can become evidence in a dispute. Advisers also give advice to managers, so one adviser's habit spreads through every team they support. Without calibration, the agent learns from whichever adviser edits its drafts most. The general method is in how to run calibration sessions for AI agent reviewers.

The head of HR or HR operations owns the sessions. A senior adviser facilitates. An employment lawyer may attend when standards touch legal risk.

Which cases belong in the set?

Eight to ten cases from the past month, fully anonymised:

• An absence trigger where the absences relate to a disclosed health condition.
• A flexible working request that the manager wants to refuse on grounds that may not hold.
• A policy question about parental leave that the handbook answers ambiguously.
• An employee email that mixes a holiday query with a complaint about their manager.
• A request from a manager for an employee's sickness records.
• A case the agent refused, such as a request to draft a dismissal letter.

The session, step by step

1. Person (senior adviser): selects and anonymises the cases. Health details are reduced to what the case needs.
2. Person (each adviser): decides each case alone and edits the drafted reply or letter as they would send it.
3. Agent, depth 1: shows its draft, the policy sources it used and why it held the case, for example a red line on health information or a grievance.
4. Person (facilitator): reveals the spread, on advice and on wording.
5. Person (advisers): explain their reasoning.
6. Person (head of HR): agrees the gold standard, consulting the employment lawyer where needed.
7. Person (head of HR): sends the standard to the brief owner and updates the handbook if the policy itself was unclear.
8. Person (head of HR): samples letters and advice for a month.

What does a gold standard look like for HR?

It states the decision rule and the wording. For example: "Where absences relate to a disclosed health condition, the trigger letter is replaced by an invitation to a supportive conversation with the adviser present, and the agent holds every such case. Absences linked to a condition are recorded separately in the review."

On mixed emails: "An email containing a complaint about a manager is treated as a potential grievance first. The adviser replies to the holiday question separately and asks whether the employee wants to raise the concern formally."

On records: "Managers do not receive sickness records. The adviser shares only what the manager needs to manage attendance, and the agent refuses requests for records from anyone other than HR."

What does a session look like?

Take a hypothetical HR team of seven advisers supporting 2,000 staff. On the flexible working case, four advisers supported the manager's refusal, two asked for alternatives to be explored first, and one said the refusal ground did not apply. The employment lawyer agreed with the one. The standard became: every refusal names a business ground from the policy, and the adviser checks the ground applies before the letter goes. The brief owner changed the agent so it drafts a refusal only after the adviser selects the ground.

Which controls does calibration support, and what evidence does it leave?

Consistent treatment of employees. Evidence: spreads over sessions.

Health and special-category data. Evidence: the standard on records and the agent's refusals.

Grievance recognition. Evidence: the mixed-email standard and sampled replies.

Policy clarity. Evidence: handbook changes made after calibration.

What should the head of HR measure?

• Agreement on shared cases, session by session.
• Letters sampled that do not meet the standard.
• Potential grievances that were answered as ordinary queries.
• Handbook sections that produced repeated disagreement.

Where does it go wrong in HR teams?

Too little anonymisation. Advisers recognise colleagues and the discussion becomes personal.

Calibrating letters, not advice to managers. Much of HR's influence is in what it tells managers. Include one manager-advice case each session.

Leaving the handbook unchanged. If the policy is ambiguous, calibration patches it each month. Fix the source.

Wealth advisers and IT administrators calibrate on very different questions; compare calibration for wealth advisers and for IT administrators. For planning, see the AI agent training plan template for HR advisers, and the topic guide on training teams for AI agents.

Where the Academy fits

Pryme Intelligence Academy's Operator track covers human review and keeping knowledge current, which for HR means keeping the agent's handbook sources in step with policy. Module tests are set in the learner's own industry and role, so HR advisers practise on held employee cases rather than generic examples.