Keep one standard and deliver it locally. Write the red lines, the bar for sign-off and the pattern for reasons once, in a single reference language. Teach them in each team's own language with local examples. Then calibrate across languages on the same cases, so a decline in Lisbon and a decline in Lagos mean the same thing.
What is the problem with several languages?
An organisation running agents across countries often has reviewers working in several languages, and an agent that drafts in all of them. Three things can drift apart. The standard: what counts as a red line or a good decision. The delivery: how well each team understood it. And the judgement: whether reviewers in different languages actually decide alike.
Translation solves only the first, and only partly. A policy translated word for word can still be read differently, and an example that makes sense in one market may be meaningless in another. The aim is a single standard, understood locally and applied consistently.
The head of operations or the global learning lead owns the programme. A local lead in each language team runs delivery. A small group of bilingual senior reviewers acts as calibrators across languages.
What must be the same everywhere, and what can differ?
The same everywhere: the red lines; the sign-off bar; the reason pattern; the meaning of each verdict (allowed, held, refused, withheld); the concepts of depth and provenance; the language of the record, or a rule that every reason can be read by whoever audits it.
Local: the examples, which should use the market's own documents, customer phrasing and pressure tactics; the delivery format; and local rules that sit on top. Under the EU AI Act, Article 4 asks providers and deployers to ensure a sufficient level of AI literacy among staff dealing with AI systems, which applies to teams in EU markets whatever language they work in. This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
The steps
- Person (global lead): writes the standard in one reference language: red lines, sign-off bar, reason pattern, verdict definitions.
- Person (local leads): translate and test the standard, checking each red line with two reviewers to confirm the meaning survived.
- Person (local leads): build local examples from their own market's cases for every concept.
- Person (local teams): complete the learning in their own language.
- Person (calibrators): assemble a shared calibration set of 30 cases, each available in every working language with the same facts.
- Person (all reviewers): decide the shared set independently; results are compared across languages, not just within teams.
- Agent, depth 2: drafts live replies in each language; reviewers in that language decide in shadow while a certified local reviewer decides for real.
- Person (calibrators): cross-read a sample of reasons and decisions from each language team monthly.
- Agent, depth 1: logs decisions with the language recorded, so differences between teams are visible.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Reply in the customer's language | Drafts from approved content in that language | 2 | Reviewer fluent in the language sends or edits |
| Routine answer in a certified language | Answers | 3 | Local reviewer samples |
| Held action | Prepares and holds | held | Local reviewer with rights decides |
| Request in an unsupported language | Refuses and hands over | refused | A person handles it or arranges help |
| Cross-language sample | Selects decisions per language | 1 | Calibrators compare |
How do you calibrate across languages?
Use the same thirty cases with identical facts, each rendered naturally in every working language, not machine-literal. Ask every reviewer to decide alone. Then compare the rates of approval, edit and decline by language team, and read the reasons.
Differences usually come from one of three sources. A translated red line that shifted meaning. A cultural norm, such as how directly a refusal can be worded. Or a local team with a weaker understanding of a concept, often provenance. Each needs a different fix: correcting the translation, agreeing a local tone guide within the standard, or reteaching the concept. Calibrators should say which it is.
Which controls matter, and what evidence do they leave?
- Single reference standard. Evidence: the versioned standard and each translation with its check.
- Shared calibration set. Evidence: decisions by language team and the agreed answers.
- Cross-language sampling. Evidence: the calibrators' monthly findings.
- Fluent reviewers only. Evidence: the admin record of which reviewers may approve which languages.
- Language recorded with each decision. Evidence: the language field in the record.
What should you measure?
Agreement on the shared set within and across language teams. Findings from cross-language sampling, by cause. Red-line misses by team, which must be zero everywhere. And complaints by language, which can reveal a team or a translation that is falling behind.
Where does it usually go wrong?
The standard is written in the reference language and translated once, never tested. Meaning drifts silently.
Calibration happens only inside each team, so each language develops its own standard.
Reviewers approve drafts in a language they read only partly, because the queue is long. Approval rights should follow fluency.
How agents themselves handle languages is covered in whether an agent can give support in several languages and answering customers in several languages with an agent. For teams split by country as well as language, see training a team across several countries. Practical session design is in running calibration sessions for patient services teams. The guide to training teams for AI agents covers sequencing.
Where the Academy fits
Pryme Intelligence Academy teaches one standard for every learner, with module tests set in the learner's own industry and role. Lessons can be read or heard as audio, with highlights and notes, which helps reviewers working in a second language. Certificates are issued through VerifyDoc.ai and can be checked at source by number or QR code, wherever the reviewer works.
Questions people ask
Should the record be kept in one language?
It helps auditing if it is, but it is not essential. What matters is that anyone who must review a decision can read its reason, through a shared language or a reliable translation process.
Can one bilingual reviewer cover two language teams?
Yes, and such reviewers make good calibrators. Make sure they are not the only approver for either language, or absence will stall the queue.
Do local regulations change the standard?
They can add to it. The global standard is a floor; a market's rules may require more, and local leads should document any addition.