Kiwi QA testers can move into AI operations by testing and overseeing agents: agent evaluation analyst, agent reviewer and, in smaller teams, a combined role that covers both. Software firms, banks, insurers and government digital teams adding agents need someone to prove they behave. Your testing skills are the foundation, and a certificate employers can check backs them up.
What does QA work look like in New Zealand today?
New Zealand testers work in software product companies, banks and insurers, government agencies' digital teams, and consultancies that deliver projects for them. Teams are often small, so a tester may own the whole test approach for a product: planning, manual and automated testing, accessibility checks, test data and release support. Contracting is common, and many testers move between projects and sectors.
That generalist profile suits the way smaller organisations adopt agents. They rarely hire a separate evaluation team and review team. They look for one person who can design the tests, run the review queue and explain the results to managers.
What changes as agents arrive?
- Test planning, depth 2. An agent drafts a test plan from requirements. You check scope and add risks the agent cannot know about, such as a known fragile integration.
- Accessibility checks, depth 1. The agent scans pages and flags likely issues. People confirm and prioritise.
- Regression runs, depth 3. Within bounds, the agent runs suites and files defects.
- Release, depth 4. Stays with a named owner.
- Testing agents that serve the public. For a government or bank service agent, you test that it stays within scope, gives no advice it is not allowed to give, escalates to a person when someone is vulnerable or upset, and grounds answers in current, approved sources.
Escalation is the test that matters most for public-facing agents. A service agent that keeps answering politely when a person says they cannot pay their rent has failed, even if every answer is accurate. A tester who writes cases for those moments, and checks the agent hands over, is doing oversight work.
Which AI operations jobs are open to QA testers?
Titles vary across New Zealand employers: "AI quality analyst", "test analyst (AI)", "AI operations", "digital assurance". Read the duties: evaluation, review, exceptions, quality and escalation.
| Role | What it involves | Where it is found |
|---|---|---|
| Agent evaluation analyst | Builds case sets and scores agents against floors | Software firms, banks, insurers |
| Agent reviewer | Clears held decisions with a reason | Organisations running agents in operations |
| Oversight generalist | Tests, reviews, keeps sources current, reports | Smaller organisations and agencies |
| Agent builder | Designs, trains and certifies agents | Teams building their own agents |
Our guide to AI agent jobs and careers maps these roles, and which industries in New Zealand are hiring people to supervise AI agents shows where to look.
What skills and certificate do employers look for?
They want testers who can design cases for agent failure modes, judge live decisions, and explain results plainly to non-technical managers. Contract roles in particular reward evidence that can be checked quickly. The overlap with testing is covered in why QA testers make strong AI agent reviewers.
Certified Pryme Operator, from Pryme Intelligence Academy, requires a written exam (40 questions in 60 minutes, pass mark four in five) and a two-week practicum of review decisions, nineteen in twenty correct with no missed red line, signed off by an assessor. It is issued through VerifyDoc.ai and checkable by number or QR code. It is a professional certificate, not a government accreditation. For practicum tips, see how QA testers can complete the operator practicum.
Which rules shape oversight work in New Zealand?
New Zealand has no AI-specific law. Its privacy law sets principles for collecting, using, storing and disclosing personal information, enforced by an independent privacy regulator. For testers, the principle that personal information is used only for the purpose it was collected matters most: production data copied into a test environment for an agent needs care, and synthetic or masked data is usually safer.
Public agencies have committed to principles on transparent and accountable use of algorithms, including human oversight and clear explanation, which shape how government agents are tested and released. Organisations serving Australian, EU or UK users may also fall under those markets' rules. Security testing can draw on the OWASP Top 10 for LLM Applications, and the NIST AI Risk Management Framework offers a voluntary structure.
This is an explanation of the rules, not legal advice; confirm how they apply to you with counsel.
What does a 90-day plan look like?
- Weeks 1 to 4. Complete the free Foundations track. Write escalation and refusal test cases for an imaginary public service agent.
- Weeks 5 to 8. Study the Operator modules with tests set in software. Start the practicum, on a simulated queue in a free workspace if needed.
- Weeks 9 to 13. Finish the practicum, sit the exam and share the certificate. Use how to get an AI operations job in New Zealand and how to find and win an AI operations job in New Zealand for the search.
Questions people ask
Do contract testers benefit from this?
Yes. A certificate that can be checked at source shortens vetting for each new contract, and agent testing skills widen the projects you can take.
Is this only for government work?
No. Banks, insurers, software firms and retailers also run agents. Government work adds strong expectations on transparency.
Should I learn automation first?
Not for review or evaluation analyst roles. It helps for larger evaluation programmes and builder work.