Software companies that run governed AI agents in support, billing and account management need people to review escalations, approve held refunds and account access changes, build the test cases agents are certified against and keep documentation current. Support specialists, customer success staff, technical writers and QA testers are best placed, and most of these roles do not involve writing code.
Why do software firms need non-engineering AI roles?
Software companies were early to put agents in front of customers, so they met the problems early. A support agent can answer how-to questions from the documentation, triage bugs, issue refunds for failed renewals and reset access. Each task is easy until it is not. A customer locked out of an account may be the owner or an attacker. A refund request may be fair or part of a pattern.
In a governed setup, routine answers flow at depth 3 within the agent's certified scope. Account ownership changes, refunds above the firm's limit and anything touching security are held or refused. The engineering team builds the product. The people-side operations team decides what the agent's held work means.
What are the new jobs?
| Role | What it involves | Often comes from |
|---|---|---|
| Support escalation reviewer | Takes over cases the agent held or refused, and checks its answers on hard tickets | Senior support specialists |
| Billing and refunds reviewer | Approves held refunds, credits and plan changes | Billing support, customer success |
| Account access reviewer | Decides held ownership transfers and access recovery requests | Support, trust and safety staff |
| Agent test case curator | Builds the set of real past tickets used to certify the agent and to catch drift | QA testers, support leads |
| Documentation keeper | Keeps the help articles and internal notes the agent reads accurate | Technical writers, support specialists |
| Agent owner | Sets the agent's bounds and argues for more depth on evidence | Support operations managers |
Our article on what AI agent jobs exist in software lists more titles and variants.
What does the test case curator do?
This role is particular to software, where firms often have years of ticket history. Certification replays an agent against real cases from the business's own operation and scores each answer against a floor. Someone has to choose those cases: a fair mix of routine, tricky and dangerous tickets, refreshed as the product changes. If the test set is stale, the agent's certificate says little about how it will behave on next month's tickets.
Take a QA tester we will call Aidan at a mid-sized software company. When the product adds single sign-on, he pulls the first hundred tickets about it, writes the correct answer for each with the support lead, and adds them to the certification set. The agent fails twelve of them on its next run and drops to read-only for that topic the same day. The docs keeper fixes the article it was misreading, the agent passes, and sign-on questions go back to the agent. Aidan's work set the bar.
What rules and standards shape these roles?
Software firms selling to businesses often hold a SOC 2 report or ISO/IEC 27001 certification, and their customers expect agents to fit within those controls. The OWASP Top 10 for LLM Applications describes risks such as prompt injection and excessive agency, which account access reviewers need to understand: a support ticket can contain instructions aimed at the agent rather than at a person. Customer data brings GDPR in the EU, UK GDPR in the UK and state laws such as California's CCPA as amended by the CPRA in the US.
Our pieces on AI operations jobs in software in Ireland and AI operations jobs in software in the US cover how those rules play out in each market.
How does software compare with retail?
Both serve large numbers of customers with repetitive questions, and both deal with refunds and account issues. Retail's hard cases are physical, such as damaged goods and missing stock; see the new AI operations jobs in retail. Software's hard cases are about access, security and product behaviour. Reviewers need to understand how the product works well enough to know when the agent's answer is wrong.
What is the trade-off?
These roles sit close to engineering but are not engineering. People who want to move into development may find they help, since they teach you how the product fails in practice. Others may feel stuck between support and engineering without fully belonging to either. Ask how the role is valued before you take it.
There is also a pace problem. Software products change often, so docs and test cases go stale quickly. Keepers and curators can feel they are always catching up. That is the job, and the people who do it well become central to how the company supports customers.
How do you get into one of these roles?
Inside a software firm, offer to curate test cases from the tickets you handle or to own a section of the help centre. Track the agent errors you catch. From outside, show experience with support, QA or documentation. Our guide on how to get into AI operations in software covers routes, and the AI agent jobs and careers guide maps the wider field. Pryme Intelligence Academy's Builder track, which ends with a capstone agent certified to depth 2 or higher, suits people who want to move from reviewing agents into designing them.
Questions people ask
Do I need to code for these roles?
Most do not require it. Reading logs and understanding how the product works help. Test case curation and documentation are writing and judgement jobs.
What is excessive agency?
It is the risk that an agent can take actions beyond what its task needs, listed in the OWASP Top 10 for LLM Applications. Governed agents limit this through role scope and depth.
Is support operations a dead end?
It need not be. People who understand how agents behave on real tickets move into agent ownership, support operations leadership and, with further study, product roles.