Software companies create two sets of agent jobs: those around the agents they ship to customers, and those around the agents they run internally. Both need agent builders, evaluation testers, support and operations reviewers, AI operations analysts, agent owners and security staff. Not all of these are engineering roles; testers, support staff and product people move into many of them.
How is software different from other sectors?
In most industries, agents are bought and run. In software, they are also built and sold. That adds roles on the build side: designing agents, testing their behaviour and proving they work before a customer relies on them. It also means support teams handle questions about agents that customers run, which needs people who understand how agents fail.
Which jobs exist around software agents?
- AI agent builder. Turns a job description into an agent: its brief, sources, connections and bounds. Then trains it on real cases and gets it certified. Builders need product sense and patience more than deep machine learning.
- Evaluation tester. Designs the cases that test an agent's behaviour: does it refuse out-of-scope requests, escalate risky ones, stay grounded in its sources? QA testers move into this naturally.
- Support reviewer. Checks what the company's own support agent holds: refunds, account changes, technical answers it is unsure of.
- AI operations analyst. Reads the record across many customers or tickets, spots drift, proposes changes.
- AI agent owner. Answers for an agent's scope, sources and certification. Often a product manager or support operations lead.
- Security and trust staff. Test for risks such as prompt injection, excessive agency and sensitive information disclosure, which the OWASP Top 10 for LLM Applications lists.
What does evaluation work involve?
Pryme's Attest certifies an agent by replaying it against real cases from the customer's own operation and scoring every answer against a floor. Seven gates must each clear on their own: grounded, refuses, depth, escalates, scope, adversarial and stable. Each gate needs well-designed cases and people who can judge the answers.
Take an evaluation tester we will call Yusuf at a company that sells billing software with a built-in support agent. He builds a set of hard cases: a customer asking for another customer's invoice, a refund request with instructions hidden in a pasted document, a question the knowledge base does not answer. The agent fails one: it follows the hidden instruction. Yusuf writes the failure up with the case and the evidence, and the builder changes the agent's bounds. Because a run that clears no depth drops an agent to depth 1 the same day, the failure has an immediate consequence.
What stays with people in software?
- Deciding what an agent is for and what it must never do.
- Judging whether a certification result is good enough to ship.
- Customer-facing decisions the agent holds, such as refunds and account changes.
- Security reviews and incident response.
- Explaining to customers how the agent decides, which they will ask.
Which controls matter most?
| Control | Who works on it |
|---|---|
| Brief and bounds | Builder, agent owner |
| Evaluation cases and certification | Evaluation tester, builder |
| Source provenance | Builder, operations analyst |
| Record of every verdict | Operations analyst, support reviewer |
| Security testing | Security staff |
How do you move into these jobs?
QA testers, support engineers, technical writers and product people all have a route in. The new AI operations jobs in software covers each role in depth, and market detail is in AI operations jobs in software in Australia and in Singapore. Software companies selling into online retail face many of the same questions; see AI agent jobs in e-commerce. The guide to AI agent jobs and careers covers all sectors.
Where the Academy fits
Pryme Intelligence Academy's Builder track is the closest fit for software roles: about 24 hours plus a capstone agent certified in Attest to depth 2 or higher, leading to Certified Pryme Builder, valid for 12 months. Every lab runs in a free Build workspace, so you can practise without an employer. Support reviewers would start with Operator instead.
Questions people ask
Do I need to be a software engineer to build agents?
Not always. Building a governed agent is mostly about defining the job, the sources and the bounds, and proving it on real cases. Engineering helps with integrations.
Are evaluation roles separate from QA?
Sometimes. In smaller companies, QA testers take on agent evaluation. Larger ones may create dedicated evaluation or trust roles.
What should I show in a portfolio?
An agent you built or tested, the cases you used, what failed and what you changed. A checkable certificate adds independent evidence.