Document review is the paralegal skill that maps most directly onto agent review. You become an AI operations analyst by checking a legal agent's clause flags, disclosure calls and due diligence summaries, learning how its bounds keep it away from advice and privilege, and reading its record to show where it misses things and why.
What does an AI operations analyst do in a legal team?
Law firms and in-house legal teams now use agents to extract clauses from contracts, flag unusual terms, sort documents for disclosure by likely relevance, summarise data rooms and fill standard forms. Lawyers still decide privilege, give advice and sign anything that leaves the firm. The agent's outputs are drafts or flags for a person to confirm.
Reviewers confirm or correct those flags. Analysts read across them: which clause types the agent misses, how often documents it marked irrelevant turn out to matter, which matter types produce the most corrections, and whether the sources it relies on, such as playbooks and standard positions, are current. The firm owns the legal risk; the analyst owns the measurement and the recommended fix. For the general route, see how you become an AI operations analyst; the guide to AI agent jobs and careers maps the wider field.
What steps take you from paralegal to analyst?
- You, weeks 1 to 2: write down your review standards. What makes a clause unusual, how you decide relevance, when you flag possible privilege to a lawyer.
- You, weeks 2 to 4: learn the decision model. Depth from 1 (read) to 2 (draft, a person sends) for most legal work, the four verdicts, and why advice and privilege sit outside every agent's role scope.
- You, weeks 4 to 7: review a legal agent's flags. In your firm if one is running, otherwise on a simulated queue. Record every correction and its reason.
- You, weeks 7 to 9: earn a certificate employers can check by number or QR code.
- You, weeks 9 to 11: write one analysis. Compare the agent's flags with your corrections on a sample. Report which clause types or document types it handles worst, and recommend a fix with an approver.
- You, weeks 11 to 13: apply. Legal operations, litigation support and knowledge teams are natural homes.
Where does a legal agent act, and where do people?
Take an illustrative commercial firm handling an acquisition and a dispute.
| Step | The agent does | Depth | Stays with a person |
|---|---|---|---|
| Extract key clauses from target contracts | Lists change-of-control, termination and liability terms | 1 | Paralegal checks a sample; analyst measures misses |
| Flag terms outside the firm's playbook | Drafts a flag with the playbook reference | 2 | Lawyer decides whether it matters |
| Sort disclosure documents by likely relevance | Proposes a relevance call for each | 2 | Reviewers confirm; analyst tracks overturned calls |
| Mark a document as possibly privileged | Holds it for a lawyer | 2 | A lawyer decides privilege |
| Disclose documents to the other side | Refused: outside the agent's role scope | none | The lawyer with conduct of the matter |
The analyst's key number is how often the agent's calls are overturned, and which way. An agent that marks too much as relevant wastes review time; one that marks too little risks missing a document that matters. The second is far worse, and your analysis should say so.
How is analyst work different from paralegal review?
As a paralegal you review documents. As an analyst you review the reviewer: the agent. That means sampling deliberately, not just checking what you are given. Take a random set of documents the agent marked irrelevant and check them. Count clauses it missed by type. Report the pattern with enough detail that a lawyer can decide whether to change the playbook, the agent's sources or its limits.
Report the misses in a way lawyers can use. A supervising associate does not need every error; they need to know that the agent misses termination clauses buried in schedules, that the misses come from one template, and that a change to the playbook or the extraction instructions would fix it. One page with that, and a sample to check, is a finding they can act on.
Provenance matters too. A contract taken from the client's own document system is a stronger source than a copy forwarded by email, which may be an earlier draft. An analyst who notices that misses cluster on emailed copies has found a cause, not just a symptom.
How do you measure progress?
Count flags reviewed with reasons and agreement with a marked standard, aiming for nineteen in twenty. Count sampling exercises you have designed and run. Count analyses that separate serious misses from minor ones. Nobody can promise the role, but a paralegal who brings a measured sample of an agent's errors is doing analyst work already.
Where does the move usually go wrong?
- Checking only what the agent flagged. The misses are in what it did not flag.
- Treating all errors alike. A missed change-of-control clause matters more than a misread date format.
- Recommending legal positions. Analysts recommend changes to sources and limits; lawyers decide positions.
- Not comparing routes. Former business analysts and former fraud analysts bring different strengths to the same seat. Pryme Intelligence Academy's Operator track is one way to certify the review skill.
Questions people ask
Does this role suit paralegals who want to become lawyers?
It can, as experience of how legal work is changing. But it is a different path: analysts specialise in how agents perform, not in legal advice. Choose based on which work you want to do in five years.
Is legal operations work common in the US?
Many US firms and in-house teams have legal operations functions, and litigation support has long used technology for document review. For local points, see becoming an AI operations analyst in the US.
Do I need to know how the agent's relevance scoring works?
No. You need to measure whether its calls are right and explain the pattern of errors. The people who build the agent use your findings to change it.