Legal operations teams need a competency matrix that keeps legal judgement with lawyers while agents take on intake, contract triage, first-pass reviews and deadline tracking. Paralegals must check every agent comment against the playbook, spot when a request strays into advice, and know which drafts they may send and which need a lawyer.
What does the matrix protect against?
Two failures specific to legal work. The first is the confident wrong clause: an agent reviewing a supplier contract flags a liability cap as acceptable because it misread the playbook, and the paralegal passes it on because the comment reads well. The second is drift into advice: a business colleague asks the intake agent whether they can terminate a contract, and the agent answers rather than refusing and routing to a lawyer.
Both are skill problems more than technology problems. A paralegal who checks that each comment is grounded in the current playbook version catches the first. A team that knows what a correct refusal looks like catches the second. The matrix names who must hold each skill, and it sits beneath the broader map of AI agent skills by role.
How do you fill it in?
Use a simple three-step scale per skill: Aware, Competent and Expert. Then do three things:
- List the agent's capabilities and their depth. In most legal teams the agent reads at depth 1 and drafts at depth 2; nothing leaves the team without a person.
- For each capability, write the skill a person needs to oversee it, and set targets for paralegals, the legal operations manager, and the supervising lawyer.
- Record evidence and a review date. Legal playbooks change often, so tie re-checks to playbook updates as well as to the calendar.
The template
| Agent capability (depth) | Skill needed | Paralegal | Legal ops manager | Supervising lawyer | Evidence | Next review |
|---|---|---|---|---|---|---|
| Matter intake and routing (1) | Recognises an advice request the agent should refuse | |||||
| NDA triage against playbook (2) | Checks each flagged clause against the current playbook | |||||
| First-pass contract review (2) | Weighs the comment's source and confidence | |||||
| Deadline and renewal tracking (1) | Confirms dates against the signed contract | |||||
| Outside counsel invoice review (2) | Checks rates and hours against the engagement terms | |||||
| All | Reads why an item was held or refused | |||||
| All | Reconstructs a sent draft from the record | |||||
| All | Keeps the playbook source current |
A worked example
Take a hypothetical in-house legal team at a logistics company: four paralegals, one legal operations manager and two supervising lawyers.
| Capability | Skill | Paralegal | Ops manager | Lawyer | Evidence | Next review |
|---|---|---|---|---|---|---|
| Intake (1) | Spots advice requests | Competent | Expert | Expert | Scenario test | Quarterly |
| NDA triage (2) | Checks against playbook | Competent | Competent | Expert | 25 NDAs re-marked | On playbook change |
| Contract review (2) | Weighs source | Aware | Competent | Expert | Lawyer sample | On playbook change |
| Deadline tracking (1) | Confirms dates | Competent | Expert | Aware | Monthly check | Monthly |
| Invoice review (2) | Rates and hours | Competent | Expert | Aware | Sample check | Quarterly |
| All | Reads held or refused | Competent | Expert | Competent | Foundations test | Yearly |
| All | Reconstructs a draft | Aware | Expert | Competent | Walkthrough | Yearly |
| All | Playbook source current | Aware | Expert | Competent | Source log | On change |
Paralegals are only Aware on full contract review. They can read the agent's comments, but the decision to accept a non-standard clause stays with a lawyer. For NDAs, which follow a tight playbook, paralegals reach Competent and may send the agent's mark-up once they have checked it.
The legal operations manager holds Expert on keeping the playbook source current. If the playbook changes and the agent's source is not updated, every review that follows is grounded in the wrong document. That is a process failure, and the manager owns it.
The wider operations version of this matrix is the AI agent competency matrix for the operations team. Who covers the queue on which day belongs in the reviewer rota for the legal operations team, and which roles across a firm need training at all is covered in which roles in legal services need AI agent training.
Where the Academy fits
Pryme Intelligence Academy's Operator track trains paralegals and legal operations managers in human review and in keeping knowledge current, two modules that map directly to the rows above. Its module tests are set in legal services for legal staff, so practice cases look like their own. Supervising lawyers usually need Foundations only, unless they review the queue daily.
Questions people ask
Can a paralegal approve an agent's contract mark-up?
For documents covered by a tight playbook, such as standard NDAs, many teams allow it once the paralegal is Competent. For anything with negotiated terms, the mark-up is a draft for a lawyer.
What should the agent do with a request for legal advice?
Refuse it and route it to a lawyer, with a short note saying why. A correct refusal is a success, and paralegals should be able to recognise one.
Does legal privilege change the skills needed?
It adds one: knowing what the agent's record holds and who can see it. The legal operations manager should understand where agent drafts are stored and how access is controlled.