What is retrieval-augmented generation?

Retrieval-augmented generation, or RAG, is a technique in which a system first retrieves documents or records relevant to a request and then passes them to a language model with the request, so the model answers from that material rather than from memory. It is how most AI agents get access to a business's own policies, products and records without retraining a model.

Why does RAG matter once agents act?

Because it decides what the agent knows at the moment it acts. An agent answering "can this customer switch tariff mid-contract?" needs the current tariff rules and that customer's contract. RAG fetches them. If it fetches the wrong version, a draft, or another customer's contract, the agent acts on the wrong facts with full confidence.

RAG also keeps knowledge current without touching the model. Change a policy, re-index it, and the agent uses the new version on its next request. No retraining is needed.

How does retrieval-augmented generation work?

Four steps:

1. Index. Documents are split into chunks and stored, usually as embeddings in a vector store, with metadata: source, version, owner, access rights.
2. Retrieve. The request is used to find the most relevant chunks, by meaning, keywords or both.
3. Filter. Chunks the agent is not permitted to use, or that are out of date, are removed.
4. Generate. The remaining chunks go into the model's context with the request, and the model answers, ideally citing which chunk supports each claim.

The filter step is the one most often missing. Without it, retrieval returns whatever is most similar, which may be a superseded policy or a document the agent should never see.

What does that look like in an example?

An energy supplier's billing agent is asked why a customer's direct debit rose. RAG retrieves the customer's last two bills from the billing system, the current direct debit review policy, and a help article. The filter drops an older version of the review policy that scored higher on similarity. The agent explains the increase from the bills and the current policy, citing both. A proposed change to the debit amount is drafted for the customer to confirm.

How is RAG different from prompt engineering?

Prompt engineering is about how you instruct the model. RAG is about what information you give it. A well-written prompt cannot make up for retrieving the wrong policy, and the right policy can still be misused with a poor prompt. Agents need both. RAG is also different from fine-tuning, which changes the model's weights; RAG leaves the model untouched and changes its inputs.

Does RAG replace the systems it reads from?

No. RAG reads from your systems of record and document stores; it does not become one. The ledger stays the authority on balances and the policy owner stays the authority on policy. A copy in a retrieval index is a convenience, and it should carry where it came from and when it was read. See does an agentic operating system replace your existing systems?

Where Pryme fits

The Pryme Context Protocol stamps every retrieved fact with its source and provenance class before the model sees it, so an action can be checked against what it rests on. A fact retrieved from an external page cannot authorise a payment, however relevant it looked. See the governed agentic operating system, explained.