RAG (retrieval-augmented generation)


Updated 14 October 20262 min read

Retrieval-augmented generation (RAG) is a technique where an AI system first retrieves relevant passages from your own documents, then has a language model answer using only those passages, ideally citing them.

RAG, short for retrieval-augmented generation, is the standard way to make a language model answer questions about documents it was never trained on. Instead of hoping the model remembers the answer, the system first retrieves the most relevant passages from your documents and puts them in the prompt, then asks the model to answer from that context.

A RAG system has two phases. At ingestion, each document's text is extracted, split into chunks of a few hundred words with some overlap, and each chunk is turned into an embedding: a vector of numbers that captures its meaning. The vectors are stored in an index. At question time, the question is embedded the same way, the closest chunks are found with vector search, and the top few are passed to the model along with instructions to answer from them.

Done well, RAG gives answers that are specific, current and checkable. Because the system knows which chunks it supplied, it can number them and show citations, so a reader can jump to the passage behind each statement. That is the main defence against confident but invented answers: if a claim has no citation, treat it with suspicion.

RAG has clear limits. Retrieval only finds what the text layer contains, so scanned PDFs need OCR first. Chunking can split a table or separate a heading from its content. Questions that need every page, such as summarising a whole long report, are poorly served by retrieving a handful of chunks and work better as several narrower questions. And RAG is the wrong tool for numbers that live in a database: summing a column from retrieved text snippets is unreliable, while a database query is exact.

That is why wuup uses both approaches. Documents you upload go through RAG and answers cite the numbered passages they used. Structured data in MongoDB or Google Sheets is answered by running a real query or reading a real range, with the query shown. One chat can combine both, with each kind of source cited in its own way.

Example
Question: What notice period does the Northwind contract require?

Retrieved passages:
[1] Northwind-MSA-2026.pdf, section 11.2: "Either party may terminate at renewal by giving sixty (60) days' written notice..."
[2] Northwind-MSA-2026.pdf, section 11.4: "Notices must be sent to the addresses in Schedule B..."

Answer: The contract requires 60 days' written notice before renewal [1], sent to the addresses in Schedule B [2].
A worked example of RAG (retrieval-augmented generation).

Questions,
answered

From definitions
to answers.