Retrieval-augmented generation (RAG)

Retrieval-augmented generation, usually shortened to RAG, is a way of making a language model answer from specific documents instead of from whatever it absorbed during training. When a question arrives, the system first retrieves the passages most likely to contain the answer, then asks the model to generate a reply using only those passages.

Why it exists

A language model on its own knows a great deal about the world in general and nothing about your shop in particular. It has never seen your price list, your returns policy or this morning's stock. Asked about them, it will produce something fluent and plausible, which is the problem. RAG gives the model the relevant passages at the moment of the question, so its answer can be about your catalogue rather than a guess at a typical one.

How a RAG chatbot works

  1. Your content is prepared. Pages, files and product listings are split into passages (chunking) and each passage is turned into an embedding, a numerical fingerprint of its meaning.
  2. The question is searched. The question gets an embedding too, and the passages closest to it in meaning are gathered, usually with a keyword search alongside.
  3. Weak matches are dropped. Each passage has a score, and anything below a threshold is discarded.
  4. The answer is written. The model receives the surviving passages and an instruction to answer from them, and to say so when they are not enough.

Where RAG goes wrong

Most failures blamed on "hallucination" are retrieval failures. The right passage was never in the content, was split badly, was never retrieved, or was retrieved and scored below the threshold. Each of those has a different fix, and none of them is fixed by changing the model. The honest test of a RAG system is whether it lets you see which of those happened.

RAG in a shopping assistant

A store adds two complications. Prices and stock change hourly, so the passages alone are not enough: a good shopping assistant reads live price and stock from the store at the moment of the answer. And a recommendation must be a product the store actually sells, so the catalogue needs to be retrieved as products, not as prose.

How ChatWidget uses it: the assistant retrieves from your catalogue and pages, drops weak matches at a threshold, reads live price and stock from your store, and keeps a trace of every passage it considered.

Related terms

  • Chunking

    Chunking is the step in a retrieval-augmented system where documents are split into passages, or chunks, small enough to be retrieved individually. It happens once, when content is added, and it quietly sets the ceiling on how good every later answer can be.

  • Embedding

    An embedding is a list of numbers that represents the meaning of a piece of text. Passages that mean similar things get similar lists of numbers, even when they share no words at all. That is what lets an AI assistant search by meaning instead of by matching words.

  • Similarity threshold

    The similarity threshold is the minimum score a retrieved passage must reach before an AI assistant is allowed to use it in an answer. Everything above the line can be used; everything below it is dropped. It is the single setting that decides whether an assistant answers a question or declines it.

  • Deflection rate

    Deflection rate is the share of conversations that an automated assistant handled without passing them to a person. If 1,000 people start a chat and 700 never reach a human, the deflection rate is 70 per cent. It is one of the most quoted numbers in customer service software, and one of the easiest to flatter.

  • Resolution rate

    Resolution rate is meant to be the share of conversations in which the customer's problem was actually solved. It sounds like the right thing to measure, and it is, which is why it matters so much how each vendor defines "solved". Two vendors quoting the same percentage can be measuring completely different things.

See it on a real answer

The trace is where the vocabulary becomes a line between what was used and what was dropped.