Why it exists
A language model on its own knows a great deal about the world in general and nothing about your shop in particular. It has never seen your price list, your returns policy or this morning's stock. Asked about them, it will produce something fluent and plausible, which is the problem. RAG gives the model the relevant passages at the moment of the question, so its answer can be about your catalogue rather than a guess at a typical one.
How a RAG chatbot works
- Your content is prepared. Pages, files and product listings are split into passages (chunking) and each passage is turned into an embedding, a numerical fingerprint of its meaning.
- The question is searched. The question gets an embedding too, and the passages closest to it in meaning are gathered, usually with a keyword search alongside.
- Weak matches are dropped. Each passage has a score, and anything below a threshold is discarded.
- The answer is written. The model receives the surviving passages and an instruction to answer from them, and to say so when they are not enough.
Where RAG goes wrong
Most failures blamed on "hallucination" are retrieval failures. The right passage was never in the content, was split badly, was never retrieved, or was retrieved and scored below the threshold. Each of those has a different fix, and none of them is fixed by changing the model. The honest test of a RAG system is whether it lets you see which of those happened.
RAG in a shopping assistant
A store adds two complications. Prices and stock change hourly, so the passages alone are not enough: a good shopping assistant reads live price and stock from the store at the moment of the answer. And a recommendation must be a product the store actually sells, so the catalogue needs to be retrieved as products, not as prose.
How ChatWidget uses it: the assistant retrieves from your catalogue and pages, drops weak matches at a threshold, reads live price and stock from your store, and keeps a trace of every passage it considered.