Embedding

An embedding is a list of numbers that represents the meaning of a piece of text. Passages that mean similar things get similar lists of numbers, even when they share no words at all. That is what lets an AI assistant search by meaning instead of by matching words.

A worked example

A shopper asks for "something to fry pakoras in". Your product page says "cold-pressed groundnut oil, high smoke point, ideal for deep frying". A keyword search finds nothing in common except perhaps "fry". An embedding search places the question and the product close together, because their meanings are close, and the oil is retrieved.

How it is used in a RAG system

Every passage of your content is turned into an embedding once, when it is added. Every question is turned into an embedding when it arrives. The search then finds the passages whose embeddings are nearest to the question's, and scores each one by how close it is. Those scores are what a similarity threshold is applied to.

What embeddings are bad at

  • Exact identifiers. Order numbers, SKUs and pincodes carry little "meaning", so embeddings match them poorly. A good system runs a keyword search alongside to catch them.
  • Negation. "Is this suitable for diabetics?" and "this is not suitable for diabetics" are close in meaning-space, which is why the score alone should never decide what an answer says.
  • Numbers and quantities. "500g" and "5kg" look similar to an embedding. Sizes and prices should come from structured product data, not from similarity.

Why a store should care

Embeddings decide which of your passages an answer is even allowed to consider. When a shopper's wording is far from yours, the right passage can score too low to be used. The fix is usually to add the words shoppers actually use to your page, or to write a verified answer, rather than to change anything technical.

Embeddings and languages

Modern embedding models place questions in different languages close to passages with the same meaning, so a question in Hindi can find an English page about the same thing. It works well for common topics and less well for regional product names and brand terms, which is one more reason a keyword search runs alongside, and one more reason to include the words shoppers actually use on your own pages, in the languages they use them.

How ChatWidget uses them: every passage and every question is embedded, a keyword search runs alongside for exact terms, and the score of each candidate appears in the trace.

Related terms

  • Retrieval-augmented generation (RAG)

    Retrieval-augmented generation, usually shortened to RAG, is a way of making a language model answer from specific documents instead of from whatever it absorbed during training. When a question arrives, the system first retrieves the passages most likely to contain the answer, then asks the model to generate a reply using only those passages.

  • Chunking

    Chunking is the step in a retrieval-augmented system where documents are split into passages, or chunks, small enough to be retrieved individually. It happens once, when content is added, and it quietly sets the ceiling on how good every later answer can be.

  • Similarity threshold

    The similarity threshold is the minimum score a retrieved passage must reach before an AI assistant is allowed to use it in an answer. Everything above the line can be used; everything below it is dropped. It is the single setting that decides whether an assistant answers a question or declines it.

  • Deflection rate

    Deflection rate is the share of conversations that an automated assistant handled without passing them to a person. If 1,000 people start a chat and 700 never reach a human, the deflection rate is 70 per cent. It is one of the most quoted numbers in customer service software, and one of the easiest to flatter.

  • Resolution rate

    Resolution rate is meant to be the share of conversations in which the customer's problem was actually solved. It sounds like the right thing to measure, and it is, which is why it matters so much how each vendor defines "solved". Two vendors quoting the same percentage can be measuring completely different things.

See it on a real answer

The trace is where the vocabulary becomes a line between what was used and what was dropped.

See the trace panel All terms