Similarity threshold

The similarity threshold is the minimum score a retrieved passage must reach before an AI assistant is allowed to use it in an answer. Everything above the line can be used; everything below it is dropped. It is the single setting that decides whether an assistant answers a question or declines it.

Why the line matters

Every search returns something. Ask a store's assistant about a product it has never stocked, and the search will still find the nearest passages, perhaps a similar product or a general page about the range. Without a threshold, the assistant would write an answer from those weak matches, and that answer would sound exactly as confident as a correct one. The threshold is what lets it say "we do not have that" instead.

Two ways to get it wrong

  • Too low. Weak matches are used, and the assistant answers questions it should have declined, using passages that are only loosely related. This is how most confident wrong answers are made.
  • Too high. Good matches are dropped, and the assistant declines questions your content could have answered, often because a shopper worded the question differently from your page.

There is no universal right value

Scores depend on the embedding model, on how your content is written and on how your shoppers ask. A value tuned on one store routinely fails on another. The right way to choose it is to measure it on your own questions: collect real questions with known answers, and see where the passages that answer them score compared with the ones that do not.

Seeing it in action

A trace draws the threshold as a line through the candidates. When an answer is wrong, the first question is where the right passage sits relative to that line: absent, a little below it, or above it and still misread. Each points to a different fix.

What to do when it is wrong

When the assistant declines questions it should answer, look at the dropped candidates: if the right passage is there, a little under the line, rewrite that passage in the words shoppers use, or add a verified answer. When it answers questions it should decline, look at what it used: a loosely related passage above the line usually means the content is too broad, not that the line is too low.

How ChatWidget uses it: every trace shows the threshold as a line, with the passages that were used above it and the ones that were dropped below it, dashed.

Related terms

  • Retrieval-augmented generation (RAG)

    Retrieval-augmented generation, usually shortened to RAG, is a way of making a language model answer from specific documents instead of from whatever it absorbed during training. When a question arrives, the system first retrieves the passages most likely to contain the answer, then asks the model to generate a reply using only those passages.

  • Chunking

    Chunking is the step in a retrieval-augmented system where documents are split into passages, or chunks, small enough to be retrieved individually. It happens once, when content is added, and it quietly sets the ceiling on how good every later answer can be.

  • Embedding

    An embedding is a list of numbers that represents the meaning of a piece of text. Passages that mean similar things get similar lists of numbers, even when they share no words at all. That is what lets an AI assistant search by meaning instead of by matching words.

  • Deflection rate

    Deflection rate is the share of conversations that an automated assistant handled without passing them to a person. If 1,000 people start a chat and 700 never reach a human, the deflection rate is 70 per cent. It is one of the most quoted numbers in customer service software, and one of the easiest to flatter.

  • Resolution rate

    Resolution rate is meant to be the share of conversations in which the customer's problem was actually solved. It sounds like the right thing to measure, and it is, which is why it matters so much how each vendor defines "solved". Two vendors quoting the same percentage can be measuring completely different things.

See it on a real answer

The trace is where the vocabulary becomes a line between what was used and what was dropped.

See the trace panel All terms