Choosing a RAG similarity threshold, by measurement

Retrieval scores every candidate passage for similarity to the question, then discards everything below a threshold. That single number decides whether your assistant answers from your content or fills the gap from general knowledge.

It is usually set by feel. It is measurable in about an hour, and the measurement is worth far more than the number anyone else can give you.

What goes wrong at each extreme

Too high. Real content is discarded. The assistant receives nothing, and a model given nothing produces something plausible, which is what everyone then reports as a hallucination. The passage existed. It scored 0.36 against a cut of 0.45 and was thrown away.

Too low. Loosely related passages get through. A blog post announcing a spring sale becomes a pricing source. The assistant is now grounded in the wrong document, which is harder to spot than being grounded in nothing, because the answer cites something real.

The second failure is worse in practice, because the first at least announces itself.

The measurement

You need two lists of questions from your own domain, twenty or thirty each.

List A, questions your content should answer. Real ones. Take them from support tickets, site search logs, or the sales inbox. Do not write them by looking at your content, because you will unconsciously phrase them to match and every score will be flattering.

List B, questions it should decline. Things adjacent to your business that you have no material on. Competitor comparisons, tax advice, “who is your CEO”, requests for a discount. These matter as much as list A: the threshold’s job is to separate them.

Run both. For each question, record the similarity score of the best candidate, not the average. The threshold acts on the top of the ranking, so that is the number that decides the outcome.

You now have two distributions. Plot them if you like, but the shape is usually obvious from a sorted list.

Reading the result

Here is a real measurement, taken on a grocery catalogue of 88 products:

list A, product questions   ("do you have pickles", "dosa mix", "masala powders")   0.36 – 0.55
list B, support questions    ("return policy", "track my order", "talk to a human")  0.12 – 0.28

There is a gap between 0.28 and 0.36, and the threshold belongs inside it. We chose 0.32, roughly the middle, which leaves margin on both sides.

Before that measurement the value was 0.45, which sits inside list A. Category questions like “what pickles do you have” scored 0.36, fell under the cut, and returned nothing. The assistant then invented a product line. The instruction was fine; the constant was wrong.

Our numbers will not work for you

This is the part most worth taking away. That gap is a property of one catalogue, in one language, in one product category, with an embedding model chosen at a particular time. Change any of those and it moves:

• A catalogue of part numbers, M8-1.25-30-A2, produces much flatter similarity, because part numbers carry little semantic signal. Your bands may overlap almost entirely.
• Long documents split into large chunks score differently from short product records.
• Different embedding models produce different absolute scales. A threshold is not portable between them, even if both are “good”.
• Multilingual content, or questions asked in one language about content written in another, shifts everything.

Copy the method. Do not copy 0.32.

When the bands overlap

Sometimes there is no clean gap. That is informative rather than a dead end, and it means one of these:

Your content does not cover list A. The honest fix is content, not tuning. No threshold separates questions you cannot answer from questions you cannot answer.

Your chunks are too large. A 2,000-word page chunked whole scores moderately against everything and strongly against nothing. Splitting on structure sharpens the distribution more reliably than moving the cut.

Similarity alone is not enough for your domain. Part numbers are the classic case. Blending in keyword matching, or boosting the catalogue over blog content, separates cases that pure similarity cannot.

If the bands still overlap after all three, prefer the higher threshold and let the assistant decline more often. A decline costs you one handover. A confidently wrong price costs you the order and the trust.

Re-measure when the shape of the content changes

Not on a schedule, on an event. Adding a product line, translating the site, switching embedding models, or replacing long pages with short ones all move the distributions. Adding twelve more products of the same kind does not.

What to check afterwards

Take ten questions you know your content answers, ask them, and read the trace rather than the answer. You are looking for one line:

0 chunks above threshold

Every appearance of that on a question you believe is answerable is either a content gap or a threshold set too high, and the trace tells you which by showing what the best candidate scored. That is a five-minute diagnosis for a class of bug that otherwise gets escalated as “the AI is unreliable”.