A shopper asked for “spice”. The assistant asked a sensible clarifying question back. Underneath it, three product cards appeared: a green pepper pickle, a curd chilli, and one rasam powder.
The reply had named no product at all. The cards were noise, and worse, two of them contradicted the thing the visitor had just read.
Why it happened
Cards were chosen from whatever retrieval returned. That sounds reasonable until you watch an embedding handle a one-word query. “Spice” sits close to anything spicy, so a pickle scores near a powder. Retrieval was behaving correctly. The mistake was treating its output as the answer’s subject.
The fix, and the fix to the fix
Cards are now chosen from the retrieved set by how much of each product’s name the answer actually uses. If the reply names nothing, no cards appear.
The first version of that scored by counting matched words, which was wrong in a way worth recording. Given a reply listing four powders, a raw count carded only “Kashmiri Lal Mirch Powder” (three words matched) and dropped “Rasam Powder”, which the answer had named in full but which has only two words to match. Counting rewards long names.
Scoring by the fraction of the name used fixes it. Half is the cut: it keeps a name the model trimmed, “Sambar Powder” for “777 Madras Sambar Powder”, and drops one that shares a single generic word out of four.
spice → 0 cards (the reply names no product)
spice powder → 4 cards (exactly the four it lists)
what pickles do you have → 3 cards (the three it names)
shipping charges → 0 cardsThe general lesson
Two systems describing the same subject drift apart unless one is derived from the other. The text and the cards were both generated from retrieval, independently, and that independence was the bug. Deriving the cards from the text made disagreement impossible rather than unlikely.