Resolution rate

Resolution rate is meant to be the share of conversations in which the customer's problem was actually solved. It sounds like the right thing to measure, and it is, which is why it matters so much how each vendor defines "solved". Two vendors quoting the same percentage can be measuring completely different things.

Common definitions, from strictest to loosest

  • Confirmed by the customer. The customer said the answer helped, or rated it. Strict, but most customers never rate anything, so the sample is small and skewed.
  • No follow-up. The customer did not come back or contact support within a set period. Reasonable, but a customer who gave up also never comes back.
  • No handover. The conversation ended without reaching a person. This is deflection under a better-sounding name.
  • The model said so. An AI judges its own conversation as resolved. The least independent measure of all.

How to compare vendors fairly

Ask for the definition in writing, the time window, whether abandoned conversations are included, and whether the figure is an average across all customers or from one favourable case study. A resolution rate without those four answers cannot be compared with any other.

For a shopping assistant

A store has a better signal than any resolution rate: whether the shopper bought something. Until a product can measure that honestly, from the cart to a completed order, it should not claim a sales figure either. In the meantime, you can read the conversations yourself.

Why the number varies so much

Published resolution rates in this market range widely, and most of the difference is in the definition, not the product. A figure measured over a vendor's best customer, in one language, on simple questions, will always beat a figure measured across every customer on every question. That does not make either number false; it makes them incomparable. Treat any single resolution rate as a claim to investigate rather than a result to compare.

A measure you can check yourself

Take fifty real conversations from your first week, read them, and mark each as helped, not helped or should have gone to a person. It takes an hour, and it tells you more than any vendor's percentage, because it is your shoppers and your catalogue.

How ChatWidget reports it: we publish no resolution rate. The console shows grounded answers, ratings, open gaps, handovers, and completed sign-ins and orders that started in the chat, and you can open any conversation with its trace.

Related terms

  • Retrieval-augmented generation (RAG)

    Retrieval-augmented generation, usually shortened to RAG, is a way of making a language model answer from specific documents instead of from whatever it absorbed during training. When a question arrives, the system first retrieves the passages most likely to contain the answer, then asks the model to generate a reply using only those passages.

  • Chunking

    Chunking is the step in a retrieval-augmented system where documents are split into passages, or chunks, small enough to be retrieved individually. It happens once, when content is added, and it quietly sets the ceiling on how good every later answer can be.

  • Embedding

    An embedding is a list of numbers that represents the meaning of a piece of text. Passages that mean similar things get similar lists of numbers, even when they share no words at all. That is what lets an AI assistant search by meaning instead of by matching words.

  • Similarity threshold

    The similarity threshold is the minimum score a retrieved passage must reach before an AI assistant is allowed to use it in an answer. Everything above the line can be used; everything below it is dropped. It is the single setting that decides whether an assistant answers a question or declines it.

  • Deflection rate

    Deflection rate is the share of conversations that an automated assistant handled without passing them to a person. If 1,000 people start a chat and 700 never reach a human, the deflection rate is 70 per cent. It is one of the most quoted numbers in customer service software, and one of the easiest to flatter.

See it on a real answer

The trace is where the vocabulary becomes a line between what was used and what was dropped.

See the trace panel All terms