Deflection rate

Deflection rate is the share of conversations that an automated assistant handled without passing them to a person. If 1,000 people start a chat and 700 never reach a human, the deflection rate is 70 per cent. It is one of the most quoted numbers in customer service software, and one of the easiest to flatter.

What it actually measures

Deflection measures what did not happen: a conversation did not reach your team. It does not measure what did happen. A shopper who got the right answer and bought the product is deflected. So is a shopper who got a wrong answer and left. So is a shopper who gave up because the bot kept misunderstanding them. All three count the same.

How it gets flattered

  • Hiding the way to a person. Make the "talk to someone" option hard to find, and deflection rises while service gets worse.
  • Counting abandonment as success. A visitor who closes the chat without asking for help is often counted as deflected.
  • Choosing the denominator. Leaving out conversations that were "out of scope" can raise the rate without changing a single answer.

What to ask a vendor who quotes it

How is a conversation counted as deflected? Is abandonment included? How many steps does it take a customer to reach a person? And, most usefully: can you show me the conversations that were deflected, so I can read whether they were answered well?

Better questions for a store

For a shopping assistant, the questions that matter are whether shoppers found the right product, whether answers came from your own content, and how many questions it could not answer. Those can be counted from the conversations themselves.

When deflection is still useful

Deflection is not meaningless. For a team drowning in repetitive questions, knowing how many never reached a person is a fair measure of time saved, as long as it is read beside a measure of quality. The mistake is treating it as a measure of customer satisfaction, which it was never designed to be. Read it as "how much work the assistant took on", never as "how well it did that work".

A useful pairing

Read deflection beside the number of questions the assistant could not answer. Rising deflection with falling gaps is progress; rising deflection with rising gaps is a warning.

How ChatWidget reports it: we do not publish a deflection rate. The console shows conversations, how many answers were grounded in your own content, thumbs up and down, open gaps and handovers, so you can judge the quality directly.

Related terms

  • Retrieval-augmented generation (RAG)

    Retrieval-augmented generation, usually shortened to RAG, is a way of making a language model answer from specific documents instead of from whatever it absorbed during training. When a question arrives, the system first retrieves the passages most likely to contain the answer, then asks the model to generate a reply using only those passages.

  • Chunking

    Chunking is the step in a retrieval-augmented system where documents are split into passages, or chunks, small enough to be retrieved individually. It happens once, when content is added, and it quietly sets the ceiling on how good every later answer can be.

  • Embedding

    An embedding is a list of numbers that represents the meaning of a piece of text. Passages that mean similar things get similar lists of numbers, even when they share no words at all. That is what lets an AI assistant search by meaning instead of by matching words.

  • Similarity threshold

    The similarity threshold is the minimum score a retrieved passage must reach before an AI assistant is allowed to use it in an answer. Everything above the line can be used; everything below it is dropped. It is the single setting that decides whether an assistant answers a question or declines it.

  • Resolution rate

    Resolution rate is meant to be the share of conversations in which the customer's problem was actually solved. It sounds like the right thing to measure, and it is, which is why it matters so much how each vendor defines "solved". Two vendors quoting the same percentage can be measuring completely different things.

See it on a real answer

The trace is where the vocabulary becomes a line between what was used and what was dropped.

See the trace panel All terms