Skip to course content
Free RAG fundamentals course

Introduction to RAG and Knowledge Assistants

Unit 04.04: When rewriting makes retrieval worse

A rewrite can move a query away from the corpus rather than towards it, and the pipeline gives no indication that it happened.

Two ways to lose the question

Over-expansion piles on synonyms until the query no longer resembles the vocabulary anything was written in. Wrong narrowing adds a constraint the user never stated, ruling out the chunk that had the answer.

The example below runs an original query and both damaged versions against a one-chunk corpus.

CORPUS  "Refunds are allowed within 7 days of purchase."

REWRITE                                                        RESULT
original          "refund within 7 days"                        found
over-expanded     "money reimbursement compensation payment
                   restitution repayment"                       NOTHING
wrongly narrowed  "refund enterprise sale items exclusion"      NOTHING

Over-expansion drifted off the document's vocabulary. Wrong narrowing added
a constraint the user never stated. Both fail silently: retrieval returns
nothing and the model answers from whatever it received.

"money reimbursement compensation payment restitution repayment" retrieves NOTHING from a corpus that plainly contains a refund policy. Every added term is a plausible synonym and the aggregate has drifted off the document's vocabulary.

"refund enterprise sale items exclusion" fails differently: it invents a constraint. The user asked about refunds; the rewrite asked about enterprise refund exclusions, which is a narrower question with a different answer.

Both failures are silent. Retrieval returns an empty or irrelevant set and the model answers from whatever it received.

The mistake this prevents

The mistake is shipping a rewriter without a fallback. If the rewritten query returns nothing, retry with the original before refusing - otherwise a rewriting bug presents to users as "the documents do not cover that," which is the one message that stops them asking again.

Takeaway

Always compare the rewrite against the original on the eval set, and keep the original as a fallback at query time. A rewrite is a guess, and a system should not stake a refusal on a guess it can cheaply check.