Skip to course content
Free RAG fundamentals course

Introduction to RAG and Knowledge Assistants

Unit 04.00: The question the user typed is rarely the query you need

Users write in their own vocabulary. Documents are written in the organisation's. Retrieval sits between the two and, left alone, fails at exactly the moment the vocabularies diverge.

Two vocabularies, one search

"I want my money back" and "refunds are allowed within 7 days of purchase" are about the same thing and share no useful word. A retriever matching on terms returns nothing; an embedding does better but still degrades as the gap widens.

The example below runs the typed question and a rewritten version against the same corpus.

CORPUS  "Refunds are allowed within 7 days of purchase."

QUERY AS TYPED       "i want my money back"          ->  NOTHING
QUERY REWRITTEN      "refund policy purchase window" ->  found

The corpus did not change and the retriever did not change. Only the words
going in.

That makes rewriting a translation between the user's vocabulary and the
document's -- and a place where the question can be lost entirely.

As typed: NOTHING. Rewritten: ['c1']. The corpus never changed and the retriever never changed - only the words going in.

That makes rewriting a translation step between the user's language and the document's. It also makes it a place where the question can be lost entirely, because a rewrite is a *guess* about what the user meant. The next four units are about making that guess deliberate and measurable rather than incidental.

The mistake this prevents

Conversation makes this sharper. In a multi-turn chat the user's next message is often "what about the other one?" or "does that apply to us?" - sentences that carry meaning only alongside the turns before them. Sent to retrieval on their own they contain no retrievable content at all, and the pronoun is the whole question. Resolving referents against the conversation before retrieving is usually the single largest retrieval win in a chat interface, and it is a rewrite like any other: log what it produced, because when it resolves the pronoun wrongly the system answers a different question confidently.

The mistake is treating an empty retrieval result as evidence that the corpus lacks the answer. Very often the corpus has it and the wording missed it - a distinction Module 6 makes central, because the two demand opposite responses. One means fix the documents; the other means fix the query.

Takeaway

The user's question and the retrieval query are different objects. Treating them as the same thing works only while user vocabulary happens to match document vocabulary, which is not a property you can rely on.