Skip to course content
Free RAG fundamentals course

Introduction to RAG and Knowledge Assistants

Unit 10.02: Failure paths and what the user sees

The strings a system shows when it cannot answer are part of its design, and they are usually written last, by whoever was closest to the code.

Distinguishable, and careful about what they reveal

Five failure states, five different things the user should do next. If two states share a message, the user cannot tell whether to rephrase, wait, ask a colleague, or request access.

The table below lists each state with its message.

no_match    The available documents do not cover that.
conflict    The documents disagree on this. Both are shown below for review.
stale       The best available answer is over a year old. Treat with caution.
restricted  Some relevant material exists but is outside your access.
error       Retrieval is unavailable. No answer has been generated.

Two rules for these strings.

They must be distinguishable. "Something went wrong" for both no_match and
error means a user cannot tell whether to rephrase or to come back later.

And `restricted` must not confirm content. "A document about executive salaries
exists but you cannot see it" has already leaked the thing it withheld.

no_match and error are the pair most often collapsed into "something went wrong." They demand opposite responses: one means the documents do not cover it, so try elsewhere; the other means the system is broken, so try again later. One message for both wastes the user's time on whichever case they guess wrong.

restricted needs the most care. "Some relevant material exists but is outside your access" already confirms that material exists - which for some corpora is itself the sensitive fact, and the message has to be vaguer than feels helpful.

The mistake this prevents

The mistake is writing these strings during implementation, one at a time, in whatever wording felt natural at that line of code. They are the entire user experience of failure, they are what a screenshot of the system will show, and they deserve to be written together and reviewed as a set.

Takeaway

Write the failure messages as a set, check that no two are confusable, and check that none of them confirms the existence of something the user is not allowed to know about.