Skip to course content
Free RAG fundamentals course

Introduction to RAG and Knowledge Assistants

Unit 10.01: Where humans stay in the loop

Answer, refuse, escalate. Three outcomes, and the third is the one most systems leave out - which forces every ambiguous case into one of the other two.

Routing on signals you already compute

The routing rule needs nothing new. Whether anything was retrieved, whether the chunks conflict, whether the answer contains unsupported terms, and how strong the top score is - all four were built in earlier modules.

The example below routes four cases through those rules.

clear      -> answer
weak       -> escalate
conflict   -> escalate
nothing    -> refuse

The weak case is the interesting one. Something was retrieved, nothing conflicts, no unsupported terms - and the top score is 0.3. The system *can* answer, and the evidence is thin enough that a person should decide.

A two-outcome system has nowhere to put that case. It either answers from weak evidence or refuses despite having something, and both are worse than handing it to a human with the reasons attached.

The mistake this prevents

The mistake is treating escalation as a failure state to be minimised. Escalation is a designed outcome, and driving its rate to zero means the ambiguous cases went somewhere - into confident answers. Track the escalation rate, but do not target a low one.

Takeaway

Build three routing outcomes, not two, and compute the routing from signals you already have. The cases that need a human are precisely the ones the evidence cannot settle.