Skip to course content
Free RAG fundamentals course

Introduction to RAG and Knowledge Assistants

Unit 07.01: Two documents that disagree

When two retrieved chunks contradict each other, the model will pick one. Which one, and on what basis, is not something you have specified.

Detecting the disagreement before answering

Contradiction is invisible to similarity - both chunks are about refunds, so both rank. But some contradictions are mechanically detectable: two chunks on the same topic stating different numbers.

The example below extracts numeric claims from each retrieved chunk and compares them.

RETRIEVED
  c1   updated 2026-06-14   Refunds are allowed within 7 days.
  c2   updated 2024-01-02   Refunds are allowed within 30 days.

numbers claimed by c1:   7
numbers claimed by c2:   30
the chunks disagree:     yes

newest is c1 (2026-06-14)

Report BOTH and say which is newer. Silently taking the newest is a policy
decision the user cannot see, and it is sometimes wrong -- an older document
may be the one still in force.

c1 says 7 days, c2 says 30. The sets differ, so conflict is True, and the system now knows something it could not learn from the scores.

What to do with that knowledge is the interesting part. Taking the newest chunk is tempting and is a policy decision the user cannot see - and it is sometimes wrong, because an older document can be the one still in force while a newer draft sits in the corpus. Reporting both, with dates, keeps the decision where it belongs.

The mistake this prevents

The mistake is resolving conflicts silently. Whichever rule you pick - newest wins, highest score wins, first wins - it will be wrong sometimes, and a silent rule means nobody finds out. Surface the conflict and let the reader see both sides.

Takeaway

Numeric disagreement between top chunks is detectable with a regular expression and should trigger a different response path. Report both chunks with their dates rather than resolving the conflict invisibly.