Skip to course content
Free RAG fundamentals course

Introduction to RAG and Knowledge Assistants

Unit 07.02: Stale sources presented as current

A three-year-old policy ranks exactly as well as the current one. Retrieval has no opinion about time.

Age is metadata, not a score

Similarity is computed on text. A chunk describing a superseded policy is textually excellent evidence for a question about that policy - it is on-topic, well-worded and completely out of date.

The table below computes the age of two retrieved chunks against a fixed date and flags anything over a year old.

c1     45 days old  ok     Refunds within 7 days.
c2   1274 days old  STALE  Refunds within 30 days.

Nothing in a retrieval score reflects age. A three-year-old chunk about a
policy that changed will rank exactly as well as the current one, and the
answer will quote it with the same confidence.

Carry `updated` on every chunk, surface it in the answer, and decide a
staleness threshold before a user finds one for you.

c2 is over three years old and would be quoted with exactly the same confidence as c1. Nothing in the retrieval score reflects that, and nothing in the answer would either unless the updated field is carried through and surfaced.

The threshold - a year here - is a judgement about your document set. Policies that change quarterly need a tighter one; reference material that rarely changes needs a looser one. Decide it before a user finds one for you.

The mistake this prevents

The mistake is relying on ingestion to remove stale documents. Corpora accumulate: old versions stay because deleting them loses the audit trail, drafts get indexed by accident, and archives get included by a glob that was too broad. Assume stale chunks are present and detect them at query time.

Takeaway

Carry updated on every chunk, compute age at retrieval, and surface it in the answer. Staleness is invisible to similarity and cheap to check with a subtraction.