Unit 04.01: Knowledge sources and their staleness

A knowledge base does not know how old it is, and neither does the agent quoting from it.

A staleness budget per source, not one global number

How long a document stays trustworthy depends on how often that kind of document changes. A quarterly policy and a rarely-touched FAQ need different thresholds.

The code checks three sources against budgets derived from their change frequency.

from datetime import date

SOURCES = [
    {"name": "refund-policy.md",   "updated": date(2026, 6, 14), "changes": "quarterly"},
    {"name": "pricing-2024.pdf",   "updated": date(2024, 1, 10), "changes": "yearly"},
    {"name": "onboarding-faq.md",  "updated": date(2026, 7, 20), "changes": "rarely"},
]
today = date(2026, 7, 29)
BUDGET_DAYS = {"quarterly": 120, "yearly": 400, "rarely": 900}

print(f"{'source':22} {'age':>6} {'budget':>7}  status")
for s in SOURCES:
    age = (today - s["updated"]).days
    budget = BUDGET_DAYS[s["changes"]]
    print(f"{s['name']:22} {age:>5}d {budget:>6}d  "
          f"{'STALE' if age > budget else 'ok'}")

# The staleness budget depends on how often the source changes, not on one
# global number. A pricing document from 2024 in a crew's knowledge base will
# be quoted with full confidence, because nothing in retrieval knows about time.

The 2024 pricing document is well past its budget and will be quoted with exactly the same confidence as the current policy, because nothing in retrieval represents time.

Per-source budgets matter because a single global threshold is either too tight for stable reference material - flagging things that are fine - or too loose for fast-moving policy, which is where staleness actually costs you.

The mistake this prevents

The mistake is assuming ingestion removes old documents. Knowledge bases accumulate: old versions stay for audit, drafts get indexed by accident, and archives get pulled in by a glob that was too broad. Assume stale material is present and detect it at query time.

Takeaway

Give each knowledge source a staleness budget based on how often it changes, and check age at retrieval. Similarity has no opinion about time, so nothing else will surface it.

Mark complete