Unit 08.02: Caching without serving a stale answer

A cache keyed on the question alone will eventually serve last month's policy, and may serve one user's answer to another.

Everything that changes the answer belongs in the key

The question, the policy version, the prompt version, and the user's groups.

The code shows three keys differing in one field each.

import hashlib

CACHE = {}


def cache_key(question, policy_version, prompt_version, user_groups):
    """Everything that changes the answer belongs in the key."""
    material = f"{question}|{policy_version}|{prompt_version}|{sorted(user_groups)}"
    return hashlib.sha256(material.encode()).hexdigest()[:16]


base = ("What is the refund window?", "v4", "answer-v3", {"public"})
print("same everything      :", cache_key(*base))
print("policy version bumped:", cache_key(base[0], "v5", base[2], base[3]))
print("different user groups:", cache_key(base[0], base[1], base[2],
                                          {"public", "staff"}))

print("""
Three different keys. The second is why a cache keyed on the question alone
serves last month's policy after the document is updated.

The third is more serious: a staff user's answer served to a public user is a
permissions failure that a cache introduced, and the retrieval-time filter
never runs on a cache hit.
""")

The policy version in the key is what stops the cache serving a superseded answer after the document is updated. Without it, the cache holds the old answer until it expires, and expiry is measured in whatever you set rather than in when the policy changed.

The user groups are more serious. Two users asking the same question are entitled to different answers, and a cache hit skips the retrieval-time access filter entirely - so the cache reintroduces a permissions failure the retrieval layer was carefully designed to prevent.

The mistake this prevents

The mistake is adding a cache late, as a performance optimisation, without revisiting the access control. The filter runs at retrieval, the cache sits in front of retrieval, and nothing about the original design anticipated a path that skips it.

Takeaway

Key the cache on everything that changes the answer, including policy version and user groups. A cache in front of retrieval bypasses the access filter unless the key carries the permissions.

Mark complete