Unit 09.03: Falling back to something useful

Degradation has several rungs between working and broken, and most systems build only the top and the bottom.

The rung most systems skip

Four fallbacks in order of usefulness.

The code lists them.

LADDER = [
    ("serve a cached answer if one is fresh enough", "best"),
    ("return the retrieved source text without a generated summary", "good"),
    ("state that the assistant is unavailable and link the document", "acceptable"),
    ("show a spinner until it times out", "worst"),
]
print(f"{'fallback':58} quality")
for option, quality in LADDER:
    print(f"{option:58} {quality}")

print("""
The second rung is the one most systems never build, and it is usually
available: retrieval still works when the model provider does not, and source
text with a citation answers most questions -- just not in prose.

The bottom rung gives the user nothing and costs them the wait.
""")

The second rung - returning the source text without a generated summary - is usually available and rarely built. Retrieval still works when the model provider does not, and source text with a citation answers most questions in a less polished form.

The bottom rung gives the user nothing and costs them the wait, which is strictly worse than saying so immediately.

The mistake this prevents

The mistake is building only the cached-answer fallback, which covers repeated questions and nothing else. The novel question during an outage is exactly the case a cache cannot serve.

Takeaway

Build the middle rungs. Retrieval survives a model outage, and source text with a citation is a real answer - just not a written one.

Mark complete