Unit 03.03: Falling back to code when the model is unsure

Validation tells you the model's output is unusable. The next question is what the node does about it, and "raise" is rarely the best answer.

A deterministic fallback, and a field that records which ran

When the model output fails validation, a keyword rule can often produce a usable-if-cruder answer. The graph continues, degraded rather than stopped.

The code classifies three texts, with the model call standing in as a failure so the fallback path runs.

from typing import TypedDict


class State(TypedDict):
    text: str
    category: str
    decided_by: str


KEYWORDS = {"refund": "billing", "error": "technical", "password": "account"}


def classify(state: State):
    """Model step, with a deterministic fallback when it fails validation."""
    model_output = None            # stand-in for a call that returned junk
    if model_output in {"billing", "technical", "account"}:
        return {"category": model_output, "decided_by": "model"}

    text = state["text"].lower()
    for keyword, category in KEYWORDS.items():
        if keyword in text:
            return {"category": category, "decided_by": "keyword fallback"}
    return {"category": "other", "decided_by": "default"}


for text in ["I want a refund", "my password broke", "hello"]:
    out = classify({"text": text, "category": "", "decided_by": ""})
    print(f"{text:22} -> {out['category']:10} via {out['decided_by']}")

# `decided_by` is the field that makes this debuggable. Without it you cannot
# tell a working model from a fallback that has been carrying the graph for
# three weeks because someone rotated an API key.

decided_by is the field that makes this safe. Without it, a working model and a fallback carrying the whole graph look identical in the output - and the second situation can persist for weeks after someone rotates an API key.

With it, you can count fallback usage. A fallback rate that jumps from 2% to 90% is an incident; without the field it is an unexplained drop in answer quality that nobody can attribute.

The mistake this prevents

The mistake is adding a fallback and not measuring how often it fires. A silent fallback is a system that degrades permanently the first time something breaks, because nothing alerts and the outputs still look reasonable.

Takeaway

Give every model step a deterministic fallback and record which path ran. The recording is what turns graceful degradation into something you can monitor rather than something that hides a failure.

Mark complete