Unit 04.04: Testing every branch on purpose

A graph can have every node executed by a single test and still have an untested edge - and the untested edge is always a failure path.

Branch coverage, not line coverage

The unit of coverage in a graph is the routing decision, not the line. One test case per destination the router can return, plus an assertion on which branch was taken.

The code runs three cases and reports which destinations were covered.

from typing import TypedDict
from langgraph.graph import StateGraph, START, END


class State(TypedDict):
    rows: int
    error: str
    taken: str


def route(state: State):
    if state["error"]:
        return "error_path"
    return "empty_path" if state["rows"] == 0 else "normal_path"


g = StateGraph(State)
g.add_node("load", lambda s: {})
for name in ("error_path", "empty_path", "normal_path"):
    g.add_node(name, (lambda n: (lambda s: {"taken": n}))(name))
    g.add_edge(name, END)
g.add_edge(START, "load")
g.add_conditional_edges("load", route, {n: n for n in
                        ("error_path", "empty_path", "normal_path")})
app = g.compile()

CASES = [
    ({"rows": 3, "error": ""},        "normal_path"),
    ({"rows": 0, "error": ""},        "empty_path"),
    ({"rows": 3, "error": "timeout"}, "error_path"),
]
covered = set()
for state, expected in CASES:
    taken = app.invoke({**state, "taken": ""})["taken"]
    covered.add(taken)
    print(f"{'PASS' if taken == expected else 'FAIL'}  expected {expected:12} got {taken}")

targets = {"error_path", "empty_path", "normal_path"}
print(f"\nbranch coverage: {len(covered)}/{len(targets)}, missing {sorted(targets - covered) or 'none'}")

# Branch coverage, not line coverage. A graph can have every node executed by
# one test and still have an untested edge, and the untested edge is always
# the failure path.

The taken field is what makes this assertable. Without it you are checking the final output, which can be identical across two branches - a test that passes whichever path ran is not testing the routing.

Computing coverage explicitly is worth the three lines. It turns "we have tests for this graph" into a number, and the number is what reveals that the error path has never once been executed.

The mistake this prevents

The mistake is measuring line coverage and believing it. A graph reaches high line coverage easily, because the happy path touches most nodes. The failure branch is a few lines that no test constructs the state to reach, and it will run for the first time in production.

Takeaway

Assert on which branch was taken, not only on the output, and compute coverage over the router's destinations. Untested branches in a graph are reliably the error paths.

Mark complete