Unit 10.01: Reading a trace backwards

Traces are printed forwards and are best read backwards, starting from the symptom.

Walk back until a value should not be what it is

Reading forwards, you evaluate each step on its own terms and everything looks defensible. Reading backwards from the wrong output, you ask of each step whether its output was correct *given its input*.

The code shows the same trace read both ways.

TRACE = [
    {"node": "load",     "ms": 40,   "out": {"rows": 120}},
    {"node": "filter",   "ms": 12,   "out": {"rows": 0}},
    {"node": "classify", "ms": 1840, "out": {"label": "other"}},
    {"node": "report",   "ms": 30,   "out": {"text": "no items matched"}},
]

print("forwards -- everything looks fine:")
for step in TRACE:
    print(f"  {step['node']:9} {step['ms']:>5}ms {step['out']}")

print("\nbackwards, from the symptom:")
print("  report said 'no items matched'   <- correct given its input")
print("  classify labelled 'other'        <- correct given zero rows")
print("  filter produced rows=0           <- FIRST WRONG VALUE, start here")
print("  load produced rows=120           <- fine")

# Start at the output and walk back to the first value that should not be what
# it is. Reading forwards you spend the whole trace on `classify`, because it
# took 1.8 seconds and looks like where the work happens.

Forwards, classify draws the eye: it took 1.84 seconds, orders of magnitude more than everything else, and it is where the model call is. It is also completely correct - labelling zero rows as other is the right answer.

Backwards, three steps are quickly cleared and the fourth is not: filter turned 120 rows into 0. That is the first value that should not be what it is, and it took twelve milliseconds, which is why nothing about it attracted attention.

The mistake this prevents

The mistake is starting the investigation at the slowest or most complex step. Duration and complexity are not evidence about correctness, and in an LLM workflow the model call is nearly always both the slowest and the first suspect - which makes it a reliable distraction.

Takeaway

Read traces backwards from the symptom, asking of each step whether its output was right given its input. The first step that fails that question is the bug; the slow step is usually innocent.

Mark complete