Unit 10.01: Finding where a wrong answer entered

Reading a trace forwards, every step looks defensible. Reading it backwards from the symptom, one is not.

The first value that should not be what it is

Ask of each step whether its output was correct *given its input*, starting from the wrong answer and walking back.

The code shows the same trace read both ways.

SPANS = [
    ("retriever",   {"ids": ["c2"], "scores": [0.31]}),
    ("format_docs", {"context": "[c2] Exchanges are allowed within 30 days."}),
    ("model",       {"answer": "Refunds are allowed within 30 days. [c2]"}),
]

print("forwards -- each step looks fine:")
for name, output in SPANS:
    print(f"   {name:13} {output}")

print("\nbackwards from the symptom:")
print("   model      said 30 days      <- correct given its context")
print("   format_docs passed exchanges <- correct given what it received")
print("   retriever  returned c2 at 0.31 <- FIRST WRONG VALUE, start here")

# The model is not the bug. It faithfully summarised an exchanges chunk that
# retrieval should not have returned at a score of 0.31 -- which is a threshold
# problem, not a prompt problem.

The model said thirty days, which is correct given a context about exchanges. format_docs passed exchanges, which is correct given what retrieval returned. The retriever returned c2 at a score of 0.31, which is where the fault is.

That is a threshold problem, not a prompt problem. Reading forwards you would spend the investigation on the model, because the model is where the wrong number appears and where the interesting work happens.

The mistake this prevents

The mistake is debugging a wrong answer by editing the prompt. If retrieval handed the model the wrong document, no wording fixes it - and prompt edits that appear to fix it have usually just moved the failure to a question you did not test.

Takeaway

Read traces backwards from the symptom, asking whether each step was right given its input. Steps downstream of a bug behave correctly on bad data, which makes them look guilty.

Mark complete