Unit 07.01: Tracing which agent decided what

In a crew, the wrong number in the output was usually produced by a different agent from the one that made the decision.

Per-task outputs, read in order

With each task's output recorded, you can walk forward and find the first one that is wrong rather than inferring from the end.

The code shows a three-task trace where the guardrail catches the third.

import json

trace = [
    {"task": "classify", "agent": "Triage analyst",
     "output": {"team": "billing"}, "tokens": 1_240, "guardrail": "passed"},
    {"task": "decide", "agent": "Billing analyst",
     "output": {"decision": "not_eligible", "days": 9}, "tokens": 2_050,
     "guardrail": "passed"},
    {"task": "draft", "agent": "Support writer",
     "output": {"reply": "...30 days..."}, "tokens": 3_980,
     "guardrail": "FAILED: reply states 30 days, policy says 7"},
]
print(json.dumps(trace, indent=1))

failed = [s for s in trace if s["guardrail"].startswith("FAILED")]
print(f"\nfirst failure: task={failed[0]['task']} agent={failed[0]['agent']}")
print(f"upstream decision was correct: {trace[1]['output']}")

# The draft contradicts a decision that was right. Without per-task outputs you
# would debug the decision step, which is working, because the wrong number is
# what you can see.

The decision was correct: not eligible, nine days. The draft then says thirty days - so the writer contradicted an upstream result that was right.

Without per-task outputs you would debug the decision step, because the wrong number is what you can see and the decision is what produced the number in your mental model. The trace shows the decision was fine and sends you to the writer's inputs instead, which is where the fix is: the policy text was never passed to it.

The mistake this prevents

The mistake is trusting agent role names to tell you who is responsible. The Policy reviewer is not necessarily where a policy error originated, and the Billing analyst is not necessarily where a billing number went wrong. Read the outputs in order.

Takeaway

Record every task's output and read them in order to find the first wrong one. In a crew the visible error and its cause are usually in different agents.

Mark complete