Unit 09.00: Agents that agree with each other into a wrong answer

A crew can produce a wrong answer with every agent doing its job correctly. That is the failure mode most specific to multi-agent systems.

Checking against the wrong thing

An error introduced early propagates forward, and a reviewer that checks consistency rather than truth will stamp it as approved.

The code follows one wrong figure through three agents.

CHAIN = [
    ("Researcher", "The refund window is 30 days.", "wrong, from training data"),
    ("Writer",     "Refunds are available within 30 days.", "faithful to input"),
    ("Reviewer",   "Approved -- consistent with the research.", "checked the wrong thing"),
]

print(f"{'agent':12} {'output':44} note")
for agent, output, note in CHAIN:
    print(f"{agent:12} {output:44} {note}")

print("""
Each agent did its job. The reviewer checked the draft against the research
rather than against the policy document, so the error propagated with a stamp
of approval on it.

The fix is that a checking agent must check against a SOURCE, not against the
previous agent's output. Give the reviewer the policy text and a rule: approve
nothing the source does not state.
""")

The researcher was wrong - thirty days came from training data rather than from the policy. The writer was faithful to its input. The reviewer checked the draft against the research, found them consistent, and approved.

Each agent behaved correctly under its own instructions. The output is wrong and carries a stamp of approval, which makes it more dangerous than an unreviewed wrong answer.

The mistake this prevents

The mistake is giving a checking agent the previous agent's output and calling it review. A reviewer must check against a *source* - the policy text itself - with a rule saying approve nothing the source does not state. Consistency checking is not verification.

Takeaway

Give checking agents the source, not the previous agent's work, and a rule that approves nothing the source does not state. Agents agreeing with each other is not evidence.

Mark complete