Unit 03.01: Hierarchical: a manager that can also be wrong

The hierarchical process adds a manager agent that decides which worker handles what. The manager is a model call, so its decisions are model decisions.

Routing as a thing that can be wrong

In sequential, routing is something you wrote. In hierarchical it is something inferred at run time, which means it has an error rate.

The code shows four requests and how the manager routed them.

# Hierarchical adds a manager agent that decides which worker handles what.
# The manager is itself a model call, so it is a decision that can be wrong.

DELEGATIONS = [
    ("refund question",     "Billing analyst",  True),
    ("password reset",      "Billing analyst",  False),
    ("invoice dispute",     "Billing analyst",  True),
    ("outage complaint",    "Support writer",   False),
]

print(f"{'request':20} {'manager sent it to':20} correct?")
for request, target, correct in DELEGATIONS:
    print(f"{request:20} {target:20} {'yes' if correct else 'NO'}")

wrong = sum(1 for _, _, c in DELEGATIONS if not c)
print(f"\n{wrong}/{len(DELEGATIONS)} misrouted by the manager")

# Every misroute costs the wrong agent's tokens plus the retry. And the manager
# has no ground truth to learn from -- it routes again next run with the same
# instructions. A deterministic router (Module 6) does not have this failure.

Two of four are misrouted. Each misroute costs the wrong agent's tokens, then the retry, then possibly a second wrong route - so the cost of a routing error is several times the cost of the routing decision itself.

The part that does not improve is worth stating: the manager has no ground truth and no feedback loop. It will route the same way next run, with the same instructions, because nothing recorded that it was wrong.

The mistake this prevents

The mistake is choosing hierarchical because the workflow feels like it needs a coordinator. Ask instead whether the routing rule can be written down. If it can, a deterministic router does it correctly every time for no tokens, and the manager is buying you an error rate.

Takeaway

Hierarchical makes routing a model decision with an error rate that does not improve on its own. It earns its cost only where the routing genuinely needs judgement.

Mark complete