Unit 09.02: Hallucinated tools and invented handoffs

An agent asked to do something its tools cannot do will often call a tool that does not exist.

Reject by name, and read the rate as a signal

The framework rejects an unregistered name, the agent retries, and the run burns tokens converging on nothing.

The code checks four proposed calls against a registry of two.

REGISTERED = {"read_account", "issue_refund"}

PROPOSED = ["read_account", "send_email", "issue_refund", "escalate_to_manager"]

print(f"{'proposed call':22} registered?")
for name in PROPOSED:
    known = name in REGISTERED
    print(f"{name:22} {'yes' if known else 'NO -- reject, do not improvise'}")

invented = [n for n in PROPOSED if n not in REGISTERED]
print(f"\n{len(invented)} invented: {invented}")

print("""
An agent asked to do something its tools cannot do will often produce a
plausible call to a tool that does not exist. The framework rejects it, the
agent retries, and the run burns tokens converging on nothing.

Two responses. Reject unregistered names explicitly, with the list of real ones
in the error, so the retry has information. And treat a high invention rate as
a signal that the agent's goal exceeds its tools.
""")

send_email and escalate_to_manager are both entirely plausible tools for this agent to want, which is why they were invented. The agent's goal implies capabilities its tool list does not contain.

That gap is the actual finding. A high invention rate is not a model quirk to be prompted away - it is a signal that the goal exceeds the tools, and the fix is either to narrow the goal or to add the tool deliberately.

The mistake this prevents

The mistake is returning a bare rejection. Include the list of real tool names in the error so the retry has information; otherwise the agent invents a second plausible name and you pay for another round.

Takeaway

Reject unregistered tool names with the real list in the error, and track the invention rate. A high rate means the agent's goal is larger than its tools.

Mark complete