Unit 05.03: Making a rejection useful to the agent

A rejection that carries no information costs a full run and teaches nothing. The agent retries with what it had and produces what it produced.

What was wrong, what is correct, what to do

Three parts. Drop any one and the retry is a guess.

The code compares three rejections.

REJECTIONS = [
    ("Rejected.", False),
    ("This is not good enough, please try again.", False),
    ("Rejected: the reply states a 30-day window. The policy line is "
     "'within 7 days of purchase'. Restate with 7 days and quote that line.", True),
]

print(f"{'rejection':78} usable?")
for text, usable in REJECTIONS:
    print(f"{text[:76]:78} {'yes' if usable else 'NO'}")

print("""
A usable rejection has three parts: what was wrong, what the correct value is,
and what to do next. Without all three the agent retries with the same
information and produces the same output, which costs a full run to learn
nothing.

This applies to human reviewers as much as to automated guardrails -- give the
reviewer a rejection form with those three fields rather than a free-text box.
""")

"Rejected" and "not good enough, please try again" both cost a full run to produce the same output. The third names the error, supplies the correct value, and states the action - so the retry has something the first attempt did not.

This applies to human reviewers as much as to automated guardrails, which is the part most systems get wrong. Give the reviewer a form with those three fields rather than a free-text box, and the rejections become usable by construction.

The mistake this prevents

The mistake is measuring the rejection rate without measuring what happens after one. A crew where every rejection leads to an identical retry has a rejection rate that looks like quality control and functions as a way of paying twice for the same output.

Takeaway

A rejection needs the error, the correct value and the next action. Structure the reviewer's form around those three fields so human rejections are as usable as automated ones.

Mark complete