Unit 05.01: Callbacks as the place to assert

A task callback runs when a task finishes and before the next one starts. That is the natural place to assert on what was produced.

Assertions that run on every task, every run

A callback receives the task output. Checking it there means the check runs automatically on every run rather than when someone remembers to look.

The code attaches a callback and shows what it sees for two outputs.

from crewai import Agent, Task

CHECKS = []


def after_task(output):
    """A task callback: runs when the task finishes, before the next one."""
    text = str(output)
    CHECKS.append({
        "length_ok": 20 < len(text) < 2000,
        "cites_policy": "7 days" in text,
        "no_hedging": "probably" not in text.lower(),
    })
    return output


agent = Agent(role="Policy reviewer", goal="Check drafts against policy",
              backstory="You quote the policy line you relied on.",
              allow_delegation=False)
task = Task(description="Review the refund reply for ACC-1187",
            expected_output="Approve or reject, quoting the policy line.",
            agent=agent, callback=after_task)

print("callback attached:", task.callback is not None)

# Simulating what the callback sees, without needing a model:
after_task("Reject: the request came 9 days after purchase; policy is 7 days.")
after_task("Probably fine.")
for i, result in enumerate(CHECKS, 1):
    failed = [k for k, v in result.items() if not v]
    print(f"output {i}: {'all checks pass' if not failed else 'FAILED ' + str(failed)}")

Three checks, all cheap: plausible length, the policy is actually cited, and no hedging language. The second output fails two of them.

The hedging check is the one worth copying. "Probably" in an output that is supposed to be a decision is a signal the agent did not have what it needed - and catching that at the callback tells you to fix the task's inputs rather than to re-run and hope.

The mistake this prevents

The mistake is doing this checking by reading outputs during development and stopping once the crew works. The checks that matter are the ones running six months later, on runs nobody is watching, when a prompt or a model version has changed underneath you.

Takeaway

Put assertions in task callbacks so they run on every task of every run. Hedging language in a decision output is a particularly useful signal that the task's inputs were insufficient.

Mark complete