Unit 08.01: Streaming tokens versus streaming steps

There is more than one thing to stream, and choosing the wrong one produces a worse experience than not streaming at all.

Three modes, three purposes

Updates emit one event per finished node. Values emit the whole state after each node. Messages emit tokens as a model produces them.

The code lays out all three with what each is for.

MODES = [
    ("updates", "one event per finished node",  "progress through a workflow"),
    ("values",  "the whole state after each node", "debugging, or a live state view"),
    ("messages", "tokens as the model produces them", "a chat-style answer appearing"),
]

print(f"{'mode':10} {'emits':34} best for")
for mode, emits, best in MODES:
    print(f"{mode:10} {emits:34} {best}")

print("""
The mistake is streaming tokens for a workflow that takes 90 seconds across
six nodes. The user watches one node's output appear and has no idea five more
are coming.

Stream steps for workflows and tokens for answers. A long agentic run usually
wants both: step events for the progress line, token events only for the final
node that writes something the user reads.
""")

The mismatch to avoid is streaming tokens for a six-node workflow. The user watches one node's text appear, assumes that is the answer, and has no idea five more nodes are still to run - so the perceived completion time is worse than with no streaming at all.

A long agentic run usually wants both: updates for the progress line throughout, and messages only for the final node that writes something the user reads.

The mistake this prevents

One more caution, specific to workflows with an approval gate. Streaming a draft to the user while it is still on its way to review shows them text that nobody has approved, in the same place the approved result will eventually appear. People read it as the answer - and they act on it, before the reviewer has seen it, which quietly defeats the gate you built. Stream progress during those runs and hold the content until it clears review.

The mistake is choosing values for a user-facing stream because it contains the most information. It emits the entire state - including internal fields, error strings and anything a reviewer has not yet seen - on every step. It is a debugging mode.

Takeaway

Stream steps for workflows and tokens for answers, and use both when a long run ends in something readable. values is for debugging, not for users.

Mark complete