Unit 06.04: Checkpoint size and what to leave out

State size is multiplied twice over: once by the number of steps in a run, and again by the number of runs. Small differences become large numbers quickly.

Ids versus contents, at scale

The comparison is between storing three document ids and storing the three documents. Both are correct; only one scales.

The code sizes both and projects across a thousand runs of twelve steps.

import json

small = {"question": "what changed?", "doc_ids": ["d1", "d2", "d3"], "attempts": 1}
large = {**small, "documents": ["x" * 4000 for _ in range(3)]}

size_small = len(json.dumps(small))
size_large = len(json.dumps(large))

print(f"ids only     : {size_small:>7,} bytes")
print(f"with contents: {size_large:>7,} bytes  ({size_large / size_small:.0f}x)")

STEPS, RUNS = 12, 1000
print(f"\nover {RUNS:,} runs x {STEPS} checkpoints:")
for label, size in [("ids only", size_small), ("with contents", size_large)]:
    print(f"  {label:14} {size * STEPS * RUNS / 1e6:>8.1f} MB")

# Every step writes a checkpoint, so state size is multiplied by the number of
# steps and the number of runs. Store the ids and re-fetch; store the text and
# your checkpoint store becomes a second copy of the corpus.

The multiplier is the point. A state object that is a few kilobytes larger than it needs to be costs almost nothing on one run and becomes hundreds of megabytes across a thousand - in a store you now have to operate, back up and eventually prune.

The trade against re-fetching is real but usually one-sided: re-fetching costs a call on resume, which is rare, while the storage cost is paid on every step of every run. The exception is content that may change or disappear, where the id no longer resolves to what the run actually saw.

The mistake this prevents

The mistake is measuring state size on a development run with one small document. Measure it on a realistic input and multiply by steps and runs before deciding what to keep - the number that matters is not the size of one checkpoint.

Takeaway

Store ids and re-fetch, unless the content could change under you. Checkpoint size is multiplied by steps and by runs, so a few extra kilobytes per state is a storage problem rather than a rounding error.

Mark complete