Unit 01.01: Hidden state is the notebook's defining hazard

A notebook remembers what you ran. It does not remember the order, and that is where reproducibility goes to die.

Hidden state is the notebook's defining hazard

Cells can be executed in any order and re-executed at will, so the visible document is not a record of what produced the output. A name defined in a cell you later deleted is still in memory, so the notebook keeps working and a fresh kernel fails.

This is not an argument against notebooks. It is an argument for knowing what each tool is for. A notebook is for exploring, where the state is thrown away afterwards. A script is for computing, where the result is saved and reproducible. A report is for explaining, citing values the script saved.

The single habit that catches most of it: restart the kernel and run all, before believing any notebook result.

This block shows an out-of-order session and a variable that outlived its cell.

# A notebook remembers what you ran. It does not remember the ORDER.
executed = ["cell 3", "cell 1", "cell 2", "cell 3", "cell 5"]
print("Execution order in this session:", " -> ".join(executed))
print("Order a fresh reader would use  : cell 1 -> 2 -> 3 -> 4 -> 5")
print()

# Hidden state: a name defined in a cell you later deleted still exists.
threshold = 10          # imagine this cell was deleted after running
del_simulated = True
print("Cell deleted from the notebook:", del_simulated)
print("Variable still in memory      :", "threshold" in dir())
print("So the notebook runs, and a fresh kernel would fail.\n")

roles = {
    "notebook": "exploring, where you throw the state away afterwards",
    "script":   "computing, where the result is saved and reproducible",
    "report":   "explaining, citing values the script saved",
}
for name, job in roles.items():
    print(f"  {name:9s} {job}")
print("\nRestart the kernel and run all before believing any notebook result.")

The session ran cells in the order 3, 1, 2, 3, 5 — nothing like the order a fresh reader would use. Then a cell is deleted while its variable remains in memory: threshold is still defined, so the notebook runs and a clean kernel would raise NameError. The three roles at the end are worth keeping distinct, because most reproducibility failures come from one file trying to do two of them.

The mistake this prevents

The mistake is developing in a notebook for a day and assuming it reproduces. The cells that worked are in the order you happen to scroll past them, and half the state came from cells that no longer exist.

Takeaway

Explore in a notebook, compute in a script, explain in a report. Restart and run all before trusting any result, and move anything worth keeping into a script.

Mark complete