Unit 04.04: State that survives the run being interrupted
If the run has to survive a restart, every field in the state has to be writable.
Save, die, resume
State written to a store and read back by a different process. Anything that cannot be serialised fails at save time.
The example below saves, resumes, and then attempts an unserialisable field.
BEFORE THE INTERRUPTION written down as
invoice_id INV-1187
decision (not yet made)
notified false
attempts 1
--- the process stops here ---
AFTER RESUMING, a new process reads back
invoice_id INV-1187
decision (not yet made)
notified false <- the guard flag survived
attempts 1
WHAT CANNOT BE WRITTEN DOWN
an open database connection rejected: cannot be recorded
a temporary file path rejected: the file may be gone
Anything that cannot be written down cannot survive, and finding that out
when you design the state beats finding it out during an incident.
The resumed state includes notified: False, which is what lets the next step know not to notify twice. That flag only helps if it survives, which is why guard flags belong in the written-down state rather than in something the run holds only in passing.
The rejection of the unwritable field is a good failure. Discovering that a field cannot be written down happens at save time, on the first run, rather than at resume time during an incident.
The mistake this prevents
The mistake is testing resume inside one process. That passes even when a step depends on something the running process is holding that would not survive a restart - and a resume in production is nearly always a different process.
Takeaway
Write the state down and read it back in a separate process to verify. Guard flags must live in the state, and anything unserialisable should fail at save time.
