Unit 02.03: Temperature, tokens, and reproducibility

temperature=0 is what people reach for when they want reproducibility. It does not deliver it, and being clear about that shapes where you put your logic.

Consistency is not determinism

Sampling temperature is one source of variation among several. Model versions change, providers change infrastructure, and some variation persists between identical calls.

The code lists four settings with what each actually gives you.

SETTINGS = [
    ("temperature=0",   "more consistent", "NOT deterministic across versions"),
    ("temperature=1",   "varied",          "different output every call"),
    ("max_tokens=100",  "bounded cost",    "truncates mid-sentence if too low"),
    ("seed (if offered)", "closer to repeatable", "provider-specific, not guaranteed"),
]

print(f"{'setting':20} {'effect':22} caveat")
for setting, effect, caveat in SETTINGS:
    print(f"{setting:20} {effect:22} {caveat}")

print("""
`temperature=0` is the setting people reach for when they want reproducibility,
and it does not deliver it. Output can still vary across model versions, across
providers, and sometimes between calls -- sampling is only one source of
variation.

Treat it as noise reduction. Anything that must be identical every run belongs
in code, not in a model call.
""")

Read the caveat column. temperature=0 gives more consistency and no guarantee; a seed, where offered, gets closer and remains provider-specific.

The practical consequence is a rule you can apply directly: anything that must be identical every run belongs in code, not in a model call. That is the same conclusion Module 6 of the LangGraph course reaches from a different direction, and it is the most reliable principle in this whole area.

The mistake this prevents

The mistake is writing tests that assert on exact model output with temperature set to zero. They pass locally, pass in CI for a month, and fail on a provider-side model update - at which point someone disables them rather than rewriting twenty assertions.

Takeaway

Treat temperature=0 as noise reduction, never as determinism. Anything requiring identical output belongs in code, and tests should assert on structure rather than wording.

Mark complete