Unit 03.01: Paying a model to do arithmetic

Asking a language model to add three numbers is the clearest example of a step in the wrong place, and it is common because the numbers happen to be in a prompt already.

Free and exact versus slow, paid and usually right

The comparison is not close on any axis: time, cost, or correctness. The interesting part is the third one, because "usually right" is not a property sum() has at all.

The code times the deterministic version and states the model figures alongside it.

import time

rows = [{"amount": 120.0}, {"amount": 80.5}, {"amount": 12.25}]

start = time.perf_counter()
total = sum(r["amount"] for r in rows)
code_ms = (time.perf_counter() - start) * 1000

print(f"code : total {total:.2f} in {code_ms:.4f} ms, cost $0, always correct")
print(f"model: total 212.75 in ~800 ms, cost ~$0.002, correct most of the time")

print("""
The "most of the time" is what matters. A model that gets arithmetic right on
99% of calls fails one run in a hundred, silently, with a plausible number.

Arithmetic, comparisons, date parsing, string formatting, deduplication and
sorting all belong in code. Use the model for the step where the rule cannot
be written down.
""")

"Correct most of the time" is the line to sit with. A model that gets arithmetic right on 99 runs out of 100 fails the hundredth silently, with a plausible number - no exception, no log line, just a total that is wrong.

Deterministic code has a different failure profile: when it is wrong it is wrong every time, which means a test finds it. A step that fails one run in a hundred passes every test you write and fails in production.

The mistake this prevents

The mistake is judging this by average accuracy. A 99% accurate arithmetic step in a workflow running a thousand times a day produces ten wrong answers a day, each individually plausible. Accuracy is the wrong frame for a task where an exact method exists.

Takeaway

Arithmetic, comparison, date parsing, formatting, deduplication and sorting all belong in code. Reserve the model for the step where no exact method exists.

Mark complete