Unit 01.00: Defining working for a non-deterministic system

"Does it work?" has a clear answer for most software and no answer at all for a system whose output differs every run.

A rate over a set, judged on properties

Three of four runs below are correct and no two are identical. If working means "produces output X", the system fails constantly; if it means anything useful, it has to be a rate.

The code shows four runs of the same question.

RUNS = [
    {"id": 1, "answer": "Refunds are allowed within 7 days.", "ok": True},
    {"id": 2, "answer": "You have a week to request a refund.", "ok": True},
    {"id": 3, "answer": "Refunds take about 30 days.", "ok": False},
    {"id": 4, "answer": "Refunds within 7 days of purchase.", "ok": True},
]
for r in RUNS:
    print(f"run {r['id']}: {'OK  ' if r['ok'] else 'WRONG'} {r['answer']}")

rate = sum(r["ok"] for r in RUNS) / len(RUNS)
print(f"\ncorrect on {rate:.0%} of runs")

print("""
Runs 1, 2 and 4 differ in wording and agree in substance. Run 3 is wrong.

"Working" for this kind of system is therefore not "produces output X". It is
a RATE over a fixed set of cases, judged on properties rather than strings --
which means it cannot be stated at all until you have the set and the
properties.
""")

Runs 1, 2 and 4 differ in wording and agree in substance. Run 3 is wrong. Any definition of working has to separate those two situations, and no string comparison does.

So the definition needs two things you do not have yet: a fixed set of cases, and a judge that decides on properties rather than text. The next three modules build both, and until they exist "does it work?" cannot be answered even in principle.

The mistake this prevents

It is worth naming why this differs from MLOps, which solved a superficially similar problem. There, you train the model, so drift is something you monitor in your data and fix by retraining. Here the model is a third party's and you do not retrain it - behaviour is driven by prompts, by retrieved context, and by a model that can change underneath you without a version bump you notice. The artefacts under your control are different, so the practices are too.

The mistake is answering the question by trying the system a few times. Three good answers from a system that is right 75% of the time is the most likely outcome, and it feels like evidence. A rate needs a set.

Takeaway

Working means a rate over a fixed set, judged on properties rather than strings. Until the set and the judge exist, the question has no answer.

Mark complete