Unit 11.00: Testing everything except the model

Take the model as a parameter and everything else becomes testable.

One line of design, most of the coverage

A function that builds a reply, given a model.

The code calls it with a fake and asserts on the result.

def build_reply(model, policy, question):
    """The model is a parameter. That is the whole testability trick."""
    prompt = f"POLICY:\n{policy}\n\nQ: {question}"
    raw = model(prompt)
    return {"prompt_chars": len(prompt), "answer": raw.strip()}


def fake_model(prompt):
    return "Refunds are allowed within 7 days. [policy#2]"


result = build_reply(fake_model, "Refunds within 7 days.", "How long?")
print(result)

for name, ok in [("cites a source", "[policy#" in result["answer"]),
                 ("mentions the figure", "7 days" in result["answer"]),
                 ("prompt was built", result["prompt_chars"] > 20)]:
    print(f"  {'PASS' if ok else 'FAIL'} {name}")

# Prompt construction, parsing, validation, routing and error handling are all
# exercised. None of them needs a real model, and together they are where most
# bugs actually live.

Prompt construction, parsing, validation, routing and error handling are all exercised. None of them needs a real model, and together they are where most bugs actually live.

The alternative - a module-level client created at import - makes the module impossible to import without a key, which means the tests never run in CI.

The mistake this prevents

The mistake is building the client at import time because it is used everywhere. Pass it in, or create it in a factory the tests can override; the import-time version is what forces every test to need a key.

Takeaway

Take the model as a parameter. Everything except the model's own behaviour is then testable with no key, no cost and no network.

Mark complete