Unit 15.02: Testing the contract offline

The suite runs with no key, no network and no cost, which is what makes it run at all.

Ten tests, nine needing no model

Each test with what it requires.

The code lists them.

TESTS = [
    ("valid request returns 200 and the contract shape", "no model"),
    ("empty text returns 422",                           "no model"),
    ("text over the limit returns 413",                  "no model"),
    ("missing API key returns 401",                      "no model"),
    ("wrong scope returns 403",                          "no model"),
    ("card number in input is refused before any call",  "no model"),
    ("exhausted budget refuses without calling",         "no model"),
    ("invalid model output returns 503, not a bad body", "fake model"),
    ("response contains no unexpected fields",           "no model"),
    ("rate limit returns 429 with Retry-After",          "no model"),
]
print(f"{'test':52} needs")
for test, needs in TESTS:
    print(f"{test:52} {needs}")

print(f"\n{len(TESTS)} tests, {sum(1 for _, n in TESTS if n == 'no model')} needing no model at all")
print("The whole suite runs with no key, no network and no cost.")

Nine of ten need no model. Validation, authentication, authorisation, the input pre-checks, the budget refusal, the response shape and the rate limit are all exercised without a network.

The tenth uses a fake that returns invalid output, which is how you test that bad model output becomes a 503 rather than a malformed response - a path a real model will not produce on demand.

The mistake this prevents

The mistake is treating the endpoint tests as integration tests because the endpoint calls a model. Only the model call is integration; everything else is ordinary code and should be tested as such.

Takeaway

Build the suite so nearly every test runs offline, including the one that checks invalid model output. Only the live call is an integration test.

Mark complete