Unit 14.03: Health checks that mean something

Liveness and readiness answer different questions and warrant different responses.

Restart it, or stop sending it traffic

Two endpoints, one reporting the process and one reporting its dependencies.

The code shows both with one dependency failing.

from fastapi import FastAPI
from fastapi.testclient import TestClient

app = FastAPI()
DEPENDENCIES = {"database": True, "model_api": False}


@app.get("/healthz")
def liveness() -> dict:
    return {"status": "alive"}


@app.get("/readyz")
def readiness():
    failing = [name for name, ok in DEPENDENCIES.items() if not ok]
    if failing:
        return {"status": "not_ready", "failing": failing}
    return {"status": "ready"}


client = TestClient(app)
print("liveness :", client.get("/healthz").json())
print("readiness:", client.get("/readyz").json())

print("""
Liveness answers "is this process wedged?" -- restart it if not. Readiness
answers "should traffic come here?" -- and a dependency being down is a reason
to stop sending traffic, not to restart.

A health check that returns 200 unconditionally tells an orchestrator nothing
and will keep a broken instance in the load balancer.
""")

Liveness answers "is this process wedged?" - if not, restarting is the remedy. Readiness answers "should traffic come here?", and a dependency being down is a reason to stop sending traffic rather than to restart.

Conflating them means an orchestrator restarts a perfectly healthy process because a database is briefly unavailable, which helps nothing and loses in-flight requests.

The mistake this prevents

The mistake is a health check that returns 200 unconditionally. It tells the orchestrator nothing and will keep a completely broken instance in the load balancer indefinitely.

Takeaway

Separate liveness from readiness. A failing dependency should remove an instance from rotation, not restart it.

Mark complete