Unit 10.04: Keeping the model call out of the request path

Anything slower than a couple of seconds should not hold the request open.

Accept, return an id, poll or reconnect

Submission returning a job id, and a worker completing it.

The code shows both halves.

JOBS = {}


def submit(question):
    job_id = f"job-{len(JOBS) + 1}"
    JOBS[job_id] = {"status": "queued", "answer": None}
    return job_id


def worker_step(job_id):
    JOBS[job_id] = {"status": "done", "answer": "Refunds within 7 days."}


job = submit("How long for a refund?")
print(f"POST /ask   -> 202 accepted, {job}, status={JOBS[job]['status']}")
worker_step(job)
print(f"GET /ask/{job} -> {JOBS[job]}")

print("""
For anything that takes more than a couple of seconds, accept the request,
return an id, and let the client poll or reconnect.

It survives a dropped connection, a page reload and a mobile network change --
none of which a request that holds the connection open does.
""")

This survives a dropped connection, a page reload and a mobile network change - none of which a held-open request does. It also lets you retry the work without the user resubmitting.

The job record is where the request id, the versions and the outcome live, so the trace from Module 3 has somewhere natural to go.

The mistake this prevents

The mistake is holding the connection open because it is simpler. It works in development, on a fast connection, with one user - and fails on mobile networks, behind proxies with their own timeouts, and under any load.

Takeaway

For anything slow, accept the request, return an id, and let the client come back. A held-open connection fails on exactly the networks your users have.

Mark complete