Module 7 of 9 · Lesson 21 of 26

Bound background work and shut down cleanly

Work through bound background work and shut down cleanly using a runnable reference, a focused regression check and a local extension.

Read in any order. All lessons stay open, including after an unanswered or incorrect check.

In this lesson you will bound background work and shut down cleanly. Work with the Equipment Rental API in the downloadable lab. The reference is a complete solution with separate lesson checks, so you can inspect the answer, make a deliberate local change and verify its behavior.

Accepted background work consumes resources even after a response is sent. PreviewQueue sets a maximum waiting queue size and a fixed worker count. submit uses put_nowait: a full queue rejects work with 503 instead of accumulating unlimited tasks. Only immutable identifiers enter the queue; a worker opens its own database session rather than retaining a request's ORM objects.

At shutdown, the queue stops accepting work, gives outstanding items a bounded drain interval, then cancels and awaits its workers. task_done runs even when a handler fails, and failures are counted without logging payloads. A 202 acceptance response means admission to this process-local queue, not durable completion. A process crash may lose queued previews. Use a persistent queue and an explicit job state contract when work must survive restarts; this teaching preview sends no messages and makes no delivery guarantee. Bounding admission and cleanup is useful even for non-durable work. Fire-and-forget create_task calls without supervision would bypass both limits and shutdown accounting.

Worked source: equipment/jobs.py, PreviewQueue.

Run lesson21. One handler is blocked, one item waits, and a third is rejected. Shutdown cancels the blocked worker within its drain deadline, and later submission is denied. Inspect the finally block that pairs every queue.get with task_done.

pythonCopyable

class PreviewQueue:
    def __init__(self, handler, *, capacity=2, workers=1):
        if capacity < 1 or workers < 1:
            raise ValueError("Positive capacity and worker count are required")
        self.queue = asyncio.Queue(maxsize=capacity)
        self.handler = handler
        self.count = workers
        self.tasks = []
        self.accepting = False
        self.failures = 0

    async def start(self):
        self.accepting = True
        self.tasks = [asyncio.create_task(self.worker()) for _ in range(self.count)]

    def submit(self, identifier):
        if not self.accepting:
            raise DomainError("preview_unavailable", "Preview work is stopping.", 503)
        try:
            self.queue.put_nowait(str(identifier))
        except asyncio.QueueFull as error:
            raise DomainError(
                "preview_busy", "Preview work is full; try later.", 503
            ) from error

    async def worker(self):
        while True:
            identifier = await self.queue.get()
            try:
                await self.handler(identifier)
            except Exception:  # noqa: BLE001 -- worker boundary supervises failed preview jobs
                self.failures += (
                    1  # supervised failure; no payload or credentials logged
                )
            finally:
                self.queue.task_done()

    async def stop(self, deadline=0.2):
        self.accepting = False
        with suppress(TimeoutError):
            await asyncio.wait_for(self.queue.join(), timeout=deadline)
        for task in self.tasks:
            task.cancel()
        await asyncio.gather(*self.tasks, return_exceptions=True)

TerminalPython 3.13 virtual environment; Docker running; extracted lab directory

python run_checks.py -k lesson21

Expected result The selected lesson test passes against a new temporary PostgreSQL database; the container is removed afterward.

Keep for reference

Equipment Rental lab and lesson checks

ZIP containing Python source, real Alembic migrations, 26 lesson checks, a dependency lock and text instructions. Extract it before following the local exercise.

Download Equipment Rental lab and lesson checks

Practise locally

Add a handler that raises a deliberate local exception. Confirm failures increments, the worker remains available for another item and stop returns with every worker done. Then test a clean drain by releasing the handler before shutdown. Document that accepted work may be lost on process failure; do not describe the preview as an email delivery service.

The lesson check verifies the reference behavior. Add your own assertions for your change. Local practice is not uploaded or scored by this learning release.

Pause and reflect

What failure does this lesson prevent, and which assertion in lesson21 would expose it?

Use a concrete input, expected result and limitation from your local work. Saving a reflection does not certify the project.

Optional knowledge check

What does 202 mean for this process-local preview queue?

Use one worker but leave the waiting queue unlimited.

Try another answer. A fixed worker count does not bound accumulated waiting work.

The item was admitted, with no durable completion guarantee.

Correct. A restart can lose this in-memory queue; acceptance and durable delivery are distinct.

The preview is permanently stored and has already been delivered.

Try another answer. This queue has neither persistent storage nor a delivery mechanism.

This practice does not assess your project or award a certificate.

Your reading progress

Progress is saved in this browser when storage is available.