Module 7 of 9 · Lesson 20 of 26
Bound retries and recover from provider failures
Work through bound retries and recover from provider failures using a runnable reference, a focused regression check and a local extension.
Read in any order. All lessons stay open, including after an unanswered or incorrect check.
A 400 response or invalid schema is not repaired by another identical request, so those failures are not retried. Exhausted transient attempts become 503; the total deadline becomes 504. The example deliberately does not retry writes. A timed-out booking or payment may already have succeeded remotely, so repeating it requires an agreed idempotency key and recovery contract. Production traffic may also need jitter, rate limits and a circuit policy; none are silently claimed here. Test attempt count and cancellation as behavior, rather than relying on a provider to happen to be slow during a live exercise.
Run lesson20. The provider returns 503 twice and succeeds on the third GET. A 400 is attempted only once. A delayed mock exceeds a 20ms outer deadline and must report provider_deadline. The test checks both the eventual result and the exact number of calls.
async def fetch_quote(client, url, *, attempts=3, deadline=0.5):
if not 1 <= attempts <= 5 or deadline <= 0:
raise ValueError("Choose 1–5 attempts and a positive deadline")
try:
async with asyncio.timeout(deadline):
for attempt in range(attempts):
try:
response = await client.get(url, timeout=min(deadline, 0.2))
if response.status_code in (502, 503, 504):
raise httpx.ReadError("Temporary upstream failure")
response.raise_for_status()
return Quote.model_validate(response.json())
except (httpx.ConnectError, httpx.ReadError, httpx.TimeoutException):
if attempt + 1 == attempts:
raise DomainError(
"provider_unavailable",
"The quote service is unavailable.",
503,
)
await asyncio.sleep(0.01 * 2**attempt)
except (httpx.HTTPStatusError, ValidationError, ValueError) as error:
raise DomainError(
"invalid_provider_response",
"The quote service returned an unusable response.",
502,
) from error
except TimeoutError as error:
raise DomainError(
"provider_deadline", "The quote service did not respond in time.", 504
) from errorpython run_checks.py -k lesson20Expected result The selected lesson test passes against a new temporary PostgreSQL database; the container is removed afterward.
Keep for reference
Equipment Rental lab and lesson checks
ZIP containing Python source, real Alembic migrations, 26 lesson checks, a dependency lock and text instructions. Extract it before following the local exercise.
Practise locally
Add a test with an always-503 provider and attempts=2; require exactly two calls and provider_unavailable. Add a case where the deadline expires during backoff. Explain why an unprotected POST cannot reuse this retry helper. Keep test URLs under example.invalid and do not send real provider requests.
The lesson check verifies the reference behavior. Add your own assertions for your change. Local practice is not uploaded or scored by this learning release.
Pause and reflect
What failure does this lesson prevent, and which assertion in lesson20 would expose it?
Use a concrete input, expected result and limitation from your local work. Saving a reflection does not certify the project.
Optional knowledge check
When is retrying a failed request appropriate?
Always, until the provider succeeds, including unprotected writes.
Try another answer. Unbounded retries consume resources and can duplicate a write whose result is unknown.
Give each retry a fresh full deadline, ignoring previous waits.
Try another answer. A total budget includes all attempts and backoff.
When repeating is safe and a bounded attempt fits the remaining deadline.
Correct. Both operation semantics and the total budget must permit another attempt.
This practice does not assess your project or award a certificate.
Your reading progress
Progress is saved in this browser when storage is available.