Unit 12.02: Timeouts on every outbound call

Every outbound call needs a timeout, and most client libraries default to none.

One unresponsive dependency stops everything

Four kinds of outbound call with appropriate bounds.

The code lists them.

BOUNDARIES = [
    ("database query",     "5s",  "a missing timeout holds a worker forever"),
    ("model API call",     "30s", "generation is slow; bound it anyway"),
    ("internal service",   "3s",  "your own services should be fast"),
    ("object storage read", "10s", "large files, still bounded"),
]
print(f"{'outbound call':22} {'timeout':8} why")
for call, timeout, why in BOUNDARIES:
    print(f"{call:22} {timeout:8} {why}")

print("""
Every client library defaults to no timeout or to something enormous. Without
one, a single unresponsive dependency consumes every worker in turn and the
whole service stops answering -- including endpoints that never touch it.

Your timeout must also be shorter than any gateway in front of you, or the
caller gets a generic 504 instead of your error.
""")

Without a timeout, a single unresponsive dependency consumes every worker in turn and the whole service stops answering - including endpoints that never touch it.

Your timeout also has to be shorter than any gateway in front of you, or the caller receives a generic 504 from the proxy instead of your error with its request id.

The mistake this prevents

The mistake is trusting the library's default. Several popular clients default to no timeout at all, which means the failure mode is a hang rather than an error - and a hang is much harder to diagnose.

Takeaway

Set an explicit timeout on every outbound call, shorter than any gateway in front of you. Library defaults are frequently unbounded.

Mark complete