Unit 13.02: Instrumenting every request

Every request emits one record, and the fields were decided before the first real call.

Versions, retrieval, cost, budget, retention

A complete trace with content redacted.

The code prints it.

import json

record = {
    "request_id": "r-8841",
    "prompt_version": "reply-v3",
    "policy_version": "support-policies-v4",
    "model": "some-model-2026-06",
    "retrieved": ["policy#2"],
    "refused": False,
    "unsupported_terms": [],
    "input_tokens": 96, "output_tokens": 22,
    "usd": 0.0000468,
    "latency_ms": 780, "budget_ms": 3000, "over_budget": False,
    "auto_send": False,
    "content": "[redacted]",
    "retention_days": 30,
}
print(json.dumps(record, indent=2))

print(f"\nexplains the run; keeps no content; records both versions")

Both versions are recorded - the prompt and the policy - which is what lets you explain a response after either changes. over_budget makes the latency target from the scope measurable on every request rather than in an occasional review.

auto_send: False is logged too, so an audit can confirm the boundary held rather than assume it.

The mistake this prevents

The mistake is instrumenting only failures. The request that most needs investigating is usually one that completed successfully and produced something wrong - no exception, no error, nothing a failure-triggered log would catch.

Takeaway

Emit one record per request with both versions, cost, the budget comparison and the boundary flags. Instrument successes, not only failures.

Mark complete