Unit 02.00: Six dimensions, measured separately

Quality is not one number. Averaging six dimensions produces a figure that cannot fail in any specific way.

Six measurements, six bars, six verdicts

Correctness, grounding, format, safety, latency and cost. Each has its own bar and its own direction - some are better higher, some lower.

The code checks all six against their thresholds.

RESULT = {
    "correctness": 0.92,
    "grounding":   0.78,
    "format":      1.00,
    "safety":      0.99,
    "latency_p95_ms": 1840,
    "cost_per_request_usd": 0.0031,
}
BARS = {"correctness": 0.90, "grounding": 0.95, "format": 0.99,
        "safety": 0.99, "latency_p95_ms": 2000, "cost_per_request_usd": 0.005}

print(f"{'dimension':22} {'measured':>10} {'bar':>10}  verdict")
for name, value in RESULT.items():
    bar = BARS[name]
    lower_is_better = name.startswith(("latency", "cost"))
    ok = value <= bar if lower_is_better else value >= bar
    print(f"{name:22} {value:>10} {bar:>10}  {'PASS' if ok else 'FAIL'}")

print("\nOne composite score would average grounding's failure away.")

# Grounding is the only dimension below its bar, and a single "quality: 0.93"
# would hide it entirely. Six numbers, six bars, six verdicts.

Grounding is the only dimension below its bar, at 0.78 against 0.95. A composite "quality: 0.93" would absorb that completely, and the system would ship with its grounding a fifth below target.

Note that latency and cost are compared in the opposite direction. Mixing higher-is-better and lower-is-better metrics into one score is not just lossy - it requires arbitrary weighting decisions that nobody revisits.

The mistake this prevents

The mistake is reporting a composite because stakeholders want one number. Give them the one that gates the release - usually the weakest dimension - and keep the six visible. A composite moves for reasons nobody can attribute.

Takeaway

Measure six dimensions separately against six bars. A composite hides the one that is failing and requires weightings you cannot defend.

Mark complete