Unit 04.04: Measuring a segmentation against a hand-drawn mask

A segmentation needs a number, and one number is not enough.

IoU, precision and recall together

Three predictions against one ground-truth mask.

The code computes all three metrics for each.

import numpy as np

truth = np.zeros((32, 32), dtype=bool)
truth[8:24, 8:24] = True

predictions = {
    "exact":       (slice(8, 24), slice(8, 24)),
    "shifted 2px": (slice(10, 26), slice(10, 26)),
    "too big":     (slice(4, 28), slice(4, 28)),
}
print(f"{'prediction':14} {'IoU':>6} {'precision':>10} {'recall':>8}")
for name, box in predictions.items():
    pred = np.zeros_like(truth)
    pred[box] = True
    inter = (pred & truth).sum()
    union = (pred | truth).sum()
    print(f"{name:14} {inter / union:>6.2f} {inter / pred.sum():>10.2f} "
          f"{inter / truth.sum():>8.2f}")

# "Too big" has perfect recall and poor precision -- it found everything and
# a lot besides. One number hides that; IoU, precision and recall together
# tell you which way the prediction is wrong.

"Too big" has perfect recall and poor precision - it found everything and a great deal besides. "Shifted" loses both roughly equally. IoU alone would rank them without telling you which way each is wrong.

Which way it is wrong determines the fix. Over-segmentation and under-segmentation call for opposite adjustments.

The mistake this prevents

The mistake is reporting IoU alone because it is the standard metric. It is a single number summarising two independent failure directions, and the direction is what you act on.

Takeaway

Report IoU with precision and recall. IoU ranks predictions; precision and recall tell you which direction each one is wrong in.

Mark complete