Unit 11.00: Where a vision dataset's bias comes from

Every collection decision narrows what a model can generalise to, and none of them is visible in the weights.

Six decisions, six embedded assumptions

Who, when, what equipment, which cases, who labelled, and what was discarded.

The code lists what each embedded.

SOURCES = [
    ("who was photographed",   "one warehouse's staff and their handling style"),
    ("when",                   "daytime shifts only"),
    ("what equipment",         "one phone model, one camera pipeline"),
    ("which cases were kept",  "images a human already flagged as interesting"),
    ("who labelled",           "two people with a shared idea of 'damaged'"),
    ("what was discarded",     "blurry images -- which correlates with rushed shifts"),
]
print(f"{'decision':24} what it embedded")
for decision, embedded in SOURCES:
    print(f"{decision:24} {embedded}")

print("""
Every row is a collection decision, and each one narrows what the model can
generalise to. None of them is visible in the weights.

The last is the subtle one: discarding blurry images sounds like quality
control and removes the exact conditions -- rushed, understaffed shifts --
where damage is most likely.
""")

The last row is the subtle one. Discarding blurry images sounds like quality control and removes exactly the conditions - rushed, understaffed, poorly lit - where the thing you are detecting is most likely to occur.

None of these appears in a model file. Six months later, nobody can reconstruct which of them applies unless it was written down at the time.

The mistake this prevents

The mistake is thinking of bias as something that enters through the algorithm. It enters through six collection decisions, all of them made for good local reasons, before any model exists.

Takeaway

Bias enters through collection decisions, each made for a sensible local reason. Record them at the time - they cannot be recovered from the model.

Mark complete