Unit 11.02: Faces, consent and the uses you did not intend

The uses that cause harm are usually small, reasonable-sounding extensions of a system that works.

Intended, drift, prohibited, unvalidated

Five uses sorted into four categories.

The code lists them with the reason for each.

USES = [
    ("detect carton damage",        "intended", "what it was built and measured for"),
    ("count cartons per hour",      "drift",    "never evaluated for counting"),
    ("identify who handled a box",  "prohibited", "the dataset has no consent for this"),
    ("score staff performance",     "prohibited", "surveillance; not the agreed purpose"),
    ("deploy at a second warehouse", "unvalidated", "different lighting, different racking"),
]
print(f"{'use':34} {'class':12} why")
for use, cls, why in USES:
    print(f"{use:34} {cls:12} {why}")

print("""
Only the first is intended. The second and fifth are how systems actually
spread -- a small, reasonable-sounding extension of something that works.

Faces incidentally present in the training images do not make the model a face
recogniser, and they do make the third use a consent problem regardless.
""")

Counting cartons per hour is the drift case: adjacent to the trained task, never evaluated for it, and entirely plausible as a request. Deploying to a second site is the same shape.

Identifying who handled a box is prohibited for a reason that has nothing to do with capability. The dataset carries no consent for it, and faces being incidentally present does not create one.

The mistake this prevents

The mistake is treating capability as permission. A model that could distinguish staff members may not be used to, and the constraint comes from the collection agreement rather than from what the weights can do.

Takeaway

Sort proposed uses into intended, unvalidated and prohibited. Capability is not permission, and the drift cases are the ones that actually happen.

Mark complete