Unit 02.00: Reading a file without trusting its extension

A file extension is a claim about the contents. The bytes are the fact.

Sniff the header, do not trust the name

Every image format begins with a recognisable signature.

The code writes a PNG and inspects its first bytes, then lists three mismatches.

import io
import numpy as np
from PIL import Image

buffer = io.BytesIO()
Image.fromarray(np.full((8, 8, 3), 128, dtype="uint8")).save(buffer, format="PNG")
data = buffer.getvalue()

print("first 8 bytes:", data[:8])
print("PNG magic number present:", data[:4] == b"\x89PNG")

# The extension is a claim; the bytes are the fact.
for name, header, kind in [("photo.jpg", b"\xff\xd8\xff", "JPEG"),
                           ("photo.jpg", b"\x89PNG", "actually a PNG"),
                           ("photo.png", b"GIF89a", "actually a GIF")]:
    print(f"{name:12} starts {header[:4]!r:14} -> {kind}")

image = Image.open(io.BytesIO(data))
print(f"\nloaded: {image.size} {image.mode}")

# Trusting the extension gives you a decoder error deep in a training loop, on
# one file, hours in. Read the header, or let the library sniff it and fail at
# load time.

A .jpg that is actually a PNG is common - someone renamed it, or an export tool lied. The decoder usually copes, and when it does not you get an error deep in a training loop, on one file, hours in.

Letting the library sniff the format and fail at load time is the cheap fix: the failure happens where you can see which file caused it.

The mistake this prevents

The mistake is dispatching on the extension to choose a decoder. Let the library detect the format, and validate at load time rather than at use time.

Takeaway

Detect format from the bytes, not the filename, and fail at load. An error hours into training on one unnamed file is the alternative.

Mark complete