Unit 11.01: File uploads, and validating before reading

Validate an upload while reading it, not after.

Type first, size during

An upload endpoint checking the declared type and enforcing a size limit as it reads.

The code uploads a small file, an oversized one and a wrong type.

from fastapi import FastAPI, File, HTTPException, UploadFile
from fastapi.testclient import TestClient
import io

app = FastAPI()
MAX_BYTES = 1024
ALLOWED = {"text/plain"}


@app.post("/upload")
async def upload(file: UploadFile = File(...)) -> dict:
    if file.content_type not in ALLOWED:
        raise HTTPException(415, {"error": "unsupported_media_type",
                                  "got": file.content_type})
    read = 0
    while chunk := await file.read(256):
        read += len(chunk)
        if read > MAX_BYTES:
            raise HTTPException(413, {"error": "file_too_large",
                                      "limit_bytes": MAX_BYTES})
    return {"bytes": read}


client = TestClient(app)
for name, content, ctype in [("small.txt", b"x" * 100, "text/plain"),
                             ("big.txt", b"x" * 5000, "text/plain"),
                             ("evil.exe", b"MZ", "application/octet-stream")]:
    r = client.post("/upload",
                    files={"file": (name, io.BytesIO(content), ctype)})
    print(f"{name:10} {len(content):>5} bytes -> {r.status_code}")

print("\nThe size check happens WHILE reading, so a huge upload is rejected")
print("before it is all in memory.")

The size check happens inside the read loop, so an oversized upload is rejected before it is all in memory. Checking afterwards means you have already accepted it.

The content type is a declaration from the client, not a fact. It is worth rejecting on, and for anything that matters the file's actual bytes need checking too - a .txt content type does not make the contents text.

The mistake this prevents

The mistake is reading the whole file and then checking its length. A caller uploading a gigabyte has already consumed a gigabyte of your memory by the time you decide to refuse it.

Takeaway

Reject on declared type first, then enforce the size limit while reading. Checking afterwards means you have already accepted the file.

Mark complete