Unit 11.05: Model cards and responsible release boundaries

A model card documents what a model is for and, more importantly, what it is not for.

The section people omit

Intended use, training data, metrics, and limitations are all straightforward to write. The section that gets left out is out-of-scope use — the situations where the model should not be applied at all.

That section is the one a reader most needs, and the only one that constrains anything:

import json

import torch
from torch import nn

torch.manual_seed(0)
X = torch.randn(500, 4)
y = ((X[:, 0] + X[:, 1]) > 0).float().unsqueeze(1)
Xtr, ytr, Xte, yte = X[:400], y[:400], X[400:], y[400:]

model = nn.Sequential(nn.Linear(4, 32), nn.ReLU(), nn.Linear(32, 1))
opt = torch.optim.Adam(model.parameters(), lr=0.01)
for _ in range(400):
    opt.zero_grad()
    nn.BCEWithLogitsLoss()(model(Xtr), ytr).backward()
    opt.step()
with torch.no_grad():
    acc = ((torch.sigmoid(model(Xte)) > 0.5).float() == yte).float().mean().item()
    shifted = torch.randn(100, 4) + 3
    shifted_y = ((shifted[:, 0] + shifted[:, 1]) > 0).float().unsqueeze(1)
    ood = ((torch.sigmoid(model(shifted)) > 0.5).float() == shifted_y).float().mean().item()

card = {
    "model": "4-32-1 MLP, binary classifier",
    "seed": 0,
    "training_data": {"rows": len(Xtr), "source": "synthetic standard normal"},
    "metrics": {"test_accuracy": round(acc, 3),
                "majority_baseline": round(max(yte.mean().item(), 1 - yte.mean().item()), 3),
                "accuracy_under_shift": round(ood, 3)},
    "intended_use": "teaching example only",
    "out_of_scope": ["any decision about a person",
                     "inputs outside the training range, where accuracy falls but confidence does not"],
    "known_limitations": ["no fairness evaluation performed -- no group labels exist",
                          "single seed; results move with initialisation"],
}
print(json.dumps(card, indent=2))
print("\nA model card without an out-of-scope section is marketing, not documentation.")

Notice that accuracy_under_shift is reported next to test accuracy. Including a number that makes the model look worse is what makes the rest of the card credible.

The limitations here are specific and checkable: no fairness evaluation was performed, and it says why — there are no group labels.

The mistake this prevents

Writing a model card that lists only capabilities. Without an out-of-scope section it documents nothing that would stop a misuse, which is the main thing the card exists to do.

Takeaway

State what the model is for, what it scores, and where it must not be used. The last one is the deliverable.

Mark complete