Unit 12.05: Model card and next-step plan

The final deliverable: what the model is, what it scores, where it must not be used, and what you would do next.

Why the next-steps list belongs in the card

A capstone that claims completeness is less credible than one that names its own weaknesses and says how it would address them.

Here the next steps are specific and actionable — a group-aware split, a calibration curve, and three seeds with the spread reported:

import json

import torch
from torch import nn
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

torch.manual_seed(0)
X = torch.randn(700, 6)
y = ((X[:, 0] + X[:, 1] * X[:, 2]) > 0).float().unsqueeze(1)
Xtr, ytr, Xte, yte = X[:550], y[:550], X[550:], y[550:]

net = nn.Sequential(nn.Linear(6, 32), nn.ReLU(), nn.Linear(32, 1))
opt = torch.optim.Adam(net.parameters(), lr=0.01)
for _ in range(500):
    opt.zero_grad()
    nn.BCEWithLogitsLoss()(net(Xtr), ytr).backward()
    opt.step()
with torch.no_grad():
    acc = ((torch.sigmoid(net(Xte)) > 0.5).float() == yte).float().mean().item()
logreg = LogisticRegression(max_iter=1000).fit(Xtr.numpy(), ytr.numpy().ravel())
base = accuracy_score(yte.numpy().ravel(), logreg.predict(Xte.numpy()))

print(json.dumps({
    "model": "6-32-1 MLP",
    "version": "1.0", "seed": 0,
    "parameters": sum(p.numel() for p in net.parameters()),
    "data": {"train_rows": len(Xtr), "test_rows": len(Xte),
             "split": "by row -- see the dataset card for why that is a limitation here"},
    "metrics": {"test_accuracy": round(acc, 3),
                "logistic_baseline": round(float(base), 3),
                "majority_baseline": round(max(yte.mean().item(), 1 - yte.mean().item()), 3)},
    "intended_use": "flagging for human review",
    "out_of_scope": ["automated decisions", "populations unlike the training cohort"],
    "limitations": ["single seed", "no fairness evaluation", "synthetic data"],
    "next_steps": ["group-aware split", "calibration curve", "three seeds and report the spread"],
}, indent=2))
print("\nThe next-steps list is part of the deliverable. A capstone that claims")
print("it is finished is less credible than one that says what it would do next.")

Each next step maps to a stated limitation. The split note in the data section flags row-level splitting as a known weakness, and the first next step addresses exactly that.

That pairing is what makes the card honest rather than defensive.

The mistake this prevents

Ending the capstone with a claim that the work is finished. Every model has known weaknesses; a card that lists none reads as one where nobody looked.

Takeaway

Name the limitations, then name what you would do about them. That pairing is the strongest thing in the card.

Mark complete