Unit 04.06: Project step: structured-data neural-network report

The whole module in one script, producing a report a reviewer could check without asking you a question.

What a defensible write-up contains

The pipeline is standard: split, scale on training statistics, train with dropout, evaluate. What makes the output a *report* rather than a number is the surrounding record — seed, row counts, scaling method, both baselines, and the confusion matrix.

Note model.train() before training and model.eval() before evaluating: dropout must be active in one and inactive in the other.

import json

import torch
from torch import nn
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix

torch.manual_seed(0)
X = torch.randn(800, 6)
y = ((X[:, 0] + X[:, 1] * X[:, 2] - 0.5 * X[:, 3]) > 0).float().unsqueeze(1)
n = int(len(X) * 0.7)
Xtr, ytr, Xte, yte = X[:n], y[:n], X[n:], y[n:]

mean, std = Xtr.mean(0), Xtr.std(0)          # training statistics only
Xtr_s, Xte_s = (Xtr - mean) / std, (Xte - mean) / std

model = nn.Sequential(nn.Linear(6, 32), nn.ReLU(), nn.Dropout(0.1), nn.Linear(32, 1))
opt = torch.optim.Adam(model.parameters(), lr=0.01)
for _ in range(500):
    model.train(); opt.zero_grad()
    nn.BCEWithLogitsLoss()(model(Xtr_s), ytr).backward()
    opt.step()

model.eval()
with torch.no_grad():
    preds = (torch.sigmoid(model(Xte_s)) > 0.5).float()
acc = (preds == yte).float().mean().item()
baseline = LogisticRegression(max_iter=1000).fit(Xtr_s.numpy(), ytr.numpy().ravel())
base_acc = accuracy_score(yte.numpy().ravel(), baseline.predict(Xte_s.numpy()))
tn, fp, fn_, tp = confusion_matrix(yte.numpy().ravel(), preds.numpy().ravel()).ravel()

print(json.dumps({
    "seed": 0,
    "rows": {"train": len(Xtr), "test": len(Xte)},
    "scaling": "standardised with training mean and std only",
    "majority_baseline": round(max(yte.mean().item(), 1 - yte.mean().item()), 3),
    "logistic_baseline": round(float(base_acc), 3),
    "neural_network": round(acc, 3),
    "confusion": {"tn": int(tn), "fp": int(fp), "fn": int(fn_), "tp": int(tp)},
    "limitation": "synthetic data with a known interaction; real data rarely this clean",
}, indent=2))

The confusion matrix is worth more than the accuracy. It separates the two error types, and on an imbalanced problem those errors usually have very different costs.

The limitation line is not decoration. This data is synthetic with a known interaction, so the result says the pipeline works — not that it would work on real data.

The mistake this prevents

Forgetting model.eval() at evaluation. Dropout stays active, predictions become random, and the reported accuracy is lower than the model deserves — a bug that looks like a modelling failure.

Takeaway

A number, its baselines, its confusion matrix, and its limitations. Anything less is not reportable.

Mark complete