Unit 10.05: Project step: embedding exploration report

An embedding evaluation that reports usefulness and leakage together, because shipping one number without the other is not a decision anyone can make.

Three questions, one report

Does the embedding help the downstream task? Does it beat using raw features? And is a sensitive attribute recoverable from it?

All three use the same probe method, so the numbers are directly comparable:

import json

import torch
import torch.nn.functional as F
from torch import nn
from sklearn.linear_model import LogisticRegression

torch.manual_seed(0)
raw = torch.randn(500, 20)
task = (raw[:, 0] + raw[:, 1] > 0).long()
sensitive = (raw[:, 2] > 0).long()

encoder = nn.Sequential(nn.Linear(20, 32), nn.ReLU(), nn.Linear(32, 8))
encoder.eval()
with torch.no_grad():
    vectors = encoder(raw)

split = 400
def probe(features, target):
    model = LogisticRegression(max_iter=1000).fit(features[:split].numpy(), target[:split].numpy())
    return round(float(model.score(features[split:].numpy(), target[split:].numpy())), 3)

unit = F.normalize(vectors, dim=1)
sims = unit @ unit.T
sims.fill_diagonal_(-2.0)

print(json.dumps({
    "seed": 0,
    "encoder": "20 -> 32 -> 8, untrained (random projection baseline)",
    "vectors": list(vectors.shape),
    "downstream_task_accuracy": probe(vectors, task),
    "raw_feature_accuracy": probe(raw, task),
    "sensitive_attribute_recoverable": probe(vectors, sensitive),
    "neighbour_shares_task_label": round((task[sims.argmax(1)] == task).float().mean().item(), 3),
    "reading": "an untrained encoder is a random projection -- it should NOT beat raw features",
    "limitation": "synthetic data; on real embeddings run this probe per sensitive attribute",
}, indent=2))

The reading field states the expected result in advance: this encoder is untrained, so it is effectively a random projection and should *not* beat raw features. Writing that expectation down before running turns the output into a test rather than an observation.

If an untrained encoder does beat raw features, something is wrong with the evaluation.

The mistake this prevents

Reporting only downstream accuracy. An embedding that performs well and encodes a protected attribute is not ready to ship, and the accuracy figure will never tell you that.

Takeaway

Report usefulness and leakage in the same table, and state the expected result before you run.

Mark complete