Module 11 Summary

What this module established

A control group is only a comparison if it is comparable. Adopters outscored non-adopters by 6.45 against a true effect of 2 — because their baseline engagement was 52.8 against 45.2 before the feature existed.

Carry forward

• Randomisation balances the variables you never thought of. Adjustment fixes only the confounders you measured, and the observational estimate here was significant at p = 4e-49 for an effect of exactly zero.
• Every line of a pre-analysis plan blocks a specific abuse. The stopping rule is the one most often missing.
• One primary metric decides; guardrails can only veto. Retention did not move and support contacts rose by 2.3 points.
• Twenty subgroup tests on a zero effect produced exactly one 'winner' at p = 0.0498 — precisely the expected number.

Before moving on

Move on when your plan is dated, your primary metric is named, and any subgroup finding is reported with how many were tested.

Mark complete