Module 12 Worksheet: Experiments, A/B Tests, and Statistical Reporting
Scenario
You are helping a beginner analyst review a small synthetic dataset. Your job is not to sound impressive. Your job is to make the safest useful statement.
Data
Dataset: ab-test-summary.csv
| variant | visitors | signups | completion_rate | avg_time_seconds | notes |
|---|---|---|---|---|---|
| A_old_button | 6000 | 480 | 0.0800 | 96 | control variant |
| B_new_button | 5900 | 515 | 0.0873 | 101 | new variant |
| C_highlighted_button | 1200 | 111 | 0.0925 | 108 | small follow-up sample |
Your Task
Write a final evidence review memo using the A/B-test summary.
Questions
- What is the statistical question?
- What is the unit of analysis?
- Which variables matter most?
- What summary, comparison, or visual check is suitable?
- What does the evidence suggest?
- What should not be claimed?
- What limitation should be mentioned?
Write Your Conclusion
Write two to four careful sentences. Use plain English. Do not use words like proves, guarantees, or always unless the evidence truly supports them.
Self-Check
- Did I name the question?
- Did I use the right denominator or comparison?
- Did I avoid claiming more than the data can support?
- Did I mention uncertainty, bias, assumptions, or limits where needed?