Module 8 Worksheet: Hypothesis Testing and p-values
Scenario
You are helping a beginner analyst review a small synthetic dataset. Your job is not to sound impressive. Your job is to make the safest useful statement.
Data
Dataset: hypothesis-test-scenarios.csv
| study_id | question | p_value | effect_size_note | practical_importance | caution |
|---|---|---|---|---|---|
| HT01 | Did the new reminder increase worksheet completion? | 0.041 | Completion rose by 2 percentage points | low | Statistical significance may not be practically useful. |
| HT02 | Did Group A score higher than Group B? | 0.180 | Mean difference was 3 points | unclear | Sample evidence is weak; do not claim a real difference. |
| HT03 | Did the new lesson layout reduce reading time? | 0.004 | Average time fell by 40 seconds | medium | Check whether faster reading harmed understanding. |
| HT04 | Did one of ten tested messages perform best? | 0.049 | Winner beat second place by 1 percentage point | low | Multiple testing makes the result fragile. |
| HT05 | Did practice quizzes improve final review scores? | 0.030 | Mean score rose by 8 points | medium | Check assignment process before using causal language. |
Your Task
For each scenario, write the p-value meaning, the practical-size note, and one caution.
Questions
- What is the statistical question?
- What is the unit of analysis?
- Which variables matter most?
- What summary, comparison, or visual check is suitable?
- What does the evidence suggest?
- What should not be claimed?
- What limitation should be mentioned?
Write Your Conclusion
Write two to four careful sentences. Use plain English. Do not use words like proves, guarantees, or always unless the evidence truly supports them.
Self-Check
- Did I name the question?
- Did I use the right denominator or comparison?
- Did I avoid claiming more than the data can support?
- Did I mention uncertainty, bias, assumptions, or limits where needed?