Module 8 Worksheet: Hypothesis Testing and p-values
Scenario
A test result has come back and someone has already written the sentence they intend to publish.
Data
Dataset: hypothesis-test-scenarios.csv
| study_id | question | p_value | effect_size_note | practical_importance | caution |
|---|---|---|---|---|---|
| HT01 | Did the new reminder increase worksheet completion? | 0.041 | Completion rose by 2 percentage points | low | Statistical significance may not be practically useful. |
| HT02 | Did Group A score higher than Group B? | 0.180 | Mean difference was 3 points | unclear | Sample evidence is weak; do not claim a real difference. |
| HT03 | Did the new lesson layout reduce reading time? | 0.004 | Average time fell by 40 seconds | medium | Check whether faster reading harmed understanding. |
| HT04 | Did one of ten tested messages perform best? | 0.049 | Winner beat second place by 1 percentage point | low | Multiple testing makes the result fragile. |
| HT05 | Did practice quizzes improve final review scores? | 0.030 | Mean score rose by 8 points | medium | Check assignment process before using causal language. |
Your Task
For each scenario, write the p-value meaning, the practical-size note, and one caution.
Questions
- What exactly does this p-value mean?
- What does their sentence claim instead?
- How many comparisons were made in total?
- How does that change how one result should be read?
- What effect size accompanies the p-value?
- What would you want alongside it before acting?
- What is the honest one-sentence report?
Write Your Conclusion
Write the report you would defend, including the estimate, the interval, the p-value and the number of tests run.
Self-Check
- Did I state the p-value's meaning correctly?
- Did I ask how many tests were run?
- Did I report an effect size, not only significance?
- Would my sentence survive a sceptical reader?