Module 9 Lesson: Comparing Groups and Effect Sizes
Classroom Explanation
Imagine we are in class and someone puts a small table on the board. The table looks simple, so it is tempting to jump straight to the answer. In statistics, we slow down first. We ask what question the table can answer, what question it cannot answer, and what kind of claim would be safe.
In this module, the goal is to compare groups using magnitude, uncertainty, and practical meaning. The important part is not only the calculation. The important part is the thinking before and after the calculation.
Why This Matters
The common mistake is ranking groups by mean only and ignoring sample size, spread, and how people entered the group. This mistake can make a report sound more confident than the evidence deserves.
Key Ideas
1. A group comparison should name the groups, metric, and sample sizes.
A fair comparison names the groups, the metric, and how people or records entered each group. Without that, the comparison may be shallow.
2. Effect size asks how large the difference is.
Effect size asks how much difference there is. It moves the discussion from is there evidence? to how large is the difference?
3. A result can be statistically visible but practically small, or practically important but uncertain.
Practical meaning depends on the decision. A tiny change can be statistically visible and still not worth acting on.
Worked Example
The three-practice-quiz group has a higher mean score than the no-practice group. The optional review group is also high, but self-selection makes that comparison weaker.
Here is the dataset used in this module.
| group_name | sample_size | mean_score | standard_deviation | notes |
|---|---|---|---|---|
| No practice quiz | 48 | 68 | 12 | baseline group |
| One practice quiz | 52 | 72 | 11 | small improvement |
| Three practice quizzes | 50 | 79 | 10 | stronger improvement |
| Optional review session | 28 | 81 | 14 | self-selected group |
| Required review session | 55 | 76 | 9 | assigned group |
Read the table slowly. First name the unit. Then name the variables. Then name the comparison or pattern. Only after that should you write the conclusion.
How To Think Through It
- State the question in one sentence.
- Name the unit of analysis.
- Identify the outcome and comparison.
- Check the denominator, sample, uncertainty, or assumption.
- Write only what the data supports.
- Add one limit so the result does not overclaim.
Common Mistake
The common mistake is ranking groups by mean only and ignoring sample size, spread, and how people entered the group.
How To Write The Result
The three-practice-quiz group shows a larger average score than the no-practice group, but the comparison should still mention spread and assignment conditions.
Practice Prompt
Write a cautious group-comparison paragraph using mean, sample size, spread, and one limitation.
Takeaway
When a statistical result feels obvious, pause and ask: what exactly was measured, compared, and assumed?