Module 1 Lesson: Statistical Questions and Data Claims
Classroom Explanation
A statistical question is not the same as a topic. "Customer satisfaction" is a topic; "has the proportion of customers rating us 4 or 5 changed since March?" is a question. Only the second can be answered, and only the second tells you what to measure.
In this module, the goal is to turn a vague topic into a clear statistical question and a careful claim boundary. The important part is not only the calculation. The important part is the thinking before and after the calculation.
Why This Matters
The common mistake is jumping from a pattern to a cause. A comparison can suggest a difference, but it cannot prove why the difference happened unless the study design supports that claim. This mistake can make a report sound more confident than the evidence deserves.
Key Ideas
1. A data question asks what information is available.
This is the inventory question. It asks what columns, rows, records, charts, or measurements we actually have before we decide what they mean.
2. A statistical question asks about a pattern, comparison, relationship, or uncertainty.
This is the evidence question. It asks whether there is a pattern, difference, relationship, or uncertainty that can be checked from data.
3. A decision question asks what action should be taken after the evidence is reviewed.
This is the action question. It asks what someone should do next, but that action should come after the evidence and limits are clear.
Worked Example
A claim says short lessons help students learn better. A safer question asks whether learners using short lessons have higher checkpoint scores in a defined sample.
Here is the dataset used in this module.
| scenario_id | vague_claim | likely_claim_type | safer_question | boundary_note |
|---|---|---|---|---|
| C01 | Students learn better with short lessons | inferential | In this sample do students using short lessons have higher checkpoint scores than students using long lessons? | Can compare groups but cannot prove lesson length caused the difference. |
| C02 | The new button improves signups | causal | In a randomized A/B test did the new button produce a higher signup rate than the old button? | Needs random assignment and practical-size review. |
| C03 | People who study more always score higher | predictive | In this dataset how strongly are study hours associated with score? | Association is not a guarantee for every learner. |
| C04 | The support team was worse in March | descriptive | Did the March support ticket rate per active account increase compared with February? | Counts alone are not enough without active-account denominators. |
| C05 | The training program works for everyone | inferential | Among learners who completed the training what average score change was observed? | Completion bias may make the result too optimistic. |
Work backwards from the claim someone wants to make. Ask what would have to be true, then ask what data would show it. Very often the data in front of you cannot show it, and finding that out early is the useful outcome.
How To Think Through It
- Write the claim someone wants to make.
- Rewrite it as a question with a population and a measure.
- Name the data that would answer it.
- Compare that with the data you actually have.
- State the narrower claim your data supports.
Common Mistake
The common mistake is jumping from a pattern to a cause. A comparison can suggest a difference, but it cannot prove why the difference happened unless the study design supports that claim.
How To Write The Result
In this sample, the short-lesson group had higher checkpoint scores, but this comparison alone does not prove that lesson length caused the difference.
Practice Prompt
Rewrite three vague claims into statistical questions and add one boundary note for each claim.
Takeaway
A claim you cannot turn into a question is not yet something statistics can help with.