Module 05 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. Failing to reject H0 means…
- A. H0 is true
- B. the data was not sufficient to rule H0 out
- C. the test was run incorrectly
- D. the effect is zero
2. A p-value of 0.02 means…
- A. there is a 2% chance H0 is true
- B. data this extreme arises 2% of the time if H0 is true
- C. the effect is real 98% of the time
- D. 2% of the data is unusual
3. Running twenty independent tests at alpha 0.05 when all nulls are true gives…
- A. a 5% chance of a false positive
- B. about a 65% chance of at least one
- C. no chance of a false positive
- D. exactly one false positive always
4. Switching to a one-sided test after seeing the direction…
- A. is a legitimate refinement
- B. converts the false-positive rate from 5% to 10%
- C. has no effect on the error rate
- D. requires a larger sample
5. Between a small study finding a 6.21 difference at p = 0.26 and a large one finding 0.92 at p < 0.0001…
- A. the large study found the bigger effect
- B. the p-value reflects sample size, not effect size
- C. the small study is better powered
- D. the results are contradictory
---
Answer Key and Explanations
Check these only after attempting every question.
1. B — the data was not sufficient to rule H0 out
At n = 40 with SD 15 the test could not distinguish a mean of 100 from 103.
2. B — data this extreme arises 2% of the time if H0 is true
H0 is assumed true in order to compute it, so it cannot be the probability that H0 is true.
3. B — about a 65% chance of at least one
The simulation gave 0.653 against the theoretical 1 - 0.95^20 = 0.642.
4. B — converts the false-positive rate from 5% to 10%
The one-sided p is exactly half the two-sided, so choosing the direction afterwards doubles the real error rate.
5. B — the p-value reflects sample size, not effect size
A p-value measures evidence against the null. The smaller effect had the far smaller p-value because n was 500 times larger.
Practical Check
Apply this module to your own work: complete the module activity for *Hypothesis Testing and Test Selection*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.