Module 05 Knowledge Check
5 questions. Pass mark 4 out of 5. Answer every question before checking the answer key below, then retry after reading the feedback.
1. Failing to reject H0 means…
- A. H0 is true
- B. the data was not sufficient to rule H0 out
- C. the test was run incorrectly
- D. the effect is zero
2. A p-value of 0.01 means…
- A. there is a 1% chance H0 is true
- B. data this extreme arises 1% of the time if H0 is true
- C. the effect is real 99% of the time
- D. 1% of the data is unusual
3. Twenty independent tests at alpha 0.05 with all nulls true gives…
- A. a 5% chance of a false positive
- B. about a 65% chance of at least one
- C. no chance of a false positive
- D. exactly one false positive always
4. Switching to alternative='greater' after seeing the direction…
- A. is a legitimate refinement
- B. converts the false-positive rate from 5% to 10%
- C. has no effect on the error rate
- D. requires a larger sample
5. A small study finds a 4.33 difference at p = 0.43; a large one finds 0.78 at p = 0.0002. This shows that…
- A. the large study found the bigger effect
- B. the p-value reflects sample size, not effect size
- C. the small study is better powered
- D. the results contradict each other
---
Answer Key and Explanations
Check these only after attempting every question.
1. B — the data was not sufficient to rule H0 out
At n = 40 with sd = 15 the test could not distinguish a mean of 100 from 103.
2. B — data this extreme arises 1% of the time if H0 is true
H0 is assumed true in order to compute it, so it cannot be the probability that H0 is true.
3. B — about a 65% chance of at least one
The simulation gave 0.645 against the theoretical 1 − 0.95**20 = 0.642.
4. B — converts the false-positive rate from 5% to 10%
The one-sided p is exactly half the two-sided, so choosing the direction afterwards doubles the real error rate.
5. B — the p-value reflects sample size, not effect size
The smaller effect had the far smaller p-value because n was 500 times larger.
Practical Check
Apply this module to your own work: complete the module activity for *Hypothesis Testing and Test Selection*, then write one sentence naming what your result shows and one naming what it does not.
Strong Answer Pattern
A strong answer names the task, the evidence used, the check performed, and the remaining limitation. It avoids "proved", "guaranteed", or "always" unless the evidence genuinely supports it.