Math · Problem-Solving and Data Analysis
Evaluating Statistical Claims
Check two separate permissions: random sampling supports generalizing to a population, while random assignment in a well-designed experiment supports a causal comparison. A large sample alone supplies neither.
Ask how the participants were selected
A survey randomly selects 300 people from a town's resident list. Its results can support an estimate for that town's represented population, subject to sampling uncertainty and study limitations. A poll answered by 300 visitors to a fitness website has a different selection process.
The number 300 does not make the second sample representative. Volunteers may differ from nonvolunteers. In statistical claims analysis, first identify the target population and then ask whether the selection method reaches it fairly.
Ask how treatments were assigned
Researchers randomly assign volunteers to a new tutoring program or an existing program and compare their later results. Random assignment helps balance other differences between groups and supports attributing an observed treatment difference to the intervention, within the experiment's limitations.
Choosing which program to attend is not random assignment. Students who choose extra tutoring might differ in motivation, prior preparation, or available time. Those differences can confound a causal explanation.
Keep generalization separate from causation
An observational study might randomly sample residents and find that those who walk more report better sleep. It can describe an association in that population, but it does not establish that walking caused the sleep difference.
An experiment using volunteers may support a causal comparison for its participants while having limited justification for generalizing to everyone. A study can earn one kind of conclusion without earning the other.
Match the claim's scope to the design
If a study samples one school's students, a choice claiming the result applies to every teenager expands the population beyond the evidence. If it only records behavior, a choice claiming a treatment caused an outcome expands association into causation.
Underline the claim's population and causal verb. Compare each with the study design before considering whether the result sounds plausible. A plausible explanation is not a substitute for the needed evidence.
Check your understanding
Try it yourself
Choose an answer.
Evaluating Statistical Claims · Hard
Choose an answer.
Practice this skill
- Label a study observational or experimental.
- Mark random sampling and random assignment separately.
- Rewrite an overbroad claim so its population and causal language fit the design.
Print the evaluating statistical claims worksheet and worked answers.
Then use a mixed practice module to check whether you can choose this method among other question types.
Source and question labels
The bank label for this guide is "Evaluating statistical claims: Observational studies and experiments" in Problem-Solving and Data Analysis. The worked examples above are written for this guide.