The same population can look completely different depending on how you sample it. Compare a random sample against a convenience sample below — both draw 10 dots from the exact same population.
The population above is a true, fixed 50/50 split — but it’s clustered by position, the way real populations often cluster by neighborhood, age group, or interest. A convenience sample pulled from just one corner reports something wildly different from the truth, every single time, no matter how many times you redraw it. A random sample — where every dot has an equal chance of being picked — lands close to the true 50% on average, even though any one draw won’t be exact.
That distinction is the core of evaluating statistical claims: a small margin of error only tells you the sample was large, not that it was representative. A huge, precise-looking poll of the wrong people is still wrong.
The other classic trap is correlation vs. causation. Two things moving together doesn’t mean one causes the other — a hidden third factor (like weather behind ice cream sales and drownings) can drive both. Proving causation specifically requires a controlled experiment with random assignment, which spreads every other difference evenly across groups so only the variable being tested is left to explain any gap.