Sampling distributions
Almost everything in this course follows one blueprint. Start with a population, draw one sample, calculate a statistic, then ask: if we drew sample after sample, where would all those statistics land? This page lets you do that yourself.
Press +1 to draw a sample, then +50 or +500 to draw many. Click any dot to see the sample behind it. Step 1 draws one score at a time. Pick another step from the menu to change the sample size, then change the population or the true effect and draw again.
Open the simulator in a full window
Things to try
Each link opens the simulator ready to go. Before you press a button, predict what will happen.
Why does a bigger sample help?
- Means of 5 observations. Panels 1 and 3 share the same axis, so compare how far the means spread with how far the scores spread. Then move the slider to 25 and draw again.
- Compare the SD of M in the table with the true standard error, σ / √n. The standard error is the SD of all those sample means.
Does the scale of the test matter?
- A test with mean 500 and SD 100. The means are on a new scale. Now set “Show the statistic as” to z. How does this compare with a test that has mean 100 and SD 15, or a GPA with mean 3 and SD 0.5? Type those values in to check.
Why do gains help?
- Before and after. Each line in panel 2 is one person. Note how often we reject H₀. Then raise the correlation of before and after: the mean gain stays the same, but the gains get steadier, and rejections become more common.
Does the population have to be normal?
- Means from a skewed population. With n = 2 the means are still skewed. Raise n to 30 and draw again. This is the central limit theorem.
Why do we divide by n − 1?
- Samples of 5. In the table, compare SS / n with σ̂², and both with the true σ² of 225. Dividing by n falls short on average, and dividing by n − 1 lands on target. Then click a few dots: in some samples, SS / n overshoots.
- Also look at σ̂. Even when σ̂² is right on average, its square root runs a little low.
Why do we need the t-distribution?
- z with the true σ. About 5% of samples land beyond ±1.96.
- t with the estimate σ̂. Same samples, but now we use an estimate, so the t cutoffs sit farther out. Look at Beyond ±1.96 in the table: that’s how often the old z cutoffs would mislead us. Then raise n and watch the problem fade.
What does a p-value look like?
- Five percent of the time, by chance. There is no effect here, yet about 5% of samples look significant. Those are Type I errors.
- A real effect. Now the share beyond the cutoffs is the power.
Why is ANOVA a ratio of variances?
- No group differences. Compare MS within and MS between in the table. Both estimate σ², so F hovers around 1.
- Real group differences. MS within stays where it was, MS between grows, and F moves right.
- Only one group differs. Type your own group means to try other patterns. F says that the means differ, but not which ones.
Credit
This simulator was written for EDUC 15. It was inspired by the visualizations of Kristoffer Magnusson at R Psychologist, including p-values, statistical power, confidence intervals and correlation. They’re well worth exploring too.