ANOVA

# ANOVA

**Topic 10 · Thursday, February 25**

]

---

# Where are we?

.box-inv-1.small[Summarize data]

.box-inv-1.small[Visualize data]
]

.box-inv-6.small[Do these groups differ in the population?]

???

Same map. In Topic 9, the lottery let us read a gap as an effect. Today's groups are three Mindspark centers, and nobody drew lots for those. So we're back in the middle column: are these populations different? Not: what caused it?

---

# Same four steps, more means

.box-inv-8.small.sp-after-half[Step 1: State a hypothesis about the means of two or more populations]

.box-inv-8.small.sp-after-half[Step 2: Set the criteria: how large must F be to count as unexpected?]

.box-8.small.sp-after-half[Step 3: Observe a sample from each population. How unexpected is the spread between the group means, given the spread within the groups?]

.box-inv-8.small[Step 4: Reject or maintain H₀]

???

The same four steps as Topics 7, 8 and 9. Step 3 changes the most, and that's today's main idea.

---

# Plan for today

.box-1.medium.sp-after-half[Why not more t-tests?]

.box-6.medium.sp-after-half[Splitting the variability]

.box-3.medium.sp-after-half[The test, and what comes after]

???

Three blocks of about 20 minutes. Each ends with students doing something: predict, try, try.

---

# Why not more t-tests?

---

---

# Three centers, three means

.box-inv-1.small[Did the centers differ before the program, or is this sampling error?]

???

These are the three after-school centers from Topic 1. Children weren't sorted into them by a lottery, so a gap here would be a difference, not an effect. A real worry too: if centers start out different, that matters for reading anything later.

---

# More tests, more false alarms

<div class="math-display">\[
\text{Chance of at least one false alarm} = 1 - (1 - \alpha)^{\text{tests}}
\]</div>

.box-1.small[Three centers need 3 t-tests, and five groups need 10]

???

A Type 1 error is a false alarm, from Topic 7. Each t-test at α = 0.05 has a 5% risk of a Type 1 error. Across three tests, the chance that at least one goes off is 1 − 0.95³ = 0.14.

That formula treats the tests as independent. Ours share groups, so it runs a little high: a simulation of three t-tests on three identical groups gives 12%, not 14%. Either way, it's well past 5%.

Number of pairs: k(k − 1) / 2, where k is the number of groups.

---

# Green jelly beans

.picture[![Three comic panels. A person says jelly beans cause acne, and scientists are told to investigate. The scientists report no link between jelly beans and acne, p > 0.05. The person says they hear it's only a certain color that causes it, and the scientists are sent back.](img/xkcd-significant-panels.png)]

---

# Test 20 colors, and one will go off

.pull-left[
.picture[![The full comic strip. The scientists test 20 colors of jelly beans, one panel each. Nineteen find no link, p > 0.05. Green finds a link, p < 0.05.](img/xkcd-significant-strip.png)]
]

.pull-right[
.picture[![A newspaper front page: Green jelly beans linked to acne! 95% confidence. Only 5% chance of coincidence.](img/xkcd-significant-headline.png)]
]

???

With α = 0.05, 20 true null hypotheses give an average of one false alarm. That one is the headline.

---

# One question about all the means

.box-1.small.sp-after-half[One test, so α stays at 0.05 for the whole question]

???

The test covers everything at once. Statisticians call that an omnibus test. H₁ doesn't say that all the means differ: one odd group is enough.

Two or more factors at once is a factorial design. Regression picks that up in Topic 14.

---

# Same gaps, but the spread matters

.box-inv-1.small[On the left, the gaps stand out. On the right, they could be chance]

.source[After a figure in Gravetter et al., *Essentials of Statistics for the Behavioral Sciences*, 10e. Scores made up.]

???

Red ticks are the group means. This is the idea of the day, and it's the t-test's idea too: a gap only means something next to the spread.

With two groups, t compares one gap with the spread. With three or more means, there's no single gap. So ANOVA measures how spread out the means are, as a variance.

---

# F compares two variances

<div class="math-display">\[
F = \frac{\text{variance between the group means}}{\text{variance within the groups}}
= \frac{MS_{\text{between}}}{MS_{\text{within}}}
\]</div>

.pull-left[
.box-inv-1.small[**Top:** how far the group means sit apart. It holds any real difference, plus chance]
]

.pull-right[
.box-inv-1.small[**Bottom:** how far people sit from their own group's mean. It holds chance only]
]

.box-1.small[MS means mean square: ANOVA's word for a variance]

???

The bottom is the pooled variance from Topic 9, with three groups. It assumes one σ² across the populations, as the pooled t did.

Why chance only? Everyone in a group is treated the same way, so differences within a group can't come from the factor. Differences between groups could come from the factor, and they also carry chance: even identical populations give different sample means.

---

# Step 5: three populations

???

Step 5 on the blueprint. Same three panels. Here all three population means are 100, so H₀ is true, and the three curves sit on top of each other.

Panel 2 has three samples, with MS between, MS within and F written under them. Panel 3 records that one F.

---

# Predict it: No real differences

.box-inv-8.small[
All three population means are the same, so H₀ is true. Draw three samples, compute F, and repeat thousands of times. What does F average out to?
]

.options[
.option[
**A** About 0
]
.option[
**B** About 1
]
.option[
**C** About 3
]
.option[
**D** It depends on the size of σ
]
]

???

About 6 minutes, with the next two slides. Vote, turn to a neighbor, vote again, then show the answer.

A is the common answer: no differences, so zero. But sample means always differ a little.

Andy: create this poll in Poll Everywhere, then add the `.poll-bar` button from EDUC 265's 06-01 with its presenter id.

---

# Predict it: Answer

.options[
.option.wrong[
**A** About 0
]
.option.correct[
**B** About 1
]
.option.wrong[
**C** About 3
]
.option.wrong[
**D** It depends on the size of σ
]
]

.box-inv-8.small.sp-after-half[
With no real differences, the top holds only chance, just as the bottom does
]

.box-8.small[
So the top and bottom estimate the same thing, and their ratio averages about 1
]

---

---

# Under H₀, F averages about 1

???

Same world, 551 draws. Most F values fall between 0 and 2, and they average about 1. F can't go below 0, because variances can't be negative, so it has one long tail to the right. With three groups, the pile is tallest near 0. About 5% of samples land past the cutoff of 3.35. That's α.

---

# When means differ, the pile moves

???

One group's mean moves up to 115. Only one group differs, and that's enough for H₁. The share beyond the cutoff is now power.

Use the sample size slider here. To try other means, open the full simulator, because the group boxes hide on a slide.

---

# MS between grows when means differ

<table class="deck-table">
 <thead>
  <tr>
   <th style="text-align:left;"> Population means </th>
   <th style="text-align:center;"> MS within averages </th>
   <th style="text-align:center;"> MS between averages </th>
   <th style="text-align:center;"> F averages </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> 100, 100, 100 (H₀ is true) </td>
   <td style="text-align:center;"> 225 </td>
   <td style="text-align:center;"> 224 </td>
   <td style="text-align:center;"> 1.08 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 100, 100, 115 (H₁ is true) </td>
   <td style="text-align:center;"> 225 </td>
   <td style="text-align:center;"> 975 </td>
   <td style="text-align:center;"> 4.70 </td>
  </tr>
</tbody>
</table>

.box-inv-1.small.sp-after-half[Here σ² = 225. MS within estimates σ² in both worlds]

.box-inv-1.small.sp-after-half[MS between estimates σ² only when H₀ is true. If the means differ, it adds their real spread]

.box-1.small[So F near 1 fits H₀, and a large F is evidence against it]

???

Each row is 20,000 sets of three samples of 10, run in R. The true values are 225 for MS within, and 225 and then 975 for MS between. On the simulator's page, these are in the table "Estimates and the truth". That table is hidden on a slide, so it's copied here.

F averages a little above 1 even under H₀: exactly 27 / 25 = 1.08 here. With large samples, it's closer to 1.

---

# Splitting the variability

---

---

# The total splits into two parts

.box-6.medium.tree-top[Total variability]

.box-inv-6.small[Any real differences, plus chance]

.box-6.small[Goes on top of F]
]

.box-inv-6.small[Chance only]

.box-6.small[Goes on the bottom of F]
]

.source[After a figure in Gravetter et al., *Essentials of Statistics for the Behavioral Sciences*, 10e.]

???

Every score sits some distance from the overall mean. Part of that distance is its group's distance from the overall mean. The rest is its distance from its own group's mean. Three numbers to find: SS, df and MS. SS and df add up across the split. MS doesn't.

---

# First, each center's own spread

<table class="deck-table">
 <thead>
  <tr>
   <th style="text-align:left;"> Center </th>
   <th style="text-align:center;"> n </th>
   <th style="text-align:center;"> Sum of scores T </th>
   <th style="text-align:center;"> M </th>
   <th style="text-align:center;"> SS within the center </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Chhatarpur </td>
   <td style="text-align:center;"> 161 </td>
   <td style="text-align:center;"> 1,879 </td>
   <td style="text-align:center;"> 11.67 </td>
   <td style="text-align:center;"> 2,229.6 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Govindpuri </td>
   <td style="text-align:center;"> 82 </td>
   <td style="text-align:center;"> 951 </td>
   <td style="text-align:center;"> 11.60 </td>
   <td style="text-align:center;"> 2,039.7 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Tekhand </td>
   <td style="text-align:center;"> 376 </td>
   <td style="text-align:center;"> 4,054 </td>
   <td style="text-align:center;"> 10.78 </td>
   <td style="text-align:center;"> 5,040.1 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> All three </td>
   <td style="text-align:center;"> 619 </td>
   <td style="text-align:center;"> 6,884 </td>
   <td style="text-align:center;"> 11.12 </td>
   <td style="text-align:center;"> <strong>9,309.4</strong> </td>
  </tr>
</tbody>
</table>

.box-inv-6.small.sp-after-half[SS<sub>within</sub> adds up each group's SS: every score against its **own** group's mean]

.box-6.small[SS<sub>total</sub> treats everyone as one group, against the overall mean. Here it's 9,419.9]

???

Each SS is the one from Topic 3. In a spreadsheet, the computational formula from Topic 3 is the quick way.

The pooled variance assumes one σ². Govindpuri's own σ̂² is 25.18, against 13.93 in Chhatarpur and 13.44 in Tekhand, so that assumption is strained here. A version that drops the assumption, Welch's ANOVA, gives F(2, 188.0) = 3.68 and p = 0.027, so the decision doesn't change.

---

# Next, the means' own spread

<table class="deck-table">
 <thead>
  <tr>
   <th style="text-align:left;"> Center </th>
   <th style="text-align:center;"> n </th>
   <th style="text-align:center;"> M </th>
   <th style="text-align:center;"> M − overall mean </th>
   <th style="text-align:center;"> n × (M − overall mean)² </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Chhatarpur </td>
   <td style="text-align:center;"> 161 </td>
   <td style="text-align:center;"> 11.67 </td>
   <td style="text-align:center;"> 0.55 </td>
   <td style="text-align:center;"> 48.6 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Govindpuri </td>
   <td style="text-align:center;"> 82 </td>
   <td style="text-align:center;"> 11.60 </td>
   <td style="text-align:center;"> 0.48 </td>
   <td style="text-align:center;"> 18.6 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Tekhand </td>
   <td style="text-align:center;"> 376 </td>
   <td style="text-align:center;"> 10.78 </td>
   <td style="text-align:center;"> −0.34 </td>
   <td style="text-align:center;"> 43.3 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Add them up </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;"> <strong>110.5</strong> </td>
  </tr>
</tbody>
</table>

.box-inv-6.small.sp-after-half[Every student counts, so a bigger center gets more weight. The overall mean is 11.12]

.box-6.small[Check the split: 9,309.4 + 110.5 = 9,419.9. The parts add up to SS<sub>total</sub>]

???

The book's version, with our numbers: 1,879² / 161 + 951² / 82 + 4,054² / 376 − 6,884² / 619 = 110.52. It's easier on a calculator, while the table shows the idea.

Rounded means can drift by a little, so keep several decimals by hand. Another shortcut: SS between = SS total − SS within.

---

# df split the same way

<table class="deck-table">
 <thead>
  <tr>
   <th style="text-align:left;"> Source </th>
   <th style="text-align:center;"> df </th>
   <th style="text-align:center;"> Our numbers </th>
   <th style="text-align:left;"> Why </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Between </td>
   <td style="text-align:center;"> k − 1 </td>
   <td style="text-align:center;"> 3 − 1 = 2 </td>
   <td style="text-align:left;"> 3 means, and 1 df spent on the overall mean </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Within </td>
   <td style="text-align:center;"> N − k </td>
   <td style="text-align:center;"> 619 − 3 = 616 </td>
   <td style="text-align:left;"> Each group spends 1 df on its own mean </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Total </td>
   <td style="text-align:center;"> N − 1 </td>
   <td style="text-align:center;"> 619 − 1 = 618 </td>
   <td style="text-align:left;"> Everyone, around 1 overall mean </td>
  </tr>
</tbody>
</table>

.box-6.small[Check the split: 2 + 616 = 618. F has two df, and we report both]

???

Same logic as Topic 9, where two groups gave n₁ + n₂ − 2. Within is that same df, with three groups.

---

# A mean square is a variance

<div class="math-display">\[
MS_{\text{between}} = SS_{\text{between}} \,/\, df_{\text{between}} = 110.52 \,/\, 2 = 55.26
\]</div>

<div class="math-display">\[
MS_{\text{within}} = SS_{\text{within}} \,/\, df_{\text{within}} = 9309.39 \,/\, 616 = 15.11
\]</div>

<div class="math-display">\[
F = MS_{\text{between}} \,/\, MS_{\text{within}} = 55.26 \,/\, 15.11 = 3.66
\]</div>

.box-inv-6.small[MS<sub>within</sub> is the pooled variance from Topic 9, now with three groups]

???

SS / df is the recipe from Topic 3 for an estimate of a variance. It's the same recipe here, twice. Unlike SS and df, the two MS don't add up to a total, so a source table leaves that cell empty.

---

# The source table holds all of it

<table class="deck-table">
 <thead>
  <tr>
   <th style="text-align:left;"> Source </th>
   <th style="text-align:center;"> SS </th>
   <th style="text-align:center;"> df </th>
   <th style="text-align:center;"> MS </th>
   <th style="text-align:center;"> F </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Between centers </td>
   <td style="text-align:center;"> 110.52 </td>
   <td style="text-align:center;"> 2 </td>
   <td style="text-align:center;"> 55.26 </td>
   <td style="text-align:center;"> <strong>3.66</strong> </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Within centers </td>
   <td style="text-align:center;"> 9,309.39 </td>
   <td style="text-align:center;"> 616 </td>
   <td style="text-align:center;"> 15.11 </td>
   <td style="text-align:center;">  </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Total </td>
   <td style="text-align:center;"> 9,419.91 </td>
   <td style="text-align:center;"> 618 </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;">  </td>
  </tr>
</tbody>
</table>

.box-inv-6.small.sp-after-half[Read it left to right: SS / df = MS, then MS between / MS within = F]

.box-6.small[Software prints this same table. Now you can read it]

???

The book calls this an ANOVA summary table. Next up, students fill one in. Topic 14 returns to this table with regression, which gives the same F.

---

# Try it: Centers at endline

<table class="deck-table tight">
 <thead>
  <tr>
   <th style="text-align:left;"> Source </th>
   <th style="text-align:center;"> SS </th>
   <th style="text-align:center;"> df </th>
   <th style="text-align:center;"> MS </th>
   <th style="text-align:center;"> F </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Between centers </td>
   <td style="text-align:center;"> ? </td>
   <td style="text-align:center;"> ? </td>
   <td style="text-align:center;"> ? </td>
   <td style="text-align:center;"> ? </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Within centers </td>
   <td style="text-align:center;"> 18,357.4 </td>
   <td style="text-align:center;"> ? </td>
   <td style="text-align:center;"> ? </td>
   <td style="text-align:center;">  </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Total </td>
   <td style="text-align:center;"> 19,143.5 </td>
   <td style="text-align:center;"> ? </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;">  </td>
  </tr>
</tbody>
</table>

.box-inv-8.small.sp-after-half[3 centers, 539 students. The cutoff is F(2, 536) = 3.01. Fill in the table, then decide]

.box-8.small[Work alone for two minutes. Compare with a neighbor. Then we vote: reject or maintain?]

???

About 6 minutes, with calculators. Vote with hands on the decision. Common slips: using N for the total df, or dividing MS between by SS within.

---

# Try it: Answer

<table class="deck-table tight">
 <thead>
  <tr>
   <th style="text-align:left;"> Source </th>
   <th style="text-align:center;"> SS </th>
   <th style="text-align:center;"> df </th>
   <th style="text-align:center;"> MS </th>
   <th style="text-align:center;"> F </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Between centers </td>
   <td style="text-align:center;"> <strong>786.0</strong> </td>
   <td style="text-align:center;"> <strong>2</strong> </td>
   <td style="text-align:center;"> <strong>393.0</strong> </td>
   <td style="text-align:center;"> <strong>11.48</strong> </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Within centers </td>
   <td style="text-align:center;"> 18,357.4 </td>
   <td style="text-align:center;"> <strong>536</strong> </td>
   <td style="text-align:center;"> <strong>34.25</strong> </td>
   <td style="text-align:center;">  </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Total </td>
   <td style="text-align:center;"> 19,143.5 </td>
   <td style="text-align:center;"> <strong>538</strong> </td>
   <td style="text-align:center;">  </td>
   <td style="text-align:center;">  </td>
  </tr>
</tbody>
</table>

.box-inv-8.small.sp-after-half[11.48 is well past 3.01, so we reject H₀]

.box-8.small[At least one center's mean differs at endline. F doesn't say which one]

???

The between part is the total minus the within part. By hand, 393.0 / 34.25 gives 11.47. Unrounded, it's 11.48. The means are 19.11, 18.17 and 16.38 for Chhatarpur, Govindpuri and Tekhand.

Don't read this as the centers changing students. Each center's mix of Mindspark and control students isn't part of this table.

---

# The test, and what comes after

---

---

# Steps 1 and 2: set up the test

.box-3.small.sp-after-half[At baseline, do math scores differ across the three centers?]

<table class="deck-table">
 <thead>
  <tr>
   <th style="text-align:left;"> Step </th>
   <th style="text-align:left;"> In words </th>
   <th style="text-align:left;"> In symbols </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> 1. State H₀ </td>
   <td style="text-align:left;"> The three population means are the same </td>
   <td style="text-align:left;"> μ<sub>Chhatarpur</sub> = μ<sub>Govindpuri</sub> = μ<sub>Tekhand</sub> </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 2. Set the criteria </td>
   <td style="text-align:left;"> α = 0.05, and two df </td>
   <td style="text-align:left;"> df = 2 and 616 </td>
  </tr>
  <tr>
   <td style="text-align:left;">  </td>
   <td style="text-align:left;"> Reject H₀ if F is beyond the cutoff </td>
   <td style="text-align:left;"> F(2, 616) beyond 3.01 </td>
  </tr>
</tbody>
</table>

.box-inv-3.small[There's one cutoff. Differences in any direction make F bigger, so H₁ needs no direction]

???

The F table has df between across the top and df within down the side. If the exact df isn't in the table, use the next lower one. Each pair of dfs has its own curve, just as each df had its own t curve.

Don't call an F-test two-tailed. H₁ is non-directional, but we only look in the upper tail.

---

# Steps 3 and 4: F passes the cutoff

.box-3.small[3.66 is beyond 3.01, so we reject H₀. p = 0.026]

???

If the three population means were the same, an F this large would turn up about 2.6% of the time. So at least one center differs.

F can't go below zero, so there's no lower tail to worry about.

---

# How big? About 1.2% of the total

<div class="math-display">\[
\hat{\eta}^2 = \frac{SS_{\text{between}}}{SS_{\text{total}}} = \frac{110.5}{9419.9} = 0.012
\]</div>

.box-inv-3.small.sp-after-half[At baseline, math scores differed across the three centers, F(2, 616) = 3.66, p = 0.026, η̂² = 0.012]

.box-3.small[Real, but small. And no lottery made these groups, so it's a difference, not an effect]

???

Topic 8's notes promised this. η̂² is the share of the total variability that sits between the groups. With 619 students, a small difference can reach significance, as Topic 7 warned.

Greek means population, so this deck adds a hat to the book's η². The sample value is an estimate.

---

# With two groups, F is just t²

<table class="deck-table">
 <thead>
  <tr>
   <th style="text-align:left;"> Mindspark and control at endline </th>
   <th style="text-align:center;"> Topic 9: t-test </th>
   <th style="text-align:center;"> Today: ANOVA </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Test statistic </td>
   <td style="text-align:center;"> t(537) = 5.371 </td>
   <td style="text-align:center;"> F(1, 537) = 28.85 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Cutoff </td>
   <td style="text-align:center;"> ±1.964 </td>
   <td style="text-align:center;"> 3.859 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> p </td>
   <td style="text-align:center;"> Below 0.001 </td>
   <td style="text-align:center;"> Below 0.001 </td>
  </tr>
</tbody>
</table>

.box-inv-3.small.sp-after-half[5.371² = 28.85, and 1.964² = 3.859. Same test, same decision]

.box-3.small[Squaring folds t's two tails into F's one. ANOVA is the t-test, opened up to more groups]

???

This is the blueprint's point. t compares a gap with its standard error. F compares a squared gap, stretched to any number of groups, with a variance. Topic 14 shows that regression gives this same number too.

---

# F doesn't say which group differs

.box-inv-3.small.sp-after-half[With three or more groups, a significant F doesn't name the group]

.box-3.small.sp-after-half[So after a significant F, we compare pairs. These are called post hoc tests]

.box-inv-3.small[That's our t-tests again, so the false alarms are back. Post hoc tests set a stricter α]

???

Post hoc means "after this". They come after a significant F. If F isn't significant, we stop there.

There are many such tests. The book covers Tukey's and Scheffé's. We use Bonferroni's, because it's simple, and it shows the logic.

---

# Bonferroni: split α across tests

<div class="math-display">\[
\alpha_{\text{adjusted}} = \frac{\alpha}{\text{tests}} = \frac{0.05}{3} = 0.0167
\]</div>

.box-3.small[That's just under 0.05, as we wanted. Bonferroni plays it a little safe]

???

The 0.049 assumes independent tests, and these three aren't. Bonferroni keeps the overall rate at or under 0.05 either way.

Compare with 0.14 for three tests at 0.05.

Keep four decimals here. With 0.017 you'd get 0.0501, a shade over 0.05, but that's only rounding. The exact 0.05 / 3 gives 0.049.

A slightly less cautious correction, Šidák's, hits 0.05 exactly. It isn't needed in this course.

---

# Try it: Chhatarpur against Tekhand

.box-inv-8.small.sp-after-half[Chhatarpur: n = 161, M = 11.67. Tekhand: n = 376, M = 10.78. MS within = 15.11]

.box-8.small.sp-after-half[MS within is our pooled variance. Find the standard error of the gap, then t]

.box-inv-8.small.sp-after-half[With α = 0.0167, the cutoff is t(616) = ±2.40. Decide: reject or maintain?]

.box-8.small[Work alone, compare with a neighbor, then we vote]

???

About 6 minutes, with calculators. Vote with hands on the decision. This is Topic 9's t, with MS within in place of the pooled variance and df = 616. The standard error is √(MS within / n₁ + MS within / n₂). It's the biggest gap of the three.

---

# Try it: Answer

.box-inv-8.small.sp-after-half[σ̂<sub>M₁−M₂</sub> = √(15.11 / 161 + 15.11 / 376) = 0.366, so t = 0.89 / 0.366 = **2.43**: reject H₀]

<table class="deck-table tight">
 <thead>
  <tr>
   <th style="text-align:left;"> Pair </th>
   <th style="text-align:center;"> Gap </th>
   <th style="text-align:center;"> t </th>
   <th style="text-align:center;"> p </th>
   <th style="text-align:center;"> At α = 0.0167 </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Chhatarpur and Tekhand </td>
   <td style="text-align:center;"> 0.89 </td>
   <td style="text-align:center;"> 2.43 </td>
   <td style="text-align:center;"> 0.015 </td>
   <td style="text-align:center;"> <strong>Reject H₀</strong> </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Govindpuri and Tekhand </td>
   <td style="text-align:center;"> 0.82 </td>
   <td style="text-align:center;"> 1.72 </td>
   <td style="text-align:center;"> 0.086 </td>
   <td style="text-align:center;"> Maintain H₀ </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Chhatarpur and Govindpuri </td>
   <td style="text-align:center;"> 0.07 </td>
   <td style="text-align:center;"> 0.14 </td>
   <td style="text-align:center;"> 0.890 </td>
   <td style="text-align:center;"> Maintain H₀ </td>
  </tr>
</tbody>
</table>

.box-8.small[Only one pair clears the stricter bar. Tukey's test agrees: p = 0.041]

???

A close call: 2.43 against 2.40. Tukey's test is the other common choice, and software runs it. Its p already carries its adjustment, so it's compared with 0.05. The p of 0.015 is just under 0.0167. Maintaining H₀ for the other pairs doesn't show they're equal. Govindpuri has only 82 students.

Post hoc tests can disagree on a close call. The 2024 deck used Scheffé's test on this pair, and it maintained H₀, with F = 2.95 against 3.01. Scheffé's is more cautious than Bonferroni's or Tukey's.

---

# The blueprint so far

<table class="deck-table tight">
 <thead>
  <tr>
   <th style="text-align:left;"> Blueprint step </th>
   <th style="text-align:center;"> Statistic </th>
   <th style="text-align:center;"> Bottom of the fraction </th>
   <th style="text-align:center;"> df </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> 1. One score </td>
   <td style="text-align:center;"> z </td>
   <td style="text-align:center;"> σ </td>
   <td style="text-align:center;"> None </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 2. One mean, σ known </td>
   <td style="text-align:center;"> z </td>
   <td style="text-align:center;"> σ<sub>M</sub> </td>
   <td style="text-align:center;"> None </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 3a. One mean, σ unknown </td>
   <td style="text-align:center;"> t </td>
   <td style="text-align:center;"> σ̂<sub>M</sub> </td>
   <td style="text-align:center;"> n − 1 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 3b. Mean gain </td>
   <td style="text-align:center;"> t </td>
   <td style="text-align:center;"> σ̂<sub>M</sub> of the gains </td>
   <td style="text-align:center;"> n − 1 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> 4. Two means </td>
   <td style="text-align:center;"> t </td>
   <td style="text-align:center;"> σ̂<sub>M₁−M₂</sub> </td>
   <td style="text-align:center;"> n₁ + n₂ − 2 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> <strong>5. Two or more means</strong> </td>
   <td style="text-align:center;"> <strong>F</strong> </td>
   <td style="text-align:center;"> <strong>MS<sub>within</sub></strong> </td>
   <td style="text-align:center;"> <strong>k − 1 and N − k</strong> </td>
  </tr>
</tbody>
</table>

.box-8.small[Same idea every time: what we saw, over what chance alone would give]

???

Step 5 still has a fraction. The top became a variance of means, and the bottom became a variance of people. Steps 6 and 7 are next: relationships between two variables.

---

# What you can do now

.box-inv-8.small[Say why many t-tests inflate the risk of a Type 1 error]

.box-inv-8.small[Explain F as a ratio of two variances, and why it averages about 1 under H₀]

.box-inv-8.small[Split SS and df, then build a source table]

.box-inv-8.small[Run the four steps for an F-test, and report it with η̂²]

.box-inv-8.small[Use a Bonferroni correction to find which groups differ]

???

Lab 8 on Friday starts with a 15-minute individual check on two samples, then practises gains and today's ANOVA in a spreadsheet. Chapter 12's Mastery and Problem Set are due Sunday, with Chapter 11's.

---

# Next steps

A 15-minute individual check on two samples, then gains and ANOVA. Bring a calculator and a laptop
]

Correlation. Read sections 14-1 to 14-4, and skim 14-5
]

.box-3.small[Sunday, February 28, 6 PM: Chapters 11 and 12 are due in MindTap]

.picture-small img {
  display: block;
  margin: 0 auto;
  max-width: 100%;
  max-height: 230px;
}

.source a {
  color: inherit;
  text-decoration: underline;
}

.anteater img {
  display: block;
  margin: 0.15em auto 0;
  max-height: 105px;
}

/* This chunk sits at the end because any output before the first slide's
   "class:" line stops remark from reading that slide's properties */

/* Long titles get two smaller lines inside the 110 px title bar */
.remark-slide-content.long-title h1 {
  font-size: 1.05em;
  line-height: 1.05;
  margin-top: -100px;
  margin-bottom: 55px;
}

.source {
  position: absolute;
  bottom: 18px;
  left: 60px;
  font-size: 0.55em;
  color: #47505a;
}

.book-tag {
  margin-top: 0.7em;
  font-size: 0.65em;
  color: #47505a;
  text-align: right;
}

table.deck-table {
  width: 92%;
  margin: 0.4em auto;
  border-collapse: collapse;
  font-size: 0.72em;
}

table.deck-table.tight td,
table.deck-table.tight th {
  padding: 0.12em 0.6em;
}

table.deck-table th {
  border-bottom: 2px solid #244a70;
  padding: 0.3em 0.6em;
}

table.deck-table td {
  border-bottom: 1px solid #c9d1d9;
  padding: 0.3em 0.6em;
}

.options {
  display: grid;
  grid-template-columns: 1fr 1fr;
  gap: 0.35em 1em;
  margin-top: 0.6em;
  margin-bottom: 0.8em;
  font-size: 0.65em;
}

.option {
  background-color: #f3f3f3;
  border: 2px solid #cccccc;
  padding: 0.35em 0.6em;
}

.option.correct {
  background-color: #b18323;
  border-color: #b18323;
  color: white;
}

.option.wrong {
  opacity: 0.6;
}

.poll-respond {
  display: block;
  font-size: 0.5em;
  text-align: center;
  margin-top: 0.1em;
}

/* The tree of variability: one box over two, as in the book's figure */
.tree-top {
  width: 42%;
  margin: 0 auto;
}

.tree-arrows {
  display: flex;
  justify-content: space-around;
  width: 60%;
  margin: 0 auto;
  font-size: 1.1em;
  line-height: 1.1;
  color: #47505a;
}

/* The printed PDF keeps the question and drops the answer slide */
@media print {
  .remark-slide-container:has(.poll-answer) {
    display: none !important;
  }
}
</style>