Week 9: Inference for Many Means (ANOVA)

Welcome!

In this week’s coursework we are transitioning into our last topic of the quarter—comparing multiple means. These comparisons have a specific name, ANalysis of VAriance (ANOVA).

We are going to use all of the simulation-based methods we learned previously for this new context. An ANOVA relies on a new statistic, the \(F\)-statistic, to summarize how different 3 or more means are from each other.

The primary focus of an ANOVA is to detect if the means of 3 or more groups are different. Because of this we focus on hypothesis tests for a difference in the group means and do not use confidence intervals. We will generate permutation distributions of \(F\)-statistics that we could have observed if the null hypothesis was true (there is no difference in the group means).

0.1 Learning Outcomes

By the end of this coursework you should be able to:

  • describe what an ANOVA tests for
  • use a visualization to outline how the following are calculated:
    • total sum of squares
    • group sum of squares
    • residual sum of squares
  • describe why the mean squares of groups is called the “between group variability”
  • describe why the mean square error is called the “within group variability”
  • outline how an F-statistic is calculated
  • explain what a “large” or a “small” F-statistic indicates
  • describe the conditions for performing an ANOVA procedure
  • outline when it is appropriate to use an \(F\)-distribution in an ANOVA
  • explain the similarities and differences between parametric (\(F\)-based) methods and non-parametric (simulation-based) methods
  • use R to:
    • generate a permutation distribution for F-statistics
    • visualize the permutation distribution
    • calculate the observed F-statistic statistic
    • calculate a p-value for a hypothesis test

1 Textbook Reading

First, we are going to learn about one-way ANOVA!

Required Reading: Inference for Comparing Many Means

Reading Guide – Due Tuesday by the start of class (Monday is a holiday!)

Download the Word Document

Next, we’re going to do a refresher on experimental design.

Required Reading: Principles of Experimental Design

Important

There is no reading guide for this chapter, just a concept quiz that is due on Wednesday!

2 Concept Quizzes

We have two concept quizzes this week, one that is due by Tuesday (on one-way ANOVA) and one that is due by Wednesday (on experimental design).

2.1 Due Tuesday by the start of class

  1. Which of the following are true about the mean squares between groups? (Select all that apply)
  • it is a standardized measure of the variability in responses between groups
  • it compares the mean of each group to the overall mean across all groups
  • it compares the observations within each group to the mean of that group
  • it is used as the numerator in an F-statistic
  • it is used as the denominator in an F-statistic
  • it is found by dividing the sum of squares between groups by the number of groups minus 1 (\(k\) - 1)
  • it is found by dividing the sum of squares between groups by the sample size minus the number of groups (\(n - k\))
  1. Which of the following are true about the mean square errors? (Select all that apply)
  • it is a standardized measure of the variability in responses within each group
  • it compares the mean of each group to the overall mean across all groups
  • it compares the observations within each group to the mean of that group
  • it is used as the numerator in an F-statistic
  • it is used as the denominator in an F-statistic
  • it is found by dividing the sum of square errors by the number of groups minus 1 (\(k\) - 1)
  • it is found by dividing the sum of square errors by the sample size minus the number of groups (\(n - k\))
  1. An F-statistic uses which formula?

  • \(\frac{MSG}{MSE}\)

  • \(\frac{SSG}{SSE}\)

  • \(\frac{MSE}{MSG}\)

  • \(\frac{SSE}{SSG}\)

  1. Ideally, in an ANOVA we’d like to see… (select all that apply)
  • large differences in the means of the groups
  • small differences in the means of the groups
  • large variability in the observations within each group
  • small variability in the observations within each group
  1. If the null hypothesis that the means of four groups are all the same is rejected using ANOVA at a 5% significance level, then… (Select all that apply)
  • we can then conclude that all the means are different from one another.
  • the variability between groups is higher than the variability within groups.
  • we can then conclude that at least one mean is different from the others.
  • the pairwise analysis will identify at least one pair of means that are significantly different.

  1. True or false, as the total sample size increases, the degrees of freedom for the residuals increases.

  2. True or false, the constant variance condition is very important when the sample sizes of each group are different.

  3. True or false, the independence assumption can be relaxed when the total sample size is large.

  4. True or false, the normality condition is very important when the sample sizes of each group are small.

2.2 Due Wednesday by the start of class

1. Which of the following statements are true about observational studies and experiments? (Select all that apply)

  • Experiments randomly assign the explanatory variable
  • Observational studies randomly assign the explanatory variable
  • Observational studies can make causal statements about the relationship between the explanatory and response variables
  • Experiments can make causal statements about the relationship between the explanatory and response variables

2. Suppose we wanted to compare cattle weight gain on four different dietary supplements to determine which is the most effective. We conducted an experiment in which groups of eight cows are given a particular supplement for one month. A fifth group serves as the control group—they do not receive any supplements. At the end of the experiment, we measure how much weight each cow has gained over the month.

Experimental Units

Treatments

Control Group

Factor Levels

Eight Cows

Cows

No Supplement

Four Supplement Types & No Supplement

Dietary Supplements

3. Below are two brief descriptions of experimental designs, one represents replication, and the other represents pseudoreplication. Match each description to the correct design.

Pseudoreplication


Replication

A researcher applies a fertilizer treatment to five separate pots, each containing a different plant, and compare their growth to five control pots without fertilizer.

A researcher applies a fertilizer treatment to one large pot containing five plants and compares their growth to five plants in a single control pot without fertilizer.

4. Randomization means that every experimental unit has equal probability of being assigned to a treatment.

  • True
  • False

5. A good variable to include as a blocking variable would have [small / large] variation within a block, and [small / large] variation between blocks.

6. The table below summarizes the number of channel types (unittype) in each section of the HJ Andrews Experimental Forest. Based on this table, it would be reasonable to block on the section of the forest (CC, OG).

unittype CC OG
C 10199 8917
P 3339 4076
R 97 395
S 11 1
SC 2461 1925
I 0 23
IP 0 155

7. The use of a placebo pill in an experiment does which of the following? (Select all that apply)

  • Acts as a procedural control
  • Acts as a blocking variable
  • Allows researchers to measure the changes in patients resulting from simply being treated.
  • Fools the participants so they don’t know what treatment they received.
  • Gives researchers a baseline “no effect” comparison.

2.3 R Tutorial – Due Wednesday by the start of class

Required Tutorial: Comparing many means with ANOVA