Because all adults had the same chance to be among the chosen 2527, it seems reasonable to use the statistic ^p = 0.51 as an estimate of the
unknown parameter p. It’s a fact that 51% of the sample favored an amendment—we know because we asked them. We don’t know what per- centage of all adults favor an amendment, but we estimate that about 51% do.
Sampling variability
If Gallup took a second random sample of 2527 adults, the new sample would have different people in it. It is almost certain that there would not be exactly 1289 favorable responses. That is, the value of the statistic ^p will vary from sample to sample. Could it happen that one random sample finds
that 51% of adults favor an amendment and a second random sample finds that only 37% favor one? Random samples eliminate bias from the act of choosing a sample, but they can still be wrong because of the variability that results when we choose at random. If the variation when we take repeated samples from the same population is too great, we can’t trust the results of any one sample.
We are saved by the second great advantage of random samples. The first advantage is that choosing at random eliminates favoritism. That is, random sampling attacks bias. The second advantage is that if we took lots of random samples of the same size from the same population, the variation from sample to sample would follow a predictable pattern. This predictable pattern shows that results of bigger samples are less variable than the re- sults of smaller samples.
EXAMPLE 2
Lots and lots of samplesHere’s another big idea of statistics: to see how trustworthy one sample is likely to be, ask what would happen if we took many samples from the same population. Let’s try it and see. Suppose that in fact (unknown to Gallup), exactly 50% of all adults favor a constitutional amendment that would define marriage as being between a man and a woman. That is, the truth about the population is that p = 0.5. What if Gallup used the sample proportion ^p from an SRS of size 100 to estimate the unknown
value of the population proportion p?
Figure 3.1 illustrates the process of choosing many samples and find- ing ^p for each one. In the first sample, 56 of the 100 people favored the
0.35 0.40 0.45 0.50 0.55 0.60 0.65 0 50 200 150 100 Values of p Number of samples p = 0.56 p = 0.36 p = 0.61 SRS n = 100 SRS n = 100 SRS n = 100 p = 0.50
Figure 3.1 The results of many SRSs have a regular pattern. Here, we draw 1000 SRSs of size 100 from the same population. The population proportion isp= 0.5. The sample proportions vary from sample to sample, but their values center at the truth about the population.
0.35 0.40 0.45 0.50 0.55 0.60 0.65 Values of p p = 0.509 p = 0.525 p = 0.479 SRS n = 2527 SRS n = 2527 SRS n = 2527 0 100 400 500 300 200 Number of samples p = 0.50
Figure 3.2Draw 1000 SRSs of size 2527 from the same population as in Figure 3.1. The 1000 values of the sample proportion are much less spread out than was the case for smaller samples.
amendment, so ^p = 56/100 = 0.56. Only 36 in the next sample favored
the amendment, so for that sample ^p= 0.36. Choose 1000 samples and
make a plot of the 1000 values of ^p like the graph (called a histogram) at
the right of Figure 3.1. The different values of the sample proportion ^p
run along the horizontal axis. The height of each bar shows how many of our 1000 samples gave the group of values on the horizontal axis covered by the bar. For example, in Figure 3.1 the bar covering the values between 0.40 and 0.42 has a height of over 50. Thus, over 50 of our 1000 samples had values between 0.40 and 0.42.
Of course, Gallup interviewed 2527 people, not just 100. Figure 3.2 shows the results of 1000 SRSs, each of size 2527, drawn from a popula- tion in which the true sample proportion is p= 0.5. Figures 3.1 and 3.2
Sampling variability 39
are drawn on the same scale. Comparing them shows what happens when we increase the size of our samples from 100 to 2527.
Look carefully at Figures 3.1 and 3.2. We flow from the population, to many samples from the population, to the many values of ^p from these
many samples. Gather these values together and study the histograms that display them.
• In both cases, the values of the sample proportion ^p vary from sample
to sample, but the values are centered at 0.5. Recall that p= 0.5 is the true population parameter. Some samples have a ^p less than 0.5 and
some greater, but there is no tendency to be always low or always high. That is, ^p has no bias as an estimator of p. This is true for both large
and small samples.
• The values of ^p from samples of size 100 are much more spread out
than the values from samples of size 2527. In fact, 95% of our 1000 samples of size 2527 have a ^p lying between 0.4804 and 0.5204. That’s
within 0.0204 on either side of the population truth 0.5. Our samples of size 100, on the other hand, spread the middle 95% of their values between 0.40 and 0.59. That goes out 0.1 from the truth, about five times as far as the larger samples. So larger random samples have less
variability than smaller samples.
The result is that we can rely on a sample of size 2527 to almost always give an estimate ^p that is close to the truth about the population. Figure 3.2
illustrates this fact for just one value of the population proportion, but it is true for any population proportion. Samples of size 100, on the other hand, might give an estimate of 40% or 60% when the truth is 50%.
Thinking about Figures 3.1 and 3.2 helps us restate the idea of bias when we use a statistic like ^p to estimate a parameter like p. It also reminds
us that variability matters as much as bias.