Sowmya Anand University of Illinois Mindy Anderson-Knott
University of Nebraska–Lincoln H. Öztas Ayhan
Middle East Technical University Janice Ballou
Mathematica Policy Research Badi H. Baltagi
Syracuse University Laura Barberena
University of Texas at Austin Kirsten Barrett
Mathematica Policy Research Allen H. Barton
University of North Carolina at Chapel Hill
Danna Basson
Mathematica Policy Research Michael P. Battaglia
Abt Associates, Inc.
Joseph E. Bauer
American Cancer Society Christopher W. Bauman Northwestern University Sandra L. Bauman Bauman Research
René Bautista
University of Nebraska–Lincoln Patricia C. Becker
APB Associates, Inc.
Robert F. Belli
University of Nebraska–Lincoln Mildred A. Bennett
The Nielsen Company Pazit Ben-Nun
State University of New York, Stony Brook University Sandra H. Berry
RAND
Marcus Berzofsky RTI International Jonathan Best
Princeton Survey Research International
Jelke Bethlehem Statistics Netherlands Matthew Beverlin University of Kansas David A. Binder Statistics Canada George F. Bishop University of Cincinnati Steven Blixt
Bank of America
Joel David Bloom
State University of New York, Albany
Stephen J. Blumberg
Centers for Disease Control and Prevention
Georgiy Bobashev RTI International Shelley Boulianne
University of Wisconsin–Madison Ashley Bowers
University of Michigan Diane Bowers
Council for American Survey Research Organization Heather H. Boyd
University of Wisconsin–
Extension Luc Boyer
University of Waterloo J. Michael Brick Westat
Pat Dean Brick Westat
Jonathan E. Brill
University of Medicine and Dentistry of New Jersey Kimberly Diane Brown The Nielsen Company xxviii
Contributors
Contributors———xxix
Trent D. Buskirk Saint Louis University Sarah Butler
National Economic Research Associates
Mario Callegaro Knowledge Networks Pamela Campanelli The Survey Coach Patrick J. Cantwell U.S. Census Bureau Xiaoxia Cao
University of Pennsylvania Lisa Carley-Baxter
RTI International Woody Carter
University of Chicago Barbara L. Carvalho Marist College Rachel Ann Caspar RTI International Jamie Patrick Chandler City University of New York Haiying Chen
Wake Forest University Young Ik Cho
University of Illinois at Chicago Leah Melani Christian
Pew Research Center James R. Chromy RTI International M. H. Clark
Southern Illinois University–
Carbondale
Kathryn A. Cochran University of Kansas Jon Cohen
The Washington Post Michael P. Cohen Bureau of Transportation
Statistics Marjorie Connelly The New York Times Matthew Courser
Pacific Institute for Research and Evaluation
Brenda G. Cox Battelle
Douglas B. Currivan RTI International Richard T. Curtin University of Michigan Gauri Sankar Datta University of Georgia Michael Edward Davern University of Minnesota Robert P. Daves
Daves & Associates Research Bonnie D. Davis
Public Health Institute, Survey Research Group
Karen E. Davis
National Center for Health Statistics
Matthew DeBell Stanford University Femke De Keulenaer Gallup Europe Edith D. de Leeuw Methodika
David DesRoches
Mathematica Policy Research, Inc.
Dennis Dew
Loyola University Chicago Isaac Dialsingh
Pennsylvania State University Lillian Diaz-Hoffmann Westat
Bryce J. Dietrich University of Kansas Wil Dijkstra
Free University, Amsterdam Don A. Dillman
Washington State University Charles DiSogra
Knowledge Networks Sylvia Dohrmann Westat
Wolfgang Donsbach
Technische Universität Dresden Katherine A. Draughon
Draughon Research, LLC Arthur Lance Dryver
National Institute of Development Administration
Natalie E. Dupree
National Center for Health Statistics
Jennifer Dykema University of Wisconsin Asia A. Eaton
University of Chicago Murray Edelman Rutgers University
xxx———Encyclopedia of Survey Research Methods
Mansour Fahimi
Marketing Systems Group Moshe Feder
RTI International Karl G. Feld D3 Systems, Inc.
Howard Fienberg CMOR
Agnieszka Flizik BioVid
Amy Flowers Market Decisions E. Michael Foster
University of North Carolina at Chapel Hill
Kelly N. Foster University of Georgia Paul Freedman University of Virginia Marek Fuchs
University of Kassel Siegfried Gabler Universität Mannheim Matthias Ganninger Gesis-ZUMA Cecilie Gaziano
Research Solutions, Inc.
Jane F. Gentleman
National Center for Health Statistics
Amy R. Gershkoff Princeton University Malay Ghosh
University of Florida
Ryan Gibb
University of Kansas Homero Gil de Zuniga University of Texas at Austin Jason E. Gillikin
QSEC Consulting Group, LLC Lisa M. Gilman
University of Delaware Patrick Glaser
CMOR
Carroll J. Glynn Ohio State University John Goyder
University of Waterloo Ingrid Graf
University of Illinois at Chicago Eric A. Greenleaf
New York University Thomas M. Guterbock University of Virginia Erinn M. Hade Ohio State University Sabine Häder
Gesis-ZUMA John Hall
Mathematica Policy Research Janet Harkness
University of Nebraska–Lincoln Chase H. Harrison
Harvard University Rachel Harter
University of Chicago Douglas D. Heckathorn Cornell University
Dirk Heerwegh
Katholieke Universiteit Leuven Sean O. Hogan
RTI International Allyson Holbrook
University of Illinois at Chicago Gregory G. Holyk
University of Illinois at Chicago Adriaan W. Hoogendoorn Vrije Universiteit, Amsterdam Lew Horner
Ohio State University Joop Hox
Utrecht University Michael Huge Ohio State University Larry Hugick
Princeton Survey Research International
Li-Ching Hung
Mississippi State University Ronaldo Iachan
Macro International Susan S. Jack
National Center for Health Statistics
Annette Jäckle University of Essex Matthew Jans
University of Michigan Sharon E. Jarvis
University of Texas at Austin Guillermina Jasso
New York University
E. Deborah Jay
Field Research Corporation Timothy Johnson
University of Illinois at Chicago David Ross Judkins
Westat
Karen Long Jusko University of Michigan Sema A. Kalaian
Eastern Michigan University William D. Kalsbeek
University of North Carolina Rafa M. Kasim
Kent State University Randall Keesling RTI International Scott Keeter
Pew Research Center Jenny Kelly
NORC at the University of Chicago
Courtney Kennedy University of Michigan John M. Kennedy Indiana University Timothy Kennel U.S. Census Bureau Kate Kenski
University of Arizona SunWoong Kim Dongguk University Irene Klugkist Utrecht University Thomas R. Knapp
University of Rochester and Ohio State University
James R. Knaub, Jr.
U.S. Department of Energy Gerald M. Kosicki
Ohio State University Phillip S. Kott USDA/NASS John Kovar Statistics Canada Tom Krenzke Westat
Frauke Kreuter
University of Maryland Parvati Krishnamurty NORC at the University of
Chicago Karol Krotki RTI International Dale W. Kulp
Marketing Systems Group Richard Kwok
RTI International Jennie W. Lai
The Nielsen Company Dennis Lambries
University of South Carolina Gary Langer
ABC News
Michael D. Larsen Iowa State University Paul J. Lavrakas
Independent Consultant and Former Chief Research Methodologist for The Nielsen Company
Geon Lee
University of Illinois at Chicago
Hyunshik Lee Westat Sunghee Lee
University of California, Los Angeles
Jason C. Legg Iowa State University Stanley Lemeshow Ohio State University Gerty Lensvelt-Mulders Universiteit Utrecht James M. Lepkowski University of Michigan Tim F. Liao
University of Illinois at Urbana-Champaign
Michael W. Link The Nielsen Company Jani S. Little
University of Colorado Cong Liu
Hofstra University Kamala London University of Toledo Geert Loosveldt
Katholieke Universiteit Leuven Mary E. Losch
University of Northern Iowa Thomas Lumley
University of Washington Lars Lyberg
Statistics Sweden Tina Mainieri
Survey Sciences Group, LLC Contributors———xxxi
Aaron Keith Maitland University of Maryland Donald J. Malec U.S. Census Bureau Allan L. McCutcheon
University of Nebraska–Lincoln Daniel G. McDonald
Ohio State University John P. McIver
University of Colorado Douglas M. McLeod
University of Wisconsin–Madison Daniel M. Merkle
ABC News Philip Meyer
University of North Carolina at Chapel Hill
Peter V. Miller
Northwestern University Lee M. Miringoff Marist College Michael Mokrzycki Associated Press J. Quin Monson
Brigham Young University Jill M. Montaquila
Westat
Christopher Z. Mooney University of Illinois at
Springfield
Geraldine M. Mooney Mathematica Policy Research Danna L. Moore
Washington State University
David W. Moore
University of New Hampshire Jeffrey C. Moore
U.S. Census Bureau Richard Morin Pew Research Center Patricia Moy
University of Washington Mary H. Mulry
U.S. Census Bureau Ralf Münnich University of Trier Joe Murphy RTI International Gad Nathan
Hebrew University of Jerusalem Shannon C. Nelson
University of Illinois at Chicago
Thomas E. Nelson Ohio State University Traci Lynne Nelson University of Pittsburgh William L. Nicholls
U.S. Census Bureau (Retired) Matthew C. Nisbet
American University Andrew Noymer
University of California, Irvine Barbara C. O’Hare
Arbitron, Inc.
Robert W. Oldendick
University of South Carolina Randall Olsen
Ohio State University
Kristen Olson
University of Michigan Diane O’Rourke University of Illinois at
Chicago Larry Osborn Abt Associates, Inc.
Ronald E. Ostman Cornell University Mary Outwater
University of Oklahoma Linda Owens
University of Illinois at Urbana-Champaign Michael Parkin Oberlin College Jennifer A. Parsons University of Illinois at
Chicago
Jeffrey M. Pearson University of Michigan Steven Pedlow
NORC at the University of Chicago
Chao-Ying Joanne Peng Indiana University Andy Peytchev RTI International Linda Piekarski
Survey Sampling International Christine Guyer Pierce
The Nielsen Company Kathy Pilhuj
Scarborough Research Stephen R. Porter Iowa State University xxxii———Encyclopedia of Survey Research Methods
Frank Potter
Mathematica Policy Research Kevin B. Raines
Corona Research, Inc.
Susanne Rässler
Otto-Friedrich-University Bamberg Bryce B. Reeve
National Cancer Institute Lance J. Rips
Northwestern University José Elías Rodríguez Universidad de Guanajuato David James Roe
Survey Sciences Group Jennifer M. Rothgeb U.S. Census Bureau Donald B. Rubin Harvard University Tamás Rudas
Eotvos Lorand University Pedro Saavedra
ORC Macro Adam Safir RTI International Joseph W. Sakshaug University of Michigan Charles T. Salmon
Michigan State University Carla R. Scanlan
Independent Researcher Fritz Scheuren
NORC at the University of Chicago Michael F. Schober
New School for Social Research
Matthias Schonlau RAND
Paul Schroeder Abt SRBI Tricia Seifert University of Iowa William R. Shadish
University of California at Merced Dhavan V. Shah
University of Wisconsin–Madison Jacob Shamir
Hebrew University of Jerusalem Gary M. Shapiro
Westat
Joel K. Shapiro Rockman et al.
Carol Sheets Indiana University Sarah Shelton
Saint Louis University Charles D. Shuttles The Nielsen Company Samuel Shye
Hebrew University of Jerusalem Richard Sigman
Westat
Carlos Nunes Silva University of Lisbon N. Clayton Silver
University of Nevada, Las Vegas Jody Smarr
The Nielsen Company Cary Stacy Smith
Mississippi State University
Tom W. Smith
NORC at the University of Chicago Jolene D. Smyth
University of Nebraska–Lincoln Elizabeth A. Stasny
Ohio State University Jeffery A. Stec CRA International David Steel
University of Wollongong Sonya K. Sterba
University of North Carolina at Chapel Hill
Kenneth W. Steve Abt SRBI
John Stevenson
University of Wisconsin James W. Stoutenborough University of Kansas John Tarnai
Washington State University Charles Tien
City University of New York, Hunter College
Lois E. Timms-Ferrara University of Connecticut Trevor N. Tompson Associated Press Jeff Toor
San Diego State University Roger Tourangeau
University of Maryland Michael W. Traugott University of Michigan
Contributors———xxxiii
Alberto Trobia University of Palermo Norm Trussell
The Nielsen Company Clyde Tucker
U.S. Bureau of Labor Statistics Geoffrey R. Urland
Corona Research Akhil K. Vaish RTI International Melissa A. Valerio University of Michigan Wendy Van de Kerckhove Westat
Patrick Vargas
University of Illinois at Urbana-Champaign Timothy Vercellotti Rutgers University Ana Villar
University of Nebraska–Lincoln Penny Sue Visser
University of Chicago
Herbert F. Weisberg Ohio State University Eric White
University of Wisconsin Rand R. Wilcox
University of Southern California
Rick L. Williams RTI International
Gordon B. Willis
National Cancer Institute Michael B. Witt
RTI International Jonathan Wivagg PTV DataSource James Wolf
Indiana University at Indianapolis Shapard Wolf
Arizona State University
Douglas A. Wolfe Ohio State University Daniel B. Wright University of Sussex Changbao Wu
University of Waterloo Ting Yan
NORC at the University of Chicago
Y. Michael Yang University of Chicago Elaine L. Zanutto
National Analysts Worldwide Elizabeth R. Zell
Centers for Disease Control and Prevention
Weiyu Zhang
University of Pennsylvania Sonja Ziniel
University of Michigan Mary B. Ziskin
Indiana University xxxiv———Encyclopedia of Survey Research Methods
Survey research is a systematic set of methods used to gather information to generate knowledge and to help make decisions. By the second half of the 20th century, surveys were being used routinely by governments, businesses, academics, politicians, the news media, those in public health professions, and numerous other decision makers. It is not an exaggeration to state that accurate surveys have become a necessary condition for the efficient functioning of modern-day societies, and thus for our individual well-being.
Although there is a rich and expanding body of lit-erature that has been produced mostly in the past half century about the myriad methods that are used by survey researchers, heretofore there has not been a compendium with information about each of those methods to which interested parties could turn, espe-cially those new to the field of survey research. Thus, the purpose of the Encyclopedia of Survey Research Methods (ESRM) is to fill that gap by providing detailed (although not exhaustive) information about each of the many methods that survey methodologists and survey statisticians deploy in order to conduct reliable and valid surveys.
The Role of Methods and Statistics in the Field of Survey Research A survey is often contrasted to a census, and the two use many of the same methods. However, whereas a census is intended to gather information about all members of a population of interest, a survey gathers information from only some of the population mem-bers, that is, from a sample of the population. Because a survey is more limited in how much information it gathers compared to a census with a comparable scope of variables needing to be measured, a survey is less costly than a census and often is more accurate
and timelier. Due to its smaller scope, it is easy to understand why a survey is less costly and timelier than a census, but it may surprise some to learn that a survey can be more accurate than a census. That is the case because a census often is a daunting enterprise that cannot be conducted accurately across an entire population. At far less cost than a census, a survey can sample a representative subset of the population, gain a very high response rate, gather data on the same variables a census measures, and do so much more quickly than a census. Thus, given the finite resources available for information gathering, survey researchers often can allocate those resources much more effec-tively and achieve more accurate results than those conducting a census on the same topic.
There are two primary defining characteristics of a survey. One is that a sample is taken from the popula-tion and the other is that a systematic instrument—
most often a structured questionnaire—is used to gather data from each sampled member of, or unit in, the population.
However, the general methods of “surveying” are used in many ways other than their well-recognized manifestations in survey research. At the broadest level, humans are always “sampling” the physical and social environments in which they live, “gathering” informa-tion in mostly unstructured ways, and “analyzing” the information to reach decisions, albeit often imperfectly.
And although survey research is considered a quantita-tive approach for gathering information, “surveying” is routinely performed by qualitative researchers, even if many may not think of themselves as using survey methods. That is, qualitative research “samples” some members from a population of interest so as to gather information from or about them. This includes qualita-tive research that uses content analysis, focus groups, observational methods, ethnographic methods, and other quasi-scientific information-gathering approaches.
xxxv
Introduction
Whether the samples drawn for qualitative research are representative, and whether the information-gathering means are reliable, is not the primary issue here.
Instead, the issue is that qualitative research relies on
“survey methods” even if many who practice it have had no rigorous training in those methods. Also, there are many fields of inquiry in the behavioral sciences that utilize survey methods even if they do not recog-nize or acknowledge that is what is being done. For example, many psychologists draw samples and use questionnaires to gather data for their studies, even if they do not think of themselves as survey researchers or have not had rigorous training in survey methods. The same holds for many political scientists, economists, sociologists, criminologists, and other social scientists, as well as many public health researchers.
Accuracy Versus Error in Survey Research
The goal of a good survey is to utilize available resources so as to gather the most accurate informa-tion possible. No survey researcher should (or can) claim that a survey is entirely without error, that is, that it is perfectly accurate or valid. Instead, what sur-vey researchers realistically can strive for is to gather as accurate information as possible with available resources—information that has the smallest amount of “total survey error.” Ideally this will result in an amount of error that is “negligible,” that is, ignorable, for the decision-making purposes that the survey is to serve. For example, the senior executives of a corpo-ration do not need to know exactly what proportion of the population is likely to purchase their new product.
Rather, they can make a confident decision about whether to proceed with introducing the product on the basis of survey estimates that are accurate within a tolerable (negligible) level of “error.”
Broadly speaking, error in surveys takes two forms: variance and bias. Variance refers to all sources of imprecision that may affect survey data. Variance is a random form of error, which can be likened to
“noise,” and there are many approaches that can be used to reduce its size or to measure its size. Bias is a constant form of error and thus is directional: positive or negative. In some cases, bias leads to survey data that underestimate what is being measured, whereas in other cases, bias leads to overestimates. On occa-sion, different types of biases cancel out their own separate effects on survey estimates, but often it is
very difficult for researchers to know when this has occurred. There are many methods that researchers can use to try to avoid bias, as well as many that can estimate the presence, size, and nature of bias. But all of these methods add costs to survey projects, and in many cases these added costs are great indeed.
In designing a survey, researchers should strive to allocate available resources so as to reduce the impact of likely errors, measure the size of the errors, or both, and then take that knowledge into account when draw-ing conclusions with the data generated by the survey.
To accomplish this, researchers must be well aware of the various survey methods that can be used, and then they must select the ones that are most likely to achieve the most beneficial balance of both these goals. This requires survey researchers to constantly make trade-offs in choosing the “best” methods for their particular survey project. Allocating too many resources for one type of method will limit what can be allocated for other methods. If the first method addresses a source of error that is smaller in size than what will result from another source of error, then the allocation choice will have proven counterproductive in addressing total survey error concerns.
There are numerous types of possible errors that can occur with any survey, and it is the purpose of sur-vey methods to address, and ideally avoid, all of these errors. It has been found useful to categorize these possible errors into a limited number of “types,”
which logically follow the chronology of planning, conducting, and analyzing a survey. The following sequence of questions summarizes this typology:
1. What is the population that must be studied, and how well will this population be “covered” (repre-sented) by the frame (i.e., list) from which the sam-ple will be drawn? This concerns coverage error.
2. How large will be the sample of frame members chosen for measurement, and what sampling design will be deployed to select these members? This concerns sampling error.
3. Among all the sampled members of the population, how will a high response rate be achieved, and will
3. Among all the sampled members of the population, how will a high response rate be achieved, and will