print share

CHIS Data Quality

CHIS 2005 Using Neighborhood Characteristics to Assess Nonresponse Bias

Low survey response rates are commonly cited as evidence of poor quality data that lacks representativeness because of nonresponse bias. The survey methodology literature, however, does not support this perspective (see bibliography).

Response rates measure the proportion of potentially eligible respondents in a sample who actually respond to or complete a survey. Nonresponse bias occurs when responding and nonresponding sampled units systematically differ with respect to some characteristic of interest.

Bias or error due to nonresponse is only one of many potential sources of error that may exist in survey data. Surveys with low response rates may or may not result in biased estimates. There are examples of surveys with high response rates and nonresponse bias as well as surveys with low response rates that do not show estimates biased due to nonresponse (for more information on these examples, see bibliography).

In this study, CHIS nonresponse bias is assessed using administrative data to compare neighborhood characteristics among respondents and nonrespondents. CHIS 2005 administrative data that included both respondents and nonrespondents was linked to 2000 U.S. Census data at the census tract level. For every sampled telephone number, we compared neighborhood characteristics for CHIS respondents (those who completed a CHIS telephone interview) and nonrespondents (those who did not respond or refused to complete a telephone interview).

Results show little to no substantial differences in neighborhood characteristics between respondents and nonrespondents. The results are surprising and indicate that the propensity to participate in the survey varies little across a wide variety of neighborhood characteristics. (The results of this study, funded by the National Science Foundation, have been submitted to a peer-review journal and will be made available on this web site when the paper is published.)

See further documentation on this