print share

CHIS Data Quality

CHIS 2007 Nonresponse Analysis

Survey response rates have historically been used as a single indicator of data quality. Response rates measure the proportion of potentially eligible respondents in a sample who actually respond to or complete a survey. Nonresponse bias occurs when responding and nonresponding sampled units systematically differ with respect to some characteristic of interest.

Bias or error due to nonresponse is only one of many potential sources of error that may exist in survey data. Surveys with low response rates may or may not result in biased estimates. There are examples of surveys with high response rates and nonresponse bias as well as surveys with low response rates that do not show estimates biased due to nonresponse (for more information on these examples, see survey methods bibliography).

Because nonresponse bias occurs if respondents and nonrespondents systematically differ with respect to a characteristic of interest, such bias is measured by comparing responses to particular questions or estimates; it is not endemic to a survey as a whole.

To assess nonresponse bias in CHIS 2007 estimates, we conducted a special survey procedure in which we could follow up with persons who were selected for the sample but did not respond to attempts to interview them as part of the telephone survey. As part of CHIS 2007, we drew a separate area probability sample of residential mailing addresses in Los Angles County and completed 980 adult interviews. We attempted to complete a regular telephone interview of all persons in the sample for whom we could obtain a telephone number. We successfully completed regular telephone interviews with 222 respondents, and we were able to then complete follow-up interviews with 227 adults who initially did not respond to multiple telephone contacts. There were an additional 531 completed interviews in households for which we could not match a telephone number to an address.

To assess nonresponse bias, we then compared estimates for over 40 health and health related indicators among "respondents" (who responded to an initial request to conduct the survey over the telephone) and "nonrespondents" (who only participated after an in-person recruitment visit). Results indicate few differences between respondents and nonrespondents are either significant or substantial once the data are adjusted. (This study is nearing completion; when results have been published in a peer-review journal, they will be made available on this web site.)

See further documentation on this