• Summary
• Downloads
• Explanatory Notes
• Related Information
• Past Releases

Does increased effort lead to a less representative response?

Traditionally, response rates are used in survey management as an indicator of survey quality. At the Australian Bureau of Statistics (ABS), we monitor response rates on a daily basis during data collection and compare these against response rate targets. The required effort and costs of maintaining high response rates are increasing, but it is not clear what effect this is having on survey quality.

In the literature, recent debate has focussed on differentiating the level of effort between maintaining response rates and enhancing the representativeness of the response – precision versus bias.

The Operations Research and Process Improvement Section have been looking at Representativity indicators or R-indicators, as indicators of representativeness. The R-indicator is based on the variation in response probabilities. In practice, the true values of the response probabilities are not known. Therefore, they are estimated using, for example, a logistic regression model.

We produced R-indicators for some selected case studies, namely the National Health Survey, the Survey of Income and Housing, and the Survey of Mental Health and Wellbeing (all were conducted in 2007/2008). Preliminary findings indicate that for each additional contact attempt that was made, the response rate increased, while the R-indicator showed a drop in representativeness. These results would seem to indicate that making more contact attempts may just result in more of the same kind of respondents. The R-indicator may prove to be a useful indicator of non-response bias. Further research on the use of R-indicators in ABS surveys is continuing.

In future work, we also plan to investigate using R-indicators to compare different follow-up strategies of non-respondents by conducting a simulation study.

For further information, please contact Rosslyn Starick (03 9615 7055 or rosslyn.starick@abs.gov.au).