Cost Modelling for Economic Surveys
The Operations Research Unit (ORU) is undertaking a number of projects involving modelling intensive follow-up (IFU) processes for ABS economic surveys. These projects are part of a conceptual framework which links input parameters and processes, such as survey sample size, number of reminder letters to be sent, number of calls to be made to providers, to cost and statistical outcomes, such as bias of output estimates and response rates. Currently we are developing statistical models to estimate these inputs more robustly, and to link these inputs to outcomes, to enable predictions to be made, and to inform decision making on what operational practice efficiencies can be made.
The first model which we are developing is a probability of response model. The aim of this model is to determine the drivers of response patterns and to be able to predict response rates resulting from various IFU process scenarios, leading to an improved understanding of IFU best practice. For example, the model will be able to be used to inform the optimal number of calls to make, and when to make them, thus linking into other problems such as optimal interviewer allocation, and minimising the dollar cost of operations for an acceptable level of statistical quality of the output estimates.
This model will make use of survey paradata for a number of surveys and cycles. Survey paradata includes information on individual businesses' response status over the survey cycle, how many reminder letters and subsequent follow-up calls were made to providers, how many incoming calls were received from providers, and what the relative timing of each of these were, as determined by the IFU contact strategy. It also includes business characteristics such as business industry and size and their priority of contact within the IFU strategy.
Because the timing of events is an important part of the IFU process, survival analysis is being used to fit the probability of response model. Survival analysis is a technique commonly used to model time-to-event data, where some records may be censored (i.e. the timing of the event is unknown or is outside the observation window) and where the explanatory variables may vary over time. In our case, the event is receiving a response from the business (non-respondents are censored), and some of the explanatory variables (e.g. number of calls) vary over the analysis period. Such a model will allow us to predict not only the end of enumeration response rate, but also the response rate throughout the enumeration period.
The other models in the conceptual framework are the Provider Contact Unit (PCU) cost model and a soon to be developed bias model. The PCU cost model, which has been reviewed and enhanced by the ORU, can be used to forecast costs based on survey characteristics (such as sample size), a specified IFU strategy, and expected response rates over time. For the bias model, as proof of concept we are using a simulated bias based on the practical application of reducing the response rate. In the near future, a more robust approach to modelling the bias will be investigated.
Once the three component models have been linked together, PCU will be able to make better informed decisions on practical issues, for example, reducing target response rates, or how long to wait between follow-up calls. Early next year, ORU will commence work on the application of scientific methods to determine optimal practices and procedures, such as interviewer allocation/ scheduling and the number of calls which should be made to a provider.
For more information, please contact Melanie Black on (02) 6252 7241 or melanie.black@abs.gov.au
Print Page
Print All