1504.0 - Methodological News, Mar 2008  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 31/03/2008   
   Page tools: Print Print Page Print all pages in this productPrint All

The Use of Trials to Assess Changes in Operational Procedures in ABS Business Surveys

The Operations Research Unit has run three trials relating to the operational procedures behind Intensive Follow-Up (IFU). These were run on the Quarterly Economy Wide Survey (QEWS) and the Economic Activity Survey (EAS) during 2007. All trials involved the changing of the timing of the IFU procedures and were based on research looking at the length of time taken for forms to be returned.

For QEWS, the first trial (A) involved removing units from IFU until AFTER the second reminder. The units were chosen if they returned their form in the previous cycle by the second reminder, and received no outbound telephone contact.

Trial A has been run for five quarters now and each time the response rate for the trial A providers is around 98%. The number of providers in this group is usually around 7,000, meaning that no phone contact is required for those 7,000 providers. The success of this trial has resulted in the process being automated and hopefully will be implemented in a number of sub-annual collections during 2008.

The second trial (B) involved the change in the timing and staff allocation for a sample of providers. The changes in timing were as follows:

  • wait five days after the first reminder before commencing phone IFU (instead of one day);
  • wait five days after the second reminder before recommencing phone IFU; and
  • wait three days after leaving an answering machine message.

Trial B was run in the September quarter 2007. A sample of 1000 providers was chosen and compared with providers not in the trial that were of the same scope.

Ideally, all providers would be contacted during the window of opportunity, however, resources do not permit this to happen. The main recommendations from the trial were therefore to prioritise providers in IFU such that new units, previous non-respondents or late respondents are targeted in the first five days; and to use priorities to ensure that all providers are called before the second reminder rather than some getting called multiple times and others not at all. More work is needed to advise exactly how to proceed with this prioritisation and it is expected that it can also be applied to other collections.

The trial for EAS involved a sample of 3000 with the following changes in timing:
  • delay commencement of phone IFU for ten days after the first and second reminders, or until ten days after the second reminder if there has been an inbound call;
  • delay all phone contact for a minimum of five days and a maximum of 14 days after previous direct contact, whether inbound or outbound; and
  • wait three days after leaving an answering machine message.

The EAS trial was run for the 06/07 cycle. Only preliminary results are available at the moment, however the results are generally similar to those for QEWS. For example, form receival rate for those in the trial was slightly higher (90% compared to 86%), and trial providers had higher average outbound calls (1.8 compared to 1.3). In the case of EAS, the main recommendation from the trial was to move the timing of the entire EAS such that the optimal time to contact EAS providers does not overlap with any other surveys.

The overall outcome from these trials is that trial A is being implemented as general practice. The other two trials require further work to see how they will work in practice.

For further information, please contact Louise Gates on (02) 6252 6540.