• Summary
• Downloads
• Explanatory Notes
• Related Information
• Past Releases

QEWS Phase 3 sample design and system re-engineering

This article follows on from an article in the December 2008 issue of Methodological News, which described the redesign of the Quarterly Business Indicators Survey. That article described the survey's change of industry classification and scope change to include non-employers (estimated at 15% of economy based on BAS turnover records). The old and new designs of the QBIS survey will run in parallel for two quarters (March and June 2009), and then the new design will be used exclusively from September 2009 quarter onward. This article goes on to describe some of the automated system changes that are being made to improve efficiency by automating parts of the data editing and estimation processes.

We digress briefly to describe QBIS' sister survey, the Survey of Capital Expenditure, or known simply as CapEx. CapEx is also a quarterly survey that is ran in tandem with QBIS - together they form the Quarterly Economy-Wide Surveys or QEWS. CapEx, like QBIS, is undergoing a redesign to recognise the updated and modernised Australian and New Zealand Standard Industrial Classification (from ANZSIC 1993 to ANZSIC 2006), and an increase in scope to include non-employers for the first time. CapEx surveys the business community's expenditure on building and equipment capital in the past quarter, as well as their anticipated capital expenditure planned in the next 12-18 months. CapEx is run as a separate survey from QBIS due to the differing nature of the financial quantities, and because capital expenditure data can be measured with sufficient accuracy with a significantly smaller sample of businesses.

The redesign work for both QBIS and CapEx has been completed at the time of writing, and the new designs are being implemented over the first half of 2009. One key change is that the QEWS surveys are transferring from the old custom-written QEWS Phase 2 systems to the ABS Survey Facilities (ABS SF), which is the official corporate system. These changes represent an overall improvement to efficiency through the reduction of person-hours required to process survey data in any one cycle, as well as an improvement to the corporate alignment of the systems, thereby improving transparency and comparability with other surveys. The components of these changes are described below.

The ABS Survey Facilities includes a sophisticated imputation engine. Imputation is the process whereby the responses from surveyed units (businesses) that do not respond can be "estimated" by borrowing information from similar units. A typical example is Live Respondent Mean imputation where a unit's value is substituted with the average of responding units of similar characteristics (size, industry, geographical location). Imputation is one of many strategies used by the ABS to compensate for imperfect data quality, though of course the ABS always aims to have high response rates so as to minimise estimation bias associated with imputation.

The QEWS surveys will now use the standard ABS SF imputation tools, which include a selection of 39 imputation methods for different circumstances, and a user-friendly metadata interface which allows ABS officers with relatively little data integration experience to precisely specify and tune their chosen methodology. The standardised format of the metadata allows relatively easy comparison of imputation strategies between different surveys.

Occasionally, our business surveys receive unusually large dollar-value responses for individual units that were thought to be relatively small based on historical information. These outliers can significantly increase the variance (recognised as volatility by some) of our survey results, particularly if the unit has a large survey weight (i.e. the design says it represents a large number of other businesses not surveyed for the sake of calculating estimates). One way to treat such outliers is through winsorisation, which decreases a unit's contribution to estimates. Winsorisation comes at the cost of a mild downward bias on estimates, but this is a necessary compromise to limit the amount that estimates vary from quarter to quarter due to unusually large units. The QEWS surveys will now use the full version of the ABS SF winsorisation facility.

Lastly, QEWS will now use the full suite of standard error calculation engines of the ABS SF. Under the previous QEWS design, standard errors for QEWS surveys involved the running and interpretation of a specialised program, which was a cumbersome manual task particularly if repeats were requested. The new system will be able to produce standard errors as part of the existing "data refresh" processes, thereby improving efficiency. The ABS SF standard error calculation engine uses the highly flexible bootstrap replication technique, developed by the Statistical Services Branch (SSB) in MDMD. Bootstrap replication is adaptable to a wide variety of survey designs, and can be applied with little knowledge of the internal working of the survey, and hence is an ideal choice for a corporate system. The automation of standard error calculation for QEWS also opens the possibility of running multiple draft refreshes of standard error estimates to aid in data editing while it is still being finalised.

For further information, please contact Benedict Cusack on (02) 9268 4775 or benedict.cusack@abs.gov.au