UPDATE ON RECOMMENDATIONS FROM THE INDEPENDENT TECHNICAL REVIEW
INTRODUCTION
The Executive Summary of an independent technical review into the Labour Force Survey (LFS) and the ABS response to the review's recommendations were released on the ABS website on 9 December 2014. For details see the November 2014 issue of Labour Force, Australia (cat. no. 6202.0). This note provides an update on recommendations 8 and 9 of the review which related to the use of composite estimation and the Labour Force computing system respectively.
COMPOSITE ESTIMATION
Recommendation 8 of the review and the ABS response are:
Recommendation 8: An ongoing program should be established to systematically re-calculate the weights underlying composite estimation. The frequency of re-calculation should be based on an analysis of the effects of the changes in the weights between 2007 and the current time.
ABS Response in December 2014: Agree and is in progress with findings to be reported by April 2015.
Update for April 2015: The sample for the LFS is divided into eight roughly equal segments, known as rotation groups. Each rotation group is a representative random sample of the Australian population in its own right. Every month, one group is rotated into the sample while another rotates out. The newly rotated-in group stays in sample for eight months until it is rotated out. This new rotation group generally comes from the same geographic area as the outgoing one. Under this approach seven-eighths of the sample is common between successive months. This provides stability in the estimates of movement, while ensuring that no dwelling is retained in the sample for more than eight months, and that the sample reflects change over time in the dwelling population (such as construction of new dwellings).
In May 2007 a new estimation methodology, called composite estimation, was introduced to the LFS to improve the accuracy of level and movement estimates. Composite estimation was subsequently applied to all estimates from July 1991.
Composite estimation improves the accuracy of the current month's estimates by utilising the strong correlation between overlapping samples across months. It does this by applying a different factor for each rotation group in the current month as well as for the preceding six months. The composite weights from this process are then adjusted further to align with the current month's population benchmarks to produce the final weight for each current month's observation. This final weight is used to produce the estimates. For more information about composite estimation see paragraph 15 of the Explanatory Notes.
Previous investigations have found that composite estimation results in a reduction in the standard error (a measure of variability) of around 8% for estimates of level (point-in-time) and 5% for estimates of movement.
The composite estimation factors applied to each observation, based on their rotation group and time in survey, were calculated in 2007 but have not been updated since. An analysis was conducted using recent LFS data, to determine whether updating these factors would further improve the accuracy of the estimates. The analysis showed that updating the factors resulted in marginal changes to accuracy. As such, it was concluded that the current factors are still appropriate. It is planned that this analysis will be repeated with each new sample design (currently undertaken every five years).
LABOUR FORCE COMPUTING SYSTEM
Recommendation 9 of the review and the ABS response are:
Recommendation 9: The LFS system and associated collection systems need to be replaced so that proposed changes to the LFS can be formally assessed (e.g. through splitting the sample and comparing responses to new and old questionnaires). As an interim measure, a separate means of storing seasonal factors in the LFS system should be implemented so changed seasonal adjustment methods can be tested without impinging on the LFS production environment and that the seasonal factors from the SEASABS production system can be used selectively, if required.
ABS Response in December 2014: Agree but further work required before implementation to determine the feasibility and cost of the interim measure and system replacement. An update will be included with the March 2015 publication.
Update for April 2015: It is recognised that the ABS's current statistical infrastructure is aged and fragile which increases the risk of errors. However, due to the commitment of ABS staff the high quality of statistical output is maintained with relatively few errors in released estimates. The ABS is working with the government on a plan which will transform the way in which the organisation works, in particular to better manage risk to quality, enable more timely response to new information requirements and achieve efficiencies. The transformation will include the introduction of a best practice, consistent enterprise wide approach to data capture, production, use and dissemination. It is planned to replace the current LFS system and associated collection systems as part of this transformation. All replacement systems will be thoroughly tested before they are implemented.
In the meantime targeted enhancements have been made to the LFS system to improve reliability and efficiency. Data, metadata and code are now better separated and issues that had hampered the testing and release of new content in the past have been resolved. A further set of enhancements is planned to coincide with the release of the new outputs from the LFS. For further information on new Labour Force outputs, refer to "What's new in the Labour Force".
The associated collection systems, used by LFS and other household collections, have also undergone a number of targeted improvements to address the main areas of risk, reliability and efficiency. These improvements have been progressively implemented over the last year.
Recommendation 9 of the review specifically mentioned the seasonal adjustment process. While the suggestion ("a separate means of storing seasonal factors in the LFS system should be implemented") cannot be implemented in the current system, processes have been implemented to enable investigations into seasonal adjustment methods to be undertaken in a test environment rather than the production environment. This addresses one of the main concerns behind the recommendation and these processes have been successfully used in refining Labour Force supplementary survey prior corrections in the annual seasonal reanalysis incorporated with this issue.
While the ABS is not currently able to implement all desired system enhancements, significant steps have been taken to improve the LFS system and associated collection and seasonal adjustment systems, and reduce the risk of a system issue impacting the Labour Force estimates. However, any substantial change to these systems will depend on decisions about the broader ABS transformation.
Quality Declaration 
Print Page
Print All