PROGRESS WITH RECOMMENDATIONS FROM THE INDEPENDENT TECHNICAL REVIEW - JULY 2015


INTRODUCTION

The Executive Summary of the Independent Technical Review (McCarthy Review) into the ABS's Labour Force Survey (LFS) and the ABS response to the Review's recommendations were released on the ABS website on 9 December 2014. For details see the November 2014 issue of Labour Force, Australia (cat. no. 6202.0).

This note provides an update on the implementation of the recommendations. Work is continuing on several of the recommendations and a further update will be provided by the end of 2015.


OVERALL STATUS

Of the 16 recommendations arising from the review:

• the ABS has completed 8,
• five related to how we undertake future work and, as such, are ongoing,
• one will be progressed as part of the Statistical Business Transformation Program and
• three are in progress.

Recommendation 2, which focussed on the main cause of the substantive problem with the labour force series in August 2014, was implemented with the October 2014 issue of Labour Force, Australia with refinements introduced with the March 2015 issue as part of the annual seasonal reanalysis.


PROGRESS BY RECOMMENDATION

The following section sets out a summary version of each recommendation from the independent technical review of the LFS. It also sets out the status of the response to the recommendation - completed, in progress or included in work plans.


Recommendation 1:

The labour force characteristics of those responding by e-forms should continue to be analysed to determine how they compare with those being interviewed by telephone or in person and whether or not a changing take-up of e-forms could have an effect on the LFS estimates.

Response status

Completed. Findings reported in the February 2015 issue of Labour Force, Australia (cat. no. 6202.0) and will be repeated every 6 months.


Recommendation 2:

The LFS estimates should be seasonally adjusted using prior corrections for the supplementary surveys that have been assessed as having a significant effect on the seasonality of the LFS estimates from now on. The methodology should be closely examined as part of the annual seasonal reanalysis of LFS data in early 2015 to determine whether or not the method should be refined further prior to it being applied to the full length of LFS monthly series from February 1978 onwards.

Response status

Completed. Implemented in the October 2014 issue. Methodology further refined as part of the annual seasonal reanalysis implemented with the March 2015 issue.


Recommendation 3:

The ABS should warn users of LFS data of the possibility of instability in the seasonally adjusted LFS estimates in those months in which new supplementary surveys are being conducted from 2014 on and that it will be at least three years before the effects on the LFS estimates of the new supplementary survey program can be measured with any degree of certainty. Steps should be taken to identify any such effects (e.g. by analysing individual records that change LF status more than once) and adjust the core LFS estimates, if necessary.

Response status

Completed. Statement included in the February 2015 issue and in the Explanatory Notes.


Recommendation 4:

It is possible that the lower response rates may be having an effect on the estimates of employment and/or unemployment. The impact of lower response rates should be analysed further.

Response status

Completed. Findings reported in the April 2015 issue of Labour Force, Australia (cat. no. 6291.0.55.001). This analysis examined the impact of lower response rates on the labour force estimates and found that it was not significant. This is subject to ongoing vigilance in the preparation of labour force estimates each month.


Recommendation 5:

Expand the description of seasonal adjustment and trend in the 6202.0 Explanatory Notes (and in any related publications) by including the links between an original series and its seasonal, irregular and trend components (i.e. O = T * S * I) as an introduction to the more technical aspects already included in the notes.

Response status

Completed. Explanatory Notes updated from the February 2015 issue of Labour Force, Australia (cat. no. 6202.0) onwards.


Recommendation 6:

The ABS should highlight the fact that the changes in the numbers of persons employed, unemployed and not in the labour force from one month to the next are the net outcomes of considerably larger gross flows between these categories. One means of doing so would be to include some comments on the gross flows each month in the “ Force Commentary” section of Labour Force, Australia (cat. no. 6202.0).

Response status

Completed. Additional commentary included from the January 2015 issue of Labour Force, Australia (cat. no. 6202.0) onwards.


Recommendation 7:

Bayesian techniques should be considered as a means of identifying and resolving potential problems in core LFS series.

Response status

In progress. Article included in the May 2015 issue of Labour Force, Australia (cat. no. 6202.0) outlining the nature of our proposed investigation. This work is currently underway, with a further update planned for release with the November 2015 issue.


Recommendation 8:

An ongoing program should be established to systematically re-calculate the weights underlying composite estimation. The frequency of re-calculation should be based on an analysis of the effects of the changes in the weights between 2007 and the current time.

Response status

Completed. Investigation completed and findings reported in the March 2015 issue of Labour Force, Australia (cat. no. 6291.0.55.001). The investigation concluded that the current weights underlying composite estimation were still appropriate.


Recommendation 9:

The LFS system and associated collection systems need to be replaced so that proposed changes to the LFS can be formally assessed (e.g. through splitting the sample and comparing responses to new and old questionnaires). As an interim measure, a separate means of storing seasonal factors in the LFS system should be implemented so changed seasonal adjustment methods can be tested without impinging on the LFS production environment and that the seasonal factors from the SEASABS production system can be used selectively, if required.

Response status

Ongoing. Update included in the March 2015 issue. Targeted improvements have been made and further improvements will be implemented as part of the Statistical Business Transformation Program over coming years.


Recommendation 10:

Given the importance of the LFS data and the adverse effect of the associated supplementary survey program on these key economic indicators, the ABS should consider discontinuing the supplementary surveys, or conduct them in a manner that has a negligible impact on the LFS data.

Response status

In progress. The ABS has requested the views of stakeholders on continuing the supplementary survey program as the prior correction methodology (recommendation 2) allows the ABS to properly adjust for any impact on the LFS.


Recommendation 11:

The LFS processing and publication cycle should be closely monitored to determine whether or not it is necessary to delay publishing LFS data until early in the week following the current timetable on an ongoing basis or at least when changes are being made to the LFS. In particular, delaying the publication should be considered if any significant change is to be implemented.

Response status

In progress. The ABS has requested the views of stakeholders on delaying the release of labour force publications (Labour Force, Australia (cat. no. 6202.0), Labour Force, Australia, Detailed (cat. no. 6291.0.55.001) and Labour Force, Australia Detailed, Quarterly. (cat. no. 6291.0.55.003)) by one week.


Recommendation 12:

The ABS should involve key stakeholders in any proposals to change the LFS or the ways in which LFS estimates are compiled. Any proposed changes should be communicated in advance via a release on the ABS website providing details of what is happening and why.

Response status

Ongoing. This recommendation refers to how the ABS should approach change programs in the future. The ABS accepts the recommendation.


Recommendation 13:

Statistics Canada has faced similar criticisms to the ABS regarding perceived volatility in its LFS estimates. One of the responses was to publish an article on interpreting monthly changes in employment from the labour force survey to show that recent history was, in fact, less volatile than the past. Although the recent volatility in the Australian LFS had a specific cause, the internal ABS analysis of the general volatility in the Australian LFS estimates should be expanded and formalised and published to help educate users about this issue.

Response status

Completed. The findings of a report into the volatility of Australian labour force estimates over time were released with the June 2015 issue of Labour Force, Australia, Detailed (cat. no. 6291.0.55.001). The analysis found that, for key national seasonally adjusted estimates (employed persons, unemployed persons and the unemployment rate), the current level of volatility is below historical averages


Recommendation 14:

Wherever feasible, future changes to the LFS program should be introduced individually, spread out over time rather than bunched as has been the case during the past 18 months or so, to the extent they can be controlled (e.g. the timing of introducing Population Census benchmarks and benchmarking to ERP estimates is non-discretionary).

Response status

Ongoing. Timing of future changes will be discussed with key stakeholders.


Recommendation 15:

If it does become necessary to make a number of changes to the LFS in a short time, specific risk management strategies need to be put in place to assess the potential impacts of each of the changes. A small working group consisting of senior Labour Branch/LFS staff and representatives of Methodology Division, Time Series Analysis Branch and Technology Services Division should evaluate the individual and cumulative impacts of the proposed changes and provide a formal risk management report to the relevant FAS and Deputy Statistician.

Response status

Ongoing. The ABS accepts this recommendation including the need to assess the combined statistical risks.


Recommendation 16:

As part of any future Quality Incident Response Plan (QIRP), the effects on LFS seasonality of changes in institutional arrangements (e.g. changes in the LFS questionnaire or in the supplementary survey program or in the incidence of response by e-forms) should be considered explicitly.

Response status

Ongoing. This recommendation refers to how the ABS should approach quality issues programs in the future. The ABS accepts the recommendation.