Feature Article - Telephone Interviewing - Effect on Labour Force Survey Estimates
Introduction
The ABS is changing the way it conducts the monthly Labour Force Survey (LFS). Up to July 1996, the survey was conducted "face to face" that is, interviewers would visit each selected dwelling to fill out the survey questionnaires.
From August 1996, telephone interviewing has been introduced progressively as the main method of gathering information from selected dwellings
to complete LFS questionnaires.
Selected dwellings remain in the survey for eight months. For the first month, face to face interviewing (i.e. a visit to the dwelling) will be retained. But under the new methodology, for the second and subsequent months, the survey will be conducted by telephone if feasible and acceptable to the residents of the selected dwelling.
Telephone interviewing has been phased in. One eighth of the sample is converted to telephone interviewing each month. By December 1996, telephone interviewing was the principal method for five eighths of the sample of dwellings. The phase-in will be complete for the February 1997 survey.
The ABS has been closely monitoring the contributions to key labour force estimates from the telephone interviewing component and the face to face component. It appears that there is a discernible and consistent difference in the estimates of employment under the new methodology.
The following questions and answers explain the changes the ABS has made to the way it conducts the survey, and discusses the impact on key estimates.
Why Is Telephone Interviewing Being Introduced?
Telephone interviewing is a well established method used by international statistical agencies, market research companies and other agencies conducting surveys. For the ABS it offers substantial savings in operational costs of the monthly Labour Force Survey. Overseas experience and local testing showed that these savings could be achieved with little or no detrimental impact on quality, reliability and timeliness of the survey results.
How Have Employment Estimates Been Affected?
Analysis conducted to date suggests that telephone interviewing has had a discernible effect on aggregate employment estimates. The effect has been, on average over the five months to December 1996, to lower aggregate employment by an amount between 6,000 and 9,000 persons, for each of the five months.
After five months of phasing in telephone interviewing, the cumulative effect on the estimate of total employment is a reduction of the order of 30,000 to 45,000. That is, the Labour Force Survey estimates have underestimated employment growth from July 1996 to December 1996 by an amount of that magnitude.
The impact on individual months is more difficult to assess. The size of the effect is too small to permit conclusions to be drawn about the effect on employment estimates month by month.
What Is The Effect On Estimates Of Persons Not Employed?
An effect on employment estimates must be compensated by a similar effect but in the opposite direction, on estimates of those persons not employed i.e. the unemployed, and persons not in the labour force. <N>However the analysis to date has not shown any consistent pattern in the effect on estimates of unemployment and of persons not in the labour force. On average, the estimate of persons not in the labour force has shown an increase, but the magnitude is not statistically significant. Nor has there been any reliable evidence to indicate that there is a consistent and significant effect on estimates of unemployment.
For Employment, Can The Effect On Components Be Identified?
Because the change in the estimate of total employment is relatively small, it is not possible to produce reliable measures of the impact for component series (e.g. by sex, by age, or by State/Territory). However, evidence suggests the effect is spread over age and sex groups, and is broadly consistent across States/Territories. Sampling variability tends to swamp any further dissection of the estimated effects, preventing any finer level conclusions being drawn.
Is The Observed Effect On Employment A Permanent One?
While the observed effect on employment estimates has been consistent and statistically significant over the phase-in period to date, it is not possible to ascertain whether the effect on employment is an intrinsic part of the telephone interviewing methodology, or if it is temporary in nature and associated with the phase-in period. If the effect is temporary, then estimates would be expected to return to levels consistent with those produced under the former face to face methodology.
Why Has Telephone Interviewing Had An Effect On Employment Estimates?
The effect on the employment estimates is very small and at this stage of implementation, it is not possible to ascertain why the new methodology has produced the observed effects. However, the following theories have been put forward:
- A marginal change in response rates in the telephone interviewing methodology, which may mean that there is a slightly different mix of households responding to the survey in the new environment;
- Changes in the proportion of proxy interviewing. While survey information is collected for all usual residents of the selected dwelling, interviewers generally obtain responses from one responsible adult in the household, which is known as the "proxy respondent" methodology. In the telephone interviewing environment, the proportion of proxy interviewing is marginally higher which may indicate a slightly different set of people are responding over the phone. Furthermore, these respondents may be answering survey questions differently than they might in the personal interview environment;
- A learning/training effect for interviewers, which will dissipate after an initial period.
Will The ABS Be Revising Or Adjusting Series?
At this stage, it is not possible to state whether the ABS will be revising or adjusting employment data. When telephone interviewing is fully implemented and the analysis is complete, the ABS will be evaluating the implications of the telephone interviewing effect and assessing whether revisions are warranted and feasible.
Which Method Produces Most Accurate Estimates?
Telephone interviewing is a widely accepted survey methodology used in many countries for the collection of labour force statistics. It has strong acceptance among official statistical agencies and private sector survey organisations throughout the world. Telephone interviewing has been used in labour force surveys in the USA since 1954; in Canada since 1973; and in the UK since 1984.
Overseas research indicates that telephone interviewing produces estimates of comparable reliability to those from face to face interviews and for some variables the quality may be better.
What Analytical Methods Have Been Used To Analyse The Effect?
The phase-in approach provides a relatively powerful means for the identification and measurement of possible impacts. The telephone interviewing impact has been analysed by comparing that part of the sample which has been converted to telephone interviewing, with the remainder of the sample. A wide range of analytical techniques have been employed, including:
- The comparison of the estimates from different parts of the sample;
- Analysing the month to month movement of the estimates as phoning was being introduced;
- Studying the change in the historical pattern of the series during the phase-in period.
These techniques have led to consistent results.
Conclusion
The ABS will continue to monitor the implementation of telephone interviewing and continue to assess its effect on key labour force series. During the remainder of the phase-in period, further information will be published in Labour Force, Australia, Preliminary (cat. no. 6202.0). The ABS will also be releasing a technical paper once the implementation and analysis is complete. That paper will provide detailed information about the analytical techniques employed to distinguish the effect, as well as the findings from the analysis.
For further information regarding telephone interviewing in the Labour Force Survey, contact the Director, Labour Force Section on 02 6252 5489.
Print Page