Feature Article - Can Labour Force Estimates be Improved Using Matched Sample Estimates?
Introduction
Many users of Labour Force Survey (LFS) estimates are interested in the movement between successive monthly estimates. The current LFS design is very good for estimating this monthly movement. There have, however, been recent suggestions that estimates of movement could be improved by the use of so-called ‘matched sample’ estimates. This article considers the usefulness of matched sample estimates and describes the properties of such alternative estimates. It also makes recommendations on appropriate use of the LFS estimates.
Why use the matched sample?
The ‘matched sample’ between two months consists of individuals who were interviewed in the survey in both months. An estimate of movement based only on
these individuals is called a matched sample estimate. This estimate is intuitively attractive compared to the movement based on the whole sample, which is affected by changes in the individuals sampled between the months. By using only the matched sample these changes are avoided, reducing the volatility of the estimates. However, the common sample has a reduced sample size, and this tends to increase the sampling error partially
offsetting the reduced volatility on the matched sample estimates.
Matched sample estimates are biased
The LFS procedures aim to ensure that the individuals surveyed at each time point are a representative sample of individuals. The procedures do not ensure that the set of individuals interviewed in both of two successive months is a representative sample of the population at either time point. In practice, the matched sample will tend to omit individuals who are harder to contact or more mobile. Such people are more likely to have been missed or to have moved house between the surveys. They are also likely to have labour force characteristics that differ from the rest of the population. Thus matched sample estimates of movement will not represent these classes of people as adequately as the original estimates do. That is, they are open to ‘bias’, in that they are likely to understate the movement in the whole population.
The ABS does not produce matched sample estimates from the labour force survey. However, some matched sample data are published in ‘gross flows’ tables in Labour Force, Australia (6203.0).
These tables are not aimed at producing estimates for the whole population, so they have not been specifically adjusted for that purpose. This means that they not only under-represent more mobile individuals, but also may have a different age and sex mix from that of the whole population. Matched sample estimates from this source would therefore be subject to additional variation or bias.
Is a better estimate of movement available?
The LFS sample is actually divided into eight groups, known as rotation groups. Each rotation group is a representative sample of the Australian population in
its own right. Estimates from these rotation groups can be used to produce alternative labour force estimates that are not biased in the ways just described.
Between any pair of successive months, the sample in one of the rotation groups is changed. The other seven rotation groups contain the same sample of
dwellings (though they will include some different individuals due to mobility of the population). An estimate of movement could be produced based on the seven ‘matched rotation groups’. (These rotation groups contain individuals other than those in the matched sample. This allows them to be representative of the population, though with slightly lower month-to-month correlations than the matched sample alone.)
Calculations show that matched rotation group estimates of movement of employed persons would have standard error about 10% lower than that of a movement based on the published estimates. (Standard error is a measure of the sampling error on an estimate-for details on standard error in the LFS see Labour Force Australia (6203.0)). The improvement for estimates of unemployed persons would be smaller (under 2%) because they have lower correlation between months.
Why doesn’t the ABS use matched rotation group estimates of movement?
The ABS could produce valid estimates of movement using the matched rotation group approach. However, publishing these estimates would raise some difficult issues. First, many users would prefer the estimate of movement to equal the difference between the published monthly estimates of level. To achieve this would require changing the monthly estimates, so that each monthly estimate of level is obtained by adding the estimate of movement to the level estimate from the previous month. However, the monthly estimates of level produced by this method would have higher sampling error than the current ones, with sampling error increasing over time. These estimates could stray significantly over time from the best available estimates of the level of the series.
Second, the ABS does not encourage users to base their decisions on the individual month-to-month movements. Decisions about the direction of labour force series are better made by examining the trend series. Month-to-month movements often go in a different direction from the underlying direction of the time series, and so can be misleading. Given the importance of the level and trend series, it would be poor practice to base the series on an estimate optimised for monthly movement at the expense of the trend estimate.
(As a side issue, some users prefer to look at quarterly averages in order to obtain a broad picture of the direction of the economy. While this is valid, the trend estimates published by the ABS tend to detect and confirm turning points in a monthly series much earlier than an examination of the quarterly averages.)
Third, the cost and confusion that could be caused by a change to ABS estimation practices would offset the relatively small improvements that apply to movement estimates.
Finally, the matched rotation group estimates of movement are just one example of a whole class of estimates known as composite estimates. If a composite estimate is used, there are more sophisticated approaches that yield greater gain.
Composite estimates
While matched sample estimators give all the weight to the matched sample, and none to the sample dwellings which change between months, composite
estimators give most weight to the common dwellings, but also put some weight on the changed dwellings. Composite estimators have been derived that achieve improvements for monthly estimates of level,movement and trend (for employed persons the improvements in standard error are about 6%, 12% and 6%, respectively).
These composite estimators achieve consistency between the different estimates by revising the last few monthly estimates when new data arrives. This takes advantage of the correlation of successive monthly estimates to improve earlier estimates when new information arrives. While this approach is reasonable, it is possible that regular revisions would be unacceptable to users. The need to gain user acceptance of revisions is a factor against using this form of composite estimation.
A comparison of published, matched rotation group, and composite estimates
Matched rotation group estimates and composite estimates were produced for comparison with published figures for experimental purposes. The variable studied was the proportion of the population employed. Proportion employed is used because it has a higher correlation month to month than proportion unemployed, and hence greater scope for improving the movement estimates. The matched rotation group estimates are based on adding successive estimates of monthly movement to an initial published figure from November 1990. Graph 1 shows the resulting matched rotation group series alongside the original series and the composite estimate series.
The graph suggests a small systematic difference between the non matched rotation groups and the remaining groups, with the difference accumulating over time, in addition to any difference arising from sampling error. Sensitivity to such a systematic difference is an unfortunate property of the matched rotation group estimates. The composite estimates do not have this problem.
To use the matched rotation group estimates over a long period would require at least occasional revision of the series to bring it into line with the best estimate
of level. One approach is to calculate the average monthly discrepancy over a long period, and add it in to each matched rotation group movement estimate.
This is equivalent to adjusting each matched rotation group movement by the same amount so that they add to the long term movement of the original series.
This ‘adjusted matched rotation group’ series is closer to the original series, but still strays from it for extended periods. This is clear from Graph 2, which
gives the difference from the original series of both the adjusted matched rotation group estimates and the composite estimates. It can be seen that the
adjusted matched rotation group series is above the original series for the period around late 1990 to early 1993, and then below for a period from early 1993 to
late 1995. The differences are too large to be explained by the sampling error on the original series, given the number of successive time points for which
the discrepancy was large. This suggests that a single adjustment for the whole series was insufficient, and more regular adjustments (and hence revisions) would be needed to give a good series.
In contrast, the composite estimator does not stray far from the original series for long. It is interesting to note that the composite estimator often moves in the same way as the matched rotation group estimator, but not as far, or for as long. In a sense, the composite estimator is a compromise between the two
estimates, having short-term properties like the matched rotation group estimator and long-term properties more like the original series.
GRAPH 1. COMPARISON OF ESTIMATES, PROPORTION EMPLOYED
GRAPH 2. DIFFERENCE FROM ORIGINAL ESTIMATE, PROPORTION EMPLOYED
(percentage points)
GRAPH 3. COMPARISON OF ESTIMATES, PROPORTION EMPLOYED
Note, however, that the graphs give the composite estimator at its final value after revisions - they do not illustrate the sequence of small revisions which each
composite estimate undergoes in the first few months after it appears. The total of these revisions was measured for each month in the series - the average
magnitude of these total revisions (ignoring whether they are positive or negative) was 0.07 percentage points, with the largest total revision being 0.25
percentage points. For comparison, the average month to month movement (ignoring whether they are positive or negative) over the same period was 0.5
percentage points.
Measuring longer term movements
The matched rotation group estimates are somewhat better than the original estimates as indicators of month-to-month movement. This remains true for adding a small number of these movements together to obtain a longer term movement, for periods up to about six months. The composite estimator is better still, for both short and longer term movements.
For measuring movements over more than six months, the matched rotation group estimates quickly deteriorate compared to the original estimates. For movement between months a year apart the matched rotation group estimator has 16% greater standard error than the original. (This compares poorly with the
composite estimator, for which yearly movement has a 14% lower standard error than the original.)
Graph 3 shows a particularly bad period (in 1993) where adding successive matched rotation group movements to the January 1993 figure would significantly understate the movement over a twelve month period. This occurs because over this period the newly selected dwellings tended to have higher employment than those they replaced. These upward influences were completely discounted by the matched estimates. The composite estimates make better use of the data, and take values similar to the original for this period.
Conclusion
Estimates of movement based on matched samples of persons can achieve lower sampling error than the published estimates, but are open to significant bias.
As a result, the ABS does not recommend use of estimates based only on the matched sample (as published in ‘gross flows’ tables). This article described a less biased estimate based on matched rotation groups, and discussed its properties. A still better method known as composite estimation was also discussed.
All the alternative estimators discussed have drawbacks compared to the current published estimates. In particular, if an estimate that does not drift from the best level estimate is sought, revision is required. For this reason the ABS official estimates will continue to be based on the current approach. Composite estimates show some promise, and the ABS will investigate these further. Users interested in the direction of the labour force series are advised to use the published trend series rather than placing too much emphasis on the month-to-month movements.
This feature article was contributed by Phillip Bell, Labour Force, ABS.
Print Page