TECHNICAL NOTE


IN THIS SECTION

Reliability of Estimates
Non-Sampling Error
Sampling Error
	Standard Error
	Relative Standard Error
Examples of Calculations
	Level Standard Errors
	Movement Standard Errors
	Differences between Estimates

RELIABILITY OF ESTIMATES

1 Estimates from the LFS, as with data from all surveys, are subject to error. The data presented in this publication are subject to two sources of error:

• non-sampling error, which arises from imperfections in reporting, recording or processing of data that can occur in any survey or census.

• sampling error, which occurs because data were obtained from a sample rather than the entire population.

NON-SAMPLING ERROR

2 The main sources of non-sampling error are response errors and non-response bias. These may occur in any enumeration whether it is a full count or a sample.

3 Response errors include errors on the part of both respondents and interviewers. These reporting errors may arise through inappropriate wording of questions, misunderstanding of what data are required, inability or unwillingness to provide accurate information, and mistakes in answers to questions.

4 Non-response bias arises because the persons for whom no response is available may have different characteristics in relation to labour market behaviour than persons who responded in the survey.

5 Non-sampling errors are difficult to quantify in any collection. However, every effort is made to minimise these errors in the LFS by careful design of questionnaires, intensive training and supervision of interviewers and efficient operating procedures. Non-response bias is minimised by call-backs to those households which do not respond, and is compensated for in the estimation process.

6 There are a number of other issues associated with collecting information from Indigenous persons in communities in remote areas. Although special procedures are used in some Indigenous communities, there may still be some cultural and practical difficulties in applying standard labour force concepts in these communities. Operational issues include the high turnover of trained interviewers in remote areas, the seasonal fluctuations in population numbers as well as in employment opportunities, and high population mobility.

7 Responses in the LFS may be given by any responsible adult in each selected household. Reporting errors may arise when the respondent provides information for another member of the household without being fully aware of their labour force details. Although this is a minor issue for the survey in general, the higher mobility of Indigenous household members may affect the accurate reporting of details such as active job search or availability for work.
SAMPLING ERROR

8 The LFS estimates are based on information obtained from a sample of the population, and are subject to sampling error. Sampling error is the difference between the estimate obtained from a particular sample and the value that would have been obtained if the whole population were enumerated under the same procedures (referred to as the 'population value'). Standard error

9 The most commonly used measure of sampling error is the standard error (SE). This measure indicates the extent to which a survey estimate is likely to deviate from the true population value by chance. There is a 67% chance (2/3) that the sample estimate will differ by less than one standard error from the estimates that would have been obtained if all dwellings had been included in the survey, and a 95% chance (19/20) that the difference will be less than two standard errors.

10 The magnitude of the sampling error depends on the sample design, the sample size and the population variability. The larger the sample on which the estimates are based, the smaller the sampling error. The main contribution to sampling error for the Indigenous labour force estimates is the sample size.

11 Movements in the level of an estimate are also subject to sampling variability. The standard error of the movement depends on the levels of the estimates from which the movement is obtained rather than the size of the movement. The standard errors for both level estimates and movements between annual estimates are provided in Tables 1 to 5 (available in Downloads). The standard errors have been derived using the group jack-knife method. Relative standard error

12 Another measure of sampling error is the relative standard error, which is obtained by expressing the standard error as a percentage of the estimate to which it refers. The smaller the sample estimate, the higher the relative standard error. The small sample size of Indigenous persons results in estimates of labour force characteristics which are considerably less precise and less stable than comparable aggregate estimates for non-Indigenous persons. This is reflected in the relatively high standard errors for the survey estimates derived for Indigenous persons.

13 Very small estimates are subject to such high standard errors, relative to the size of the estimate, as to detract seriously from their value for most reasonable uses. In Tables 1 to 5 (available in Downloads), only estimates with relative standard errors of less than 25%, are considered sufficiently reliable for most purposes. Accordingly, while included in the tables, estimates with relative standard errors of 25% or more are annotated (highlighted by a red triangle), to indicate they are subject to high standard errors and should be used with caution.


14 Proportions and percentages (for example, unemployment rates) formed from the ratio of two estimates are also subject to sampling error. The size of the error depends on the accuracy of both the numerator and the denominator. A formula to approximate the RSE of a proportion or percentage is:

15 This formula is only valid when x is a subset of y.
EXAMPLES OF CALCULATIONS

Level standard errors

16 As an example of the calculation and use of standard errors, consider the estimate of 166,100 Indigenous persons employed in 2010. The standard error for this estimate is 6,600 (see L1 of Table 1, available in Downloads). This indicates that there is a 67% chance that the true value (the number that would have been obtained if the whole population had been included in the survey) is within the range 159,500 to 172,700 (that is, 166,100 + or - 6,600). There is a 95% chance that the true value is in the range 152,900 to 179,300 (that is, 166,100 + or - 13,200).Movement standard errors

17 Standard errors can also be used to interpret the reliability of annual movement estimates. For example, in 2008 there were an estimated 91,200 Indigenous males in employment, decreasing to 89,100 in 2009 (a movement of -2,100). The associated standard error for the movement estimate is 4,900 (see M1 of Table 1, available in Downloads) . This indicates that there is a 67% chance that the true value of the movement is within the range -7,000 to 2,800 (that is, -2,100 + or - 4,900) and there is a 95% chance that the true value is in the range -11,900 to 7,700 (that is, -2,100 + or - 9,800). Differences between estimates

18 Published estimates may also be used to calculate the difference between two survey estimates (numbers or percentages). Such an estimate is subject to sampling error. The sampling error of the difference between two estimates depends on their SEs and the relationship (correlation) between them. An approximate SE of the difference between two estimates (x-y) may be calculated by the following formula:

19 While this formula will only be exact for differences between separate and uncorrelated characteristics or subpopulations, it is expected to provide a good approximation for all differences likely to be of interest in this publication.

20 For example, in 2009, the participation rate of Indigenous males was 62.8%, 14.3 percentage points higher than the rate of 48.5% for Indigenous females. The approximate standard error of the difference between these two estimates can be calculated as follows: