TECHNICAL NOTE 1: DATA QUALITY
DATA QUALITY
When interpreting the results of a survey it is important to take into account factors that may affect the reliability of estimates. Such factors can be classified as either sampling error or non-sampling error.
SAMPLING ERROR
Estimates in this publication are based on information collected for a sample of registered motor vehicles, rather than a full enumeration, and are therefore subject to sampling error. They may differ from the figures that would have been produced if the information had been obtained for all registered motor vehicles. Examples of the sampling error for selected estimates from the Survey of Motor Vehicle Use (SMVU) for the 12 months ended 31 October 2000 and 2001 are included below. The sampling error associated with any estimate can be calculated from the sample results. One measure of sampling error is given by the standard error, which indicates the extent to which an estimate might have varied by chance because only a sample of vehicles was included. There are about two chances in three that a sample estimate will differ by less than one standard error from the figure that would have been obtained if all vehicles had been included, and about 19 chances in 20 that the difference will be less than two standard errors.
Another measure of sampling variability is the relative standard error (RSE) which is obtained by expressing the standard error as a percentage of the estimate to which it refers. The RSE is a useful measure in that it provides an immediate indication of the percentage error likely to have occurred due to sampling. In this publication, only estimates with a RSE of less than 25% are considered sufficiently reliable for most purposes. Estimates with a RSE between 25% and 50% are preceded by a single asterisk (*) and should be used with caution while those with an RSE of greater than 50% are preceded by two asterisks (**) and are considered too unreliable for general use.
All tables in this publication contain estimates from the 2000 and 2001 SMVUs. The SMVU is not designed to minimise the standard errors of the movements between reference periods so care should be taken in drawing inferences from changes in data over these two years.
NON-SAMPLING ERROR
Non-sampling error covers the range of errors that are not caused by sampling and can occur in any statistical collection whether it is based on full enumeration or a sample. For example, non-sampling error can occur because of non-response to the statistical collection, errors in reporting by providers, definition or classification difficulties, errors in transcribing and processing data and under-coverage of the frame from which the sample was selected. If these errors are systematic (not random) then the survey results will be distorted in one direction and therefore unrepresentative of the target population. Systematic errors are called bias.
Two steps undertaken to help minimise non-sampling error are pre-advice and the reduction in the reporting of rounded data. The pre-advice methodology involves vehicle owners receiving early advice about their inclusion in the survey. This encourages a higher degree of record keeping. In addition, the reporting of odometer readings taken at the start and end of the survey periods (approximately three months apart) provide reliable estimates of total distance travelled without a recall bias.
The second step is the reduction in the reporting of rounded data for total distance travelled. Such rounding could cause significant errors, especially with the prevalence of certain distances which could be seen as arbitrary guesses on the part of the provider. Where rounding is identified, providers are contacted and the estimate of their total distance travelled is queried. Distances considered to be rounded are every 1,000 kilometres in the range 1,000 kilometres up to 10,000 kilometres and every 5,000 kilometres for distances over 10,000 kilometres.
Response and non-response
A potentially important factor relating to non-sampling error is the response rate achieved. Responses were received from 80% of selections for both 2000 and 2001 SMVU. When vehicles found to be deregistered or out of scope are removed, the live response rate for both the 2000 and 2001 SMVU is 79%.
The ABS makes all reasonable efforts to maximise response rates. Where appropriate, mail reminders and telephone follow-up are used to attempt to contact non-responding vehicle owners.
A large non-response increases the potential for non-response bias, which occurs if the usage patterns of the non-responding vehicles differ significantly from those of the responding vehicles. For the SMVU, it is assumed that the characteristics of non-responding vehicles including the proportion of deregistered, out of scope and nil use vehicles are the same as for responding vehicles.
RESPONSE AND NON-RESPONSE, By category
(a) Includes deregistration, out of scope and duplicates.
(b) Includes responses that were unusable because of unresolved queries or where the vehicle was sold during the reference quarter and the reported data covered less than 14 days; and non-response where no listing could be found to enable contact by telephone, owner contacted by telephone but response still not secured and refusals.
Imputation
The need for imputation of unfilled items on the returned questionnaires, as for previous surveys, remained quite high. Imputation is the process whereby a value is generated for missing data items by averaging the responses for similar vehicles which were operating for the reference period. Of the questionnaires returned for 2000, 14% of those reporting some vehicle use needed imputation of one or more items apart from the average rate of fuel consumption. The imputation for average rate of fuel consumption for 2000 was 25%. For the 2001 questionnaires the imputation rates were 15% and 24% respectively.
Adjustments
The SMVU measures the use of all vehicles registered during the reference year. Because selections are taken from vehicles registered some time before the beginning of each collection period, adjustments and additional selections from New Motor Vehicle Registrations are made to account for the change in size of the registered motor vehicle fleet since the population frame was created. This involved two categories:
- re-registrations-older vehicles that are returning to the registered vehicle fleet after a period of deregistration
- new motor vehicles-vehicles which have not been previously registered.
These activities occur continuously and the adjustments are made to account for the registrations that are estimated to have been added to the registered vehicle fleet between the population frame date and the reference period.
Refer to Technical Note 2: Methodological Review for details of changes made as a result of the review.
Fuel estimates - petrol
Introduction of lead replacement petrol (LRP) occurred progressively over the 2001 SMVU reference period. As LRP was not identified as a separate category on the collection form it is not known whether data providers categorised LRP as leaded or unleaded petrol. However, the large reduction in leaded petrol suggests that a significant proportion of providers did classify LRP as unleaded. For this reason caution should be taken in interpreting the ratio of leaded to unleaded petrol in 2001.
Users should contact the ABS if they have any queries on the quality and reliability of estimates for particular purposes.
Print Page
Print All
