9210.0.55.001 - Survey of Motor Vehicle Use: Data Cubes, Australia, 12 months ended 31 October 2001  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 24/04/2003   
   Page tools: Print Print Page Print all pages in this productPrint All

TECHNICAL NOTE 2: METHODOLOGICAL REVIEW


INTRODUCTION

A review of the methodology used for the Survey of Motor Vehicle Use (SMVU) was undertaken in 2002 to address data quality issues raised in relation to previously published data. This review identified deficiencies in the SMVU population frame which resulted in the selection of a sample that was not representative of the registered vehicle population. This deficiency has been rectified for the selection of the sample for SMVU 2003.

The review identified some minor errors in the adjustments used to account for re-registration. These errors have now been rectified. The review also investigated new vehicle provision calculations. While no errors were identified a number of options to improve these calculations were investigated and implemented.

This Technical Note specifically outlines the investigations that led to the identification of the frame deficiency and the post-stratification technique used to correct it.

    FRAME PROBLEM

    To ensure the SMVU sample was representative of the population, random selection was used within each stratum. For the SMVU, the random selection process allocated a random number to each unit on the frame. To select the sample, the frame was sorted by random number and a start point was randomly selected. A number of units were selected in order, depending on the number of selection units required for a particular stratum.

    An investigation of the SMVU frame revealed a large number of units which had duplicate random numbers. Duplicate random numbers will not produce a bias in a sample as long as the duplicates contain a random assortment of units.

    The SMVU frame investigation showed however, that the characteristics of certain variables differed between those units with unique random numbers and those with duplicated random numbers. This was particularly the case with Year of Manufacture. Therefore, the distribution of these variables within the resulting sample was dependant on whether the random start and the units selected incorporated duplicate random numbers. All estimates produced from samples selected under this scenario would contain bias, with the direction of this bias dependant on the inclusion of duplicates.

    POST-STRATIFICATION

    The collection of SMVU data for 2000 and 2001 had already been completed before the concerns with the frame were identified. To correct for the unrepresentative sample, a process of post-stratification was used.

    Post-stratification is a method of stratifying a sample after the responses have been received. It is used to improve the quality of results through stratifying by variables that were not used at the time of sample design.

    In the case of SMVU the frame investigation identified six variables to be used in the post-stratification. These variables were State, Vehicle type, Year of Manufacture, Body Code, Fuel Class and Number of Cylinders. Once post-stratification was applied to the SMVU data, the weights of each unit were adjusted based on the particular post-stratum of that unit to realign sample totals to be representative of population totals. Both 2000 and 2001 were post-stratified independently and the post-stratification will vary from one year to the next.

    Caution needs to be taken in making comparisons between 2000 and 2001. Comparisons at the broad level are more reliable than those at the detailed level.

    Previously published 1998 and 1999 data will be re-estimated using post-stratification     and published with post-stratified 2002 SMVU data in late 2003.

    IMPACT

    The impact of post-stratification on the estimates for the main data items is summarised in the following table. The size of the change varies by data item.
    IMPACT OF METHODOLOGICAL REVIEW ON SMVU DATA, Australia
    Before review
    After review
    % change
    2000
    Total kilometres travelled (million)
    180,782
    184,593
    2.11
    Total tonne-kilometres travelled (million)
    128,702
    134,378
    4.41
    Total fuel (million litres)
    24,926
    25,853
    3.72
    2001
    Total kilometres travelled (million)
    187,819
    190,152
    1.24
    Total tonne-kilometres travelled (million)
    132,756
    132,422
    -0.25
    Total fuel (million litres)
    25,931
    25,948
    0.07

    It is important to understand that the percentage change before and after the review can vary significantly between state and vehicle type. Therefore, the percentage change figures in the above table for all vehicles at the national level cannot be used at the state or vehicle type level to calculate the changes due to post-stratification.

    For 2000 and 2001 SMVU data care should be taken in drawing inferences from changes in data over these two years as movements may be subject to high relative standard errors. Therefore the resulting estimates of movements may not be considered statistically significant. There is also potential for increased volatility in the estimates due to the changes that have been implemented as a result of the methodological review.

    Users should contact the ABS if they have any queries on the methodological review.