9208.0.55.001 - Survey of Motor Vehicle Use Fitness for Purpose Review: Information Paper, 2004  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 29/11/2004   
   Page tools: Print Print Page Print all pages in this productPrint All  
Contents >> 2.0 Key data issues

2.0 Key data issues


2.1 Authoritative data set

Description of the data issue (scope)

Feedback from the TSUG meeting identified a need for a single set of figures (at national and state level) that is accepted as providing reliable annual estimates of vehicle kilometres travelled, by broad vehicle type. Users consider the primary importance of the SMVU is to publish basic data time series which is broadly consistent with other key trend data (GDP, employment etc). External users want data sets to be published at regular intervals, with a focus on supporting time series analysis and comparisons. External users want consultative input in determining the frequency with which individual data items are collected/published.

Questions for stakeholders:

  • Is the current SMVU providing data on the data items of interest to stakeholders prioritised specific requirements? If not, what data items are not of use/benefit to these prioritised needs?
  • What additional SMVU data items would stakeholders benefit from and how do stakeholders intend to use these additional data items?
  • Do stakeholders analyse/combine SMVU data with other data sets? What are the associated difficulties with combining SMVU and other data sources that users would like the ABS to consider addressing?


Potential options for addressing shortcomings
  • Options will be identified when issues have been clarified by stakeholders as part of the feedback process.


2.2 Measuring level movements in key variables

Description of the data issue (scope)

The SMVU sample allocation is currently designed to minimise both provider load and the standard error of the level estimates of key variables. TSUG members have commented that users require an emphasis on the measurement of annual changes for the key variable of vehicle kilometres travelled by broad vehicle type (passenger vehicles, rigid freight vehicles, articulated freight vehicles) and geographic region (preferably national, state and capital city region).

The current SMVU uses a non-overlapping sample. An overlapping sample is generally required to minimise the standard errors associated with movements in the estimate between years. Redesigning the SMVU sample to improve the accuracy of estimate movements between years can be done, however it may affect the accuracy of the level estimates.

Questions for stakeholders:
  • How important is it to users that the SMVU be designed with the purpose of controlling movement RSEs between years?
  • Would 'movement controlled' SMVU data be used in a manner that the current data is not being used?
  • There may be a trade-off (in terms of accuracy) between controlling for movements compared to level estimates (see discussion below under ('Potential options for addressing the level movement data issue'). What are users preferences in regard to this trade-off?

Potential options for addressing the level movement data issue

Options to address this issue could include:

  • Increasing the sample size of non-overlapping samples to improve the accuracy of level estimates and therefore the accuracy of the associated movement between years. Preliminary investigations by ABS Methodology Division show that this option produces a better accuracy outcome for both time periods, compared to an overlapping sample.
  • Produce overlapping samples for the SMVU. This option may require increased provider burden, however 'familiar' respondents may provide more consistent data over time. A trade off may exist regarding controlling for level and movement RSEs and this depends on what type of sample design is used. If the current sample design is used with sample overlap between consecutive years, then the level RSEs can theoretically remain the same with improvement in movement RSEs. However, it is also theoretically possible to design the sample to provide optimal movement estimates, yet result in increased RSEs for the level estimates.

These two options would need detailed investigation before we can determine which is feasible and which might be the most appropriate way to improve movement RSEs. Selecting the same respondents in consecutive years can affect the response rate which in turn affects the data accuracy, so this would also need to be considered.


2.3 Data volatility

Description of the data issue (scope)

Some users are unhappy with the current sample size for SMVU - approximately 16,400 vehicles in the 2003 survey. For example, the stability of the kilometres travelled time series has been criticised by these users, who highlight the correlation between increased variability (measured by the estimated standard errors) and smaller sample sizes. If necessary to achieve improved accuracy, users would accept some of the SMVU data items to be published less frequently and with data items reduced to a core set. External users want to be consulted for input to the re-specification of the survey's objectives and design.

It should be noted that methodological changes aimed at reducing 'recall bias' have arguably reduced the non-sampling error associated with these estimates. Moreover an internal methodological review in 2002 identified some erroneous aspects of the SMVU survey frame and new vehicle provision methodology, both of which have now been rectified.

Questions for stakeholders:
  • Is the data quality concern smoothness of the series rather than standard error and sample size?
  • Is the current SMVU providing data accuracy (standard errors) on the data items of interest to your specific organisation requirements? The 2003 SMVU RSEs are provided in Appendix D for reference. Please comment on which RSEs are too high for user requirements. The ABS is interested in the minimum RSE requirements necessary for data to be of value.
  • Do users want the ABS to produce a technical paper discussing the time series and summarising the changes that have occurred over time? What time periods or issues would users require information on?


Potential options for addressing data quality issues

Options to address this issue could include:

  • Develop suggestions or techniques for smoothing the current SMVU level estimates, allowing users to incorporate trend data into models. The ABS needs to investigate the conceptual issues, feasibility and cost of this option.
  • The ABS will consider further options available to overcome the issues raised, in response to external stakeholder feedback.

Further questions for state transport departments and motor vehicle registration authorities:

The inclusion of odometer readings on the administrative data sets provided to the ABS would improve the accuracy of SMVU estimates. Can state stakeholders work with their counterpart motor vehicle registration authorities in identifying the barriers precluding the inclusion of vehicle odometer readings on the administrative data sets provided to the ABS? How can the respective organisations overcome the constraints of:
  • Costs to the organisation of including this data?
  • Likely developmental lead time?
  • Data quality issues?

A dimension of data quality is constrained by the consistency of the survey frames the SMVU samples are selected from. The state motor vehicle registration authorities do not currently apply consistent classificatory standards to the stratification variable (vehicle type) used by the SMVU. This introduces an element of non-sampling error into SMVU estimates. Can state stakeholders work with their counterpart motor vehicle registration authorities in identifying a strategy for the adoption of common standards (used by all motor vehicle registration authorities) to be applied to the state vehicle registers?


2.4 Geographic patterns of activity

Description of the data issue (scope)

In describing transport activities, geographic detail is required by users below the level of national and state aggregates. Data for individual coherent physical regions (eg for each 'capital city and surrounding catchment'), would be valuable. This data requirement would require the use of geographic area classifications. How would the ABS definition of metro area compare with different users of this state specific data? Data at Statistical Division level would be useful to users but of lower priority for inclusion in nationally collected data sets. If modelling is to be used in order to impute estimates for low-level geographies, users would need some raw data to be available for the low level classifications being investigated.

External data users have a need for some small area transport information to be available - including data that supports the national level modelling work that describes transport characteristics at geographic regions between local and capital city regions. TSUG have acknowledged that the primary role of the ABS is to provide stable national and state SMVU data. TSUG have also acknowledged that it is not the role of the ABS to provide intra-regional transport data or transport corridor data. The ABS may have a role in assisting the modelling work for some sub-state data by providing some limited regional data. To this end, the ABS would need to know which region(s) to collect data for, how to define the region, what frequency of data availability would be required and what the associated data quality requirements would be. These questions need to be determined by stakeholder input.


Questions for stakeholders:

  • Is the current data accuracy (eg standard errors) of sufficient standard at the state estimates level? See Appendix D for the SMVU 2003 RSEs.
  • Is sub state data more important to users than improved data quality at the state level?
  • How substantial is the data requirement for sub state data?
  • What type of sub state data is of greatest importance, metropolitan or regional?
  • What would the minimum data quality requirements (eg standard errors) be for sub state data?
  • What frequency of data availability would be required for sub state data?
  • If it is feasible to produce sub state data, what criteria should the ABS use, to decide on which particular regions to collect?


Potential options for addressing geographic issues
  • Options will be identified when issues have been clarified by stakeholders as part of the feedback process.


2.5 Impact of unregistered vehicles

Description of the data issue (scope)

Users have raised concerns that unregistered vehicles make a contribution to motor vehicle usage in Australia. These vehicles are not covered by the survey as they are not available for selection from the frame. The concerns centre around what contribution these unregistered vehicles make to total fuel consumption and general motor vehicle use. The extent of contribution of unregistered vehicles is not known. There have been reports that unregistered vehicles illegally used on Queensland roads, could make up 4.5% of vehicles used on the road, whereas in NSW it is estimated to be around 2%. The extent of legal use of unregistered vehicles in industry is also not known, and we do not know if it is stable between years.

Does the proportion of unregistered vehicles provide valid explanation for the criticisms of the estimates of fuel use and perceptions of underestimating total kilometres travelled? To answer this, we need to know if the proportion of unregistered to registered vehicles is the same across all states and stable over time. Volatility in the number of unregistered vehicles may well be a causal factor in some of the volatility of the SMVU estimates. Some investigation into the unregistered vehicle issue is recommended.

Questions for stakeholders:

Anecdotal evidence from Victoria suggests the number of illegally unregistered vehicles (actually in use) in Victoria are between 5% and 10%.
  • Do any of the SMVU stakeholders or state motor vehicle registration authorities have any data/evidence on the number and type of unregistered vehicles that are illegally used on roads?
  • Are stakeholders aware of estimates relating to the legal use of unregistered vehicles in industry (eg Agriculture, Mining)?

Potential options to address the issue of unregistered vehicles

Options to address this issue could include:

  • Undertake a separate analysis to identify the characteristics of unregistered vehicle use (using data from police records, state motor vehicle registration authorities/fines/convictions, mining and agricultural industry groups), the results of which can be implemented by users independently of published data.
  • Assume that the number of unregistered vehicles that are actually in use, is constant and/or insignificant.


2.6 Quality of fuel use estimates

Description of the data issue (scope)

SMVU fuel use estimates are used in a number of different applications, and have significant ramifications for governmental income and expenditures. In order to forecast income from items such as fuel taxes, it is important that Commonwealth and State governments have data describing characteristics of the fuel use to which taxes and charges are applied. Each year the National Road Transport Commission (NRTC) sets heavy vehicle levies, on fuel and on registrations, for the purpose of covering the estimated damage these vehicles do to roads. Greenhouse gas emission (GHG) estimates are modelled on the basis of fuel use estimates. These uses represent a substantial ongoing need for SMVU data.

Quality of the SMVU fuel estimates was not specifically mentioned at the TSUG meeting, however the ABS is aware of the criticism that differences exist between SMVU's fuel use figures and those from fuel excise data sources.

Questions for stakeholders:
  • In the absence of SMVU fuel estimates, could users needs be met by fuel excise data?
  • What are the respective advantages/disadvantages of using SMVU fuel estimates and fuel excise data?
  • Would users prefer the ABS investigate the possibility of tailoring the collection of fuel data toward complementing the availability of excise data (addressing any identified data gaps associated with fuel excise data), rather than producing independent fuel estimates?
  • Do users have suggestions on how the ABS may tailor the SMVU collection to complement fuel excise data?
  • The SMVU scope excludes some types of registered vehicles as well as unregistered vehicles. Therefore it can not provide a measure of total fuel consumption in Australia. Do users consider the fuel consumption of out of scope vehicles is stable over time or varies?


Potential options for addressing shortcomings

The ABS needs to investigate the conceptual issues, feasibility and cost of improving fuel use estimates associated with the following potential options:

  • Dropping the SMVU fuel estimates collection, allowing users to rely on available fuel excise data.
  • Relying on fuel excise data for the total estimate of fuel use and designing the SMVU to provide accurate fuel use proration factors by vehicle type (accurate splits of the national fuel excise total by vehicle type etc).
  • A longer term solution focuses on the suite of possibilities relating to improved accuracy from incorporation of odometer readings and ABN on survey frames (these issues are discussed separately in 3.2 'Capturing transport activity by industry').
  • Research the impact of out of scope vehicles and whether their associated fuel use is stable over time. The result of such an investigation may be to recommend attempting to include some unregistered vehicles within the SMVU scope.
  • Develop a different strategy for the collection of fuel data, thus looking to reduce the non sampling error associated with SMVU fuel use estimates.



Previous PageNext Page