9208.0.55.003 - Information Paper: Survey of Motor Vehicle Use Fitness for Purpose Review, Jan 2005
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 18/02/2005
Page tools:
 Print Page
 Print All
| |||
2.0 Data issues this review will address 1. Issues relating to what the SMVU is measuring, eg. vehicle kms travelled (VKT) and tonnes kms travelled (TKM). 2. Issues relating to how the data is dissagreggated, eg. the vehicle type classification used and the level of geographic dissaggregation available. 3. Issues associated with the quality of the published survey estimates, eg. sampling error and non-sampling error associated with the estimates.
A. Description of the data issue B. Links to policy and decision making C. Suitability of current SMVU data D. Recommendations
Feedback from the July 2003 TSUG meeting and the SMVU review consultation process identified that users require reliable estimates by vehicle type, at the national, state and area of operation level, for the following key estimates: A. Kilometres travelled. B. Tonne kilometres travelled, by broad commodity type, by road freight vehicles. C. Fuel consumption rates (L/100km). Tonnes kilometres data by commodity and by area of operation is not currently available from the SMVU. Data on commodity type and area of operation information are collected separately and a sound methodology for publishing a combination of the two is yet to be established.
Suitability of current SMVU data Most users are content with the current data items available and the level of dissaggregation of SMVU data. The type of additional data that would add value to the SMVU as an authoritative dataset and allow for enhanced decision making is:
Regarding the 'reliability' of the SMVU data, as noted in the description of this issue, there are concerns about some disparities between SMVU data and some other industry transport measures.
Regarding the additional data items requested:
2.2 Data volatility Description of the data issue During the phase 1 stakeholder consultation process, some users expressed dissatisfaction with the current sample size for SMVU - approximately 16,400 vehicles in the 2003 survey. For example, the stability of the VKT time series has been criticised by these users, who highlight the correlation between increased variability (measured by the estimated standard errors) and smaller sample sizes. Users would accept some of the SMVU data items to be published less frequently and with data items reduced to a core set, if necessary to achieve significantly improved accuracy. The opinions of key external stakeholders were sought on aspects of data frequency and accuracy, the objective being improvement of overall data quality. It should be noted that methodological changes aimed at reducing 'recall bias' have arguably reduced the non-sampling error associated with these estimates. Moreover an internal methodological review in 2002 identified some erroneous aspects of the SMVU survey frame and new vehicle provision methodology, both of which have now been rectified. A methodological review into optimal survey stratification was conducted in 2003 and the ABS anticipates some minor quality benefits to be identifiable with the release of the 2004 SMVU. Users would find useful a technical paper, discussing the time series and summarising the changes that have occurred over time, particularly from the period 1995 onwards. Stakeholders did point out that such a paper would not be the panacea for their data quality concerns, increased sample size producing lower Relative Standard Errors (RSEs) was considered paramount. Stakeholders did acknowledge that any potential sample increase may be at the expense of data frequency (see discussion in section 2.5 'Data frequency'). Suitability of current SMVU data Stakeholders have some useability issues on the key SMVU estimates at all levels of dissaggregation. Some users (smaller jurisdictions) stated that they wanted to see standard error consistency across the states/territories. The consistent message relating to RSE requirements on level estimates are as follows:
Links to policy and decision making The main use of SMVU key aggregates are for modelling and validation purposes. See also the discussion on Links to policy and decision making in section 2.3 'Measuring level movements'. Recommendations
ABS to investigate publishing a technical paper which discusses the time series aspects of the SMVU and the changes that have occurred over time, particularly from the 1995 survey onwards. These include changes to the survey methodology, sample size, systems, processes and questions. 2.3 Measuring level movements in key variables Description of the data issue The SMVU sample allocation is currently designed to minimise both provider load and the standard error of the level estimates of key variables. TSUG members have commented that users require an emphasis on the measurement of annual changes for the key variable of vehicle kilometres travelled by broad vehicle type (passenger vehicles, rigid trucks, articulated trucks, etc.) and geographic region (preferably national, state and capital city region). Acknowledging the resource constraint faced by the ABS, key stakeholders generally stated a preference for enhanced accuracy of level estimates over enhanced movement accuracy. Some of the smaller jurisdictions expressed a preference for controlling movement estimates due to the concern that movements between years may be due to sampling variability rather than real changes (see discussion in section 2.2 'Data volatility'). The current SMVU uses a non-overlapping sample. An overlapping sample is generally required to minimise the standard errors associated with movements in the estimates between years. It may be possible to introduce a level of overlap between SMVU samples to improve the accuracy of estimated movements between years. Users indicated that movement controlled estimates would not be used any differently to the way in which the current SMVU estimates are used. Links to policy and decision making Stakeholders expressed a preference for accurate SMVU level estimates where the data is used as input for transport modelling purposes. In instances where the SMVU is used for validation purposes (state/territory specific pre-existing models or annual transport surveys) users have expressed a desire for RSE controlled movement estimates. The NTC uses SMVU data to identify variations between vehicle and geographic usage associated with different parts of the road network. For example, it is important to identify changes in the proportion of road use at the state/territory level and also changes in the proportion of road use between the sub-state urban/rural areas of operation. Changes in the proportion of road use between different vehicle types are important for assessing regulatory initiatives. These data needs are best achieved via enhancements to level estimate data quality. Suitability of current SMVU data Some stakeholders have useability issues relating to the movement of some key SMVU estimates. Users generally have a preference for enhanced accuracy of level estimates.
This option will require detailed investigation to determine feasible options on what might be the best way to reduce the RSEs of movements between years RSEs. Selecting the same respondents in consecutive years can affect the response rate, which in turn affects the data accuracy, and would also need to be considered. 2.4 Geographic patterns of activity Description of the data issue In modelling transport activity and related impacts, users frequently require data at a level of detail below the national and state/territory level. In particular, data at the level of capital city and surrounding population catchment would be valuable. Such data would require the use of geographic area classifications in the SMVU. Data at the Statistical Division level would be of use to users, but is lower priority for inclusion in nationally collected data sets. If modelling is to be used to impute estimates for finer level geographic areas, users would need some raw data upon which to base these finer level estimates. For many stakeholders sub-state level data is more important than state/territory level data. Sub-state level data is generally used for transport network and environmental modelling, particularly metropolitan versus rest of state/territory data. For example, the majority of the growth in road transport is occurring in the metropolitan areas. Given policy and community interest in issues such as fuel-based pollutant emissions, traffic congestion, urban sprawl, land use patterns, and other population based traffic issues, metropolitan/urban level data is of greater importance than regional data. In addition, fine level regional data on some aspects of vehicle use can be obtained from some state/territory road authority traffic count data. TSUG have acknowledged that the primary role of the ABS is to provide stable national and state/territory SMVU data. TSUG have also acknowledged that it is not the role of the ABS to provide intra-regional transport data or transport corridor data. The ABS may have a role in assisting the modelling work for some sub-state data by providing some limited regional data. The ABS is not currently able to produce regional specific data due to resource constraints, however, investigations into improving the data quality of 'metropolitan', 'urban' and 'rest of state' areas of operation will be undertaken (see later in this section under 'Recommendations'). Links to policy and decision making The preference for enhanced data quality at the state/territory or sub-state level does appear to depend upon the nature of the stakeholder. The majority of stakeholders who provided feedback to the SMVU review, revealed a preference for enhanced data quality at the sub-state level (particularly the greater metropolitan regions). These users have a focus on infrastructure planning and modelling type work and have a preference for sub-state (particularly metropolitan) data. This is especially the case for the larger jurisdictions which are subject to higher traffic congestion in the greater metropolitan regions. For other users, a state/territory dimension of transport activity is more important than sub-state data. These users have a focus on Key Performance Indicators (KPIs) at the state/territory level. Accurate (consistent) state/territory level data is important to these users, since funding for road repair/construction is determined on the basis of state/territory aggregates. Charges upon heavy vehicle use and vehicle registrations are dependant upon this state/territory specific information. It is important for these users that funding and charges are subject to changes in performance and activity rather than being subject to sampling variability. Suitability of current SMVU data The current SMVU generally has the required data items to fulfil the spectrum of user requirements for the given geographic dissaggregation. Some minor exceptions relate to requests for expanded freight commodity detail and data relating to passenger kilometres travelled. Users also require enhanced data accuracy, generally at the greater metropolitan regions. Users have the expectation that greater data accuracy at the Greater Metropolitan Region (GMR) would correlate to increased accuracy at the state/territory level. The required accuracy for the geographic delineation can be found in section 2.2 'Data volatility'. In some cases State and Territory specific stakeholders have user specific definitions of an optimal GMR, however, the current SMVU geographical definitions are meeting user needs. The definition of GMR, or any decision of which particular sub-state region to collect, should be based on population density. Recommendations
ABS to investigate sample size ramifications for improving data accuracy at the sub-state level (the results of these investigations will link into those proposed investigations within the recommendations discussion in section 2.5 'Data frequency'). The ramifications for provider load and ABS survey processing costs will need to be assessed. Any final decision on sample size will also be considered in light of the investigation into data frequency (as mentioned in section 2.5 'Data frequency'. It may also be possible to add a sub-state dimension to the survey stratification (using the metropolitan - rest of state breakdown of interest to users) to improve sub-state estimates. Note that it may be difficult to meet the user requirements for the specified RSEs for the given dissaggregations outlined in section 2.2 'Data volatility'. Note that the data volatility issue recommendations will address possible improvements to the survey data at the state/territory level. Significant improvements to measured standard errors at the sub-state level may be difficult to achieve given the SMVU budget constraints. 2.5 Data frequency Description of the data issue Data frequency is discussed here in terms of modifying the time lapse between the collection of data for specific data items, in order to re-direct resources toward other survey improvements. External users want datasets to be published at regular intervals, with a focus on supporting time series analysis and the ability to compare with other available data. It is important that data series be up-to-date, so that policy work can be based on current information. It might be cost effective to create a 'data gap' by collecting some data items less frequently, but with a larger sample size when it is collected, allowing for increased accuracy. Some users need annual data, therefore it may be prudent to collect selected data items annually at national and state/territory levels, (eg vehicle kilometres travelled, fuel use), and to collect data on other items less frequently (eg data breakdown by vehicle type or freight data by commodity carried). Analysts may then use the annually supplied data to model annual estimates for data that is available less frequently. This approach would require a transparent schedule for availability of the data gaps. TSUG participants recognise that there is high priority in having data that is accurate and reliable - even if cost considerations mean that the frequency of published data must be reduced. Key stakeholders were provided with the opportunity to state their preferences in terms of data item frequency. Most stakeholders indicated that their preference was for key SMVU estimates to be published annually and with increased accuracy. All stakeholders acknowledged the budget constraints imposed on the SMVU and the consequent trade-off between increased accuracy and reduced data frequency. Some stakeholders are encouraging the ABS to choose the trade-off in favour of increased accuracy. Others are generally supportive of this trade-off, if the review investigations are able to demonstrate that there would be significant improvements to accuracy. Based on stability of the time series and user data requirements, certain data items are more suitable candidates to be published less frequently than others. Average fuel consumption rates and passenger vehicle data are a possibility for publication less frequently, but with greater accuracy. The trade-off for freight vehicles may be more problematic due to the desire to capture the changes in transport activity associated with changes in overall economic activity. The ABS will investigate the time series stability of specific data items and the likely user impacts, prior to identifying potential data items for this trade-off. The ABS will not consider altering the frequency of any key data items without being able to demonstrate a significant improvement in accuracy. Biennial data can be incorporated within transport models for data items that are generally relatively stable over a two year period. Links to policy and decision making As discussed previously in section 2.1, some users stated a preference for annual data, due to reasons of 'policy sensitivity' and to capture structural changes that are occurring in the transport industry (such as supply chain management). For some stakeholders this policy and industry structure focus is based on regional areas and for others the policy emphasis is on capturing changes at the GMR. Suitability of current SMVU data The consistent message from stakeholders is the current SMVU is producing the right data items, frequency and level of detail, but enhanced accuracy is required. As stated above, stakeholders are prepared to trade-off significant improvements in accuracy for less frequent data availability for certain data items. Recommendations
Scenario 1. No change to the SMVU sample size or frequency of data availability. Scenario 2. Maintaining current sample size (over a two year period approximately 34,000) devoting the entire 17,000 sample to freight vehicles in the first year and the other 17,000 to passenger and other vehicles in the second year. By focusing the sample increase (for the vehicle type) at the sub-state level, the intention is to produce estimates with improved accuracy, stability and reliability at the state/territory and the sub-state level. Scenario 3. Combination of scenarios one and two above. Maintaining current sample size of approximately 34,000 over a two year period, devoting the majority of the sample to freight vehicles in year one (say 12,000 vehicles) to produce enhanced accuracy at all levels of aggregation. The remaining 5,000 sample units could be devoted to providing an accurate national estimate only for passenger and other vehicles during their 'off' year. Passenger and other vehicles would be sampled in year two (with a sample size of say 12,000 vehicles), producing enhanced accuracy at all levels of aggregation. The remaining 5,000 sample units could be devoted to providing an accurate national estimate only for freight vehicles during their 'off' year. Other potential solutions will also be investigated, ranging from annual to triennial data frequency options.
2.6 Quality of fuel use estimates
It does appear possible that the data need for fuel use estimates may be met by non SMVU sources. Total fuel sales data, producing accurate totals by fuel type, has already been mentioned above. The Australian Greenhouse Office (AGO) provides fuel consumption rate information for new vehicles which are obtained from the vehicle manufacturer in accordance with Australian Standards testing. Older vehicles are not covered by the AGO data, yet it would be feasible to produce a model which tracks changes in the consumption rate with vehicle age, this data may already be available. It is possible that SMVU fuel use data is used as an input to these other alternative fuel use sources, this will need to be investigated. Vehicle counts by vehicle age and type are available from the Motor Vehicle Census. Therefore it appears highly feasible that total fuel use estimates by vehicle type and age can be determined by non SMVU data. This requires further investigation. In addition to fuel use data requirements already discussed, the NTC requires data on where fuel is consumed. It does appear that the SMVU is the only source of such data. The relationship between where fuel is sold and where it is consumed would need to be investigated to identify the critical nature of this data gap, should this review recommend not continuing with fuel use estimates. The other aspect which will need to be considered is the difference, if any, between fuel consumption rates as determined in testing conditions and fuel consumption incorporating different driver behaviour, traffic conditions etc. Links to policy and decision making As above. Suitability of current SMVU data Stakeholders have expressed concern that the SMVU fuel use estimates are consistently lower than other data sources such as total fuel sales data (in the order of 10%-15%). Recommendations
This investigation is closely related to the 'out of scope' and unregistered vehicle use investigation (see section 2.7 'Impact of unregistered vehicles and out of scope vehicles').
The results of this investigation could establish any of the following:
Any identified provider load and survey processing savings could then be re-allocated to improving other key SMVU estimates.
2.7 Impact of unregistered vehicles and out of scope vehicles Description of the data issue Users have raised concerns that unregistered vehicles contribute to total motor vehicle use in Australia, but that these vehicles are not covered by the SMVU as they are not available for selection in the frame. The extent of unregistered vehicle use is unclear. Evidence from Queensland has estimated unregistered vehicles to be around 4.5 per cent of vehicles on the road, whereas in New South Wales there is evidence that unregistered vehicles are around 2 per cent of vehicles on the road. The extent of legal use of unregistered vehicles by industry is also not known, and variability in use of such vehicles is also unclear. The questions that arise in the context of the SMVU are:
To answer these questions it is necessary to understand whether the proportion of unregistered to registered vehicles is the same across all state/territories and stable over time. An investigation into the number and use of unregistered vehicles is recommended. The evidence gathered on unregistered vehicle use during the consultative phase of this review was largely anecdotal. It is likely that some raw data exists within State and Territory MVRs, Police departments, transport portfolios and road agencies. A number of partial investigations have also been conducted in recent years which are vehicle or industry specific, however, this work is yet to be brought together in a complete investigation of unregistered vehicle use. The BTRE estimates VKT by long-term unregistered vehicles to be around 2-3 per cent of national VKT, (and that the actual value would be unlikely to fall outside the range 1 to 5 per cent of national VKT). The proportion of cars on the road that are technically unregistered would be higher (with a likely national average of 6-7 per cent), but many of these vehicles will only be 'late payers', and thus will still be within scope of the SMVU sample frame (unless they always pay late and so are not on the MVR records provided to the ABS). Given this partial analysis, it is unlikely that unregistered vehicles provide a complete explanation for some of the criticism that SMVU data has not matched industry expectations. However, users do want the ABS to attempt to quantify the problem rather than to assume the issue is insignificant. With respect to out of scope vehicles, the Australian Bureau of Agricultural and Resource Economics (ABARE) have estimates of unregistered vehicle, fuel and machinery use by industry sector. The Department of Defence will also have estimates of unregistered military vehicle use. Suitability of current SMVU data Some stakeholders agreed that a systematic analysis aimed at identifying the significance and the degree of volatility of unregistered vehicle use is warranted. A complete investigation of the unregistered vehicle use would necessarily include a section on the level and variability of 'out of scope' vehicles. Recommendations
Undertake a separate analysis to identify the characteristics of unregistered vehicle use (using data from police records, State and Territory MVR authorities, mining and agricultural industry groups), the results of which could be published as a 'feature article' within the SMVU and implemented by users independently of published data. 2.8 Vehicle classification used by the ABS Description of the data issue The vehicle classification used by the ABS for the SMVU has 27 vehicle and 8 trailer categories. This is aggregated into seven vehicle types in the published survey estimates. Vehicles are differentiated by chassis type and axle configuration. Vehicle type is also a stratification variable in the SMVU. The AustRoads vehicle classification, which was agreed to by all State and Territory road authorities following a review in the early 1990s, has 12 vehicle categories covering passenger vehicles, rigid trucks and articulated trucks. Vehicles are differentiated by the number of axles. The AustRoads classification was designed primarily to collect heavy vehicle road use data via weigh-in-motion axle detectors. The AustRoads vehicle classification doesn't have separate categories for motorcycles or buses and coaches, which are required for the SMVU. Users of transport data would prefer the two vehicle classifications be compatible to allow greater comparison and integration of different data sets produced by different Australian agencies, such as administrative data created by the State and Territory MVRs. The majority of users consulted during phase 1 of the review use the AustRoads vehicle classification. The AustRoads classification is generally used for monitoring and charging purposes, whereas the ABS SMVU classification is used for more detailed modelling work. The NTC is one agency that has to use both classifications, relying on data from the SMVU to set charges for vehicle types defined within the AustRoads classification. Stakeholders generally find the respective classifications satisfactory to their needs. The main requirement is for consistency between different classifications, thereby allowing disparate data sources to be used together. Stakeholders should note that the SMVU has a number of classification variables which may be available at finer levels of detail than can be found in the publication. Data based on these classification variables is available from the ABS via special request. Some minor differences between the classifications are evident. For example, the AustRoads classification does not separate motorcycles from light vehicles or buses and coaches from trucks. The ABS classification classes rigid trucks with attached trailers to rigid truck categories, however, under the AustRoads classification rigid trucks with trailers are classified to AustRoads vehicle classes 6 to 9, which include 3-6 axle articulated trucks. The ABS classification allows the separate identification of passenger and load carrying four-wheel drives (4WD), although this data is not generally published in the SMVU. Some stakeholders use the AustRoads classification for screen line vehicle count data and the ABS classification for differentiating between rigid and articulated trucks. It is highly desirable that the more detailed ABS classification can be collapsed into the AustRoads classification. This could require some adjustments to both the current ABS and AustRoads classifications. A concordance between the respective classifications would then be used by data users to combine data using the different classifications. Users would like the ABS to consider including some of the following vehicle breakdowns in standard publications:
Some users may find data on 4WD vehicles useful, but this was not identified as a major data need. Links to policy and decision making The classification is the fundamental basis upon which data from the SMVU is collected and disseminated. The links to policy therefore include every link to policy and planning referred to in this paper. Suitability of current SMVU data Some stakeholders have useability issues relating to the movement of some key SMVU estimates. Users generally have a preference for enhanced accuracy of level estimates.
Begin concordance mapping work between the ABS vehicle classification and the AustRoads vehicle classification. The aim of this investigation should be to identify how closely the 35-bin ABS classification can aggregate to the AustRoads 12-bin classification and recommend changes that allow for complete integration between the two classifications. (Note that the ABS uses 35-bin classification, however, SMVU results are published at a more aggregated level, ie. 7 vehicle types are published.) 2.9 Question changes on SMVU survey forms Description of the issue Reducing the non-sampling error (and therefore increasing the accuracy) of the SMVU estimates may be achieved by improving the quality of particular questions on the survey form to more accurately collect the data required. The questions outlined below are known to be difficult for some users to answer.
The review stakeholders did not identify any additional survey questions that this review should investigate improving. Links to policy and decision making The policy and decision making uses of the existing data items outlined above are discussed in section 2.1 'Authoritative data set'.
This may involve a type of 'post-enumeration survey' where a sample of data providers are contacted after responding and asked some detailed questions to assess their understanding of specific survey questions. It should be noted that any changes to the survey questions has the potential to affect the continuity of the time series data. Continuity are important to SMVU data users and this will be taken into account when any decisions on improving the questions is made.
|
|||