1001.0 - Annual Report - ABS Annual Report, 2005-06
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 11/10/2006
Page tools:
 Print Page
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Chapter 12 - Quality and timeliness
RELEVANCE The relevance of statistical information reflects the degree to which it meets the real needs of the users of the information. It is concerned with whether the available information sheds light on the issues most important to researchers and policy-makers. The outputs produced, the concepts and classifications used and the scope of the collection can all affect the relevance of the data. A detailed understanding of the users of statistical information and their requirements is a very important part of the statistical process within the ABS, and it has a range of mechanisms to achieve this. The Australian Statistics Advisory Council assists the ABS with this. There are also a range of other groups and mechanisms, as described in chapter 9. For particular surveys, key stakeholders are identified and consulted before and during the development. Further, each survey is regularly evaluated to assess the degree to which it meets user requirements. The gaps between user requirements and statistical output are formally considered by the ABS senior executive each year. Other specialised reviews are also conducted regularly, for example, adequacy of data outputs from monthly retail trade, quarterly business indicators and new capital expenditure. ACCURACY The accuracy of statistical information is the degree to which the information correctly describes the phenomena it was designed to measure. Most statistics produced by the ABS are obtained from a sample of households or businesses. This process results in some uncertainty as to the accuracy of the estimates published. For example, the estimate from the sample may not be the same as would have been obtained if information had been collected from the whole population – this is known as sampling error. There are also other sources of error that potentially cause inaccuracy including the level of non-response, the magnitude of revisions made as additional information is received, and errors from other parts of the collection process (non-sampling error). INFORMATION ABOUT ACCURACY Users will want to use statistical information for different purposes so it is important to make information available to enable users to make their own assessment of quality. Descriptions of accuracy, as well as extensive information on the statistical methods used in collections, are routinely provided in concepts, sources and methods publications, the explanatory notes in publications, and at the Statistical Clearing House through the ABS web site. In addition, major changes to methodology are explained in feature articles or information papers. Some examples include a feature article on a new methodology for deriving counts of Australian exporters in International Trade in Goods and Services, Australia, February 2006 (cat. no. 5368.0), and Information Paper: Improved Methods for Estimating Net Overseas Migration, 2006 (cat. no. 3107.0.55.003). The ABS has made few significant mistakes in the statistics it has released. On the infrequent occasions when substantial processing errors are found, it is ABS policy to publish corrected data as soon as possible. A serious error was discovered in the production of retail trade estimates in May 2005. The ABS commissioned a statistical expert, Mr John Cornish, to independently review the causes leading to the error. His report and the ABS management response were published on the ABS web site in July 2005. The ABS has been implementing a range of the recommendations from this review to help prevent or minimise the risk of a repeat occurrence of similar errors, especially when methodological or system changes are being made. More recently, Producer Price Indexes, Australia, March 2006 (cat. no. 6427.0) and International Trade Price Indexes, Australia, March 2006 (cat. no. 6457.0) had to be reissued due to a processing error that resulted in incorrect exchange rates being used in converting prices expressed in foreign currencies (mainly $US prices for imports) into $A. Again, the ABS was open about the error, and will ensure that processes are examined to ensure that any weaknesses are addressed. NON-SAMPLING ERROR Non-sampling error is a general term that describes all sources of error other than the error introduced by the sampling process. Sampling error can be measured by using the mathematical properties of the selected sample. Non-sampling error is much harder to measure. Some sources of non-sampling error that are most relevant to statistical surveys include: non-response error; errors in identifying and contacting the population of interest for a survey; errors introduced by the questionnaire design, such as misunderstanding the question, inadvertently missing questions, or phrasing questions that predispose a respondent to answer in a particular way; and data capture, processing and coding errors. The ABS minimises the impact of non-sampling errors by use of better practice procedures in questionnaire design, interview procedures, data validation and repair, and processing. Any significant changes to questionnaire wording or data collection methods are carefully trialled and evaluated before they are implemented. SAMPLING ERROR The relative standard error (RSE) is a measure of the sampling error associated with an estimate. The magnitude of standard errors varies between collections and between data items within a collection due to factors such as the responding sample size and the nature of the data item. The RSE is a useful indicator for comparing the accuracy of estimates between surveys. Table 12.1 presents a summary view of the estimated RSEs for key statistics from a number of major ABS surveys. Further detailed information is included with each ABS publication, as well as in the concepts, sources and methods publications released by the ABS. Relative standard errors are affected by the size of the sample used, the sample design used for the survey, and by the underlying variability of the indicator in the population. Sample size influences the level of accuracy that can be attained. For example, RSEs for estimates of Indigenous Australians will generally be higher than RSEs for estimates of all persons, as there are generally fewer Indigenous Australians contributing to the sample. This is shown in Table 12.1 below – the sample size for the estimate from the National Health Survey was 25,900 and the resulting RSE was 1.0%, while the sample size for the National Aboriginal and Torres Strait Islander Health Survey was 10,400 and the RSE was 3.0%. ABS sample designs for business surveys use groups of similar businesses (strata) as the basis of sample selection to improve the efficiency of estimation. Information such as employment size or annual sales can be used in this grouping. Many indicators, such as annual turnover or value of building work done, are closely related to the variables used in stratification, allowing these indicators to be estimated with relatively high accuracy. Other variables, such as capital expenditure or job vacancies, are not as closely related, and so cannot be estimated with the same accuracy. As well as differences between surveys, RSE can also change with time for any given survey. These changes may be due to changes in the way the survey is conducted, for example, changes in the sample size or the method of producing estimates, or may be due to changes in the population being studied, for example, a change in the prevalence of a particular characteristic. An improvement was made in the accuracy of retail trade turnover for 2004–05 (to 0.8% from 0.9% in 2003–04) due to an improved sample design and estimation methodology even though the overall sample size was reduced. For more information, refer to Information paper: Changes to the Retail Trade Series, July 2004 (cat. no. 8501.0.55.002). The RSE for job vacancies is relatively large due to the underlying variability. That is, the number of job vacancies can vary considerably from business to business, and for any business it can vary considerably from month to month, and so a very large sample would be required to measure job vacancies with high precision. In the table below, the estimated RSE for job vacancies for 2004–05 of 5.5% represents a decrease from 6.8% for 2003–04. This was not due to an improvement in survey methodology, but rather due to an increase in the number of vacancies, which has made the error smaller in relative terms. TABLE 12.1: RELATIVE STANDARD ERRORS (RSE) FOR SELECTED INDICATORS (a)
REVISIONS TO DATA One measurable component of statistical accuracy is revisions to data made after initial publication, resulting from additional information becoming available. Revisions are generally measured by their size and frequency over time. Revisions are applied to statistical series to ensure that there is an appropriate balance between accuracy and timeliness in the release of statistics. Revisions could be avoided – but this would mean that either the release of statistics is substantially delayed, or that any inaccuracies subsequently discovered in released statistics are not corrected. The ABS aims to maximise the overall quality of the released statistics by releasing accurate statistics in a timely manner, while subsequently improving the accuracy through revisions as new data becomes available. It is also ABS policy to inform users of any significant revisions and, where appropriate, to revise past time series and advise users accordingly. The tables below provide, for two key series, the mean revision and the mean absolute revision for the past six years. The mean revision shows the percentage difference between the first estimate published, and that estimate one year later, averaged over the four quarters for the year. The mean absolute revision shows the average absolute values of the mean revision. Table 12.2 describes the revisions to quarterly gross domestic product (GDP). In particular, it shows the difference between the first estimate of GDP and that estimate one year later, in terms of the mean revision and the mean absolute revision expressed as percentage points. The figures continue to show that revisions to quarterly GDP in recent years remain relatively small (mean absolute revision). Zero mean revision figures indicate that the revisions to quarterly GDP over the year have been offsetting. Despite the revisions to quarterly GDP being quite small, efforts to further improve the estimates are ongoing. TABLE 12.2: REVISIONS TO GROSS DOMESTIC PRODUCT, PERCENTAGE CHANGE (a)
(b) Figures based on three quarters of GDP data A decreasing trend in the mean absolute revisions to the quarterly current account transactions since 1999-2000 is shown in Table 12.3. The revisions to the current account deficit are expressed in percentage terms, rather than percentage points as is the case with the revisions to GDP. TABLE 12.3: REVISIONS TO QUARTERLY CURRENT ACCOUNT TRANSACTIONS (a)
(b) Figures based on three quarters of the data TIMELINESS The timeliness of statistical information can be measured by the gap between the reference period (the period the data relates to) and the date of release of results. The ABS continues to adhere to pre-announced release dates and make improvements, where possible, to the timeliness achieved. Tables 12.4 and 12.5 presents information on the timeliness for ABS monthly and quarterly tabular data for main economic indicator statistics, and other general releases. Table 12.6 reports on the timeliness of confidentialised unit record files (CURFs). There has been little change to the timeliness of statistical tables in 2005–06, apart from the increase in average number of elapsed days for the release of other quarterly data. The increase is due to the re-issue of small area tourist accommodation data for March 2005 in October 2005. The re-issue affected eight separate products (small area tourist accommodation statistics for eight states and territories), and so contributed eight times to the average. The re-issue was planned as part of expanding the scope of the survey to include additional types of accommodation. The original release of March data reflected the old scope. In October 2005, data for the new scope was released for both March and June quarters. This approach was taken to ensure data quality. For more detail, refer to the description of scope in the explanatory notes of Tourist Accommodation, Australia (ABS cat. no. 8635.0). TABLE 12.4: TIME BETWEEN END OF REFERENCE PERIOD AND RELEASE OF TABULAR DATA (AVERAGE NUMBER OF ELAPSED DAYS) (a)
TABLE 12.5: TIME BETWEEN END OF REFERENCE PERIOD AND RELEASE OF TABULAR DATA FOR SELECTED PUBLICATIONS
The timeliness of release of information depends on a number of factors, including the amount and complexity of information being collected, the source of the data (for example, whether directly collected or sourced from administrative records), and the amount of processing or validation of the information required before release. Labour force statistics are released very quickly after the end of the reference month. Part of the explanation for this is that the data collection is completed before the end of the reference month. Interviews are generally conducted over a two week period commencing on the Monday between the 6th and 12th of each month. The information collected from each survey respondent is relatively small compared to other surveys conducted by the ABS, and an 'any responsible adult' methodology is used to allow one member of a household to respond on behalf of other household members, so that subsequent appointments with specific respondents can be avoided. In addition, Labour Force Survey processes have been enhanced and refined over time so they are now very efficient and a large collection is processed in a relatively short timeframe. This all helps in the very timely release of statistics. Demographic statistics report on Australia's population, components of population growth, and distribution among states and territories. The quarterly changes to population statistics are based on a variety of administrative sources, such as registrations of births and deaths, and passenger cards completed at Australia's borders, and modelled estimates of interstate migration (using information from Medicare card registration address changes, delayed by three months as registration often takes place after the actual move). It takes around five months before estimates can be published due to the time needed to acquire and process the administrative data, particularly with the delay of three months for the Medicare card data. The average number of elapsed days between the end of the reference period and the supply of confidentialised unit record file (CURF) data has improved significantly in recent years. Note that the information for the 2004–05 reference year only includes CURF data made available prior to 1 July 2006. TABLE 12.6: TIME BETWEEN END OF REFERENCE PERIOD AND RELEASE OF CONFIDENTIALISED UNIT RECORD FILE (CURF)
ACCESSIBILITY The accessibility of statistical information refers to the ease with which it can be referenced. This includes the ease with which the existence of information can be ascertained, as well as the suitability of the form or medium through which the information can be accessed. The cost of the information may also be an aspect of accessibility for some users. All statistics on the ABS web site can now be accessed free of charge. The new policy was announced by the Treasurer, The Hon. Peter Costello MP as an ABS centenary tribute to the people of Australia in December 2005. The change means that all publications, spreadsheets and census data on the ABS web site are now available free to any member of the public with Internet access. However, people who require their own paper copies of ABS publications, information on CD-ROM, or information more detailed than that published, will be charged under the ABS pricing policy. Confidentialised unit record files (CURFs) are a product that allow approved researchers with a valid statistical purpose to access individual survey responses. The data files are confidentialised and access is carefully controlled to ensure that no individual or organisation can be identified. The price of CURF access has been reduced from 1 July 2006 to $1,320 per application in order to improve the accessibility of this information. The ABS has also worked to improve the accessibility of CURFs, including through the ABS Remote Access Data LaboratoryTM (RADLTM). For more information see chapter 13.
INTERPRETABILITY The interpretability of statistical information reflects the availability of the supplementary information and metadata necessary to interpret and utilise it appropriately. This information normally covers the availability and clarity of metadata, including concepts, classifications and measures of accuracy. In addition, interpretability includes the appropriate presentation of data such that it aids in the correct interpretation of the data. ABS releases are accompanied by extensive explanatory notes to aid the interpretation of statistical information. There is also a range of material available on the ABS web site detailing the methods, classifications, concepts and standards used by the ABS. During 2005–06, the Australian Consumer Price Index: Concepts, Sources and Methods, 2005 (cat. no. 6461.0) was released for the first time and Labour Statistics: Concepts, Sources and Methods, 2006 (cat. no. 6102.0.55.001) was updated. The ABS is currently working to improve the metadata available for ABS collections. For more information see chapter 15. COHERENCE The coherence of statistical information reflects the degree to which it can be successfully brought together with other statistical information within a broad analytic framework and over time. Coherence encompasses the internal consistency of a collection as well as its comparability both over time and with other data sources. The use of standard concepts, classifications and target populations promotes coherence, as does the use of common methodology across surveys. Coherence of ABS output requires the use of nationally and internationally agreed concepts and classifications. Standard concepts and classifications are not only used extensively within the ABS, but also promoted to other producers of statistical information within Australia. Information on statistical standards, concepts, classifications and methodologies are readily accessible through the ABS web site. For more information refer to the chapter 15. The Statistical Clearing House (SCH) provides approval to conduct surveys that are directed to 50 or more businesses and that are conducted by or on behalf of any Australian Government agency, to ensure that surveys are necessary, well designed and place minimum burden on business respondents. One of the criteria used by the SCH is the coherence of the statistical information that will be produced. In particular, surveys are assessed on their use of standard methodologies, concepts and classifications, their consistency with past or future surveys, and the extent to which outputs can be compared or jointly used with other sources of data. For more information about the SCH see chapter 11. Any changes that may impact on the coherence of ABS statistics are detailed in the explanatory notes that accompany each release. Significant changes may lead to series breaks in time series, or adjustments to past data. An example of a series break is the trend adjustment made in the release of Balance of Payments and International Investment Position, Australia, June quarter 2005 (cat. no. 5302.0) to account for significant increases in the prices of coal and iron ore export commodities during this quarter.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
