5206.0 - Australian National Accounts: National Income, Expenditure and Product, Mar 2003  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 04/06/2003   
   Page tools: Print Print Page Print all pages in this productPrint All
  • About this Release
  • A Quality Assessment of The National Accounts Information System (Feature Article)

Feature Article - A Quality Assessment of The National Accounts Information System


This article was published in Australian National Accounts: National Income, Expenditure and Product (cat. no. 5206.0), March Quarter 2003

SUMMARY

This article describes the International Monetary Fund's (IMF) Data Quality Assessment Framework (DQAF) and broadly assesses the effectiveness of Australia's national accounts statistical program against the DQAF's five dimensions of quality. Overall, Australia's national accounts perform well in all areas of the DQAF with particular strength in the areas of integrity, methodological soundness, serviceability and accuracy and reliability. In the other area of quality, accessibility, the national accounts perform well but improvements can be made as user requirements continue to change.

The ABS is planning to do a comprehensive assessment against the IMF's DQAF for the national accounts. The assessment will be publicly available and is likely to be completed by early 2004.

The development of an ABS data quality framework and associated quality measures for its economic accounts is also planned. In this work the ABS will draw upon the IMF's DQAF, but will also modify and extend aspects of quality to suit local circumstances and user needs. The proposed ABS quality framework and its application to the national accounts statistical program will be the subject of a future article.

BACKGROUND

Work on an approach to assessing data quality was begun in 1997 by the Statistics Department of the IMF. Building on this earlier work, in 2001 the IMF developed its DQAF for assessing the quality of data used for macroeconomic analysis. The DQAF is an integrated and flexible framework that serves as an umbrella for dataset-specific frameworks, including national accounts. The System of National Accounts 1993 (SNA93) is used as the benchmark for definitions, concepts, classifications and valuation methods. Data quality is assessed using a six part structure that spans institutional structures, statistical processes and characteristics of the statistical products, recognising the need for trade-offs among the various aspects of data quality.

Three main uses were envisaged in developing the IMF's DQAF. First, as a tool for guiding countries' efforts to strengthen their statistical systems though the provision of a means of self-assessment which would facilitate the identification of areas for improvement. Second, as a guide for IMF staff in assessing the quality of data provided by member countries and in designing programs of technical assistance and third, to assist data users to assess data quality for their own purposes.

The DQAF has not been designed to be a checklist or audit of statistical practices as it does not cover exhaustively all issues at the detailed level. Further, in developing its assessment framework, the IMF explicitly recognised the need for member countries to take into account their own particular circumstances. The IMF expects that the frameworks will be applied flexibly with the objective of pointing to relevant areas that may need attention so that an action plan and resource requirements can be identified.

The IMF's work on quality frameworks has the strong support of the ABS. The ABS regards quality assessments of its statistics and methods as extremely important, both to determine areas that are deficient so that improvements can be made, and to assist users in their understanding of the statistics so that they can use them more effectively. As the ABS generally complies with the SNA93 benchmark standards underlying the DQAF, the IMF quality framework provides a good foundation on which to build a comprehensive quality assessment system for Australia's national accounts.

THE IMF DATA QUALITY ASSESSMENT FRAMEWORK

The IMF DQAF covers five dimensions of quality and a set of prerequisites for the assessment of data quality. The framework recognises that data quality is intrinsically bound with that of the institution producing it as well as the 'quality of the individual data product'. Thus the DQAF also includes some elements and indicators which, while not constituting a quality dimension, are preconditions for quality. They generally refer to desirable attributes of the agencies operating the statistical system. The following briefly describes the set of prerequisites and the statistical practices associated with each of the five dimensions of quality.

  • Prerequisites of quality—the environment is supportive of statistics, resources are commensurate with the needs of statistical programs, and quality is a cornerstone of statistical work.
  • Integrity—statistical policies and practices are guided by professional principles, statistical policies and practices are transparent, and policies and practices are guided by ethical standards.
  • Methodological soundness—concepts and definitions used are in accord with internationally accepted statistical frameworks; the scope is in accord with internationally accepted standards, guidelines, or good practices; classification and sectorisation systems are in accord with internationally accepted standards, guidelines, or good practices; and flows and stocks are valued and recorded according to internationally accepted standards, guidelines, or good practices.
  • Accuracy and reliability—source data available provide an adequate basis to compile statistics; statistical techniques employed conform with sound statistical procedures; source data are regularly assessed and validated; intermediate results and statistical outputs are regularly assessed and validated; and revisions, as a gauge of reliability, are tracked and mined for the information they may provide.
  • Serviceability—statistics cover relevant information on the subject field; timeliness and periodicity follow internationally accepted dissemination standards; statistics are consistent within the data set, over time, and with other major data sets; and data revisions follow a regular and publicised procedure.
  • Accessibility—statistics are presented in a clear and understandable manner, forms of dissemination are adequate, and statistics are made available on an impartial basis; up-to-date and pertinent metadata are made available; and a prompt and knowledgeable support service is available.

ASSESSMENT OF THE ABS NATIONAL ACCOUNTS PROGRAM AGAINST THE IMF FRAMEWORK

Given the importance of the national accounts for economic policy formulation and for monitoring the impact of those policies, it is important that they are of a high quality. The quality of a particular statistic is determined by its 'fitness' for whatever purposes users have for the statistics.

Because the compilation of the national accounts is a complex task involving the use of many diverse data sources and methods, it is not possible to provide a single, comprehensive measure of the quality of the estimates. Nonetheless, it is possible to gain an insight into their quality by analysing each aspect of quality separately and then bringing them together to obtain an overall picture.

In understanding quality, it is also important to recognise that there are often trade-offs across the various aspects of quality. For example, statistics that are very timely are often less accurate than those that are less timely. Trade-offs can also occur between accuracy and relevance. It is also important to recognise that, in general, the higher the quality of an estimate, the more costly it is to compile it. Ultimately the assessment of whether data are of high quality, i.e. whether they are 'fit for purpose', must lie with the user.

The ABS considers that it observes the statistical practices associated with each of the IMF's quality dimensions. A brief discussion of each dimension follows. The 'prerequisites of quality' and the 'integrity' dimension are only touched on very briefly in this article as they are essentially corporate in nature. Users who would like further information regarding either of these dimensions from an ABS wide perspective should refer to the ABS Annual Report and the ABS Corporate Plan, both of which are available on the ABS web site.

Prerequisites of quality

The legislative framework, organisational structure, planning and quality management practices of a statistical agency all affect the ability of a statistical program to meet user needs efficiently and effectively. The ABS operates within the framework of a basic statistics act and an additional act relating specifically to its statistical program. The principal legislation determining the functions and responsibilities of the ABS are the Australian Bureau of Statistics Act 1975 and the Census and Statistics Act 1905.

The ABS is headed by the Australian Statistician—a statutory office. The Australian Statistician determines which statistics are collected, after full discussion with users and the Australian Statistics Advisory Council, and makes the results widely available. The independent status of the Australian Statistician is specified in law. This helps to ensure the ABS's impartiality and independence from political influence.

The Census and Statistics Act 1905 provides the Statistician with the authority to conduct statistical collections and, when necessary, to direct a person to provide statistical information. The same Act imposes on the ABS obligations to publish and disseminate compilations and analyses of statistical information and to maintain the confidentiality of information collected under the Act.

The Australian Bureau of Statistics Act 1975 requires the Statistician to prepare an annual report on the operations of the ABS for presentation to Parliament. This Act also established the Australian Statistical Advisory Council to assist the ABS fulfil its role. The Council is the key advisory body to the ABS and provides valuable input to the directions and priorities of the ABS work program. The ABS is financed via Parliamentary appropriations for its administration and programs. Its financial statements are audited annually by the Australian National Audit Office. Together these elements provide an environment conducive to the production of high quality national accounts and other statistics.

While working within this sound and established framework, the ABS regularly reviews its organisation and structure to ensure that its operations remain efficient, that resources are focussed on providing 'statistical leadership' and that its program remains well placed to take advantage of emerging opportunities, including making the best use of all available data.

Integrity
    This dimension of quality relates to the need for statistical systems to be based on the principles of objectivity in the collection, compilation and dissemination of data to ensure unbiased statistics which are not subject to confidentiality breaches or premature release. The dimension covers those aspects of the statistical program that relate to policy, transparency and ethical standards.

    The ABS has an excellent reputation, both at home and abroad, for providing a statistical service of quality and integrity. This, in particular, is recognised by those who influence opinion such as parliamentarians, government agency heads, media commentators and the international statistical community. A key reason for this reputation is that the ABS is willing to subject its operations and performance to both internal and external scrutiny. Some of the ways that this is done include:
    • The advertisement of all scheduled release dates for publications up to twelve months in advance. Daily press and media releases inform users of publications being released each day. This information is also available on the ABS web site. Release of all publications is subject to a strict publicly known embargo policy that ensures impartiality, an essential element of integrity.
    • The publication of the ABS three year Forward Work Program demonstrates the open statistical process by describing for each ABS program the resources, outputs, clients and uses of statistical information, and the proposed main developments over the next three years. The Forward Work Program is published in hard copy each year and is also available on the ABS web site.
    • The release of information about statistical standards, frameworks, concepts, sources and methods in a range of information papers and other publications in hard copy or electronically to ensure that the public are informed about changes to statistical processes. Details of major revisions to published data are described in the explanatory notes of the relevant publication.

    Methodological soundness
      This dimension of quality covers the need for the methodological basis for the production of statistics to be firmly based on international standards, guidelines and generally accepted practices. Within this dimension the DQAF highlights five aspects of quality namely, frameworks, concepts and definitions, scope and coverage, sectorisation and classifications. Overall, the Australian national accounts cover these five aspects well.

      The Australian system of national accounts (ASNA) is based on the international standard, the SNA93. Aside from the core national accounts of GDP and income, the ASNA includes both national and sectoral financial accounts, capital accounts, and balance sheets. The ASNA also encompasses input-output tables, multifactor productivity statistics, and accounts for each state and territory. Many Australian national accounts estimates are also disaggregated to provide information about economic assets and activities for sectors, industries, and commodities, and about different types of assets, liabilities, transactions and other economic events. The ultimate scope of the ASNA encompasses the full range of statistics that SNA93 recommends for a complete national accounting system. However, at this stage of its development, the ASNA does not include all of the elements of the SNA framework, although Australia's implementation of SNA93 is extensive relative to most other countries. The ABS is continuing to work towards further improving and extending the range of its statistics.

      The concepts and definitions used in the ASNA generally conform with those set out in SNA93 but some minor variations have been adopted to allow for particular Australian data supply conditions or user requirements. Such variations are relatively minor and are noted at appropriate points in the Australian System of National Accounts, Concepts, Sources and Methods (cat. no. 5216.0).

      Standard classifications and definitions of statistical units and items are essential elements underlying the compilation and presentation of statistics produced by national statistical offices. The use of such standards ensures that statistics are comparable across industry and sector boundaries and can be aggregated from various collections. The ABS has a range of standard economic classifications, including those for institutional sector, industry and commodity. These classifications are used in the national accounts and in other ABS statistics and, occasionally, in non-ABS statistics. The ABS places great emphasis on ensuring that its economic classifications align with SNA93 and other international standards to ensure comparability across its own economic collections and outputs as well as with those of other statistical organisations.

      Accuracy and Reliability
        For most users, accuracy and reliability are among the most sought after attributes of data. Accuracy can be defined as the proximity of an estimate to some notional true value. The true value is considered notional because, in practice, most statistics cannot be measured with perfect accuracy. Reliability is the proximity of initial and intermediate estimates for a particular period to the 'final' estimate for that period. In practice, at least in the Australian national accounts, accuracy and reliability tend to be interwoven and reinforcing. Ideally, as the estimate for a particular period passes through a sequence of revisions the size of the revisions gets smaller (the statistics become more reliable) and the estimate moves closer to the true value (the statistics become more accurate), but in practice this is sometimes not the case. By being aware of the factors that influence accuracy, judgements can be made as to the extent of error likely to be associated with an estimate.

        For assessing the accuracy and reliability of the Australian national accounts users can consider two information sources. First, 5216.0 provides a detailed account of the sources and methods underlying the data compiled as well as an overall assessment of data quality for each of the key components of GDP. Second, periodically, the ABS undertakes an analysis of the revisions that it makes to estimates of GDP and its components. Such an analysis was released in an article published in the July 1998 edition of Australian Economic Indicators (cat.no.1350.0). Another analysis is planned in the next 12 months. A recent study by staff at the Reserve Bank of Australia showed that although revisions to quarterly GDP growth rates have been significant in some quarters the pattern of revisions has shown a marked improvement in the last decade. It is our belief that this is due to the introduction of improved methods and data sources since 1990.

        Improvements in accuracy and reliability continue to be an aim. Currently, the ABS is embarking on a major program of implementing new ways of processing business statistics, new organisational arrangements, new methods and new technologies, all of which are expected to have a positive impact on the national accounts. Internal processes will also be improved through the application of quantifiable indicators of quality throughout the statistical processing cycle.

        Serviceability
          This dimension of quality relates to the need to ensure that:
          • data are produced and disseminated in a timely and regular manner
          • data are consistent both internally and with other related datasets
          • that a predictable revisions policy is used.

          Release practices: In releasing national accounts statistics, the ABS follows established principles that results should be made available as soon as practicable after the reference period, and should be available equally to all users. Australian national accounts statistics include major economic indicators that are in strong demand and can influence financial markets. Therefore, care is taken to ensure that no user receives the statistics before the designated release time, with a small number of exceptions. These exceptions relate to designated officers in certain government departments who are required to prepare briefing material on the statistics for their ministers; they are subject to a strict embargo until the official release of the national accounts. Importantly, the ABS announces the dates of its national accounts releases six months in advance and sticks to these announced dates in all but the most exceptional circumstances.

          Revisions policy: It is ABS policy to inform users of any significant revisions, and where appropriate to revise past time series and advise users accordingly. A comprehensive revisions policy for national accounts statistics can be obtained on request.

          Timeliness: The timeliness of data releases is measured by the gap between the reference period and the date of publication of results. There is an important trade-off between accuracy and detail on the one hand and timeliness of the release of statistics on the other. The source data that are used to compile the national accounts are available with varying degrees of timeliness, frequency, accuracy and detail. The data sources providing more detailed and accurate data tend to be those that are less frequent and/or less timely. Because of this, within a given level of resources, improvements in timeliness can generally only be made at the expense of accuracy and detail.

          The trade-off between timeliness, and accuracy and detail, is accommodated by way of the sequence of releases of national accounts publications. The first published quarterly estimates, which have considerable detail, are usually made available about 65 days after the reference period. The detailed annual estimates are released between 4 and 5 months after the reference period. The most detailed national accounts estimates relating to production, income and expenditure are those contained in the input-output tables which typically become available about 3.5 years after the reference period. Australia's national accounts rate well against those of other countries in terms of their timeliness - particularly when the level of detail made available is taken into consideration.

          Relevance: The ABS recognises that, in order to be relevant to informed decision making, debate and discussions, its national accounts statistics must be timely and relatable to other data. To support this they are placed in an appropriate statistical framework. Apart from the extensive range of existing data, the ABS continues to monitor and anticipate users needs. Ensuring ABS national accounts outputs support decision-making is also achieved by extensive consultation with state and territory governments and other program-specific user groups. The national accounts program proposes to increase its range of statistical output by expanding the range of data released and increasing the range of electronic releases.

          Training and assistance: Part of the ABS vision is to provide leadership by assisting decision makers to better understand how to access, use and interpret national accounts information to improve overall decision making or research activities. In line with this objective, the ABS regularly provides training sessions within Australia covering the concepts underlying national accounts data and the sources and methods used in their derivation. In addition, statistical advice and technical assistance is regularly provided to national accountants in other countries.

          In future, the ABS also plans to further expand its service to users through the provision of courses covering the various dimensions of quality and how knowledge of quality should inform the use of statistics. It is hoped that, in turn, this will encourage users of to provide feedback to the ABS about areas of its statistical program which they feel need further improvement.

          Comparability: For the most part, SNA93 (in conjunction with related international standards, such as the IMF's Balance of Payments Manual) is used as the framework for all ABS economic statistics, not just the national accounts. The use of this common framework helps to achieve comparability between the national accounts and other ABS economic statistics. In addition, the ABS has a range of standard economic classifications including those for institutional sector, industry and commodity (product). These classifications are used in the national accounts and in other ABS statistics and, occasionally, in non-ABS statistics.

          Another way in which the comparability of the national accounts with other ABS statistics is achieved is through the use of a common statistical units model. Statistics (particularly those pertaining to industry) from two sources are more comparable when both sources use the same statistical unit. However, for a number of reasons it is not always possible to use the same type of statistical unit in all collections and thus the ABS's units model provides for a number of different, but relatable, types of statistical units. Statistical comparability is also enhanced when the populations for statistical collections are obtained from a common source; the ABS's Business Register is such a source for the ABS economic collections.

          ABS national accounts are likely to be more compatible with other ABS statistics than with non-ABS statistics. Accordingly, when the national accounts are compared with non-ABS statistics, users need to be aware of any conceptual or methodological differences that may impact upon comparability.

          Accessibility
            This quality dimension relates to the need to ensure that clearly presented statistics are readily available. It also relates to the need for clear information on how the data estimates are derived and how to interpret them correctly, including the need for prompt and knowledgeable support to users of the statistics from the data producers.

            The most well-known national accounts outputs are the quarterly publication, Australian National Accounts: National Income, Expenditure and Product (cat.no.5206.0) and the annual publications, Australian System of National Accounts (cat.no.5204.0) and Australian National Accounts: State Accounts (cat.no.5220.0). Other national accounts releases include the Australian National Accounts: Financial Accounts (cat.no.5232.0), Australian National Accounts: Input Output Tables (cat. no. 5209.0), the closely related Australian National Accounts: Input Output Tables (Commodity Details) (cat.no.5215.0) and the Australian National Accounts: Tourism Satellite Account (cat.no.5249.0).

            In order to make national accounts statistics readily accessible, they are usually released in a number of formats. All of the outputs are released as traditional paper publications, which are available by mail and 'over the counter' in each capital city, and copies of the key national accounts publications are provided to public libraries around Australia.

            The paper publications are supplemented by a variety of electronic releases, including AusStats and tailored time-series releases, for catalogue numbers 5206.0, 5204.0 and 5220.0. Increasing emphasis on electronic dissemination is accepted as a strategic direction for the release of ABS data. Accordingly, the national accounts program has considerably expanded its electronic release of data over the past year. Currently, 118 tables are released electronically each quarter for 5206.0, many containing data back to 1959–60. Annually, 103 tables are released for 5204.0 and 37 tables for 5220.0. These electronic spreadsheets are available on subscription but can also be purchased on-line through the ABS web site.

            The main features section of each national accounts publication is available on the ABS web site free of charge. The ABS also makes its national accounts statistics available to secondary providers for inclusion in their products. National accounts statistics are also provided to the media, who in turn provide extensive coverage of the statistics. Users requiring more detail than that provided in the standard national accounts releases can seek information on request and, subject to cost-recovery charges, the information will be provided if it is available. Information about the full range of ABS national accounts publications and products is provided in the Catalogue of Publications and Products (cat.no.1101.0) or the ABS web site.

            Information about the national accounts

            The ABS, like other statistical offices, has an extensive range of ways to inform users about its published national accounts data. The concepts, sources and methods publication outlines the major concepts and definitions, describes the data sources and methods used to prepare the estimates and provides information about the ways in which the relevant international concepts and standards have been implemented in Australia. It also discusses the accuracy and reliability of national accounting concepts and estimates. It is a reference document suitable for data users requiring an in-depth understanding of data concepts and quality. The publication is available from the ABS's web site or on CD-ROM. Supplementing and updating this publication are an assortment of information, working papers and feature articles which are used to draw attention to issues impacting on the data such as changes to the classifications, systems, concepts or standards, major data revisions and changes in data dissemination practices.

            Significant sources of information about data quality are the annual and quarterly publications. Paper publications include text that describes their content, accuracy and the production process. Such information is necessary for users to interpret the data and make sensible judgments about their relevance, coherence and accuracy. The front pages contain key figures and key points which draw attention to the most relevant data. Information is included about future releases and about any changes in this issue. Main features describe the major elements of the data, the tables and graphs contain footnotes to alert users to revisions and other relevant issues that impact the data series. The final pages of both the quarterly and annual publications contain 'Explanatory Notes' and a 'Glossary' which describe in detail scope and coverage, data item definitions and classifications, chain volume measures and revisions policy. In accordance with ABS policy, each publication also explicitly names a person who can be approached for further information.

            Future directions

            The ABS is strongly committed to maintaining and improving the quality of all aspects of its statistical program, including the provision of enhanced information on the quality of its statistics. Involvement in the development of international quality frameworks will continue, in particular for those being developed by the IMF for macro-economic statistics. The ABS will also continue to regularly review its processes and procedures to ensure their effectiveness and reliability and to ensure that the adoption of 'best practice' continues across all its statistical programs.

            While significant amounts of information about the different dimensions of national accounts data quality are already available, including concepts, sources and methods publications, user guides and the explanatory notes in publications, it is comparatively more difficult to obtain a short qualitative summary of all this information, especially electronically. Additionally, much of the information collected about data quality is only referenced internally as part of the continuous improvement process.

            The challenge in the future will be to integrate the information that is available in internal systems into public quality declarations. The ABS is proposing to develop Quality Summaries for each of its statistical programs that will be concise enough to be useful to the user. Through the use of explicit links between these external perceptions of data quality and the internal processes which generate statistical information, the proposed quality summaries will also be able to identify quality gains from the implementation of new procedures and processes as well as identify areas requiring further improvement. Summaries will be made available on the internet via the ABS web site, and will be automatically supplied with purchased tables, as well as being incorporated into existing explanatory notes within ABS publications.

            Further advances in electronic dissemination will offer additional opportunities to develop and enhance presentation methods. Finally, it will remain a priority of the national accounts program to continue to provide a high quality service to its users by keeping them informed of statistical developments that impact on the national accounts through the use of information papers, seminars and special articles in publications.

            Further information

            Chapter 20 of 5216.0 contains further information about the quality of Australia's national accounts. For further information on the national accounts quality program contact Margaret Hausknecht on (02)62526250 or by email margaret.hausknecht@abs.gov.au.