Accuracy

24.28    For most users, accuracy is the most sought-after attribute of data. Accuracy can be defined as the proximity of an estimate to some notional true value. It is not possible to produce an objective overall measure of accuracy of the accounts because the national accounts draw data from a wide variety of sources, reflecting varying valuations, coverage, frequency, detail and timeliness. Assessments need to be made instead of individual component items within the accounts. Even at this level, the use of multiple data sources in estimating a single item, their variable accuracy over time, and changing compilation methods complicate the picture. As a result, assessment of the accuracy of an item requires a high degree of subjective judgement based on knowledge of the sources, the data and the compilation methods used.

24.29    It can also be useful to make a distinction between the concepts of accuracy and reliability in considering the quality of national accounts statistics. Accuracy is the proximity of an estimate to some notional true value, while reliability is the proximity of initial and intermediate estimates for a particular period to the 'final' estimate for that period. A series which is never revised is reliable, but it may not be accurate. Although reliability can be objectively measured by an analysis of revisions, it is a relative term, and users are likely to have some tolerance to revisions, given the trade-offs with other characteristics of quality.

24.30    In practice, accuracy and reliability tend to be interwoven and reinforcing, at least in Australia's national accounts. Ideally, the size of revisions gets smaller (and the statistics become more reliable) as the estimate for a particular period passes through a sequence of revisions, and the estimate moves closer to the true value (and the statistics become more accurate). In practice, this may not always be the case. Revisions can be reduced by delaying the release of statistics until all or most 'final' data sources are available, but this would mean that the statistics would be less relevant to users. ABS policy is to always aim for the most accurate estimate, even though this may be at the expense of more frequent revisions.

24.31    Judgements can be made as to the extent of error likely to be associated with an estimate by being aware of the factors influencing accuracy. Additionally, historical analyses of the revisions that the ABS makes to its estimates of GDP and its components can provide a quantitative guide to the reliability of the statistics produced.

24.32    The range of factors that can influence the accuracy of the national accounts include:

• coverage deficiencies
• input data errors, which include sampling error and error due to the inability of data providers to report on the correct basis, mistakes in the reporting of data, and error due to non-response
• error introduced during the processing of data
• methodological deficiencies
• output error due to inadequate editing and data confrontation.

24.33    The scope of the Australian national accounts is exhaustive within the production boundary recommended in the 2008 SNA, with the exception of illegal production. Illegal activity (which relates mainly to illicit drugs) is omitted from Australia's official statistics because it is difficult to measure with sufficient accuracy.

24.34    Illegal activity is one part of the Non-Observed Economy (NOE). The NOE also includes:

• underground production, which is activity which is legal but is deliberately concealed from authorities
• household production for own final use, such as crops and livestock, own-account fixed capital formation and owner-occupied dwelling services
• informal economy production
• the statistical underground, which is production that is missed due to deficiencies in data collection, such as survey non-response.

24.35    By its very nature, the NOE cannot be directly measured. Estimates of it rely on limited indicative information and a variety of indirect methods, all of which can be regarded as contentious. It is likely that an unknown proportion of underground production would be captured in the official GDP estimates due to the data sources used and the estimation methods employed. Nevertheless, it is accepted that additional adjustments for non-observed activity are required.

24.36    The ABS does not attempt to distinguish between the formal and informal sector, as informal production is relatively unimportant in Australia. Estimates, however, are included for the value of dwelling services consumed by owner-occupiers and owner-built construction activities and goods produced by households for own consumption, such as food consumed on farms and home-grown fruit and vegetables. Volunteer services and unpaid housework (such as cooking and cleaning) are excluded, consistent with the recommendations of the 2008 SNA. Estimates for these services have been prepared as satellite accounts; see Chapter 23 for more detail.

24.37    Explicit upward adjustments are made to account for underground activity which is legal, but is conducted in such a way as to avoid detection by taxation and other government authorities. The method used to obtain an estimation of underground activity was to systemically analyse each component of GDP and make judgements as to the maximum feasible level of underground activity, given anecdotal and available evidence from the Australian Taxation Office.

24.38    Issues involved in measuring the Non-Observed Economy are articulated in Information Paper: The Non-Observed Economy and Australia’s GDP.

24.39    The scope of some data collections may be drawn more narrowly for cost or other reasons. For example, the scope may exclude non-employing businesses; some industries; or business under a certain size. A narrower scope is more likely for monthly or quarterly surveys. Data from the taxation system and other sources allows these gaps to be filled. The annual economic collections are designed around the availability of taxation data, and the supplementation brought to the coverage of small businesses, including non-employing businesses. Non-profit institutions are generally exempt from the payment of income tax, such that data for those units has to be collected by the ABS via the annual economic survey.

24.40    The ABS undertakes a large number of collections that directly feed into the national accounts compilation process. National accounts requirements are a key consideration in the design of these surveys. A range of non-ABS data are either integrated into the survey outputs or are used independently to compile the national accounts. Examples include data from the taxation system, the financial institution regulation system, government financial reporting data, and retail scanner data. External data sources can potentially have issues for quality assurance and measurement, being beyond the control and responsibility of the ABS. Considering these points, the ABS and external sources enter into explicit arrangements concerning roles and responsibilities of the respective agencies; questionnaire content and design; timely data and transference procedures; editing and querying protocols; and appropriate feedback loops to the source agency.

24.41    The quality of ABS statistics is dependent on the application of good statistical methods during the selection and collection phases of a survey. The ABS puts substantial effort into developing standards in terms of classifications, concepts, data item definitions and question modules. All ABS surveys must use these standards. The collection instrument must be well tested and evaluated and this process is supported through documented standards in forms design and forms evaluation.

24.42    A high proportion of information used in compiling the Australian national accounts comes from surveys using the ABS Business Register (ABSBR) to provide the statistical frame from which representative samples are drawn. The ABSBR is based on the Australian Taxation Office's Australian Business Register (ABR), which contains all businesses with an Australian Business Number (ABN). It is expected to be comprehensive because businesses are required by law to obtain an ABN, with very limited exceptions. The ABS has adopted a strategy of building and maintaining its own records for large and complex businesses, with information for the remaining businesses sourced from the ABR. Although the ABS register is comprehensive and current, there are known to be some problems with the quality of the industry coding, which would have implications for the quality of the data for individual industries used in the national accounts.

24.43    Sample design and estimation systems are developed by specialist areas in accordance with internationally accepted practices. Accuracy is considered in terms of both sampling and non-sampling error. The ABS publishes information on the relative standard errors (RSEs) for its various sample surveys. These can provide an indication of the accuracy of the national accounts components to which they relate. It has not been possible to systematically calculate the impact that RSEs have on the various national accounting aggregates, because of the transformations of survey data, and the aggregations made in order to compile the national accounts.

24.44    An important potential source of non-sampling error can result from the inability of some data providers to report on the correct basis, considering the data requirements underlying the national accounts. Data providers can make errors with regard to the content, timing and valuation of their transactions. It is inevitable that some data providers will include extraneous items in their survey responses, and exclude relevant items, although every effort is made to match survey data items with business accounting practices. Survey forms are tested with a small number of providers before approved for use. Sophisticated techniques are used to edit provider responses, but errors can remain undetected. Because the national accounts is a closed system, such errors can lead to inconsistencies, affecting the coherence of the accounts.

24.45    The ABS uses best practices in survey design and operation, given the data intensive needs of the national accounts system, and the cost and respondent load imperatives that apply in survey design. National accounts compilers make do with data that are of acceptable (but less than ideal) quality, particularly for data at a finer level of detail, where the standard errors are often higher. Limited detail might be collected in some cases, or only collected infrequently (such as product details). The ongoing challenge is to ensure the adequacy of input data quality, in order to meet national accounts requirements. The national accounts process itself is designed to mitigate input data problems through data confrontation, as well as balancing in the supply-use tables, which are used to benchmark the national accounts. High quality input data, however, are essential to high quality national accounts data.

24.46    The ABS periodically reviews its economic survey strategy to ensure data requirements are met within the constraints of resource availability. The national accounts requirements for data are regarded as of very high importance.

24.47    The data sources and methods used in preparing ABS national accounts statistics are regularly reviewed, and changes are made periodically to the basis of compilation of an item. A major methodological improvement to the Australian national accounts was the introduction of supply-use benchmarking and annually re-weighted volume estimates in 1998. The former results in greater coherence of the accounts, and the latter provides a superior measure of economic growth to the previously available volume estimates based on five-yearly reweighting.

24.48    Seasonal adjustment methods used by the ABS are based on the United States Bureau of the Census (USBC) Method II Seasonal Adjustment program, X-11 variant. This approach decomposes original estimates into a combination of modelled trend, seasonal and irregular components. Seasonally adjusted estimates are created by adjusting original estimates using seasonal factors to remove the seasonal component, and trend estimates are obtained by removing the irregular component from seasonally adjusted estimates. Since September 2006, the ABS has usually re-analysed and re-estimated seasonal factors for national accounts time series with each quarterly publication—a procedure referred to as concurrent adjustment. This process generally results in the best quality estimates because the method is most responsive to emerging changes in seasonal patterns. However, in response to the economic disruption caused by the COVID-19 pandemic from 2020 onwards, some components have returned to the previous process of re-estimating and fixing seasonal factors annually, to better manage large movements near the current end of the series.   

24.49    As the national accounts are compiled within a comprehensive framework, it is possible to reduce the impact of data errors through the confrontation of the various estimates in the national accounts. Data confrontation is built around the conceptual relationships between data items. The best known data confrontation exercise is the compilation of the annual supply-use tables. Estimates of the supply (production) and use (demand) for commodities are compared in the compilation of these tables, with differences in the initial estimates being eliminated. At the same time, estimates of the value of production are compared with estimates of incomes attributable to production, and differences eliminated (further details are included in Chapter 8 on coherence of the national accounts).

24.50    Each collection area is required to confront its data with other data held by the ABS and other organisations as an important part of the process of ensuring the coherence of ABS statistics. Clearance meetings are held for all the major economic collections used to compile the national accounts, as a means of assuring data consistency between those collections and the national accounts, with emphasis on the most recent reference period.

24.51    The national accounts quarterly compilation process incorporates a review process designed to highlight inconsistencies and improbable data movements. Problems are identified, investigated, and resolved in the process of finalising the GDP estimates. There is a feedback loop to the data collection areas, culminating in the clearance procedures mentioned above.

24.52    Since 2005, the ABS has developed quarterly Supply and Use (QSU) tables as an editing tool to assist in the preparation of the quarterly national accounts. Preliminary quarterly estimates for the production and income components are used as inputs to the model which generates estimates of product supply and use in a time-series format. The QSU model enables inconsistencies between the different measures of GDP to be identified and investigated more systemically, and at a greater level of detail than is possible by simply examining the aggregate estimates. At the present stage of its development, the QSU model is used as an aid in the compilation of the seasonally adjusted production and expenditure-based estimates of GDP. A description of the initial QSU model (and how it is used) is contained in the research paper, A Supply and Use Model for Editing the Quarterly National Accounts, Australia. The model has recently been updated to increase the level of product detail and incorporate current price measures.

24.53    Accuracy is concerned with the proximity of an estimate to the 'true' but unknown value of the component being measured. It is concerned with the degree of precision associated with the estimate. As already discussed, the complex nature of the national accounts makes it extremely difficult to produce a tangible benchmark against which to measure accuracy. In practice, the accuracy of the national accounts is evaluated by considering the potential sources of error, and whether those sources have been minimised to the greatest extent possible. One approach to tie all the information related to data quality together is to assign subjective accuracy ratings to each series.

24.54    Subjective accuracy ratings are not derived by a particularly rigorous process, but represent an intuitive assessment by national accounts compilers. A consensus was reached taking into account knowledge about the standard errors on key survey inputs; impressions about the coverage and reliability of administrative data sources; and revisions to initial estimates of growth.

24.55    The next three tables contain an assessment for the initial quarterly estimates of movement for the current price and chain volume estimates of expenditure, income and production components of GDP. These have been chosen as they are generally the most anticipated of the national accounts estimates. Each component has been assigned one of the following grades:

A good

B fair

C poor

D very poor

24.56    A more objective (but still limited) indicator of accuracy is provided by examining the differences between the conceptually equivalent estimates of GDP, measured by the expenditure, production and income approaches. Quarterly GDP in the Australian national accounts is derived as the average of the three measures, with accounting balance being achieved by an explicit statistical discrepancy item for each of the three elements.

24.57    The three measures of GDP are balanced in annual terms using the supply-use approach, up to the year prior to the most recent complete year (and latest two years in the June quarter release). This process eliminates the statistical discrepancies in annual terms, except for the latest year. The quarterly estimates of GDP are benchmarked to the annual estimates, but within-year inconsistencies remain.

24.58    Large and persistent statistical discrepancies relative to GDP can indicate gaps in the coverage of the measures or other quality issues. Because the measures are balanced annually in a supply-use table, the magnitude of the quarterly statistical discrepancies is small relative to the level of GDP, and they cancel out over the year. Any significant data coverage issues have been addressed over time. Of more interest in the Australian context is that quarter-to-quarter changes in the level of the statistical discrepancies are associated with inconsistencies between the growth rate of GDP, and its conceptually equivalent measures. 

24.59    Figure 24.1 shows the discrepancy between quarterly growth rates of chain volume GDP as measured by the expenditure, production and income approaches and their average, GDP(A). In this graph, positive discrepancies indicate that a given measure has a higher growth rate than the average measure.

Figure 24.1 Quarterly growth in GDP measures

24.60    The statistical discrepancies between the three measures of GDP provide one indicator of the accuracy of the national accounts; the net errors and omissions item in the financial account provides another. It represents the difference between the net lending derived in the capital account, and the conceptually-equivalent net lending item derived in the financial account – the net errors and omissions item is required to balance them.

24.61    Net lending in the capital account is derived as a residual and reflects the expenditure, production and income components of GDP as well as other items such as net property income and net secondary income. The financial account also derives net lending as a residual but using totally independent data – it is the difference between the acquisition of financial assets and the incurrence of liabilities.

24.62   Figure 24.2 plots the net errors and omissions item in annual terms. Some year-to-year differences are evident, reflecting the difficulties inherent in measuring residuals. It reflects timing and other errors in the estimation of income, expenditure and capital flows on the one hand and financial assets, liabilities and financial flows on the other.

Figure 24.2 Net errors and omissions

24.63    The magnitude and volatility of the net errors and omissions item and the statistical discrepancy can also be used to indicate the quality of the saving estimates in the national accounts. National net saving is derived in the income and use of income accounts by deducting final consumption expenditure and consumption of fixed capital from gross disposable income. Its quality is particularly sensitive to inaccuracies in the series from which it is derived, as a relatively small residual measure derived from very large aggregates. This point needs to be borne in mind when using the data for analysis.

24.64    Saving is a source of funds for both acquisition of financial assets and investment. An alternative and largely independent measure of saving can be derived using acquisition of financial assets from the financial account as the starting point. The accounting relationships within the national accounts system mean that the alternative measure can be calculated by deducting the net errors and omissions item and the statistical discrepancy between GDP(I) and GDP(E) from the official measure of saving. Figure 24.3 plots the official measure of annual saving and the alternative measure as a percentage of GDP, using data expressed in current prices. The series track each other closely, although there are some year-to-year differences.

Figure 24.3 National net savings, ASNA and alternative measures