Data quality

11.1 Among national statistical agencies, quality is generally accepted as "fitness for purpose". Fitness for purpose implies an assessment of an output, with specific reference to its intended objectives or aims. Quality is therefore a multidimensional concept which does not only include the accuracy of statistics, but also includes other aspects such as relevance and interpretability.

11.2 The ABS Data Quality Framework uses seven dimensions of quality, reflecting a broad and inclusive approach to quality definition and assessment. The seven dimensions of quality are institutional environment, relevance, timeliness, accuracy, coherence, interpretability and accessibility.

11.3 There are often trade-offs between the different dimensions of quality. In order to make economic statistics timely enough to be relevant indicators for the analysis of current and recent economic conditions this is likely to be at the expense of some degree of accuracy. The ABS, in consultation with data users, aims to optimise the different dimensions of quality.

11.4 This chapter will assess the quality of international merchandise trade statistics using the seven dimensions of quality. When assessing the quality of statistics the ABS and international organisations recommend that all seven dimensions should be considered together but some statistical users may place more weight on particular dimensions.

11.5 International merchandise trade statistics are produced using administrative by-product information from The Department of Home Affairs. Exports and imports data are collected under the Customs Act 1901 with exporters and importers having a legal obligation to provide documentation to the Department of Home Affairs. The Department of Home Affair's legislation also allows the information to be passed to the ABS for statistical purposes.

11.6 The ABS and the Department of Home Affairs foster a close relationship to ensure ongoing cooperation. An Agency Heads Agreement between the two organisations, signed by the Australian Statistician and the Chief Executive Officer of the Department of Home Affairs, is in place. The agreement formalises the relationship between the two agencies and refers to a set of subsidiary agreements covering a range of issues including: the provision of import and export declarations to the ABS; and management of system changes which affect the provision of data between the agencies. In addition, the Deputy Agency Heads meet annually to review the relationship and discuss strategic issues.

11.7 Australian international merchandise trade statistics are compiled within an institutional environment which focusses on quality and emphasises objectivity and professionalism. The environment in which the Department of Home Affairs operates is similarly underpinned by legislation to ensure the quality of customs declarations and the continued supply of these data to the ABS.

11.8 For information on the legislative obligations of the ABS, financing and governance arrangements, and mechanisms for scrutiny of ABS operations, please see ABS Institutional Environment.

11.9 Relevance refers to how well the statistics meet the needs of users. It can be assessed by considering the purpose or aim of the data collection, compliance against relevant international standards and the use of the statistics.

11.10 International merchandise trade statistics are compiled from customs declarations.

11.11 Details of exports and imports of goods with values greater than or equal to the respective thresholds ($2,000 and $1,000) must be entered on customs declarations prior to exit from or entry to Australia. The detailed information provided on customs declarations ensures the relevance of the data as it includes: the commodity classification and a description of the goods, the value, quantity, gross weight, country of origin or country of final destination, and where appropriate, port of loading, port of discharge, mode of transport export goods type and duty related fields.

11.12 The exclusion of small value records has historically had a limited impact on the relevance of the statistics but the impact has increased in recent years due to the ease of individuals purchasing goods over the internet. Estimates of the value of trade below the thresholds are included in Balance of Payments statistics and the ABS has recently reviewed the quality of those estimates.

11.13 The ABS actively promotes the use of international standards for compiling international merchandise trade and other economic statistics to ensure comparability across its own economic collections and outputs and those of other statistical organisations. The conceptual framework followed for Australia's international merchandise trade statistics is broadly consistent with the international standards set out in the UN publication IMTS 2010. The various editions of this standard have been developed over many years and involved the input of international organisations, national statistical agencies and other experts (academics and users). Periodic updates are necessary to accommodate changes in the way international merchandise trade is conducted e.g. increasing globalisation.

11.14 Australia's international merchandise trade statistics align with many of the recommendations contained in IMTS 2010. The standards are applied with consideration to the availability of timely and accurate information and the requirements of Australian data providers and users. For example, to enable accurate reporting by exporters and importers the unit of quantity required on customs declarations reflects the unit in use and most relevant to the Australian industry.

11.15 A wide variety of organisations (government, business, national and international) and individuals use international merchandise trade statistics. The data use varies from information about a single commodity to access to the complete aggregate (and confidentialised) dataset. While some statistical users access the data periodically, many users of international merchandise trade statistics subscribe to and use monthly data on a regular basis.

11.16 International merchandise trade statistics are based on a large number of individual transactions and are released at a very detailed aggregated level e.g. 8-digit exports and 10-digit imports commodity by Australian port, country and overseas port. The statistics, in conjunction with other related economic statistics, are used to analyse Australia's external trade performance, formulate and evaluate macroeconomic policy (including trade policy, trade treaties and foreign investment policy), analyse trends in income, capital flows and patterns of investment into and out of the economy, and to undertake international comparisons. As the ABS has limited resources available for ensuring the quality of all the detailed international merchandise trade statistics, resources are focussed on the data required for macroeconomic statistics.

11.17 Import and export statistics have been released by the ABS for over 100 years. The longevity of the statistics indicates the ongoing usefulness of the data.

11.18 The relevance of international merchandise trade statistics are enhanced by the inclusion of feature articles or additional releases about the statistics. A list of articles is included on the ABS website.

11.19 Timeliness refers to the delay between the reference period to which the data pertains and the date on which the data become available. There is an important trade-off between the accuracy of the estimates and the timeliness of their release. As international merchandise trade data are important inputs to major economic indicators and there is widespread use of the data for other purposes, there is a need to publish the statistics on a timely basis, consistent with achieving acceptable levels of accuracy.

11.20 To the extent that complete or accurate data are not available within the desired time frame, there will be compromises in data quality. For example, exporters who have been accorded Confirming Exporter Status by the Department of Home Affairs can revise the value, quantity and weight details after the goods have been included in the initial export estimates, see revisions analysis below.

11.21 The timeliness of public release of Australia's international merchandise trade statistics rates favourably against similar, detailed statistics produced by our major trading partners.

11.22 Accuracy refers to the degree to which the data correctly describe the phenomenon they were designed to measure

11.23 The statistics are compiled from export and import declarations and are not collected by sample survey so they are not subject to sampling errors. However, they are subject to reporter error. While the Department of Home Affairs apply various logical and legality checks and are able to audit business records they operate in a self-regulation environment which mainly targets declarations with perceived risk.

11.24 There has been significant growth in the volume of imports with values below the import declaration threshold (see the Low Value threshold update (Technical Note) in the August 2014 issue of International Trade in Goods (cat. no. 5368.0) for more information). Goods valued $1,000 or less are excluded from international merchandise trade statistics. The number and value of export transactions which are excluded from the statistics are much lower despite the higher export declaration threshold ($2,000).

11.25 For details regarding the difference between scope (economic territory) and coverage (statistical territory), refer to Table 2.1 in the Scope, Coverage and Treatments chapter.

11.26 The ABS has implemented best practice input and output processing systems, processes and procedures to ensure the quality of international merchandise trade statistics. These include: 

• Quality strategies to ensure the data is fit for purpose e.g. edit checks, significance editing, data confrontation with other sources.
• In cases where initial unconfirmed estimates of data provided by exporters who have been accorded Confirming Exporter Status by the Department of Home affairs are identified as inconsistent with confirmed data, careful estimates based on previous trends are made. Such estimates replace unconfirmed data until confirmed data are provided.
• Testing and evaluation plans prior to the introduction of any new or updated processes/and or systems to ensure errors and potential quality issues are identified and resolved.
• Training and development of staff to ensure they have the skills and abilities to identify key statistical issues and potential quality impacts.
• Risk mitigation and assessment processes e.g. quality gates and data management information to ensure all data are received, loaded, processed and accounted for at each processing stage.

11.27 Methodological changes can affect the comparability of data before and after the changes are implemented but they are generally directed towards improving the overall quality of the statistics. Historically there have been a small number of significant methodological changes to international merchandise trade statistics. In the last 25 years the most significant changes occurred in 1988 with the introduction of the Harmonized System and in 1992 when exports were converted from a processing date to a departure date basis. Both these changes were significant at the point of introduction but represented major improvements to the quality and international comparability of the statistics.

11.28 Wherever possible the ABS attempts to assist users to understand changes to methods or classifications by:

• notifying changes in advance
• preparing and releasing information about the impact of changes
• preparing correspondences showing the links between the new and old classifications
• providing information on both bases for a limited period or less frequently using look-up tables to re-classify information to the previous classification or method.

11.29 The accuracy and accessibility of international merchandise trade statistics is affected by confidentiality restrictions. International merchandise trade statistics are subject to confidentiality restrictions which means statistics are able to be released unless, and until, an individual or organisation demonstrates that disclosure would be likely to enable the identification of that person or organisation. Where a request has been made to confidentialise data, the full detailed dataset is not available to users, but the value of confidential data is included in total exports and imports.

11.30 By carefully checking all large value transactions, most cases of significant over-valuation error are detected and corrected by the ABS, prior to publication of the statistics. Significant under-valuation, and smaller value inaccuracies, however, may not be detected by the Department of Home Affairs and ABS quality checking procedures and may therefore contribute to inaccuracies in the statistics.

11.31 For imports the Department of Home Affairs system uses official exchange rates to convert values at the time of export from the overseas country of export, this minimises currency conversion errors. Exporters are required to declare goods prior to export and these are converted to Australian dollars by the ABS from RBA data.

11.32 Exporters may undertake currency conversions using a different exchange rate, or one applying on a different day to that at the time of exportation. This includes practices such as hedging which are designed to offset adverse currency movements. However, as the majority of exports are negotiated in one of the reportable currencies, any inappropriate conversion practices in the remaining exports will have a very limited impact on the statistics.

11.33 The ABS compiles import and export declarations, as lodged with The Department of Home Affairs through the Integrated Cargo System, for the purpose of creating economic statistics. While The Department of Home Affairs and the ABS apply various logic and quality assurance checks in order to ensure that trade statistics are timely and fit for purpose at aggregate levels, ABS editors focus on the most significant transactions. As such, care should be exercised in the use and interpretation of detailed international merchandise trade estimates. In addition, the ABS International Merchandise Trade statistics are subject to confidentiality restrictions. Any detailed International Merchandise Trade data should be read in conjunction with the Confidential Commodities List which is available in Table 92 of International Trade in Goods (located in the ‘Data downloads’ section).

11.34 A revisions analysis for the period July 2008 to June 2012 (48 months) is included as another way to assess the quality of international merchandise trade statistics. The analysis measures the mean bias and dispersion between the average initial monthly published result for the period and the average final published result.

11.35 Bias is a measure of the extent to which initial estimates are lower or higher than the final estimate and gives an indication of the direction of revisions. Unscaled bias is calculated as the average of the differences, over the observation period, between the initial estimate of a category for each month and the latest available estimate for the same month, with positive and negative revisions being netted against each other. This measure can be described as the average of the values of all revisions taking account of sign. Scaled bias is calculated by expressing the average revision for each category as a percentage of the average initial value of that category for the month, and then calculating the average of these percentage revisions.

11.36 Dispersion is a measure of the 'spread' between the initial and final estimates and indicates the magnitude of revisions. Unscaled dispersion is calculated as the average of the differences, over the observation period, between the initial estimate of a category for each month and the latest available estimate for the same month, however positive and negative revisions are not netted against each other. This measure can be described as the average of the absolute values of all revisions, without regard to sign. Scaled dispersion is calculated as the average of the absolute value of the percentage revisions.

11.37 As measurements of dispersion are based on absolute values, the value of dispersion measures is generally larger than the equivalent measure of bias. Only if all revisions were in the same direction, would the results be the same for both measures. The larger the difference between the absolute values of the two measures for a particular series, the greater the variability in the direction of revisions.

11.38 Table 11.3 below shows that the revisions to initial published monthly export estimates had on average, a positive bias of $490 million or 2.21% for the period July 2012 to June 2014. Negatively biased revisions indicate that the initial estimate tends to overstate the latest estimate. The dispersion measures were generally higher, indicating that the individual monthly revisions were both positive and negative. The average magnitude of revisions to total exports, without regard to sign, was also $490 million or 2.21% of the average monthly estimate.

11.39 The main contributors to the negatively biased revisions were Crude materials, inedible, except fuels and Mineral fuels, lubricants and related materials. The unscaled and scaled dispersion measures were also higher for these SITC sections due to:

• relatively higher values and a greater contribution to total exports of the commodities included in these SITC sections.
• at the time of initial reporting to the Department of Home Affairs the quantity and value information for these commodities is estimated and revised in later months when the actual transaction values are available. This contributed significantly to revisions when:
• the timing of export contracts for these commodities changed from annual to quarterly and then monthly
• the exchange rate depreciated significantly
• Commodity prices, in particular Iron Ore depreciated significantly.

11.40 As crude material prices dropped in 2014, reporters initial estimates to the Department of Home Affairs were often overstated. In response, the Australian Bureau of Statistics applied an adjustment to the originally reported data, which was subject to revision, based on historical trends. The figures are revised in subsequent months, once actual transaction values are available. This method yields significantly more accurate results in comparison to solely relying on administrative Department of Home Affairs data, at both aggregate and transactional levels. These changes are not reflected in the current analysis as they were only implemented in January 2015. However, post-January 2015 outputs have shown a significant reduction in data revisions.

11.41 The average revisions for all other SITC sections were downward as shown by the negative bias measures in Table 11.3 below.

11.42 Table 11.4 below shows that the revisions to initial published monthly import estimates had on average a negative bias of $118 million or 0.58% for the period July 2012 to June 2014, indicating that the initial estimate overstated the final estimate. This pattern is also reflected in the revisions at the SITC section level. The dispersion measures were slightly higher, indicating that the individual monthly revisions were both positive and negative.

11.43 Coherence refers to the internal consistency of data at a point in time and over time, as well as its comparability with other sources of information.

11.44 Coherence is an important component of quality as it provides information about any limitations with comparing the data with other data sources. The use of standard concepts and definitions and robust methods promote coherence. Coherence with other statistics does not on its own mean the statistics are accurate, as the statistics that are being compared may suffer from similar sized errors in the same direction.

11.45 International merchandise trade data are almost exclusively compiled from customs declarations. This together with a robust processing system (with separate environments for testing and training) and established processes for receiving, editing, aggregating, confidentialising, finalising and disseminating data ensures a high level of coherence.

11.46 There are a range of checks applied at various stages in the ABS compilation processes to ensure the consistency of the data. For example, there are edits which may identify errors in either the gross weight or quantity when the unit of quantity is grams, kilograms or tonnes. Other edits applied at the individual transaction level check the validity of the relationship between related fields, for example, if the mode of transport is sea but the goods were loaded in Switzerland. These inconsistencies are detected and corrected prior to the release of initial estimates.

11.47 International merchandise imports and exports are one of the first releases of monthly economic statistics so there are limited published sources available to confront against the estimates. Consequently, considerable attention is given to ensuring the consistency of the monthly aggregates at the HS 6 digit and SITC 2 digit level with previous monthly results. Unusual movements may be verified or corrected using publicly available information (e.g. press reports, company websites) or following contact with the exporter or importer.

11.48 Because the ABS publishes a large range of economic data it strives to ensure coherence across these estimates. To achieve this processes have been implemented including regular quarterly meetings between representatives from the various economic collections. At these meetings there are discussions which lead to agreement on the statistical treatment (including valuation, timing and classification) of recent economic events such as major resource projects.

11.49 Any significant changes to the established processes must be reviewed and approved by a Methods Board. The Methods Board consists of a panel of ABS subject-matter experts who are responsible for endorsing new methodologies, and enhancing the coherence of statistics across the ABS.

11.50 The HS was adopted internationally by the Customs and Co-operation Council in 1983 and entered into force in January 1988. The Department of Home Affairs and the ABS have used the HS classification since that time so comparison of the data (up to the 6 digit level) with other countries using the HS is possible from that date. The ABS also publishes international merchandise trade statistics by SITC for comparison at the international level. [Confidentiality restrictions can change the availability of data at any point so care should be taken when comparing data over time and with other countries. Confidentiality restrictions are not applied to the value of total trade (imports, import clearances or exports)].

11.51 Australia's international merchandise trade statistics are compiled substantially in accordance with IMTS 2010 which is also broadly consistent with SNA 2008 and BPM6. Australia, like many of its trading partners, publishes the details of its compliance with the international standards so statistical users are able to make informed decisions about the level of data comparability. Australia's proposed stance against the international standards are set out in the Information Paper: Proposed Implementation of the New International Standard for International Merchandise Trade Statistics (cat. no. 5368.0.55.020). Reasons for differences in the statistics released by each country are also available, for example, IMTS 2010 recommends that imports are valued on a CIF basis. While Australia does compile imports on a CIF basis and have these available for release, the published imports valuation is customs value (a FOB type valuation, see paragraph 3.12 in the Trade System, Valuation and Time of Recording chapter) which is the value on which customs duty is applied and is of a higher quality than the CIF value at the commodity level. In Australia, a FOB type value is used for both international merchandise trade, balance of payments and national accounts statistics so coherence between these data is maintained. However, comparisons of Australia's exports to a country which publishes its imports (from Australia) on a CIF basis will be affected because of the different valuation basis (as well as other differences including timing, coverage (e.g. lower or higher thresholds) and country classification).

11.52 The ABS has conducted bilateral reconciliations of international merchandise trade data with the United States of America, Japan, New Zealand and the European Union. These studies demonstrated that the main reasons for differences in the statistics of Australia and its major trading partners are due to the conceptual and methodological factors underlying the compilation of the data. The ABS does not have the resources to conduct further bilateral reconciliation studies. Statistical users interested in conducting their own bilateral studies should see:

• the International Trade Releases page on the ABS website which contains links to international merchandise trade statistical publications and previous bilateral studies.
• DFAT Trade Publications
• UNdata - Commodity Trade Statistics Database
• the websites of Australia's main trading partners e.g. USA Trade Online.

11.53 Interpretability refers to the availability of information to help provide insight into the data. Interpretability is an important component of quality as it enables the information to be understood and utilised appropriately. Statistical information that users cannot understand, or can easily misunderstand, has limited value and may have negative value. Providing sufficient information to allow users to properly interpret statistical information is therefore essential. Information that describes and defines data is known as metadata. Managing interpretability is primarily concerned with the provision of metadata.

11.54 The information needed to understand statistical data falls under three broad headings: 

• the concepts, definitions and classifications that underlie the data
• the sources and methods used to collect and compile the data
• indicators of accuracy of the data.

The description of methods also serves as a surrogate indicator of accuracy – it allows users to assess suitability for purpose.

11.55 There are close relationships between these three headings and the other dimensions of quality. The underlying concepts and classifications used are also a prime determinant of coherence and the degree to which they conform with national and international standards should be apparent from the metadata.

11.56 This publication defines and discusses the major concepts, definitions and classifications that underlie international merchandise trade statistics. It also describes the methods used to transform input data into statistical outputs and discusses the accuracy of the estimates. Statistical users can draw valid comparisons with merchandise trade data produced by other countries, by taking into account any differences e.g. different thresholds or time of recording.

11.57 Supplementing this publication are an assortment of information papers and feature articles which draw attention to issues impacting on the data such as changes to the classifications, systems, concepts or standards, major data revisions and changes in data dissemination practices. Feature articles and technical notes are written on a regular basis to inform users of emerging issues and method changes and their impact on the statistics. Information papers report on various aspects of research undertaken on topics relevant to Australia's international merchandise trade statistics, see International Trade Releases.

11.58 Accessibility refers to the ease of access to data by users, including the ease with which the existence of information can be ascertained, as well as the suitability of the form or medium through which information can be accessed. The cost of information may also represent an aspect of accessibility for some users. This is a key component of quality as it relates directly to the capacity of users to identify the availability of relevant information, and then access it in a convenient and suitable manner.

11.59 In discussing accessibility of international merchandise trade statistics, the following aspects will be covered: data availability, metadata availability (i.e. information about the data) and the degree of interpretive assistance available to users of the data. Data availability involves issues of data presentation and distribution media as well as the availability of non-standard data. Metadata availability is whether information concerning the concepts, sources and methods associated with the data is provided to statistical users. Accessibility also encompasses whether the information surrounding or supporting the data is openly available to the public.

11.60 Most aspects of accessibility are determined by ABS wide dissemination policies and delivery systems.

11.61 The ABS provides users with ready access to international merchandise trade statistics. All publications and standard products are available free of charge on the ABS website and this includes international merchandise trade information.

11.62 Users requiring more detail than available on the ABS website can submit a data request and if the information is available it will be provided on a fee-for-service basis.

11.63 The ABS has a range of ways of providing users with information about international merchandise trade statistics. This publication explains the concepts, sources and methods which underlie the compilation of Australia's international merchandise trade statistics.

11.64 Each international merchandise trade data release also includes metadata information in the explanatory notes, main features, changes in this issue, feature articles and in the actual tables, spreadsheets and data cubes.

11.65 Changes to classifications or methodologies are advised to users well in advance of implementation through specially prepared information papers and information in the data releases.

11.66 Release dates for international merchandise trade publications are announced well in advance on the ABS website, in publications and in the Release Advice for ABS Publications.

11.67 In releasing statistics, the ABS adheres to long established principles that the results of statistical collections should be made available as soon as practicable and should be available to all users at the same time. An embargo is placed on the release of statistics until 11.30 am (Canberra time) on the designated day of release. There are strict security procedures to ensure that there is no unauthorised access to statistics prior to release.

11.68 Each international merchandise trade release contains information about who to contact if users need more information about the statistics. Depending on the nature of the inquiry ABS staff in the National Information and Referral Service or a subject matter expert can be contacted.

11.69 The National Information and Referral Service provides help to find or access ABS products and services by telephone (1300 135 070) or secure email, see ABS Inquiry Form. Limited free data and information is given over the phone. The National Relay Service provides services for the hearing and speech impaired.