• Summary
• Downloads
• Explanatory Notes
• Related Information
• Past Releases

15.15. Compilation of balance of payments and international investment statistics is a complex task. Given the variety of data sources and methods used, there is no single comprehensive measure of the quality of these estimates. Nevertheless, each of the quality indicators described below provides a partial insight into the quality of the statistics. To get an overall picture, all measures need to be viewed together while taking account of their limitations. At best, such an assessment can only be subjective.

Measures to monitor quality

Relative standard errors

15.16. Sampling errors, generated from the use of sampling techniques to derive population estimates for a range of items, can be expressed as relative standard errors, which provide an indication of accuracy for particular items. These relative standard errors are kept under regular review and are considered to be acceptable. When sampling errors are considered too high for aggregates or sub-aggregates, the next opportunity will be taken to adjust sample selection (selecting more units in categories showing higher variability) to reduce future sampling error. The ABS publishes these sampling errors from time to time to make users aware of them and to help users to assess data quality.

Net errors and omissions

15.17. An examination of the size and direction of the net errors and omissions item may shed some light on the accuracy of estimates. As noted earlier, the adoption of the double entry accounting system of recording means that, in principle, the net sum of all credit and debit entries should equal zero. In practice such equality rarely exists, and any differences are recorded in the net errors and omissions item. The item reflects the net effect of differences in coverage, timing and valuation, as well as errors and omissions which occur in compiling all the individual component series. Therefore both users and statisticians can focus on the item as an immediate and systematic indicator of the quality of the balance of payments statistics.

15.18. Persistently large figures in one direction (negative or positive) may be taken as an indication of serious and systematic errors. However, a small figure does not necessarily mean that only small errors and omissions have occurred, since large positive and negative errors may be offsetting. Offsetting errors may be either related or unrelated, resulting from a measurement problem affecting either both sides or only one side of a transaction. If positive and negative net errors and omissions tend to offset each other in successive periods, errors may be due to timing differences in data reported by the different sources used to estimate the credit and debit sides of a transaction.

15.19 The fourth edition of the IMF’s Balance of Payments Manual suggested, as an empirical rule of thumb, that a net errors and omissions component in excess of five per cent of the gross sum of merchandise exports and imports was cause for concern. While the relevance of such a measure has diminished since the fourth edition was published in the late 1970s, when financial flows were generally more constrained and services and income flows were of less significance, it still gives some indication of accuracy, especially when compared to broader balance of payments aggregates. Australia’s experience with the relative size of the net errors and omissions item, expressed as a proportion of gross goods credits and debits, and gross current account credits and debits, is given in table 15.1.

15.20. Over the 14-year period from 1945-46 to 1958-59 net errors and omissions, expressed as an annual average percentage, represented 3.0 per cent and 2.4 per cent of gross goods and current account transactions, respectively. The annual average ratio to gross goods remained steady through to the 1980s while the ratio to the gross current account transactions improved slightly to 2.1 per cent. During the 1990s to date, the ratio to gross goods has been reduced by 70 per cent to 0.9 per cent, while the ratio to gross current account transactions has shown significant (60 per cent) improvement.

	15.1. NET ERRORS AND OMISSIONS ITEM AS A PROPORTION OF GROSS GOODS AND GROSS CURRENT ACCOUNT TRANSACTIONS
					Credits plus debits			Net errors and omissions item as a proportion of
		Net errors and				Current			Current
	Year	omissions item			Goods	account		Goods	account
			$m		$m	$m		%	%
	Annual average (a)
	1945-46 to 1958-59		77		2,548	3,268		3.0	2.4
	1959-60 to 1979-80		362		11,366	16,266		3.1	2.2
	1980-81 to 1989-90		2,018		64,902	97,543		3.1	2.1
	1990-91 to 1996-97		1,142		130,804	204,919		0.9	0.6
	1990-91		35		102,249	163,359		0.0	0.0
	1991-92		-90		106,896	167,828		-0.1	-0.1
	1992-93		482		120,538	187,189		0.4	0.3
	1993-94		1,786		129,282	200,075		1.4	0.9
	1994-95		257		142,418	223,845		0.2	0.1
	1995-96		-90		153,875	240,580		-0.1	0.0
	1996-97		-1,238		160,372	251,559		-0.8	-0.5
	(a) The annual averages are derived without regard to sign. Source: Balance of Payments and International Investment Position, Australia, March Quarter 1998 (Cat. No. 5302.0) except for the 1945-46 to 1958-59 averages, which are taken from the 1995-96 annual publication.

Examination of the statistical processes

15.21. The compilation processes involved in business and household surveys and in the final balance of payments and international investment compilation were described in Chapter 5. Obviously the processes are complex, and poor procedures at any one step in the process may lead to an inaccurate result. Therefore, management of these processes involves ensuring that, at each phase, objectives are set, monitored and evaluated. Careful attention is paid to:

• identifying the population of businesses and households undertaking certain types of international activity;

• ensuring that sample frames and/or collection frames and estimation procedures are up-to-date;

• design and testing of questionnaires to ensure that they are understood and can be completed by respondents;

• obtaining high response rates (95 per cent or more), with complete response from large enterprises;

• implementing and monitoring standards for data verification and analysis, and publication procedures; and

• ensuring that overall management procedures and computer systems are efficient and effective.

Examination of series over time

15.23. Series are analysed to check whether their behaviour over time provides a good explanation of economic activity, or whether they behave in erratic and inexplicable ways. Often, related series are examined to determine whether a ratio formed by two or more series behaves in an appropriate way. Examples of such ratios are: freight rates (comparing freight earned to imports f.o.b.), and investment yields (comparing investment income to the level of international investment, for various types of instruments of investment).

Data confrontation

15.24. Data confrontation studies, in which the same or similar data items provided in different collections are compared and reconciled, if possible on an enterprise by enterprise basis, but also for groups of data providers, have proved useful in improving quality. Data confrontation at this detailed level has been very useful, for example, in identifying deficiencies and improving data quality in international investment position and financial accounts statistics. It is also useful in resolving problems with the definitions of units reporting in the different surveys, or where classifications are employed differently. That is, it also helps in identifying errors from statistical compilation as well as reporting differences by providers.

Partner country comparisons

15.25. Another form of data confrontation is to examine Australia’s balance of payments and international investment statistics by partner country with those countries’ corresponding data for Australia. There are many conceptual and practical difficulties in undertaking such comparisons. However, bilateral studies are regularly undertaken comparing Australian international trade statistics with particular countries against those countries’ corresponding data. Such work undertaken in bilateral comparisons of data by region with our trading partners, both within the framework of the Asia Pacific Economic Co-operation (APEC) initiative (see paragraph 18.23) and more generally, has reaffirmed the quality of Australia’s merchandise trade aggregates. For example, several joint studies in bilateral reconciliation between the ABS and the US Bureau of the Census, covering merchandise trade flows (on an international merchandise trade basis) between Australia and the United States of America, demonstrated that a significant part of the asymmetry in the bilateral results came from conceptual factors underlying the compilation of the statistics. Actual data errors proved to be insignificant. The residual (unexplained) discrepancies represented 4.8 per cent and 6.3 per cent, respectively, of Australia’s published merchandise import and export trade with the USA for 1994. Bilateral merchandise trade reconciliations have also been carried out with Japan, New Zealand and the European Union.

Assessment of revisability

15.26. From an analytical perspective, users want to know how much reliance they can place on an estimate. If it is likely to be revised, how much is that revision likely to be and in which direction? Are decisions made on the basis of initial estimates likely to prove ill-founded in the light of later revisions? To help answer these questions, an analysis of the revisability of the various series, measured by the extent to which the estimate published initially is revised as it is republished, may be undertaken in a variety of ways. The two measures on which most focus is placed, namely bias and dispersion, are summarised in box 15.2.

	15.2. BIAS AND DISPERSION
	Bias is a measure of the extent to which the initial estimate is generally lower or higher than the latest estimate, and indicates the direction of revisions. Dispersion is a measure of the spread of latest estimates about the initial estimates, and indicates the magnitude of revisions. The ABS Information Paper: Quality of Australian Balance of Payments Statistics, 1996 (Cat. no. 5342.0), provides a thorough review of the revisability of these statistics for the period from 1986 to 1994. It concluded that: initial estimates of most items were negatively biased, i.e. the initial estimates were understated, and that generally positive revisions would be required in later estimates; the size of revisions required was highest for income debits and credits, making them the least reliable of all the current account components for all frequencies, which reflected the errors contained in the reinvested earnings (income credits) estimates; and initial estimates of the current account deficit tended to be relatively closely dispersed around the final estimate, while initial estimates of financial assets carried a strong negative bias (implying an understatement) and the behaviour of revisions was erratic. This means that the initial estimates of the net errors and omissions item, and subsequent revisions to it, largely reflect deficiencies in measuring net financial account transactions.

• the direction of revisions within the year immediately following initial release of an estimate (will the estimate go up or down as it is revised?);

• the magnitude of those revisions (are the initial estimates in the right ball park?); and

• whether those revisions generally move the series in the right direction (will the early revisions provide data of improved quality on which to make decisions?).

• revisability analysis inevitably reflects past experience and may not be a good indicator of the likely behaviour of the statistics in the immediate future;

• the findings for aggregate or net series should be treated with caution as they will reflect the varying impacts of revisions on their component series;

• substantial change in the volume and size of transactions for an aggregate occurring over a relatively short time will make the effect on the revisability of the series difficult to predict;

• the latest estimate is assumed to be a better approximation of the notional true value than earlier estimates; and

• significant changes in a concept or methodology need to be considered carefully in quality assessment. For example, a better approximation of a concept will improve accuracy, but at the cost of revising series and impairing revisability measures unless these impacts can be isolated.

Subjective assessment of accuracy

15.30. Analyses of the statistical processes used in the ABS, observation of the types of error occurring, examination of residuals and of consistency in the behaviour of series, and comparisons with partner country data, together with the revisions history of the series, have led to the subjective assessments of quality shown in table 15.3. The assessments relate to the first published monthly goods and services estimates, the quarterly and annual estimates for the principal balance of payments aggregates, and the first published quarterly and annual estimates for the principal international investment position aggregates. To give an idea of the relative importance of each item, 1996-97 values are also shown. The ratings given are current assessments of the quality of the estimates in terms of (i) the possible discrepancy between the estimated value and the true value, and (ii) the upper bounds in which revisions may occur from time to time.

	15.3. SUBJECTIVE ACCURACY RATINGS OF PRINCIPAL BALANCE OF PAYMENTS AND INTERNATIONAL INVESTMENT POSITION AGGREGATES (a)
					Accuracy of 1995-96 estimates
		Aggregate	Value in 1996-97 ($m)		Initial monthly estimates in Cat. no. 5368.0	Initial quarterly estimates in Cat. no. 5302.0	Initial annual estimates in Cat. no. 5363.0
	CURRENT ACCOUNT Goods
		credits	80,934		A	A	A
		debits	-79,438		A	A	A
	Services
		credits	24,384		B	B	A
		debits	-24,103		B	B	A
	Income
		credits	8,319		. .	C	B
		debits	-27,753		. .	B	A
	Current transfers
		credits	3,377		. .	C	C
		debits	-3,251		. .	C	C
	Balance on current account		-17,531		. .	B	B
	CAPITAL ACCOUNT		1,318		. .	C	B
	FINANCIAL ACCOUNT
	Direct investment
		abroad	-5,908		. .	C	B
		in Australia	11,282		. .	C	B
	Portfolio investment
		assets	-3,548		. .	C	C
		liabilities	16,927		. .	D	C
	Other investment
		assets	-3,558		. .	D	C
		liabilities	7,480		. .	D	C
	Reserve assets		-5,224		. .	A	A
	Balance on financial account		-17,451		. .	D	C
	INTERNATIONAL INVESTMENT POSITION
	Australian investment abroad
		Direct investment abroad	-67,766		. .	B	B
		Portfolio investment assets	-69,441		. .	C	C
		Other investment assets	-38,656		. .	C	C
		Reserve assets	-22,790		. .	A	A
	Total foreign assets		-198,653		. .	C	B
	Foreign investment in Australia
		Direct investment in Australia	151,115		. .	B	B
		Portfolio investment liabilities	295,816		. .	C	B
		Other investment liabilities	63,363		. .	C	B
	Total foreign liabilities		510,295		. .	C	B
	Net international investment position		311,642		. .	C	B
	(a) The accuracy ratings used relate to the following approximate margins of error:      A  less than 5 per cent;      B 5 per cent to less than 10 per cent;      C 10 per cent to less than 15 per cent;      D 15 per cent and greater. Source: Balance of Payments and International Investment Position, Australia, March quarter 1998 (Cat. no. 5302.0).