6523.0 - Household Income and Income Distribution, Australia, 2000-01

ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 23/07/2003

Page tools: Print

Print Page Print all pages in this product

APPENDIX 4 - NEW BENEFIT TRANSFER BENCHMARK

INTRODUCTION

In compiling this publication, an unexpected and significant decline was observed in the coverage of government pensions and allowances (or cash benefit transfers) as an income source in the SIHC results. The problem was initially reported in the feature article titled 'Upgrading Household Income Distribution Statistics', published in the April 2002 issue of Australian Economic Indicators (cat. no. 1350.0). This Appendix describes the results of the investigations undertaken and the steps taken to resolve the problem.

As identified in the April 2002 article, the 'coverage' of benefit transfers in the 1999-2000 SIHC had fallen significantly, to 81% of the aggregate totals published by the Department of Family and Community Services (FaCS) and the Department of Veterans' Affairs (DVA). In the four SIHC surveys to 1997-1998 coverage had been relatively stable at around 85%. Benefit transfers coverage in the then yet to be published 2000-01 SIHC data had fallen further, to 78%. If the lower coverage of SIHC benefit transfers largely related to missing payments made predominantly to households represented in the lowest two income quintiles, the change in understatement would have impacted very significantly on several of the measures used to assess income inequality.

UNDERCOVERAGE OF BENEFIT TRANSFERS

Population scope

The SIHC does not attempt to capture all benefit transfers. The scope of the SIHC is restricted to urban and rural areas of Australia, excluding remote and sparsely settled areas of the Northern Territory, and includes only the usual residents of private dwellings, such as houses, flats, units, caravans, tents and other private structures that are places of usual residence at the time of interview. Persons living in non-private dwellings, such as hotels, boarding schools, nursing homes and other institutions, are excluded. Persons residing abroad and receiving Australian government benefit transfers are also excluded from the scope of the SIHC.

Underreporting

The SIHC fails to collect some benefit payments that are made to people in scope of the survey. In some cases, respondents fail to report all their income, including government benefits. Respondents are asked to report the latest amount received as benefit transfers. These amounts are likely to be reported in SIHC, at least in part, as the net cash transfers usually received by the respondent. Amounts that are deducted at source, such as tax, rent or other regular commitments for which arrangements have been made for automatic deduction by Centrelink, may be excluded by some respondents. Amounts that are received less frequently than fortnightly, such as a quarterly telephone allowance, may also be excluded. Respondents may also fail to report all their income for a variety of other reasons, such as privacy concerns, difficulties in remembering income details, and unwillingness to reveal fraudulent or other illegal activity.

Non-response bias

Survey results are expanded to estimates for the whole population by applying weights to survey responses. In calculating the weight to be applied to each respondent, benchmarking procedures are used to ensure that the expanded estimates are consistent with the demographic characteristics of the population as established by Population Censuses and intercensal demographic estimates. It is then assumed that survey respondents are representative of all people in the population.

While demographic benchmarking ensures consistency for certain demographic characteristics, this may not be the case for other characteristics being collected in the survey, such as income and source of income. The most problematic aspect is the extent to which survey respondents may differ from people who reside in dwellings selected in the survey but from whom responses were not obtained. Such differences are called non-response bias. Non-response bias may result in undercoverage or overcoverage in final survey estimates. In the case of the SIHC, aggregate estimates of total benefit transfers may therefore exhibit undercoverage or overcoverage because of non-response bias.

Undercoverage over time

The net effect of scope restrictions, incomplete reporting and the population benchmarking adopted, was a substantial but stable difference from 1994-95 to 1997-98 between aggregate government benefits estimated from the SIHC and aggregate benefits paid by government agencies. Variations from year to year were within the range expected to be associated with sampling error. Such undercoverage of real world income flows to households impacts on other sources of income in similar ways. The extent of undercoverage of each income source will affect the estimation of household income levels and the measurement of income distribution at any point in time. As long as undercoverage is relatively stable over time, the impact on measuring changes in income and its distribution will be limited.

However, benefit transfers coverage declined significantly over the two SIHC cycles after 1997-98, as can be seen in the top segment of table A3. If the increased SIHC undercoverage was due to reporting error by individuals, or processing error, or a real world change not captured in individual reporting through SIHC methodology, there was the potential for significant misrepresentation of the changes in income distribution in Australia. In addition, analysis by life cycle groups was likely to be affected by such a major omission of one income source that is more significant to certain groups.

INVESTIGATING WELFARE TRANSFERS ESTIMATION

A number of different avenues have been investigated in seeking to understand and correct for the decline in benefit transfers coverage. These include possible systems errors, appropriateness of the coverage comparison being made between aggregate SIHC estimates and aggregate benefits paid by government agencies, changes in the way that benefit transfers are made which might not be captured in the SIHC, changes in the quality of reporting by households, and options for and appropriateness of the weighting methods used to compile aggregate results.

Processing error

The SIHC processing system had been relatively stable since its inception in 1994-95. A review of the system did not identify either any system changes that might only have impacted on 1999-2000 and 2000-01 benefit transfers estimates, or systems errors that might only be reflected systematically only in estimates for the most recent two survey cycles.

Coverage comparison between SIHC estimates and aggregate benefits paid by government agencies

Because of the audit scrutiny associated with government outlays, there is little likelihood of significant error in the published aggregate benefit amounts. It was possible that changes in the nature of accounting for the expenditures, changes in the population composition of benefit recipients, or changes in the way that recipients were provided with their benefits may have impacted on the validity of the coverage analysis being undertaken. However, ABS investigations showed that a stable relationship could be expected, over the period 1994-95 to 2000-01, between SIHC measures of benefit transfers and the aggregate transfers values published by the Departments of Family and Community Services and Veterans’ Affairs because:

the proportion of benefit recipients in special dwellings or overseas had been stable over the period when SIHC coverage declined
while accrual accounting was introduced as the basis of compilation for published benefit transfer aggregates from 1998-99, the nature of the changes were not such that they would have had an adverse impact on apparent SIHC coverage
published aggregate transfer values only relate to benefits paid, and so do not include, for example, administrative costs which may have increased in recent years
analysis of movements in selected aggregates, such as age pension payments, tracked announced changes in both benefit levels and eligibility criteria
even if all affected respondents failed to include the value of the automatic deductions made on their behalf by Centrelink, such as tax or rent, the scale of the increase in such deductions was not sufficient to have a marked effect on the coverage ratios over the period under analysis.

There was one error identified through this analysis. Under the income concept used in the SIHC, the survey had failed to collect information about the one-off payment to seniors paid in 2000-01 to income support recipients who had reached age pension age. Correcting this error accounted for 1.1 percentage point of the 3.5 percentage point deterioration in coverage in that year, but does not account for any of the deterioration in 1999-2000.

In summary, except for the one-off payment to seniors, no errors were found in the process of comparing the SIHC benefit transfer estimates with published aggregates.

Misreporting by SIHC respondents (measurement error)

Possible causes for respondent error contributing to the declining coverage of benefit transfers reported in SIHC included:

respondents increasingly understating the amount of benefit transfers that they receive
respondents increasingly declining to acknowledge that they were recipients of benefit transfers, whether from a privacy perspective, from a desire to hide fraudulent activity, or otherwise.

To assess the accuracy of respondents' reporting, the benefits reported by individuals were compared to estimates of apparent benefit entitlements modelled on the basis of other reported information such as age, non-benefit income, and number of children. The analysis did not reveal any obvious decline in the average individual benefit level being reported relative to the apparent benefits entitlement. If a decline had been detected it might have suggested an increasing tendency to understate the individual amounts received. Nor did this analysis identify any increase in people not reporting welfare transfers when they had no other significant sources of income. For example, the number of persons reporting that they received the age pension in SIHC was a constant proportion of the total number of persons in the SIHC sample who were of age pension age and also had little other income.

It is possible that persons who are not entitled to receive benefit transfers, perhaps because they receive other incomes, but nevertheless claimed and received benefits, do not report the fraudulently claimed benefit income to the ABS. While this possibility is plausible for some benefit types, no evidence of an increase in fraud was identified. And no plausible explanation was identified for fraud to be the cause of an across-the-board decline in coverage of all major pension types in 1999-2000, including age pensions, disability pensions and service pensions, nor why that level of fraud would accelerate in 2000-01.

In summary, although there may well be some misreporting by SIHC respondents, no evidence was found for any significant deterioration over the latest two years.

Differential undercoverage and demographic benchmarks

As with other household surveys, the estimation and weighting of SIHC includes a process of benchmarking to known demographic totals (i.e. population totals of people and households, classified by age, sex, state, etc.). One of the reasons to benchmark a survey is to maximise the extent to which the survey results represent the full population being surveyed. Subgroups that responded less well to the survey are therefore given larger weights than subgroups that responded more fully. However, if non-respondents differ from respondents in characteristics other than those being benchmarked, survey estimates are still subject to non-response bias.

There were several indicators that the impact of non-response on the SIHC is changing and the profile of survey respondents is becoming less representative of that of non-respondents. As a result, the SIHC estimation methodology may not have been fully effective in accommodating changing non-response patterns, leaving the potential for bias in the coverage of incomes that might result. These indicators were:

SIHC response rates that had been relatively stable at about 90% over the period 1994-95 to 1997-98, but slipped to 85% from 1999-2000, the first year of the decline in benefit transfer coverage
an apparent and significant over-representation of children in the weighted SIHC results, indicating that households with children were more likely to respond in the SIHC than households without children
the across-the-board nature of the decline in coverage of benefits suggested that weighting to demographic benchmarks was not fully compensating for differential undercoverage in the sample responses.

Various demographic benchmarking options were analysed in trying to deal with the range of representational dimensions required in SIHC results and adjusting for the undercoverage of different demographic sub-populations. While the varying combinations of benchmarks had some impact on the level of measured benefit transfers, the variations in results were usually within one standard error of each other (and at most within two standard errors) and did not offer a solution to the coverage gap. The range of benchmarking options also had virtually no impact on any of the usual summary measures of income distribution.

In summary, while the declining response rates may be associated with changing response patterns by different types of households, it is not something that can be corrected by demographic benchmarking alone.

In arriving at the final SIHC demographic benchmarks used in the revised income distribution measures reported below, the main change has been to benchmark to the number of children in the age ranges of 0-4 years, and 5-14 years, by state. However, introducing this important improvement in benchmarking, and a desire to have an estimation regime consistent across all years, required the following benchmarks that had been previously applied to be foregone:

quarterly and half yearly benchmarking
state by household counts.

The removal of sub-annual benchmarking is not considered significant to the quality of the SIHC results. While state household counts have been removed from the benchmarking, a range of state benchmarks remain (age groups by sex, state by part of state, state by labour force status), the new state by children age groups benchmark has been introduced, and national household benchmarks remain.

BENCHMARKING TO BENEFIT TRANSFERS AGGREGATES

Following the investigation of the range of issues, discussed above, that could potentially contribute to the decline in SIHC coverage of benefit transfers, ABS concluded that the increasing SIHC undercoverage of benefit transfers resulted from an increase in the differential undercoverage of benefit recipients that could not be accommodated by demographic benchmarks alone. To directly address the undercoverage of benefit transfers the ABS has therefore introduced explicit benefit transfers benchmarks for the 1999-2000 and 2000-01 SIHC estimates. This is consistent with the general approach of benchmarking to address differential response rates and coverage deficiencies, such as not collecting data from certain geographic areas for which the populations are nevertheless incorporated in demographic benchmarks.

Several issues were considered in deciding how to benchmark to benefit transfers.

Should benchmarking be to numbers of benefit recipients or to value of benefits paid?
Should benchmarking be done at an aggregate level or by benefit type?
Should benchmarking be to 100% of the FaCS/DVA values or some lower amount?

Numbers of benefit recipients or value of benefits paid?

It was decided to benchmark to value of benefits rather than to number of recipients, because the available data on value of benefits is more reliable. While the benchmarking process ensured consistency with respect to the value of benefits, the process achieved this by increasing the survey weights assigned to respondents reporting benefits and decreasing the weights of other respondents. In other words, the benchmarking process increased the estimated number of benefit recipients, and did not amend the values of individual respondents.

Aggregate level or by benefit type?

In theory, it would have been desirable to benchmark to income from individual benefits, or at least to income from broad groups of benefits, because the undercoverage has behaved differently for different benefit types over the years that SIHC has run.

However, it is known that there is some misclassification between the benefit types by respondents, such as Newstart received while ill being reported as sickness allowance. To compound the problem, the rules defining the boundary between the two have changed over time, and the degree of misclassification is likely to be greater now than in earlier years. There have also been other structural changes in benefits over time, such as youth allowance previously being part Newstart and part Austudy.

It is not possible to translate coverage rates between components in the old structure to accurately target coverage rates in the new structure, especially when dealing concurrently with both misclassification and changes in classification. Therefore attempting to benchmark to individual benefit types would imply a greater sense of accuracy than could be achieved. An analysis of the impacts of the two choices of benchmarks showed that there would be little difference between the two approaches in practice, and so it was decided to benchmark to the total income from benefits.

To 100% of the value paid by government agencies or some lower amount?

Options also exist on whether to benchmark to 100% of aggregate benefits that are within scope of SIHC, or to some lesser amount. For the early, apparently stable part of the series, the survey was accounting for about 85% of aggregate benefits. Some part of the difference is attributable to the scope differences, discussed earlier, although the exact amount is not known.

In theory, if there is no measurement error in the data, the remaining undercoverage could be removed by benchmarking the sample to the total amount of benefits. However, there may be significant differences between the benefit reported by respondents and the actual amount of benefit transfers paid to them by government agencies, and benchmarking may not be an appropriate means of addressing this problem.

Excluding the impact of the scope differences, the undercoverage is likely to result from a combination of misreporting, or measurement error, and a failure of the benchmarking process to completely account for the impact of rising differential undercoverage. While it has been concluded that increasing measurement error does not seem to be the cause of the decline in survey coverage of benefits in recent years, measurement error may well be a significant contributor to the 'base' amount of undercoverage through the whole period. Benchmarking is not an appropriate means of correcting for measurement error if the conceptual basis of the survey response is different from that of the benchmark aggregate.

Furthermore, SIHC estimates of income other than from benefit transfers are also likely to be affected by measurement error. Correcting just the benefit income for such deficiencies, by increasing the incomes of those at the lower end of the income distribution, would alter the apparent income distribution observed in the SIHC. But it is not possible at this time to determine whether such a change would increase or decrease the accuracy of the distribution measures.

As it is not known how much of the 15% 'base' undercoverage is attributable to the impact of differential undercoverage, it was decided that the benefit value benchmark should only be applied from 1999-2000 and that it should only be used to remove the deterioration in the survey coverage of benefit transfers that occurred from that time, that is, the increase in undercoverage beyond the base amount of approximately 15%.

IMPACT OF CHANGES

Three distinct changes were made to the SIHC estimates of income as a result of the work described in this Appendix.

First, the estimates for all years prior to 2000-01 were recalculated using the most up to date demographic benchmark data available and a consistent estimation and weighting system was introduced for all years through to 2000-01. It should be noted that the demographic benchmark data are based on the 1996 Census, not the 2001 Census. Household benchmark data based on the 2001 Census are not yet available and it is essential that the person benchmark data and household benchmark data are consistent.

Second, estimates for the one-off payment to seniors were modelled and added to respondent records for 2000-01.

Third, the additional government cash benefit benchmark was introduced for 1999-2000 and 2000-01 to maintain the SIHC coverage of transfer benefits at a consistent level over time.

Impact on government benefit transfers

The impact of the changes on the SIHC coverage rates of government benefit transfers is shown in table A3. As can be seen, at the start of investigations, the 1999-2000 coverage ratio of 81.2% was substantially below that of 1997-98 but not very far below the previous lowest point of 82.9%. The 2000-01 ratio fell a further 3.0 percentage points, to 78.2%.

After revisions were made to the demographic benchmarks for the years up to 1999-2000, the introduction of identical estimation and weighting procedures for all years, and the introduction of imputed estimates for the one-off payment to seniors in 2000-01, the fall in the coverage ratio between 1997-98 and 1999-2000 was not as great as previously estimated. However, the coverage ratios still showed a clear downward trend in the two years to 2000-01. The fall was even more apparent insofar as the ratios for the first four years showed less variation, after the estimation and weighting system had been standardised, than had been apparent at the start of the investigations. The first four observations now fell within a range of 1.3 percentage points, but there was still a 2.4 percentage point decline from 1997-98 to 1999-2000 and a further 2.4 percentage point decline to 2000-01. Without the contribution of the imputed estimates for the one-off payment to seniors, there would have been a 3.5 percentage point decline to 2000-01.

By definition, the introduction of the government benefit transfer benchmark for the last two years lifted the overall coverage ratio for those years to the benchmark level, that is, 84.7%. (This is marginally higher than the average of the first four years (84.4%) because the values feeding into the benchmark calculation were derived before the estimation and weighting system had been finalised.) The benchmark was applied to total benefits excluding the one-off payment to seniors. However, it can be seen that the magnitude of the impact varied between benefit types. Of the benefit categories shown in table A3, age pension was least affected (up by 3.0 percentage points in 2000-01) and disability support pension the most (up by 7.2 percentage points in 2000-01). The differences reflect the interaction between this particular benchmark and all the demographic benchmarks.

Impact on income distribution

The introduction of the government cash benefit benchmark tended to increase the sample weights of households with relatively low income and therefore lower the weights of households with relatively high income. Consequently, the values of income at the percentile boundaries shown in table A4 were all slightly lower after the introduction of the new benchmark. There was no impact on the percentage share figures (to one decimal place). Some of the percentile ratios measured slightly less income inequality, although P80/P20 and P20/P50 measured slightly greater inequality in 1999-2000. The Gini coefficient would be slightly higher in 2000-01 if a benefit benchmark had not been introduced. In all cases, the revisions to the measures were considerably smaller than one standard error (see Appendix 3), that is, they do not make a significant difference to the interpretation of the indicators.

Similarly, the correction to include imputed values for the one-off payment to seniors decreased the measures of inequality very slightly, and slightly increased the values of income at the percentile boundaries.

A3 COVERAGE RATES OF FACS AND DVA BENEFIT TRANSFERS(a)


	1994-95	1995-96	1996-97	1997-98	1999-00	2000-01

% COVERAGE

At start of investigations
Age pension	87.1	89.3	85.9	91.6	87.8	88.3
Disability support pension	62.9	67.0	74.2	86.5	77.3	76.1
Newstart	67.3	81.7	74.5	73.0	69.9	66.8
Family benefits(c)	85.6	82.6	83.9	85.9	81.0	73.9
All benefits	84.0	82.9	85.9	86.2	81.2	78.2
After standardisation of estimation(d)
Age pension	88.0	90.9	86.0	90.3	88.7	88.3
Disability support pension	63.7	69.7	73.6	87.0	81.4	81.5
Newstart(b)	66.0	77.0	72.0	73.1	69.1	66.0
Family benefits(c)	85.6	83.4	82.4	84.4	80.3	71.8
All benefits	84.7	83.6	84.5	84.9	82.5	79.0
After imputation of values for one-off payment to seniors in 2000-01
All benefits	..	..	..	..	..	80.1
After introduction of benefit transfer benchmark in 1999-2000 and 2000-01
Age pension	..	..	..	..	90.0	91.3
Disability support pension	..	..	..	..	84.5	88.7
Newstart(b)	..	..	..	..	71.3	70.6
Family benefits(c)	..	..	..	..	82.6	75.2
All benefits	..	..	..	..	84.7	84.7

.. not applicable
(a) Government benefits paid by Departments of Family and Community Services and Veterans' Affairs that fall within the definition of income used in this publication.
(b) Includes Jobsearch and Youth Training Allowance.
(c) Includes Family Allowance, Family Payments, and Family Tax Benefit.
(d) Includes revision of demographic benchmarks for 1999-2000 and earlier years, and introduction of a standard estimation and weighting system for all years.

A4 INCOME DISTRIBUTION, EQUIVALISED DISPOSABLE HOUSEHOLD INCOME


		1999-2000		2000-01

Indicator		Without transfers benchmark	With transfers benchmark	Without transfers benchmark or OOPS (a)	With transfers benchmark but not OOPS(a)	With transfers benchmark and OOPS(a)

Income per week at top of selected income percentiles, in 2000–01 dollars(b)
20th [P20]	$	242	241	244	243	245
50th [P50]	$	407	405	416	413	414
80th [P80]	$	636	636	647	643	644
Share of total income received by persons with
High incomes(c)	%	38.4	38.4	38.5	38.5	38.5
Low incomes (d)	%	10.5	10.5	10.4	10.4	10.5
Ratios of incomes at top of selected income percentiles
P90/P10	ratio	3.90	3.89	4.03	4.01	3.97
P80/P20	ratio	2.63	2.64	2.66	2.65	2.63
P80/P50	ratio	1.57	1.57	1.56	1.56	1.56
P20/P50	ratio	0.60	0.59	0.59	0.59	0.59
Gini coefficient	no.	0.310	0.310	0.313	0.312	0.311

(a) Imputed value of one-off payment to seniors.
(b) Adjusted for changes in the Consumer Price Index.
(c) Persons in the top income quintile (9th and 10th deciles) after being ranked by their equivalised disposable household income.
(d) Persons in the 2nd and 3rd income deciles after being ranked by their equivalised disposable household income.

Back to Main Features