Appendix B The Census, Post Enumeration Survey, and the Estimated Resident Population

The Post Enumeration Survey provides an assessment of Census coverage, or how successful the Census was in counting the total Australian population. This assessment is determined in terms of overcount and undercount.

Some of the reasons why people are counted more than once or counted in error (i.e. overcounted) include:

• they were included on the Census form at the dwelling where they usually live, even though they stayed and were counted elsewhere on Census night; and
• they were overseas on Census night and so should not have been counted at all, but were included on the Census form at the dwelling where they usually live.

Some of the reasons why people may be missed (i.e. undercounted) include:

• they were travelling and were difficult to contact;
• they mistakenly thought they were counted elsewhere;
• the person completing the form thought that, for example, young babies, the elderly or visitors should not be included;
• they did not wish to participate;
• the dwelling in which they were located was missed because it was difficult to find (e.g. in a remote or non-residential area); and
• the dwelling in which they were located was mistakenly determined to be unoccupied.

Rates of overcount and undercount can also vary significantly for different population groups depending on factors such as sex, age, ethnicity, Aboriginal and Torres Strait Islander origins, and geographic location.

The Post Enumeration Survey also provides an assessment of the total number of persons present in dwellings where Census have imputed persons. Imputation is the process that attempts to fill in basic information for the people who didn’t respond to the Census. Currently, this information is a person’s age, sex, usual residence and marital status. For these dwellings, the Post Enumeration Survey makes an entirely independent assessment of the number of persons who usually reside in these dwellings, based on the subset of sampled dwellings that match to Census dwellings where imputation has occurred. The net overcount for persons imputed[1] is derived by subtracting the Post Enumeration Survey estimate from the total of Census imputed persons. If Census under imputes persons, then the result is a net undercount for persons imputed.

Because the Post Enumeration Survey cannot directly associate person records with Census imputed persons, there is no precise way to break down the components of overcount and undercount. Approximate comparisons can be made by examining the characteristics and occupancy of Post Enumeration Survey dwellings that match to Census dwellings where imputation has occurred.

Some of the reasons why Census may over or under impute include:

• private dwellings were incorrectly identified as occupied on Census night (leading to over-imputation);
• private dwellings were correctly identified as occupied on Census night, but too many or too few persons were imputed;
• non-private dwellings had too many or too few persons imputed.

Rates of over-imputation can also vary significantly for different population groups depending on factors such as sex, age, ethnicity, Aboriginal and Torres Strait Islander origins, and geographic location.

[1] For technical reasons, the Post Enumeration Survey groups dwellings where Census has imputed people together with dwellings where a Census form was received after the start of the Post Enumeration Survey collection (‘Late Returns’) and people who provided insufficient personal identifier information on their Census form, as these groups all require the Post Enumeration Survey to make an independent estimate of the true population. However, the size of net overcount for imputed people is much larger than the undercount or overcount from the other components, so the net overcount for this group can safely be interpreted as over-imputation.

The Post Enumeration Survey provides information on overcount and undercount for the following variables, and as such, provides an indication of their quality:

• Geography (national, state and/or territory, and part of state and/or territory);
• Age;
• Sex;
• Marital status;
• Country of birth; and
• Indigenous status.

Note that ‘not-stated’ or missing values for Country of birth and Indigenous status are not imputed in the Census. Those persons who have a value of ‘not-stated’ against either of these two items contribute to net undercount estimates for the category in which they should have been counted, as reported in the Post Enumeration Survey.

After every census, the Estimated Resident Population is recalibrated, or rebased. Rebasing is done by adding the net overcount or net undercount from the Post Enumeration Survey to the new Census population counts. The result is adjusted further to include Australian residents who were temporarily overseas on Census night (identified from passenger traveller records). The final step in calculating Estimated Resident Population is to backdate to 30 June of the Census year. This small adjustment is undertaken by adding the deaths and subtracting the births and net overseas migration which occurred between 1 July and the Census date. The diagram below (Figure B.2) outlines the Estimated Resident Population rebasing calculations and adjustments.

As part of the population estimates rebasing exercise, the Post Enumeration Survey adjusted age and sex distributions are compared with independent population data sources, including administrative data sources such as the Multi-Agency Data Integration Project (MADIP) and published estimated resident population which has been updated quarterly from the previous Census. Where the Post Enumeration Survey cannot provide or does not result in plausible undercount adjustments for population sub-groups, demographic adjustments can be applied for the rebased estimates based on these other data sources.

The reliability of the Post Enumeration Survey affects the extent to which estimates can be used to accurately assess the quality of the Census. As a result, it is important to understand the quality of the Post Enumeration Survey itself and the factors that may limit its usefulness as a measure. The conceptual framework for population estimates is described in Appendix C. The key sources of error in the Post Enumeration Survey are outlined below.

Sampling error is random error resulting in either an underestimate or an overestimate because the Post Enumeration Survey is a sample survey.

The Post Enumeration Survey has relatively low sampling errors at the national level due to a large sample, but the relative sampling errors will be higher for estimates for lower level geographies.

The final Margins of Error on the Post Enumeration Survey estimates is reported in the Post Enumeration Survey publication.

Correlation bias is systematic error that results in an underestimate when Census and the Post Enumeration Survey tend to miss the same people. The Post Enumeration Survey relies on statistical independence to accurately adjust for Census undercount and/or overcount. If population groups are systematically missing from both the Post Enumeration Survey and the Census, correlation bias occurs, which can lead to an underestimate of the population.

The potential to miss the same population groups increased somewhat in 2021 because the same address framework was used for both collections. To minimise this, the steps described in Section B4.1 below were taken.

Correlation bias is partially treated by fitting mathematical models to account for the drivers of Census and Post Enumeration Survey non-response in estimation, such as age characteristics. This has been shown to be effective in reducing correlation bias, although some residual bias will remain. A detailed description of the prediction regression estimator used in the Post Enumeration Survey can be found in Research Paper: An Estimating Equation Approach to Census Coverage Adjustment, May 2007.[1] Treatment for correlation bias in 2021 is detailed in Appendix C.

There is also a risk of non-response bias when selected persons respond to the Census but do not respond to the Post Enumeration Survey.

Non-response bias occurs when the people who do not respond to the Post Enumeration Survey have different characteristics to those who do respond. This risk is greater if the characteristics of the non-respondents vary significantly from respondents, especially in their propensity to complete the Census. Non-response bias is treated through the Prediction Regression Estimation model referred to above.

[1] Research Paper: Bell, P. A., Clarke, C. F., Whiting, J. P. (2007). An Estimating Equation Approach to Census Coverage Adjustment. Canberra, Australia: The Australian Bureau of Statistics. Available at https://www.abs.gov.au/ausstats/abs@.nsf/mf/1351.0.55.019

Systematic error resulting in an overestimate occurs when the Census and the Post Enumeration Survey find the same people but fail to match them, leading to those people being treated as missing from the Census. The Post Enumeration Survey relies on linking to Census to establish which persons in the Post Enumeration Survey were counted in Census and which were not.

If a person is counted in both the Post Enumeration Survey and the Census, but the link between them is missed, the Post Enumeration Survey mistakenly reports that they are missing from the Census. This can lead to an overestimate of the number in the population.

This bias is predominantly treated through a high level of scrutiny and quality assurance attached to the linking and matches processes in the Post Enumeration Survey.

In 2016, a quality adjustment was introduced to identify and treat Census records that had insufficient personal identifier information, which is required for linking to the Post Enumeration Survey. These records were moved to the Census non-contact sector (comprising dwellings deemed to be occupied on Census night, from which no Census form was received; or a Census form that was received after the Post Enumeration Survey commenced data collection) and treated in a similar fashion to late returns, to remove the potential for any upward bias on the population estimates produced by the Post Enumeration Survey. In 2021, this concept has been extended to Post Enumeration Survey records.

This occurs if the Post Enumeration Survey respondents do not accurately recall where they were enumerated on Census night.

As in 2016, the 2021 Post Enumeration Survey was completed later than previous Post Enumeration Surveys increasing the possibility of recall error.

Recall error is managed in the Post Enumeration Survey processing by using an automated linking program which enables a wider range of possible addresses (including vague addresses) to be searched in the matching process.

The Post Enumeration Survey is a household survey. Therefore, its scope is limited to private dwellings and it is unable to detect any change in the level of undercount or overcount in non-private dwellings.

The 2021 Post Enumeration Survey had a new public facing name: the Post Census Review. The name was introduced to better reflect the survey and its purpose in plain language, and was selected after qualitative testing via cognitive interviewing.

The 2021 Post Enumeration Survey sample was increased from 2016 to maintain similar levels of accuracy while accounting for population growth. Final sample design aimed to achieve 44,359 fully responding households. Sample stratification was applied to increase the expected sample of Aboriginal and Torres Strait Islander peoples. Stratification is commonly used in household surveys to increase the sample size of subpopulations of interest and improve the accuracy of estimates of subpopulations of interest.

For the 2021 Post Enumeration Survey an Address Register based frame was used, consistent with the move of all ABS household surveys to this frame. As the Address Register is also used by the Census, independent quality assurance of the frame was undertaken through an address canvassing exercise. The purpose of the exercise was to maximise the chance that any dwellings missing or misrepresented on the Address Register were identified and had a chance of selection in the Post Enumeration Survey regardless of whether they were counted in the Census.

B.4.3.1       Telephone-first model

A telephone-first enumeration model was implemented in the 2021 Post Enumeration Survey to alleviate the increasing pressure on field interviewer resources and optimise response across the sample. Maximising self-initiated telephone interview response frees up resources to enable field interviewers to focus on areas of low response. The aim of this model was to achieve around fifty to sixty per cent telephone interview uptake across the whole sample. This was exceeded with 87% of Post Enumeration Survey response achieved through telephone interview.

The ABS sent a letter and flyer informing households of their selection in the survey, explaining its purpose, and asking them to phone in and complete the survey. The ABS built on their ongoing research program into approach materials to develop and test a letter specifically to maximise this telephone take up. The letter contained design features including a prominent and simple “call to action” and supporting information designed to reduce barriers to participation.

B.4.3.2       Aboriginal and Torres Strait Islander Community enumeration

Enumeration of Aboriginal and Torres Strait Islander Communities remained as in-person interviews. For 2021, the model teamed an ABS Centre of Aboriginal and Torres Strait Islander Statistics engagement manager with a Household Survey Interviewer. The model aimed to harness the knowledge and expertise of each to bring greater understanding in Communities around the purpose of the survey while ensuring activities had minimal impact to members of the Community.

B.4.4.1       Providing further detail on coverage in Post Enumeration Survey outputs

The 2021 Post Enumeration Survey outputs will be published on the new ABS website, requiring new styles and formatting. Alongside this, the outputs will contain greater component analysis and disaggregation.

Following the 2016 Post Enumeration Survey, including engagement with the 2016 Census Independent Assurance Panel, a proposed new additional measure of coverage has been developed in addition to the standard outputs such as gross undercount, gross overcount and net undercount. The new measure is “Gross Coverage Error” and is the difference between the Post Enumeration Survey population estimate and the Census count (with gross overcount and imputed persons removed).

Conceptually, Gross Coverage identifies the population that the Census counted (or received a form for) as a proportion of the population that the Census should have counted. Gross coverage error is the difference between the Post Enumeration Survey population estimate and the gross coverage estimate. This differs from the typical net undercount headline figure produced from the Post Enumeration Survey, which subtracts overcount from undercount.

Gross Coverage will also be calculated for the 2016 Post Enumeration Survey to enable comparison with 2021 results. This is of particular note given the changes made to the Census imputation model, which mean that measures such as this are needed to make meaningful comparison over time.

Similar gross measures of coverage are published by the United Nations and Statistics New Zealand. Net undercount will continue to be reported as the headline measure in the Post Enumeration Survey publication in keeping with international practice.