2021 Census overcount and undercount methodology

Scope refers to the group of people about which information is required. For PES, the population of interest was all Australian residents who should have been counted in the Census.

To produce a measure of undercount in the Census, as well as a population estimate, the scope of the PES included all Australian residents who were in Australia on Census night.

To measure Census overcount, the PES scope was expanded to include:

• Australian residents who were outside Australia on Census night
• Overseas visitors
• Babies born after Census night.

People who were out of scope for PES include:

• All people in Other territories (e.g. Christmas Island, Norfolk Island and Jervis Bay Territory) or Australian external territories (e.g. Australian Antarctic Territory, Coral Sea Islands Territory)
• Non-Australian diplomats or people who live in a diplomatic dwelling.

For collection purposes ABS limited the PES to a smaller survey population, meaning not all people and dwellings that were in scope for the PES were also in coverage. For example, the PES survey sample did not include non-private dwellings. To ensure people in non-private dwellings on Census night (and other places not covered in the PES) were still represented within the target population, the PES applied an adjustment as part of the weighting step.

We also put in place coverage rules for PES to ensure each person in the sample only had a single chance of selection in the survey. To ensure each person was associated with only a single dwelling we asked a series of questions in the PES interview, such as where each person usually lives and whether they (or anyone else) were staying at their usual residence during the PES enumeration period.

ABS designed the survey sample to produce reliable estimates for the net undercount rates for Australia, States and Territories and the Aboriginal and Torres Strait Islander population nationally. In 2021, the sample was increased by approximately 5% on 2016 to account for population growth.

For all of Australia except Aboriginal and Torres Strait Islander communities, the ABS Address Register was used as the sampling frame for unit selection. Use of the Address Register as the sampling frame formed a list-based multi-stage clustered design. Areas were grouped into strata based on locality and proportion of Aboriginal and Torres Strait Islander people in the population, as counted in the 2016 Census. This stratification aimed to increase the sample of Aboriginal and Torres Strait Islander people resulting in lower standard errors for the undercount rate of Aboriginal and Torres Strait Islander persons, population estimates and life table statistics. A similar strategy was used in the 2011 PES sample design. Each private dwelling within the stratum had the same probability of selection.

A sample of 33 Aboriginal and Torres Strait Islander communities was also included. Once in the community, the interviewer compiled a list of all the dwellings and from this a sample was selected using systematic equal probability sampling. A selection of any outstations associated with each selected community was also included where appropriate.

In 2021, for the first time both Census and PES used the ABS Address Register to select dwellings in particular areas (mail out areas for Census and all PES sample areas except Aboriginal and Torres Strait Islander communities). In doing so a dependency was created between the two collections that if left untreated would have caused sample bias; any people in dwellings that were missing from the Address Register had a risk of being missed in both collections.

To treat this dependency and give these dwellings a chance of selection in the PES, desktop address canvassing was conducted on a subsample of PES areas. Online resources including aerial imagery, property listings, and state/territory government planning and mapping tools were used to identify dwellings missing from the Address Register. New dwellings were added to the PES sample with certainty to capture potential dwellings missing from Census. Desktop address canvassing resulted in an increase in selected PES sample dwellings of 1.2%.

The total number of dwellings selected in the sample was 57,308. After sample loss and non-response, the number of fully responding dwellings across Australia was 45,138. This represents a response rate of 89.3%.

While the Census was self-completed, the Post Enumeration Survey was conducted via interview. The advantage to interviewing is the ability to clarify the meaning of questions and ensure the survey was fully completed. Interviews were conducted with any responsible adult of the household, who responded on behalf of all household members.

For the 2021 PES a telephone-first approach was used for most areas of the sample. Approximately seven weeks after Census night, PES letters were sent to households informing them of their selection in the survey, explaining its purpose, and asking them to phone in and complete the survey. People living in very remote areas or areas with limited mail services had their letters hand-delivered by field interviewers.

Maximising self-initiated telephone interview response enabled field interviewers to focus on following up areas of low response. The aim of the telephone-first approach was to achieve around 50-60% telephone interview uptake across the whole sample. At the end of the survey, 87% of response was via telephone interview.

In Aboriginal and Torres Strait Islander communities all interviews were conducted in person. A team of two interviewers engaged a trusted advisor living in community to learn about the community and help guide the collection activities in a culturally respectful way.

The ABS undertook extensive planning to protect staff and the community during COVID-19. Most people were able to complete the Census online, with no in-person contact from the ABS. They either received instructions in the mail or had a paper form delivered to their household.

Similarly, most people selected in PES were able to complete the interview without physical contact, by phoning the call centre as instructed in the letter they received.

The ABS conducted face-to-face visits and follow-ups where it was possible, ensuring safety measures were up-to-date and met the guidelines and advice from federal, state and territory governments.

The PES questionnaire collected personal details (name, sex, date of birth, age, relationship in household, registered marital status, country of birth and Indigenous status) to support the matching of PES person records to Census person records.

For each person in the selected dwelling, the PES also asked:

• Whether they were included on a Census form (and if so, where)
• Whether they could have been included on a Census form at other addresses (and if so, where)
• Where they stayed on Census night
• A set of questions to determine whether they should have been counted in the Census.

The addresses collected in the PES were used to search Census records to determine the number of times each person was counted in the Census.

To take account of language and cultural differences, as well as differences in the way Census is collected, the survey questions were tailored for Aboriginal and Torres Strait Islander communities.

An estimate of the number of people who should have been counted in a category in the Census (according to PES) but were missed because:

• they were not included on a completed Census form at their usual residence or another dwelling
• their dwelling was not known to Census (e.g. it was not included on the ABS Address Register or was not located by a Census field officer)
• their dwelling was mistakenly deemed unoccupied on Census night.

The PES estimated that 1,032,660 persons (4.0%) were missed in the 2021 Census, compared with 1,150,588 persons (4.9%) in 2016.

In these cases, Census was unaware of these persons and dwellings and therefore no imputation was performed. This component is different from the persons imputed into non-responding dwellings which forms part of the non-contact sector.

An estimate of the excess count of persons (according to PES) either from being counted on more than one Census form or from being counted in error (e.g. where a person was overseas on Census night and therefore should not have been counted, but was included on a Census form at their usual residence). In 2021 an estimated 314,788 persons (1.2%) were overcounted, compared with 274,673 persons (1.2%) in 2016.

Occasionally the answers obtained for a person in the PES interview were inconsistent with the equivalent answer in the Census. A response can differ because:

• A question may have been interpreted differently in the Census, without the assistance of an interviewer
• A different person may have answered the Census and PES questionnaires on behalf of one person
• The correct response may have changed between the Census and PES. Changes in age are taken into account using the actual date of birth, but other changes (e.g. marital status) may not have been identified.

The PES calculates differences in classification for persons counted in the contact sector by comparing a person’s response to a given category in PES to their response on linked records in the Census. The net difference in classification is the overall effect of this on the population estimates for a given category.

For example, the difference in classification for Indigenous status in 2021 is made up of:

• 61,667 persons who were recorded as Aboriginal and Torres Strait Islander in PES but non-Indigenous in the Census, and
• 74,166 persons who were recorded as non-Indigenous in PES but Aboriginal and Torres Strait Islander in the Census.

For Aboriginal and Torres Strait Islander people, this results in a net difference of -12,499 persons, which is an overcount.

The Census contact sector also contains some dwellings which were responding in the Census but returned only a partially completed form. Values for Age, Sex, Registered marital status, and State/Territory of usual residence were imputed during Census processing where there was not a valid answer on the Census form. Missing values for other items remained not stated in the final version of Census counts (including Indigenous status and Country of birth).

Where a PES person was linked to a Census person with a not stated response, this forms part of the undercount for that characteristic. For example, in 2021 there were 211,163 not stated Census responses for Country of birth that contributed to the net undercount for Australia-born persons, based on their response in PES.

The net undercount in the contact sector is equal to the sum of persons undercounted in the Census (i.e. gross undercount), minus persons overcounted in the Census (gross overcount) plus persons with a net difference in classification between PES and Census, plus Census category not stated.

Dwellings deemed occupied by the Census but for which no Census form was received are termed ‘occupied non-responding dwellings’. In these cases people were imputed into these dwellings and have an Age, Sex, Registered marital status and State/Territory of usual residence set. Values for all other variables were set to not stated or not applicable, depending on the imputed value for Age.

Imputed records are the primary driver of net overcount in the non-contact sector and impact the overall net undercount figure for Australia. For example, if the Census count increases due to a higher number of imputed persons but all else remains the same, net undercount decreases (and vice versa). In 2021 4.2% of the total Census count for people were imputed records, compared with 5.1% in 2016.

Where a household was approached in the PES that hadn’t yet returned a Census form, we considered any subsequent response from that household to the Census as motivated by the PES. To remove any bias associated with PES motivating Census response, all Census forms received after the start of 2021 PES enumeration were deemed ‘late returns’ and classified to the Census non-contact sector. In 2021 0.5% of persons counted in the Census were deemed late, compared with 2.4% in 2016.

Confidently confirming links between Census and PES records required that linking data was present on both. Census records that had insufficient personal identifier information were treated in a similar way to late returns and included in the non-contact sector. This removed any bias introduced through linking error which would have artificially inflated the PES population estimate and the net undercount. In both 2021 and 2016, 0.1% of persons counted in the Census were considered to have insufficient information for linking and were included in the non-contact sector.

The estimate of net undercount in the non-contact sector is equal to the difference between the PES estimate of the number of people who should have been counted in the non-contact sector (as reported in the PES) and the Census count of people in the non-contact sector.

Statistics produced from the PES are subject to sampling error. Sampling error is the result of random variation and can be estimated using measures of variance in the data. In this publication we report the Standard Error (SE) for both counts and proportions. There are about two chances in three that an estimate will differ by less than one SE from the figure that would have been obtained if the whole population had been included. There are about 19 chances in 20 that an estimate will differ by less than two SEs.

Difficulties associated with the linking process mean there is a risk of failing to link people who were actually included in the Census. The effect of not linking when there should have been a link results in overstating net undercount in the Census.

If the variables used to establish the link are of poor quality (e.g. not stated or imputed), links are less likely to be made. Census records that had insufficient personal identifier information were moved to the Census non-contact sector and a small number of PES records with insufficient personal identifier information were trimmed from the file.

There is also risk of linking people incorrectly to Census records. Any link error stemming from this is considered negligible following the rigorous quality assurance processes and scrutiny of final linking outcomes.

Correlation bias occurs when there is a relationship between a person responding to Census and responding in the PES. This correlation in response could lead to an underestimation of the population estimate.

Bias can be caused by correlation in the sampling frame; if a dwelling is not on the ABS Address Register, there is a risk of it being missed on both Census and PES. This potential correlation bias was treated by address canvassing a portion of PES sample areas to identify addresses missing in the frame and including them in the sample.

Correlation bias can also come in the form of non-response bias; people who avoid being included in the Census may also avoid being included in the PES. This has been treated through the weighting process. Weighting classes are used in the estimation model such that within a weighting class there are no systematic patterns of response.

The PES provides an estimate of the total number of people who should have been counted in the Census non-contact sector (i.e. late returns, imputed persons in non-responding dwellings, and persons with insufficient personal identifier information on their Census form).

PES estimates of the population in the non-contact sector have relatively high sampling errors, mainly because Census person counts for this sector were not available to use as a weighting benchmark, but also due to the small sample size (as there were relatively few Census non-contact sector dwellings selected by chance in the PES sample).

This lack of Census person counts also means that, while the dwelling weights used for the non-contact sector were estimated from the sector itself, the adjustments applied to provide final person weights use the information observed in the contact sector. This is a potential source of non-sampling error, as is any bias arising from peculiarities of the non-respondents in this sector. Both these sources of non-sampling error are expected to be small, compared with the sampling error of the population estimate for the non-contact sector.

Aboriginal and/or Torres Strait Islander community

A geographic location bounded by physical or legal boundaries, which is inhabited or intended to be inhabited predominantly by Aboriginal and/or Torres Strait Islander peoples, with housing or infrastructure that is managed on a community basis.

Aboriginal and Torres Strait Islander people

People who identified themselves, or were identified by another household member, as being of Aboriginal origin, Torres Strait Islander origin, or both Aboriginal and Torres Strait Islander origin.

Address Register

The ABS Address Register is an up-to-date, comprehensive list of all known physical addresses within Australia. It is used to produce the Census Mail Out Frame to support the mail out of Census materials to households across Australia and the PES frame.

Automated Data Linking (ADL)

ADL is an automated linking process used to determine possible links between Census and PES data, before any clerical matching begins. It employs a probabilistic linking method that uses a range of personal and addresses characteristics to evaluate the likelihood that a PES and Census record pertain to the same individual. Freely Extensible Biomedical Record Linking (FEBRL) software is used as part of ADL.

Census count

The Census counts people where they were located on Census night and this count of the population is referred to as the place of enumeration count. A count of the population based on their usual residence is also available. The Census count includes imputed persons for non-responding dwellings with values assigned for Age, Sex, Marital Status and Usual Residence. Indigenous status and Country of Birth do not have values imputed for them.

Census field officer

Census field officers are ABS staff that deliver Census instructions and forms and visit households that have not responded to the Census.

Contact sector

• Persons in dwellings that were deemed occupied on Census night from which a form was received before the start of PES collection
• Persons in occupied dwellings that were entirely missed by Census
• Persons missed by Census because their dwellings were mistakenly deemed unoccupied on Census night.

Desktop canvassing

The processes of analysing information about addresses using online resources. This can include aerial imagery, property listings, and state/territory government planning and mapping tools.

Dwelling

A building or structure in which people may live. Examples of dwellings include: houses, flats, caravans, tents, humpies and houseboats. Houses under construction, derelict houses, vacant tents, or unoccupied converted garages, are not counted as dwellings in the Census. Dwellings are categorised as either private or non-private.

Estimated Resident Population (ERP)

The official measure of the population of Australia based on the concept of usual residence. It refers to all people, regardless of nationality, citizenship or legal status, who usually live in Australia, with the exception of foreign diplomatic personnel and their families. It includes usual residents who are overseas for less than 12 months over a 16-month period. It excludes overseas visitors who are in Australia for less than 12 months over a 16-month period.

Greater Capital City and Rest of State Regions

Greater Capital City and Rest of State Regions are part of Greater Capital City Statistical Areas (GCCSA) and are geographical areas built from Statistical Areas Level 4 (SA4).

There are 16 spatial GCCSA regions covering the whole of Australia without gaps or overlaps. These consist of 8 ‘Greater Capital Cities’ representing each of the Australian State and Territory capital cities, and 8 regions covering the rest of each State and the Northern Territory. The ACT only has one GCCSA region for the entire Territory. The category of Other Territories in GCCSA includes the Other Territories of Jervis Bay, Christmas Island, Cocos (Keeling) Island and Norfolk Island.

Gross coverage error

Gross coverage is a measure that estimates of the number of persons that were ‘covered’ by the Census; i.e. the number of unique real people that were included on a returned Census form. Gross coverage error is the difference between the PES population estimate and the gross coverage estimate.

Imputation

A statistical process for predicting values where no response was provided to a question and a response could not be derived. Census imputes persons into non-responding dwellings, and also imputes some missing values (e.g. Age and Sex) for responding persons who left these fields blank.

Late Return

A Census form returned after the start of PES collection.

Net undercount

The difference between the PES estimate of the number of people who should have been counted in the Census and the actual Census count (including persons imputed into non-responding dwellings). The estimated net undercount for a category of person is the net result of undercount, overcount, differences in classification between the PES and Census (e.g. Age, Sex, Indigenous status) and imputation error in the Census.

Non-contact sector

The Census non-contact sector comprises persons in dwellings that were deemed occupied on Census night, from which no Census form was received (imputed dwellings); persons in dwellings whose Census form was received after the commencement of PES enumeration (late returns); and persons with insufficient personal identifier information on their Census form.

Non-private dwelling

An establishment which provides a communal type of accommodation, such as a hotel, motel, hospital or other institution.

Outstation (or homeland)

A discrete Aboriginal and Torres Strait Islander community that is administered by, or linked to, an organisation such as a resources agency or larger parent discrete Aboriginal and Torres Strait Islander community for the provision and maintenance of services.

PES population estimate

The number of Australian residents who should have been counted in the Census as estimated by the PES.

Post Census Review

The Post Census Review was the public facing name of the 2021 Post Enumeration Survey. The name was introduced to better reflect the survey and its purpose in plain language and was selected after qualitative testing via cognitive interviewing.

Private dwelling

A residential structure that is self-contained, owned or rented by the occupants, and intended solely for residential use. A private dwelling may be a house, a flat, and can also be a house attached to, or rooms above, shops or offices.

Sampling error

Sampling error occurs because a sample, rather than the entire population, is surveyed. The standard error gives a measure of the likely difference resulting from not including all dwellings in the survey. There are about two chances in three that a sample estimate will differ by less than one standard error from the figure that would have been obtained if all dwellings had been included in the survey, and about nineteen chances in twenty that the difference will be less than two standard errors.

Scope

Refers to the group of people about which information is required. For PES, the population of interest is all Australian residents who should have been counted in the Census.

Standard error (SE)

The SE is a measure of the likely difference between the true value and the estimate.

Undercount adjustment factor

The ratio of the PES population estimate to the Census count. This factor can be applied to the Census counts to indicate how many people should have been counted in the Census for that category.

Usual residence

In the Census, the usual residence for a resident of Australia is defined as the place where they have lived, or where they intend to live, for six months or more, in the Census year.