Appendices

Historically, the occupancy of a dwelling in censuses was largely determined by field observation, including contact with the householder or neighbours. Occupancy was also determined when householders directly phoned the Census Contact Centre.

While occupancy rates in the 2016 Census were consistent with previous Censuses, the 2016 Post Enumeration Survey indicated that a proportion of dwellings were incorrectly deemed as occupied on Census night and therefore that the 2016 occupancy rates were over-stated. The 2016 Census Independent Assurance Panel highlighted the opportunity to improve the accuracy of occupancy determination.

New approaches implemented in the 2021 Census include:

• In 2021, householders could use the Census Digital Service to advise that a dwelling was unoccupied on Census night. There were two ways of doing this. First, through the online Census form in which householders who entered ‘0’ persons present were streamed out of the form to report the dwelling as unoccupied. This pathway was called the ‘zero stream’. A second method was to use a new self-service option on the Census website that allowed respondents to advise they were away on Census night.
• In addition, the 2021 Census used a probability model to create an indicator of occupancy. The model used administrative data and was applied to dwellings where insufficient information was obtained in the field to definitively determine if the dwelling was unoccupied. This process was applied as part of data processing.

More information about these approaches is published on the ABS website.[1]

[1] See Australian Bureau of Statistics (2020). Assessing administrative data to enhance the 2021 Census. Available at https://www.abs.gov.au/statistics/research/assessing-administrative-data-quality-enhance-2021-census

The 2016 Census dwelling imputation method was a standard nearest neighbour ‘hot-decking’ approach, which imputed people into non-responding dwellings by finding a similar dwelling, called a donor, from the responding population by matching on (limited) known information. While the method improved the overall Census counts, the 2016 Post Enumeration Survey indicated that there was some ‘age skewing’ of the imputed population, with older persons over-represented.

In response to this recommendation, the ABS used administrative data to improve the ‘hot-decking’ approach so that it chooses the donor based on similar administrative data characteristics (see Figure A.1). Based on simulations using 2016 Census data, this approach reduced the over-representation of older persons and thereby improved the overall age distribution of Census counts.

The recommendation to consider ‘post-Census’ adjustments at the small area level based on the Post Enumeration Survey was not pursued. Maintaining the current approach provides continuity of approach and data with previous Censuses.

NR  Non-Responding

R  Responding

With the move to a frame-based mail out model in 2016, field staff no longer visited every dwelling and dwelling structure data was not collected for every dwelling in the field. Instead, the 2016 Census relied on information sourced from the Address Register for mail out areas, supplemented by Census field information, where collected. At that point in time, dwelling structure data on the Address Register was largely populated by a large canvassing exercise undertaken in 2015.

The 2016 Census Independent Assurance Panel concluded that the use of the ABS Address Register likely led to the increase in the number of dwellings that did not have a structure type, as well as a decrease in the proportion of dwellings classified as flats and apartments attached to houses, noting that the proportion of dwellings affected was small.

Since the 2016 Census, dwelling structure information on the Address Register has been significantly improved through the following activities:

• applying dwelling structure information from 2016 Census data to 9.6 million addresses;
• using Building Approvals dwelling structure data for approximately one third of new dwellings and for updating existing semi-detached dwellings and low-rise apartment blocks;
• updating the dwelling structure standard resulting in two main changes: firstly, improved clarity in the definition of separate and semi-detached dwellings and secondly, addition of a ‘9 or more storey’ classification;
• developing an imputation model to assign dwelling structure when no other data is available (approximately 3.5% of residential addresses), ensuring that every residential address record on the 2021 Census frame has an assigned value for dwelling structure.

In addition to these measures on the Address Register, field staff collected dwelling structure for those dwellings visited. Where the dwelling has an imputed value, this was updated with field information.

The introduction of a new self-service option on the Census website that allowed users to complete their Census without the code and password delivered to their address, resulted in an increase in the number of dwellings without a dwelling structure. These dwellings were assigned an imputed dwelling structure where there was sufficient geographic location information for use in the imputation model.

Although the 2016 Census results for Aboriginal and Torres Strait Islander peoples were comparable with the 2011 Census, the 2016 Census Independent Assurance Panel noted that the coverage of these populations remained much lower than that of the general population and that the ABS should consider ways of improving this coverage.

The ABS undertook user-centred research early in the design of the 2021 Census to understand drivers of participation by, and avenues for increasing response from, Aboriginal and Torres Strait Islander peoples. ABS built on this research through ongoing consultation with the ABS Round Table on Aboriginal and Torres Strait Islander Statistics. Through this work the ABS developed and enacted a number of strategies to address this recommendation. The approach has included:

• Greater investment in the dedicated engagement unit enabling early and ongoing engagement at all levels, from grass roots community organisations through to governance bodies. 
• Employing locally and increasing the number of Aboriginal and Torres Strait Islander office and field staff, including having designated positions in Remote Area Teams and Census Engagement Managers working with communities. To achieve this, ABS had a dedicated recruitment team that included Aboriginal and Torres Strait Islander staff.
• Engagement of a specialist Aboriginal and Torres Strait Islander agency to help the ABS deliver culturally appropriate communication. This included tailored campaign materials, radio advertising in 19 languages, and media placement in Aboriginal and Torres Strait Islander press, radio, and television. The Census Program appointed 21 Aboriginal and Torres Strait Islander media spokespeople from within the ABS and worked closely with Aboriginal and Torres Strait Islander media organisations. A suite of information material was developed to support oral, written and visual communication, and shared this with stakeholders through engagement teams.
• Partnerships with community organisations to create culturally safe locations for assistance in form completion. Unfortunately, this measure had to be scaled back or cancelled in New South Wales and Victoria due to the COVID-19 outbreak.
• Improvements to Census forms that include ‘Aboriginal’ and ‘Torres Strait Islander’ as listed ancestries, and changes to increase reporting of Aboriginal and Torres Strait Islander languages.
• Testing online and paper forms with Aboriginal and Torres Strait Islander peoples. Testing occurred in focus groups and through ongoing consultation with the Round Table on Aboriginal and Torres Strait Islander Statistics.

In addition to the above, one of the most significant improvements was the digitisation of field operations for remote areas. This involved remote teams using a mobile app instead of paper record books and accessing near real time operational intelligence. For the first time, staff managing remote operations were able to review all dwelling and person counts in near real time.

The ABS has also researched statistical methods to reduce the undercount of Aboriginal and Torres Strait Islander peoples in the Census. Two options were developed. The first was to extend existing hot-deck dwelling imputation to include Indigenous status in addition to Age, Sex and Marital status. This research was presented to a wide range of external stakeholders, with feedback that there was merit in the approach to reduce undercount, but there were strong reservations about imputing Indigenous status. Based on this feedback Indigenous status was not included in 2021 Census hot-deck dwelling imputation. A second option was later developed as a potential contingency for low response. This involved using administrative data records for people not responding to the Census and showed further improvements to the undercount. While this latter measure was not needed for the 2021 Census, the ABS will continue this research to show potential improvements for future Censuses.

The 2016 Census Independent Assurance Panel noted that although the overall population in non-private dwellings was relatively small, the lower response rates in these dwellings had some effect on quality. Specifically, the Panel recommended improving the measure of actual occupancy in these dwellings on Census night and therefore the number of non-responding persons that should be imputed.

In 2021, field processes and systems were improved to more accurately capture the occupancy of each non-private dwelling on Census night. In addition, persons staying in non-private dwellings were also able to complete their Census through the self-service option on the Census website that allowed users to complete their Census without a code and password.

Despite these measures, the 2021 Census experienced challenges in monitoring the progress and quality of enumeration in non-private dwellings during field operations. The COVID-19 lockdowns and travel restrictions reduced travellers in many areas and resulted in lower levels of occupancy than expected.

Furthermore, COVID-19 impacted a number of specific dwellings, notably hospitals and aged care providers, which were experiencing extreme staffing restrictions and pressures. In these circumstances, a reduced set of information was collected from these dwellings.

The 2016 Census Independent Assurance Panel noted a decline in the reporting of date of birth and the reduced proportion of people choosing to have their form retained by the National Archives. They identified an opportunity for the ABS to consider how best to respond to privacy concerns about the Census and provide appropriate assurances to the public. It was also suggested that the ABS consider sourcing an external Privacy Impact Assessment for future Censuses.

As in previous Censuses, the 2021 Census had a strong emphasis on maintaining the privacy and security of Census data. The ABS has undertaken extensive stakeholder engagement, gained external privacy advice and assurance (including a whole-of-Census Privacy Impact Assessment), and implemented a strong risk management program.

In line with the ABS’s commitment to privacy and maintaining public trust, the 2021 Census Program undertook two independent PIAs, consulting with a range of stakeholders, including privacy regulators, researchers and civil society advocates:

• The ABS commissioned Galexia Pty Ltd to conduct the 2021 Census Privacy Impact Assessment. This assessment focused on potential privacy risks and issues associated with personal information collected by the ABS. All recommendations were accepted except for the recommendation to exclude the Long-term health conditions question from the Census Time Capsule – the repository of forms that respondents have agreed can be kept by the National Archives and be accessed in 100 years.
• A separate Privacy Impact Assessment was conducted on 2021 Census Administrative Data, by Information Integrity Solutions Pty Ltd. This assessment looked at how the potential use of administrative data could affect privacy. That is, using information collected by other government agencies, businesses or other organisations to support the Census. The ABS agreed with all the recommendations from this assessment.

The results of the Privacy Impact Assessments[1], as well as other privacy material has been published on the ABS website throughout the Census development period and during operations. This included the 2021 Census Privacy Statement[2] and collection notices, which provide clear information on how the ABS collects, stores and deletes personal information. The ABS also published articles explaining the use of administrative data in the Census.

[1] Australian Bureau of Statistics (2021) Privacy Impact Assessments: 2021 Census of Population and Housing. Available at https://www.abs.gov.au/about/legislation-and-policy/privacy/privacy-impact-assessments

[2] Australian Bureau of Statistics. 2021 Census Privacy Statement. Available at https://www.abs.gov.au/about/legislation-and-policy/privacy/privacy-abs/2021-census-privacy-statement

This recommendation was readily accepted by the ABS and the 2021 Census Statistical Independent Assurance Panel was established with similar Terms of Reference to the 2016 Panel and has concluded with the publication of this report.

The Post Enumeration Survey provides an assessment of Census coverage, or how successful the Census was in counting the total Australian population. This assessment is determined in terms of overcount and undercount.

Some of the reasons why people are counted more than once or counted in error (i.e. overcounted) include:

• they were included on the Census form at the dwelling where they usually live, even though they stayed and were counted elsewhere on Census night; and
• they were overseas on Census night and so should not have been counted at all, but were included on the Census form at the dwelling where they usually live.

Some of the reasons why people may be missed (i.e. undercounted) include:

• they were travelling and were difficult to contact;
• they mistakenly thought they were counted elsewhere;
• the person completing the form thought that, for example, young babies, the elderly or visitors should not be included;
• they did not wish to participate;
• the dwelling in which they were located was missed because it was difficult to find (e.g. in a remote or non-residential area); and
• the dwelling in which they were located was mistakenly determined to be unoccupied.

Rates of overcount and undercount can also vary significantly for different population groups depending on factors such as sex, age, ethnicity, Aboriginal and Torres Strait Islander origins, and geographic location.

The Post Enumeration Survey also provides an assessment of the total number of persons present in dwellings where Census have imputed persons. Imputation is the process that attempts to fill in basic information for the people who didn’t respond to the Census. Currently, this information is a person’s age, sex, usual residence and marital status. For these dwellings, the Post Enumeration Survey makes an entirely independent assessment of the number of persons who usually reside in these dwellings, based on the subset of sampled dwellings that match to Census dwellings where imputation has occurred. The net overcount for persons imputed[1] is derived by subtracting the Post Enumeration Survey estimate from the total of Census imputed persons. If Census under imputes persons, then the result is a net undercount for persons imputed.

Because the Post Enumeration Survey cannot directly associate person records with Census imputed persons, there is no precise way to break down the components of overcount and undercount. Approximate comparisons can be made by examining the characteristics and occupancy of Post Enumeration Survey dwellings that match to Census dwellings where imputation has occurred.

Some of the reasons why Census may over or under impute include:

• private dwellings were incorrectly identified as occupied on Census night (leading to over-imputation);
• private dwellings were correctly identified as occupied on Census night, but too many or too few persons were imputed;
• non-private dwellings had too many or too few persons imputed.

Rates of over-imputation can also vary significantly for different population groups depending on factors such as sex, age, ethnicity, Aboriginal and Torres Strait Islander origins, and geographic location.

[1] For technical reasons, the Post Enumeration Survey groups dwellings where Census has imputed people together with dwellings where a Census form was received after the start of the Post Enumeration Survey collection (‘Late Returns’) and people who provided insufficient personal identifier information on their Census form, as these groups all require the Post Enumeration Survey to make an independent estimate of the true population. However, the size of net overcount for imputed people is much larger than the undercount or overcount from the other components, so the net overcount for this group can safely be interpreted as over-imputation.

The Post Enumeration Survey provides information on overcount and undercount for the following variables, and as such, provides an indication of their quality:

• Geography (national, state and/or territory, and part of state and/or territory);
• Age;
• Sex;
• Marital status;
• Country of birth; and
• Indigenous status.

Note that ‘not-stated’ or missing values for Country of birth and Indigenous status are not imputed in the Census. Those persons who have a value of ‘not-stated’ against either of these two items contribute to net undercount estimates for the category in which they should have been counted, as reported in the Post Enumeration Survey.

After every census, the Estimated Resident Population is recalibrated, or rebased. Rebasing is done by adding the net overcount or net undercount from the Post Enumeration Survey to the new Census population counts. The result is adjusted further to include Australian residents who were temporarily overseas on Census night (identified from passenger traveller records). The final step in calculating Estimated Resident Population is to backdate to 30 June of the Census year. This small adjustment is undertaken by adding the deaths and subtracting the births and net overseas migration which occurred between 1 July and the Census date. The diagram below (Figure B.2) outlines the Estimated Resident Population rebasing calculations and adjustments.

As part of the population estimates rebasing exercise, the Post Enumeration Survey adjusted age and sex distributions are compared with independent population data sources, including administrative data sources such as the Multi-Agency Data Integration Project (MADIP) and published estimated resident population which has been updated quarterly from the previous Census. Where the Post Enumeration Survey cannot provide or does not result in plausible undercount adjustments for population sub-groups, demographic adjustments can be applied for the rebased estimates based on these other data sources.

The reliability of the Post Enumeration Survey affects the extent to which estimates can be used to accurately assess the quality of the Census. As a result, it is important to understand the quality of the Post Enumeration Survey itself and the factors that may limit its usefulness as a measure. The conceptual framework for population estimates is described in Appendix C. The key sources of error in the Post Enumeration Survey are outlined below.

Sampling error is random error resulting in either an underestimate or an overestimate because the Post Enumeration Survey is a sample survey.

The Post Enumeration Survey has relatively low sampling errors at the national level due to a large sample, but the relative sampling errors will be higher for estimates for lower level geographies.

The final Margins of Error on the Post Enumeration Survey estimates is reported in the Post Enumeration Survey publication.

Correlation bias is systematic error that results in an underestimate when Census and the Post Enumeration Survey tend to miss the same people. The Post Enumeration Survey relies on statistical independence to accurately adjust for Census undercount and/or overcount. If population groups are systematically missing from both the Post Enumeration Survey and the Census, correlation bias occurs, which can lead to an underestimate of the population.

The potential to miss the same population groups increased somewhat in 2021 because the same address framework was used for both collections. To minimise this, the steps described in Section B4.1 below were taken.

Correlation bias is partially treated by fitting mathematical models to account for the drivers of Census and Post Enumeration Survey non-response in estimation, such as age characteristics. This has been shown to be effective in reducing correlation bias, although some residual bias will remain. A detailed description of the prediction regression estimator used in the Post Enumeration Survey can be found in Research Paper: An Estimating Equation Approach to Census Coverage Adjustment, May 2007.[1] Treatment for correlation bias in 2021 is detailed in Appendix C.

There is also a risk of non-response bias when selected persons respond to the Census but do not respond to the Post Enumeration Survey.

Non-response bias occurs when the people who do not respond to the Post Enumeration Survey have different characteristics to those who do respond. This risk is greater if the characteristics of the non-respondents vary significantly from respondents, especially in their propensity to complete the Census. Non-response bias is treated through the Prediction Regression Estimation model referred to above.

[1] Research Paper: Bell, P. A., Clarke, C. F., Whiting, J. P. (2007). An Estimating Equation Approach to Census Coverage Adjustment. Canberra, Australia: The Australian Bureau of Statistics. Available at https://www.abs.gov.au/ausstats/abs@.nsf/mf/1351.0.55.019

Systematic error resulting in an overestimate occurs when the Census and the Post Enumeration Survey find the same people but fail to match them, leading to those people being treated as missing from the Census. The Post Enumeration Survey relies on linking to Census to establish which persons in the Post Enumeration Survey were counted in Census and which were not.

If a person is counted in both the Post Enumeration Survey and the Census, but the link between them is missed, the Post Enumeration Survey mistakenly reports that they are missing from the Census. This can lead to an overestimate of the number in the population.

This bias is predominantly treated through a high level of scrutiny and quality assurance attached to the linking and matches processes in the Post Enumeration Survey.

In 2016, a quality adjustment was introduced to identify and treat Census records that had insufficient personal identifier information, which is required for linking to the Post Enumeration Survey. These records were moved to the Census non-contact sector (comprising dwellings deemed to be occupied on Census night, from which no Census form was received; or a Census form that was received after the Post Enumeration Survey commenced data collection) and treated in a similar fashion to late returns, to remove the potential for any upward bias on the population estimates produced by the Post Enumeration Survey. In 2021, this concept has been extended to Post Enumeration Survey records.

This occurs if the Post Enumeration Survey respondents do not accurately recall where they were enumerated on Census night.

As in 2016, the 2021 Post Enumeration Survey was completed later than previous Post Enumeration Surveys increasing the possibility of recall error.

Recall error is managed in the Post Enumeration Survey processing by using an automated linking program which enables a wider range of possible addresses (including vague addresses) to be searched in the matching process.

The Post Enumeration Survey is a household survey. Therefore, its scope is limited to private dwellings and it is unable to detect any change in the level of undercount or overcount in non-private dwellings.

The 2021 Post Enumeration Survey had a new public facing name: the Post Census Review. The name was introduced to better reflect the survey and its purpose in plain language, and was selected after qualitative testing via cognitive interviewing.

The 2021 Post Enumeration Survey sample was increased from 2016 to maintain similar levels of accuracy while accounting for population growth. Final sample design aimed to achieve 44,359 fully responding households. Sample stratification was applied to increase the expected sample of Aboriginal and Torres Strait Islander peoples. Stratification is commonly used in household surveys to increase the sample size of subpopulations of interest and improve the accuracy of estimates of subpopulations of interest.

For the 2021 Post Enumeration Survey an Address Register based frame was used, consistent with the move of all ABS household surveys to this frame. As the Address Register is also used by the Census, independent quality assurance of the frame was undertaken through an address canvassing exercise. The purpose of the exercise was to maximise the chance that any dwellings missing or misrepresented on the Address Register were identified and had a chance of selection in the Post Enumeration Survey regardless of whether they were counted in the Census.

B.4.3.1       Telephone-first model

A telephone-first enumeration model was implemented in the 2021 Post Enumeration Survey to alleviate the increasing pressure on field interviewer resources and optimise response across the sample. Maximising self-initiated telephone interview response frees up resources to enable field interviewers to focus on areas of low response. The aim of this model was to achieve around fifty to sixty per cent telephone interview uptake across the whole sample. This was exceeded with 87% of Post Enumeration Survey response achieved through telephone interview.

The ABS sent a letter and flyer informing households of their selection in the survey, explaining its purpose, and asking them to phone in and complete the survey. The ABS built on their ongoing research program into approach materials to develop and test a letter specifically to maximise this telephone take up. The letter contained design features including a prominent and simple “call to action” and supporting information designed to reduce barriers to participation.

B.4.3.2       Aboriginal and Torres Strait Islander Community enumeration

Enumeration of Aboriginal and Torres Strait Islander Communities remained as in-person interviews. For 2021, the model teamed an ABS Centre of Aboriginal and Torres Strait Islander Statistics engagement manager with a Household Survey Interviewer. The model aimed to harness the knowledge and expertise of each to bring greater understanding in Communities around the purpose of the survey while ensuring activities had minimal impact to members of the Community.

B.4.4.1       Providing further detail on coverage in Post Enumeration Survey outputs

The 2021 Post Enumeration Survey outputs will be published on the new ABS website, requiring new styles and formatting. Alongside this, the outputs will contain greater component analysis and disaggregation.

Following the 2016 Post Enumeration Survey, including engagement with the 2016 Census Independent Assurance Panel, a proposed new additional measure of coverage has been developed in addition to the standard outputs such as gross undercount, gross overcount and net undercount. The new measure is “Gross Coverage Error” and is the difference between the Post Enumeration Survey population estimate and the Census count (with gross overcount and imputed persons removed).

Conceptually, Gross Coverage identifies the population that the Census counted (or received a form for) as a proportion of the population that the Census should have counted. Gross coverage error is the difference between the Post Enumeration Survey population estimate and the gross coverage estimate. This differs from the typical net undercount headline figure produced from the Post Enumeration Survey, which subtracts overcount from undercount.

Gross Coverage will also be calculated for the 2016 Post Enumeration Survey to enable comparison with 2021 results. This is of particular note given the changes made to the Census imputation model, which mean that measures such as this are needed to make meaningful comparison over time.

Similar gross measures of coverage are published by the United Nations and Statistics New Zealand. Net undercount will continue to be reported as the headline measure in the Post Enumeration Survey publication in keeping with international practice.

The first component of the accuracy framework focuses on the quality of the Census data and identifies four main groups of potential errors that can impact on quality. These are:

Non-response error occurs when people refuse to participate in the Census or do not return their Census forms in time for their data to be processed. Key considerations for this error type are:

• The distribution of non-response. If particular sub-populations such as young males are more likely than others to non-respond, then the data may not be representative of the entire population. The impact can be reduced by the imputation of persons in non-responding occupied dwellings, depending on the accuracy of the imputation process.
• The accuracy of the imputation process determines how well non-response bias is mitigated. Imputation refers to the process whereby missing or erroneous responses are inferred from likely or appropriate information. An important aspect is the determination of whether non-responding dwellings were occupied on Census night or not. This applies to both private and non-private dwellings.
• Item non-response. Some items are not completed on the Census form, either accidentally or deliberately. As forms are processed, these are coded as ‘not stated’ and will have impacts on the quality of the Census data if the item non-response rate is high. Also, for imputed persons, there will be item non-response for all items except for Age, Sex, Marital status, and Place of usual residence, which are imputed. This will add to the effective item non-response rate.

After adjusting for non-response, coverage error in the Census is the difference between the number of people and dwellings counted in the Census, compared to the actual number of people and dwellings in Australia on Census night. Coverage error can be due to overcoverage or undercoverage:

• Overcoverage of dwellings can occur when dwellings are listed or counted more than once, or out of scope dwellings (e.g. unoccupied temporary dwellings such as cabins, caravans and tents) are mistakenly included. Overcoverage of persons may occur when people are counted more than once, or when forms for people who do not exist or are outside Australia on Census night are submitted.
• Undercoverage of dwellings can occur when dwellings are missed from the count (e.g. not listed on the Address Register) or are mistakenly considered out of scope. Undercoverage of persons can occur when the Census misses people from the count, which can be due to their dwelling where they were on Census night being missed, or because they did not respond and were not correctly identified as a non-respondent.

Measurement error is the difference between what the Census questions are trying to measure and the responses people give to them. Difference can occur due to the way people interpret questions. As the questions and interpretations change over time, this can lead to challenges comparing historical series. Key considerations for this error type are:

• comparability over time;
• consistency with external data sources; and
• internal consistency within the Census data set.

Processing error encompasses all errors introduced in processing the data after collection is complete. Two key types of processing errors are:

• coding errors, which occur when a response is incorrectly coded (or misclassified) into the wrong category; and
• imputation errors, which occur when imputed values do not accurately represent the true missing value.

There is a small possibility of some error during the data capture process due to misreading of the paper forms.

The second component of the accuracy framework focuses on the accuracy of the Census data in its use in contributing to population estimates. This component looks at potential errors that may impact on the quality of population estimates from the Post Enumeration Survey and its interaction with the Census, and categorises them into six main groups:

Coverage error in the Post Enumeration Survey is the difference between the population in scope for selection in the survey and the population that ideally should have been in scope. Dwellings and people could be missed due to deficiencies in the Address Register used for the survey or imperfect field procedures, leading to undercoverage.

Sampling error is random error resulting in either an underestimate or an overestimate as the Post Enumeration Survey is a sample survey. Sampling errors will be relatively higher for the more detailed estimates.

Non-response error for the Post Enumeration Survey occurs when people selected in the survey do not respond; this is either when correspondence requesting them to undertake the interview does not reach them (and a Field Interviewer is unable to make contact with them directly), or when people refuse to participate. Key considerations for this error type are also the distribution of non-response (i.e. representativeness of the achieved sample), and the level of item non-response.

Similar to that for the Census, measurement error for population estimates is the difference between what the Census questions are trying to measure, and the responses people give to them. Of particular interest for population estimates is consistency between the way people respond to the Post Enumeration Survey and the Census.

Processing error encompasses all errors introduced in processing the data after the Post Enumeration Survey collection is complete. A key processing error is matching error, which occurs if processing does not correctly match persons counted in the Post Enumeration Survey sample to their corresponding record in the Census.

Model error occurs when the underlying assumptions in the model used to estimate Census overcoverage and undercoverage are not valid. For example, the model makes assumptions of statistical independence within population sub-groups when the Post Enumeration Survey and the Census miss the same people (see Section C.4.3). This may not be a valid assumption in practice.

Net overcount for persons imputed (where a dwelling was known to be occupied but no form was returned, or where a person was known to be staying in a non-private dwelling but no form was returned) is 2.1%, lower than 2.7% in 2016. This is a statistically significant decrease. The introduction of self-service options on the Census website made it easier for people to report if they would not be at home on Census night. In addition, an occupancy determination model was developed for 2021 which helped to more accurately determine whether a dwelling was occupied or unoccupied on Census night in the absence of reliable field information. Both innovations – combined with a less mobile population due to COVID-19 restrictions and more people being enumerated at home - have helped ameliorate the impact of restricted field activities due to the pandemic and have resulted in an improvement in data quality.

Net undercount for persons on Census forms in 2021 is 2.8%, nearly one percentage point lower than 2016 (3.7%). This is statistically significant. Gross undercount has reduced while gross overcount has remained stable. Gross undercount has decreased from 4.9% in 2016 to 4.0% driven largely by a reduction in people missed from returned Census forms. This may be due to a combination of factors including form improvements to clarify who should be included on the form, as well as the effect of COVID-19 related restrictions reducing mobility and of household members and transience of the population.

Non-response increases the risk of bias toward characteristics of the responding population. The response rate for private dwellings is calculated as 96.1% in 2021 compared with 95.1% for 2016. Upon completion of the field phase, imputation (for persons that were not included on a Census form) is the main method for dealing with potential non-response bias. Its effectiveness for dealing with non-response bias depends on the accuracy of the imputation process. There are two aspects to imputation accuracy:

1. determining which non-responding dwellings were occupied on Census night; and
2. ensuring that the imputation process accurately reflects the age, sex and marital status characteristics of non-responding dwellings.

Following the 2016 Census, the Independent Assurance Panel recommended that these areas, namely the accuracy of occupancy determination and the representativeness of age imputation, be improved for the 2021 Census. Initial results from the 2021 Post Enumeration Survey indicate that there have been improvements in both areas in the 2021 Census.

The Post Enumeration Survey provides a comparison of the number of people created through imputation and the number of people that should have been created. As in previous censuses, the ABS has created more people in 2021 than were missed. However, this has improved in the 2021 Census with over-imputation decreasing from 2.7% in 2016 to 2.1%.

Item non-response rates for the 2021 Census are lower than in 2016 and this is largely due to the lower number of imputed records where ‘not stated’ will be recorded for all Census variables except Age, Sex, Marital status and Usual residence. Excluding imputed records, item non-response rates in 2021 are also generally lower than in 2016 mainly due to the increased use of the online form.

See Section 3.3 for more information.

For the variables examined, the Panel could not identify any significant new measurement errors by comparing 2021 and 2016 Census results. The Panel observed an increase in the number of persons reporting Aboriginal and Torres Strait Islander ancestries, and noted a higher rate of non-response for the Long-term health conditions question for people aged 15 years and under who responded on the Interviewer household form. This is thought to be due to the question’s placement on that type of form. See Sections 3.5 and 3.7 for further discussion of Census data items.

The Panel did not examine the accuracy of the coding process. However, the Panel observed there was an improvement in the imputation process in the 2021 Census.

There has been no change in the target population of the Post Enumeration Survey in 2021 compared with 2016, which includes all private dwellings in Australia but excludes non-private dwellings. In 2021, a list based multi-stage sample was used based off the Address Register frame for the private dwelling sample. This is different to 2016, and earlier iterations, which used an area based multi-stage sample. The Aboriginal and Torres Strait Islander community sample continued to use an area based multi-stage sample. To ensure the Post Enumeration Survey measured coverage independently to the Census, a desktop address canvassing exercise was undertaken prior to enumeration which identified dwellings missed from the Address Register frame and added these to the Post Enumeration Survey sample.

The sample size was increased from 2016 by around five per cent to account for population growth. Sampling error has decreased slightly overall due to the increase in Census response rates and the number of responding dwellings in the Post Enumeration Survey sample. Sampling errors are shown in Table C.1.

Table C.1 presents net undercount rates and standard errors for each state and territory. As can be seen, the level of variance in 2021 is comparable with 2016.

The total number of fully responding dwellings in the 2021 Post Enumeration Survey was 45,138. This represented a response rate of 89.3% for the general population sample (a decrease from 91.2% in 2016) and 91.6% for the Aboriginal and Torres Strait Islander Community sample (a decrease from 92.7% in 2016) (see Table C.2). This increase in non-response rate increases the risk of non-response bias and correlated response bias, in particular. This could lead to an underestimation of the population estimates but there is no evidence of this being an issue.

To reduce correlation bias, weighting classes can be used in the estimation model such that within a weighting class there are no systematic patterns of response. Based on analysis of 2021 Census and Post Enumeration Survey response patterns, state-based quartiles of the Socio-Economic Indexes for Areas (SEIFA) - Index of Economic Resources[1] were included as weighting classes which effectively reduced the correlation bias in the estimates. 

A technique known as propensity analysis was also conducted on both the Census and Post Enumeration Survey to assess the main drivers of non-response. This analysis used Post Enumeration Survey data to model the propensity for a person to complete a Census form and Census data to model the propensity for a person to complete the Post Enumeration Survey. Outcomes from this analysis were used to determine benchmarks for use in the weighting used in the Post Enumeration Survey estimation process. The variables used in the weighting process in 2016 were again used in 2021 and included state and/or territory, and part of the state and/or territory of usual residence, Sex, five-year age groups, Marital status, Country of birth, Indigenous status, and whether located in a Aboriginal and Torres Strait Islander community. As a result of the 2021 propensity analysis, two additional groupings were found to be significant predictors of Census response and therefore included in the weighting process; an indicator of how the Census form was delivered (mailed out or dropped off), and a measure of Census field areas that were difficult to enumerate (based on the proportion of occupied non-responding private dwellings).

[1]  Australian Bureau of Statistics (2018). Census of Population and Housing: Socio-Economic Indexes for Areas (SEIFA), Australia, 2016.  Retrieved from https://www.abs.gov.au/ausstats/abs@.nsf/mf/2033.0.55.001

Recall error may feature because of the long Census enumeration period. However, this would be no more significant than in 2016. Recall error depends on whether there is an anchor point to assist people’s recollection. Census communication focussed on a Census period for response rather than Census night and response patterns showed high response on and immediately before Census night. While this may have affected the concept of a Census night, recall bias was assessed as being no more a risk in 2021 than in previous Post Enumeration Surveys.

• Memory effects for major events (which Census is designed to be) tend to wane after one to two weeks and would have been a little worse after ten weeks for the 2021 Post Enumeration than they would have been for previous Post Enumeration Surveys.
• In addition, the requirement to recall information about Census night in 2021 was examined in the context of Automated Data Linking. Automated Data Linkage provides an enhanced ability to search for, and match, Post Enumeration Survey persons to their Census form, utilising a variety of search addresses and personal identifier information. Therefore, the requirement for persons to recall their exact address on Census night is reduced, provided they can correctly recall whether or not they were in Australia (i.e. whether they should have been counted at all).

Matching error should be lower in the 2021 post Enumeration Survey because of the improved matching procedures.

Outcomes from linking and matching processes underwent a high level of scrutiny and quality assurance in 2021, to ensure that the Post Enumeration Survey did not miss links for Post Enumeration Survey persons who were actually counted in the Census, and to ensure that a Post Enumeration Survey person was not linked to a Census record in error.

Final match rates for the general population for persons with at least one link to 2021 Census were higher than the 2016 equivalents (94.5% and 91.1%, respectively). More high-quality links were found by Automated Data Linking in 2021 (and did not require clerical review), compared to 2016 (75.0% and 65.1%, respectively). This is likely to be a result of increased online Census uptake as well as more people being at home on Census night with mobility restricted in some states and territories.

Given a key requirement for successfully linking a Post Enumeration Survey person to their corresponding Census record is a sufficient level of data quality, a quality adjustment was continued in 2021. This method identified Census and Post Enumeration Survey records that had insufficient personal identifier information necessary for linking records (e.g. where Census data was missing or imputed for multiple linking variables, such as Name, Age or date of birth, and Sex). To remove the potential for any upward bias on the Post Enumeration Survey estimate of population totals (and level of net undercount), these records were treated in a similar fashion to late returns. This adjustment moved 31,616 Census persons or 0.12% of all persons counted in the Census.

The ABS advises that the estimation model referred to in the section on non-response/correlated response bias, which is used to produce estimates and adjust for non-response appears to have worked effectively. However, there are no measures of the residual non-response bias.

For the first time in an Australian Census, people were able to complete the 2021 Census without receiving a Census form or Census Instruction Letter delivered to their address. Anyone from across Australia with internet access could open the Census website, receive a Census Number and initiate a Census form by providing their address and a mobile phone number. This is referred to as the ‘No-Census Number’ option.  This was developed to improve user-experience, increase Census response and coverage and as a risk mitigation in the event of disruption to the delivery of materials by Australia Post or Census field staff.

The 2021 Census received 1.75 million requests for a Census Number which resulted in 1.45 million household forms being submitted online. This represents 19% of all household forms submitted online. These 1.45 million forms were submitted by approximately 1.3 million dwellings.

The introduction of the ‘No-Census Number’ option has had a positive impact on the quality of Census data. Improvements to the coverage of and response from the population are evident through increases in the number of dwellings added to the Census Frame. In the ‘Mail out’ areas where the Census frame was based on the ABS Address Register, an additional approximately 150,000 dwellings were included. This represents 1.6% of all private dwellings in these areas. The majority of these dwellings were originally not included because they were unverified addresses or under construction at the time of frame creation.

Improvements in data quality have also occurred through this option by people staying in non-private dwellings, travelling or in hard to reach locations. Most persons staying in non-private dwellings on Census night responded using the Census form provided to the establishment by a Census field officer. However, approximately 57,000 persons or 8% of all persons counted in non-private dwellings used the option (see Table E.1). The non-private dwelling type that returned the most forms from this option was hotel/motels which is also the largest type of non-private dwelling. However, the type of non-private dwelling that had the highest proportion of persons use the option was in residential colleges and halls of residence where approximately 21% of all persons counted used this option.

While dwellings in Private Dwelling Establishments such as Caravan Parks and Retirement Villages also largely responded using the materials provided by a Census field officer, approximately 26,000 forms were received via the option. This was highest in Caravan Parks and Marinas, where 19% and 38% of all forms received used the No-Census Number option (see Table E.2).

The No-Census Number option enabled households to respond without receiving a Census form. It also enabled individuals to respond separately to other people in their household. In this Census, approximately 100,000 household forms were merged with another form from the dwelling. This is largely due to use of the No-Census Number option and is expected to have contributed to improved coverage of the population. The ease of access to the option also introduced greater duplication of responses. While these have been largely removed in the processing of Census data, it is possible that a small amount of unidentified duplication remains.