Story 6: Ensuring relevant and high-quality data

Delivering high-quality data and managing risks to data quality was an important consideration throughout the Census. While most of the teams in the Census are established to ensure a smooth running Census operation, there were complementary teams that focused on ensuring the quality of Census data. This story summarises the aspects of their work.

Producing high-quality data was one of the three 2021 interdependent Census objectives.

The 2021 Census Data Quality Plan helped us make sure the Census produced high-quality data with three main goals:

1. Innovate Census processes to improve data quality.
2. Manage risk to data by implementing strong risk controls and quality gates.
3. Understand and manage the impact on Census data from changes in the Census.

The Census shows long-term trends in important aspects of the lives of Australians. We repeat topics (a grouping of questions) in each Census to collect this valuable trend data. However it is also important to make sure Census data stays relevant by adding, removing and changing topics when needed.

Following public consultation, the ABS added two new topics to the 2021 Census, the first additions since the 2006 Census:

• long-term health conditions
• service with the Australian Defence Force.

Changes to existing questions were also made to keep the Census relevant and improve the quality of the data. These included:

• removing the question on internet access in people’s homes
• adding a non-binary option to the sex question
• enhancing response options for the language and ancestry questions for Aboriginal and Torres Strait Islander peoples

The ABS ran public consultation on topics in the 2021 Census between 3 April and 30 June 2018 with results published in November 2018. More than 450 submissions were received through the ABS Consultation Hub from a range of sectors including all levels of government, academia, community and advocacy groups, industry bodies, businesses and individuals.

The ABS received 36 suggestions for new topics along with suggestions for changes or additions to the 37 existing topics. From this list we shortlisted eight topics to consider further:

• Aboriginal and Torres Strait Islander cultural identity
• current or previous service in the Australian Defence Force
• journey to place of education (including mode of transport and name and address of educational institution)
• long-term health conditions
• more contemporary measures of household and family relationships, including shared care of children where parents live apart
• non-binary sex and/or gender identity
• sexual orientation
• smoking status.

We then assessed the shortlist according to:

• strength of the need for the data
• the cost and feasibility of collecting that data
• respondents’ ability to answer
• statistical impact the topic has on the quality of Census data.

Recommendations on the topics to be included in the 2021 Census were put to the Australian Government in mid-2019. After the Government decided on the 2021 Census topics, an amendment to the Census and Statistics Regulation 2016 was tabled in both houses of Parliament in February 2020.

We used qualitative testing to assess the ability of respondents to answer the questions. Testing included:

• focus groups with key populations to look at respondents’ understanding of central concepts
• cognitive testing by observing people undertaking tasks and then interviewing them after their experience. This is to assess whether respondents understood and were accepting of the new questions
• cognitive testing in Aboriginal and Torres Strait Islander communities with a focus on the specially designed Interviewer Household Form for Indigenous communities. This was tested in Alice Springs, Yarrabah, Torres Strait and Tiwi Islands
• testing of the Special Short Form for people experiencing homelessness including online interviews with the homelessness sector
• a substantial field test in October 2019 with 40,000 dwellings to provide quantitative assessment of the possible new topics.

Census content has to be finalised more than a year in advance of the Census to allow for the paper forms to be printed and for testing of the online form. This meant we had limited time to make changes in response to the pandemic. Some small changes were made to the workplace address question just before we finalised the form to make it clearer for respondents working from home. For the online form, additional instructions were added to help people living in lockdown in Sydney and Melbourne.

The 2021 Census quality gates have been a critical control for managing Census risks. They are a set of measures that each aspect of the Census needed to pass before they could “go live”. They involved rigorous documentation of all processes, systems, operational procedures and the relevant testing results. The documentation was then peer and senior staff reviewed before official sign off.

Quality gates have been a crucial means for assuring Census processes and services are well understood, tested and are ready in time. It was particularly critical that we minimised risks to an acceptable level.

The quality gates for the 2021 Census started in July 2020 with the sign-off of printing the Census paper forms. Quality gates will continue to be processed well into 2022 before the release of all Census products. This will involve the clearance of 107 gates for the entire Census.

The Data Quality Specialist Working Group guided the development of the 2021 Census. The Working Group offered expert statistical advice to the 2021 Census teams on how decisions might affect the accuracy and relevance of the Census data. The group was made up of Senior Executives from across ABS statistical areas and has met quarterly since August 2018. They provide support and data quality advice on a wide range of issues including:

• Census content and forms
• data coding processes
• occupancy determination and imputation of the number of people in non-responding dwellings
• response window (the time between when responses open and when they close).

Story 7 – Excellence in planning and governance also details activities by the 2021 Census Statistical Independent Assurance Panel.

Administrative data is information that government departments, businesses and other organisations collect. The ABS only uses this data for statistical purposes and it is not shared, or released, in a way that could identify someone. For the 2021 Census we used it to improve the quality of the data collected and repaired in the Census. Administrative data was used to:

• help decide areas that need specialised field staff who speak common languages or have similar cultural backgrounds
• help determine whether a non-responding dwelling was unoccupied on Census day
• update our register of addresses
• give us counts of people in larger institutions like prisons and some hospitals
• quality assurance of data processing
• imputation of people in non-responding dwellings.

We commissioned a Privacy Impact Assessment into the potential use of administrative data contained here in the Privacy PIA register (released 21 July 2020). In some cases, it was decided not to use the data on the basis of the assessment’s finding (in particular, see KEY RISK AREA 3 – Electricity usage data).