Microdata and TableBuilder: Income and Housing, Australia

This product provides information about the release of microdata from the Survey of Income and Housing (SIH), Australia, including:

• The survey
• The microdata products and content
• An explanation of how to use the files
• The data item list
• The conditions of use
• Information on the quality of the microdata

The following microdata products are available from this survey:

• Detailed microdata in DataLab - approved users can access a remote desktop environment for in-depth analysis using a range of statistical software packages
  Reference periods for Income and Housing: 2000–01, 2002–03, 2003–04, 2005–06, 2007–08, 2009–10, 2011–12, 2013–14, 2015–16, 2017–18 and 2019–20
  Reference periods for Household Expenditure: 2003–04, 2009–10 and 2015–16
• Basic microdata in MicrodataDownload
  Reference periods for Income and Housing: 1975–76, 1981–82, 1984, 1986, 1988–89, 1990, 1993–94, 1994, 1994–95, 1995–96, 1996–97, 1997–98, 1998–99, 1999, 1999–2000, 2000–01, 2002–03, 2003–04, 2005–06, 2007–08, 2009–10, 2011–12, 2013–14, 2015–16 and 2017–18
  Reference periods for Household Expenditure: 1975–76, 1984, 1988–89, 1993–94, 1998–99, 2003–04, 2009–10, 2015–16 and 2017–18
• TableBuilder 
  Reference periods: 2015-16 and 2017-18

To apply for access see Microdata Entry Page.

Before you apply for access, read Responsible Use of ABS Microdata, User Guide.

Further information about the survey and the microdata product:

• List of data items available in the Data downloads of this release.
• The Quality Declaration is available from this release.
• Detailed information about the survey is available in the Survey of Income and Housing, User Guide, Australia.

The Survey of Income and Housing (SIH) was conducted during 2019–20 to facilitate analysis and monitoring of the social and economic wellbeing of Australian households.

SIH collects statistical information about household income, wealth, housing characteristics and a range of demographic and socio-economic characteristics. This survey was last run in 2017–18.

The main statistical publications for SIH are: 

• Household Income and Wealth, Australia
• Housing Occupancy and Costs, Australia

For more detailed information about SIH, users may refer to the Survey of Income and Housing, User Guide, Australia. The SIH 2019–20 User Guide includes information about the purpose of the survey, the concepts and contents, and the methods and procedures used to collect the data and derive the estimates.

The remaining sections in this product provide information that is specifically relevant to using microdata release files.

• Basic microdata contains four levels of record files: Household, Income Unit, Person and Loans.
• Detailed microdata and TableBuilder includes more levels including Superannuation, Child care and Wealth.

Different information is available for each record level.

The Household level includes information on: 

• State or territory of usual residence
• Area of residence (Greater Capital City Area/Rest of state)
• Housing characteristics such as tenure type and housing costs
• Dwelling characteristics such as the number of bedrooms in the household
• Household type and composition such as the number of families in the household
• Household income
• Household child care costs, benefits, subsidy, number of hours of child care
• Main source of income such as employee income or government pensions and allowances
• Household wealth
• Net imputed rent and gross imputed rent
• Information relating to the household reference person

The Income Unit level includes information on:

• Income unit income
• Source of income such as employee income or government pensions and allowances (as for household level)
• Weekly rent payments
• Income unit type such as couple with dependants or lone person
• Selected housing characteristics (including tenure type and landlord type)
• Child care use, costs and subsidies
• Demographic information

The Person level includes information on:

• Age
• Sex
• Marital status
• Relationship in household
• Country of birth
• Year of arrival in Australia
• Family type
• Income unit type such as couple with dependants or lone person (as for income unit level)
• Labour force details such as employment status and hours worked per week
• Occupation and industry
• Education status
• Education qualifications and education institution attending
• Income
• Source of income
• Barriers to labour force participation due to child care related reasons
• Carer payment and supplement
• Information on personal assets, accounts with financial institutions and offset accounts
• Superannuation
• Disability, including type and severity

The Loans level contains information about the characteristics of each loan such as:

• Main purpose of loan such as housing or vehicle
• Characteristics of loan such as:
  • Security
  • Amount borrowed
  • Principal outstanding
  • Weekly repayment

The following levels are only present in the detailed microdata in the DataLab.

The Child care level contains information about the characteristics such as:

• Cost of care
• Type of care
• Child Care Subsidy and Additional Child Care Subsidy
• Income units

The Superannuation level contains information about the characteristics such as:

• Age at which recipient receives regular income from account
• Type, value and duration of Superannuation account
• Income units

The Wealth level contains information about the characteristics such as:

• Value and Type of wealth

The Household, Income Unit and Person levels are hierarchical: a person is a member of an income unit, which is a member of a household. The Loans and Wealth levels are members of the household level. Superannuation and Child care are members of the person level.

There are several identifiers on records at each level of the file. Weights are also included to enable population estimates. In the basic microdata, children under 15 years do not have their own person level record on the file. However, information on the number and ages of such children was collected and is included on the household and income unit level files. A complete list of the data items available on each record level is available from the Data downloads section.

Table 1 below shows the number of records on each level of the 2019–20 SIH Basic microdata. The number of records on the unconfidentialised file are also included in the table for comparison.

Information was collected using household and individual questionnaires.

The data item list includes record level items, populations and classification details. It contains the following worksheets: 

• A Contents page with links to the different sections of the data item list.
• Subject index, which includes both item descriptions and identifiers. It can be used to locate a variable by subject when the item name is not known.
• Field index, showing item identifiers and numbers only. It is sorted alphabetically by item name, and presents a consolidated list of survey variables.
• Household level data items, which provides a full listing of items that are output at Household level.
• Income Unit level data items, which provides a full listing of items that are output at Income Unit level.
• Person level data items, which provides a full listing of items that are output at Person level.
• Loans level data items, which provides a full listing of items that are output at Loans level.
• Child care data items, which provides a full listing of items that are output at child care level.
• Superannuation data items, which provides a full listing of items that are output at Superannuation level.
• Wealth data items, which provides a full listing of items that are output at Wealth level.
• Detailed categories for the Wealth classification.
• Other Classifications.
• Contents of the microdata unit record.
• Changes from previous reference periods.

The data item list is available in the Data downloads section.

The basic microdata contains information at Household, Income Unit, Person and Loan levels.

Review the data item list to see the topics and the level of detail that is available for the basic microdata.

For confidentiality and/or usability reasons, some data item values have been collapsed and/or restricted for use on the microdata products. Additionally, data may have been masked on the basic microdata to prevent identification of persons.

The detailed microdata in DataLab product contains information at Household, Income Unit, Person, Superannuation, Wealth, Child care and Loan levels.

Review the data item list to see the topics and the level of detail that is available for the detailed microdata.

For further information about how to access detailed microdata refer to The Responsible Use of ABS Microdata, User guide.

Many of the data items included on the basic microdata are self-explanatory. The Glossary in the Survey of Income and Housing, User Guide, Australia provides links to terms and definitions for most of the survey's data items. However, some items that require further explanation are defined below.

There are several identifiers on records at each level of the file.

Each household has a unique random identifier. This identifier appears on the household level (ABSHID), and is repeated on the income unit, person, loans level records relating to that household.

Each family within the household is numbered sequentially. Non family members, single person households and persons in group households have a sequential "family number" commencing at 50. Family number (ABSFID) appears on the income unit level and the person level. The combination of household and family number uniquely identifies a family.

A family has one or more income units and each income unit within the family is numbered sequentially. Income unit number (ABSIID) appears on the income unit level and the person level. The combination of household, family and income unit number uniquely identifies an income unit.

An income unit has one or more persons and each person within the income unit is numbered sequentially. Person number (ABSPID) appears on the person level. The combination of household, family, income unit and person number uniquely identifies a person.

A household may have one or more loans and each loan within the household is numbered sequentially. Loan number (ABSLID) appears on the loans level. The combination of a household and loan number uniquely identifies a loan.

Continuing the approach from SIH 2017–18, a selection of data items and additional detail of high user interest on previous detailed microdata are now available instead on the basic microdata. These include:

• Continuous age
• Detailed occupation and industry codes
• Detailed hours worked (individual hours) and duration of unemployment (individual weeks)
• Separating NT and ACT
• Detailed age and sex of children between the ages of 0 and 14 years, represented as counts on the income unit level.

For the following, geographies on the SIH 2019–20 basic microdata, there are no boundary differences between the 2011 ASGS and 2016 ASGS:

• Area of usual residence
• State or Territory of usual residence

As a result, Area of usual residence ASGS 2011 (GCCSA) (AOURBC) and State or Territory of usual residence 2011 (STATEHEC) can also be used when 2016 geographies are required.

Imputation flags exist for each module in the questionnaire, rather than for specific data items. A value of 1 indicates partial imputation where at least one question in the module was imputed. Referring to the contents of the questionnaire module can provide an indication of whether particular data items may have included imputed data. The number of flags with a value of 1 for a particular record provides an indication of the extent of imputation in that record. Imputation flag variable names generally start with "IMP".

A number of topics in 2019–20 SIH contain multiple response data items, including housing, child care, disability and superannuation. In these instances, respondents were able to select one or more response category within a question.

Using child care as an example, in these items the output data is multi-response in nature. Child care data item on the basic microdata are:

• Income unit level - Types of formal child care income unit used in the last 4 weeks (TYPFCIUA--TYPFCIUG)

These items capture multiple responses where a person provides more than one type of child care. The first response is captured in the first, or 'A', position (e.g. TYPFCIUA), and additional responses are in the second and then third and higher, or 'B' and 'C' and higher, positions (e.g. TYPFCIUB, TYPFCIUC). If only one response is possible, for example 'none of the above' then this response may also appear in the 'A' position. Where a data item does not apply, (e.g. an income unit does not use child care) then a value of 9 or 99 for 'Not applicable' will appear in the first position (e.g. TYPFCIUA). The 'Null response' (value of 0 or 00) is a default code and should be ignored. All of these categories should be used in analysis. For specific information on the number of item repeats and the category labels and values refer to the data item list available from the Data downloads section.

The basic microdata contains information at Household, Income Unit, Person and Loan levels. More information can be found at the 'File Structure' section of this publication.

Microdata allow users to interact with survey data to their own, unique specifications. Uses include: 

• Investigating data
• Producing tabulations
• Undertaking statistical

While microdata provide a great deal of flexibility, limitations on analysis can be impacted by factors such as sample size, the data classifications used, and conditions of use for the file.

More information on the additional detail and data items on the basic microdata can be found at the 'Data Item' section of this publication.

The basic microdata is released under the Census and Statistics Act 1905. The Act allows for the release of unit record data, provided this information is not likely to identify an individual person or organisation. As a result there are no names or addresses of survey respondents on the basic microdata. Other steps, including the following list of actions, have been taken to protect the confidentiality of respondents. Basic microdata contain unit records relating to most survey respondents.

Protective provisions include:

• On the basic microdata, households with seven or more persons were reduced to a maximum size of six persons. This reduction also resulted in the deletion of several whole income units, mainly single person records.
• Most income items, and some wealth and loan data items have been protected using perturbation. During this process, selected item values are adjusted to prevent identification of survey participants while preserving the overall statistical validity of the data.
• Some variables have had values ranged or collapsed, or had maximum values reduced. The latter is a process called topcoding whereby all variable values that exceed a threshold value are reduced.
• For records with unusual combinations of characteristics, a household or person level item was changed to decrease the chance of identification. Details that were commonly changed include geography, age, country of birth, industry or occupation.

As a result of these protective measures, population estimates obtained from the basic microdata are slightly different to the other SIH products. Variation in key items and populations are addressed in the basic microdata sample reconciliation tabulations document available from the 'Data downloads' tab. This table provides a guide to the difference between basic microdata and the unconfidentialised file, using indicators such as mean, percentiles and ratios.

Steps to confidentialise the datasets that are available on the basic microdata are taken for three main purposes:

• Maintain the confidentiality of survey respondents
• Ensure the integrity of the datasets
• Maximise the value of survey content

Prior to purchase, review whether the topics and level of detail you require are available on the basic microdata.

The DataLab allows interactive (real time) access to microdata from the Survey of Income and Housing, through a portal to a secure ABS environment. The level of detail provided within DataLab exceeds that of the basic microdata unit record and provides researchers greater access to the data.

The detailed microdata is available via DataLab and the data item list can be found in the Data downloads section.

For further information about data available in this data series visit the Available Microdata page.

Detailed microdata files on the DataLab can be accessed in a secure virtual environment from your own computer or networks. All unit record data remains in the DataLab environment, and any analysis results or tables are checked by the ABS before being provided to the researcher. More information about the DataLab can be found at About the DataLab.

The detailed microdata contains the information at Household, Income Unit, Person, Superannuation, Child care, Wealth and Loan levels. More information about the detailed microdata files can be found at the File Structure and Data Items sections of this publication.

This page provides specific information relevant to the Survey of Income and Housing (SIH) TableBuilder product. It will assist users in understanding and interpreting specific data items and functions relevant to specific SIH estimates.

For general information relating to TableBuilder, or instructions on how to use features of the TableBuilder product, please refer to the TableBuilder, User Guide.

Detailed information about the survey including scope and coverage, survey collection methodology, estimation method and reliability of estimates can be accessed from the Survey of Income and Housing, User Guide, Australia, 2017–18.

The data item list for the SIH TableBuilder product is available from the Data downloads section.

The TableBuilder file contains a number of continuous data items that are available for selection from the 'Summation Options' in the 'Customise Table' panel. Continuous data items are those data items which can have a response value at any point along a numeric continuum. Examples of continuous data items are total current weekly household income from all sources, total value of household assets, usual hours of child care per week. To create tables for continuous variables, the user must first either create ranges or if interested in sums, medians or means, the user then chooses the appropriate option and adds it to the wafer, row or column.

Only one continuous item can be in the 'wafer' at any one time in TableBuilder, this is particularly important if categorical data items will be added to the row or column. A separate table needs to be created for each continuous item (one table per item with the relevant item in the 'wafer').

There are also continuous hybrid variables containing categories such as 'not applicable'. In these cases the variable can be found under 'Summation Options' and also under the appropriate level in the list of data items. Hybrid variables on the SIH file include variables such as Debt-to-gross income ratio and Capacity of solar electricity power system (Number of kW of system); a full list can be found In the data items list.

For these continuous hybrid data items there are special codes allocated for certain responses (e.g. 9999 = 'Not applicable' or 99999999 = 'Negative or zero debt'). When creating ranges in TableBuilder for such continuous items, special codes will automatically be excluded from calculations of sums, means, medians or ranges. When added to the table, it enables comparisons across populations, i.e. comparing households with a debt ratio to those without.

Limits for ranging continuous items are detailed in the data item list which can be accessed from the Data downloads section.

• Once a quantile or range has been created based on a continuous item, it can be found by selecting the ranges button (below summation options):

Below are TableBuilder outputs containing examples of data items with continuous hybrid variables:

Example 1: Debt-to-gross-income ratio - Number of households

Columns: Debt-to-gross-income ratio
Wafer: Number of households (default)

The number of households with a debt to gross income ratio are contained in the 'A valid response was recorded' column. The 'Negative or zero debt' column includes households that do not have a debt, or that have a negative debt to income ratio. These instances are excluded from the calculation of Quintiles and Custom Ranges as shown in the tables below.

Example 2: Debt to gross income ratio - Ranged - Numbers of Households

Columns: Debt-to-gross-income ratio
Rows: Ranged debt-to-gross-income ratio
Wafer: Number of households (default)

The 'Negative or zero debt' column includes households that do not have a debt or have a negative debt to income ratio (income is higher than debt). These instances are excluded from the calculation of Custom Ranges.

Example 3: Mean gross income by Family composition of household for households with and without a debt ratio

Wafer: Weighted mean of Total current weekly income from all sources (Household level)
Rows: Family composition of HH (brief)
Columns: Debt-to-gross-income ratio

In this example the hybrid continuous variable (debt-to-gross-income ratio) is cross tabulated with a categorical value (Family composition of HH (brief)). This will enable comparisons across populations, i.e., comparing households with a debt ratio to those without. To demonstrate, the mean value of a continuous item (Total current weekly HH income from all sources) was added to the wafer, and the mean value for both columns is displayed. The 'not applicable' category is included in the calculation of the totals.

Note: If a continuous variable has a very limited range on the Data Item List between 0 and 0 then this means that there are too few contributor to this variable and therefore only estimates of weighted sum, weighted mean and weighted median can be derived and ranges cannot be created.

Ranging continuous items will not provide the same output as published data which is based on the SIH (i.e. Household Income and Wealth, Australia, 2017–18 and Housing Occupancy and Costs, Australia, 2017–18). This is due to additional confidentiality measures applied to ranged continuous items.

Quintile cut-offs in TableBuilder for continuous data items are defined according to whole numbers, whereas in published estimates they are defined according to two decimal places. This will cause slight variations between TableBuilder and published estimates based on the SIH.

Flags have been created to indicate modules in the questionnaire that have imputed data in them. If imputed values are to be removed from the analysis, this can be done by only including the 'not applicable' category. The data item list contains all the flag variables.

The 2017–18 SIH TableBuilder contained seven levels of record files: Household, Income Unit, Person, Child Care, Superannuation, Loans, and Wealth. Different information is available for each record level, and each level contains counts of the number of instances, or units, of the item in each category.

For example, if a person has more than one superannuation account, there will be multiple superannuation records for that person on the superannuation level.

Mortgage(s), motor vehicle loan(s), and personal loan(s) are all examples where a household may have multiple types of items, which means there will be multiple records for that household on the Loans level.

A number of questions included in the survey allow respondents to provide more than one response. The data items resulting from these questions are referred to as 'multiple (or multi) response data items'.

The example below displays 'Types of formal child care income unit used in the last 4 weeks'.

When a multi-response data item is tabulated, the same record is counted against each response provided. As a result, some household units utilising multiple forms of formal child care are counted multiple times. Consequently, the sum of individual multi-response categories can be different to the population or actual number of people applicable to the data item, as respondents are able to select more than one response.

Multi-response data items are identified in the data item list (available via the Data downloads section).

The Summation Options section in the Customise Table panel contains the counting units/weights that are available. It is critical that the correct weight (or summation option) is used when specifying tables for counts of persons or proportions. As a general rule of thumb, use the weight which corresponds to the level of analysis you are undertaking, i.e. household level weight with household level items, the person level weight with person level items. To analyse persons in households, use the person level weight with household level items.

The Household level weight is the default in SIH TableBuilder. The weights are located under the relevant level in the Summation Options. Below is a list of the levels and corresponding weights. To change the weight from the default, click on the "Sum" box in the appropriate level:

The default summation option will be automatically added to a table when the table is being specified, therefore care needs to be taken that this is the correct weight for the particular tabulation. If the default weight is not the required weight, select the appropriate weight from the Summation Options list and add either to the 'filter' or the 'wafer' (either will override the default of household level in the 'filter') for categorical data items. This needs to be done when obtaining counts and proportions. It is not necessary (or possible) for tables which have a continuous item in the wafer.

For example, if analysing counts of persons for person level items, select 'Person - Sum' as shown below, and add to the 'wafer', before running the table.

Weighting and defining quintiles

Weighting quintiles

Quintiles need to be weighted according to the corresponding level of the continuous item. For example, for quintiles based on 'Total current weekly HH income from all sources' (a Household level item), set the 'Equal distribution of ' box to 'Household level':

Using the 'Filter by' option

The 'Filter by' option enables quintiles to be defined further, according to a categorical variable.

To illustrate, in the example above, the quintiles for gross household income are based on the Australia wide population. This means that the entire population is divided into five equal sections and the quintile cut-offs will be based on the mean values for all households in Australia. Datacubes 13 and 14 in the Household Income and Wealth, Australia, 2017-18 publication are state based, and the distributional analysis is also undertaken at the state level for these tables. This means that the quintiles are further defined by the relevant state.

To replicate this in TableBuilder, use the 'Filter by' to add the relevant state as in the following image:

This means that the quintiles will be based on the population for NSW only.This means that the quintiles will be based on the population for NSW only.

Weighting equivalised items

Equivalised items are available at the household and the person level in TableBuilder. Equivalised items are household items which have been equivalised to take all persons in the household into account. All analysis using equivalised items in publication Household Income and Wealth, Australia, 2017–18 is done using the equivalised items on the person level with person weighting. Some equivalised analysis in the publication Housing Occupancy and Costs, Australia, 2017–18 is done at the household level with household weighting.

Example 4: Mean equivalised disposable household income (EDHI) by equivalised disposable household income quintiles

This example recreates a section of Table 5.4 in publication Household Income and Wealth, Australia, 2017–18 where mean equivalised disposable household income is used at the person level, with person level weighting on the income quintiles: 2017–18

Creating the EDHI table in TableBuilder

1. Range 'Current weekly HH equivalised disposable income (Person level)' into quintiles and apply the person weight (Person level in the 'Equal distribution of' box). Select 'Create'.

2. Add the EDHI quintiles defined in step 1 to the columns via the 'Ranges' drop down box (below summation options) in the data item list.
3. Add 'Main source of current HH income' from the Household level categorical items to the rows.
4. Add 'Current weekly HH equivalised disposable income (Person Level)' - Mean - to the wafer.
5. Select 'Retrieve Data':

Note: Slight variations between TableBuilder and estimates published in the Household Income and Wealth, Australia, 2017–18 publication are evident in this example. This is because quintile cut-offs in TableBuilder are defined according to whole numbers, whereas in the publication they are to two decimal places.

More information regarding the use of household or person weights for equivalised items is located in publication Survey of Income and Housing, User Guide, Australia, 2017–18 (refer to the 'Summary indicators of income and wealth distributions' and the 'Weights' sections).

To minimise the risk of identifying individuals in aggregate statistics, a technique called perturbation is used in TableBuilder to randomly adjust cell values. Perturbation involves small random adjustments to the statistics and is considered the most satisfactory technique for avoiding the release of identifiable statistics while maximising the range of information that can be released. These adjustments have a negligible impact on the underlying pattern of the statistics. After perturbation, a given published cell value will be consistent across all tables. However, adding up cell values to derive a total will not necessarily give the same result as published totals.

Tables generated from sample surveys will sometimes contain cells with zero values because there were no respondents who satisfied the parameters of the cell in the survey sample. There may, however, still be persons or households within the general population with these characteristics. If all persons within the population had been enumerated (ie, a Census), there may have well been a value for that cell. This is an example of sampling variability which occurs with all sample surveys. Relative Standard Errors cannot be generated for cells with zero values.

It is possible to search for data items using the search function below the data item list. When searching for a data item label using the search box under summation options, the search will default back to displaying household level, but will be performed across all levels.

The Microdata Entry Page on the ABS website contains links to microdata related information to assist users to understand and access microdata. For inquiries about these and related statistics, contact the Customer Assistance Service via the ABS website Contact Us page. The ABS Privacy Policy outlines how the ABS will handle any personal information that you provide to us.

This release previously used catalogue number 6541.0.30.001.