Weighting, Benchmarking and Estimation

Weighting is the process of adjusting results from a sample survey to infer results for the total in-scope population. To do this, a 'weight' is allocated to each sample unit corresponding to the level at which population statistics are produced. For the 2021-22 Personal Safety Survey (PSS), this is at a person level. The weight can be considered an indication of how many population units are represented by the sample unit.

Selection weights

The first step in calculating weights for each person was to assign an initial weight. The initial person weight was derived from the initial household weight (inverse of the probability of the household being selected in the survey) multiplied by the total number of in-scope males or females in the household (depending on the allocated sex for the household). For example, if the probability of a household being selected in the survey was 1 in 600, and the household contained 3 in-scope females, then the respondent (i.e. selected person) within the household would have an initial weight of 1,800 (i.e. 600 x 3). That is, the respondent in the sample represents 1,800 other persons in the population. Initial person weights took into account an increase in male-only household representation which was a result of households that were subject to sex selection flipping from their pre-allocated sex.

Using information based on observations by interviewers at the dwelling, as well as additional information collected from non-fully responding respondents as part of the compulsory component of the survey, analysis was undertaken to ascertain whether there were any particular categories of persons that were over- or under- represented in the sample. This over- or under-representation in the sample can be corrected using a non-response adjustment and/or through calibrating the weights to population benchmarks. The calibration of weights to population benchmarks was used in the 2021-22 PSS.

Benchmarks are independent estimates of the size of the population of interest. Weights are calibrated against independent population benchmarks to ensure that the survey estimates conform to the independently estimated distribution of the population, with respect to the benchmark categories, rather than to the distribution within the responding sample itself.

The 2021-22 PSS survey estimates were benchmarked to the Australian estimated resident population aged 18 years and over who were living in private dwellings (excluding very remote areas of Australia), simultaneously using the following benchmark categories:

Number of persons by:

1. State or Territory by Age groups by Sex
2. State or Territory by Capital city/Balance of state
3. State or Territory by Social marital status (Married (incl. registered or de facto) and Not married) by Sex.

Note for male estimates: State and territories have not been benchmarked to their state/territory male populations, and therefore contribute to national estimates only. Also, as outlined in the Sample Design and Selection section of the Sampling chapter, the adjusted sample was not designed to produce estimates for states/territories for males, and therefore detailed estimates are likely to have associated high standard errors (for more details on standard errors, see Data Quality and Technical Notes chapter of this publication).

Two benchmark categories (1 and 2) were benchmarked to the estimated resident population living in private dwellings in Australia (excluding very remote areas) as at September 2021 based on the 2021 Census of Population and Housing. The in-scope estimated resident population was estimated to be 19,443,424 as at September 2021. The benchmarks, and hence the estimates from the survey, do not (and are not intended to) match estimates of the total Australian estimated resident population (which include persons living in very remote areas of Australia and persons in non-private dwellings, such as hotels) obtained from other sources.

The third benchmark category was based on survey estimated benchmarks. Benchmarks are considered to be survey estimated if they are obtained from a sample survey and as such, have a non-negligible level of sample error associated with them. The monthly Labour Force Survey (as per March 2021 to June 2022) provided the survey estimated benchmarks for labour force status, social marital status, and broad country of birth categories. The survey estimated benchmarks were aligned to the in-scope resident population aged 18 years or over, who were living in private dwellings in each state and territory (excluding very remote areas of Australia), as at September 2021. They were also made to represent the same population as the demographic benchmarks with respect to state/territory (although note difference identified previously for males), section of state, age group and sex. The sample error associated with the survey estimated benchmarks was incorporated into the standard error estimation.

The 'state' population benchmark for benchmark 1, for females, consisted of all six states and the two territories. Males were not benchmarked to the state/territory component of this benchmark.

For benchmark 1, 'Age group' for females was benchmarked against 5-year age groups, between the 18-20 years and 76 years and over groups. ‘Age group’ for males was benchmarked against 10-year age groups, between 18-25 years and 76 years and over groups.

Estimation is a technique used to produce information about a population of interest, based on a sample of units (i.e. persons) from that population. Each record in the 2021-22 PSS has a person weight. Information for sampled persons is multiplied by the weights to produce estimates for the whole population (or the population of interest).

The application of weights ensures that estimates will conform to an independently estimated distribution of the population by certain categories including state, age, and sex, rather than to the distributions within the sample itself.

Replicate weights have also been calculated - 60 person replicate weights. The purpose of these replicate weights is to enable calculation of the relative standard error (RSE) for each estimate produced from the survey. Further information on replicate weights is provided in the Data Quality and Technical Notes chapter of this publication.

Users should take into consideration the quality of the estimates when interpreting data from the PSS. For further information about how to calculate RSE's and conduct significance testing, refer to the Data Quality and Technical Notes chapter of this publication. For other related information on other factors to consider when interpreting results, refer to the individual topic chapters in this publication.

Perturbation of estimates

To minimise the risk of identifying individuals in aggregate statistics, a technique called perturbation is used to randomly adjust cell values. Perturbation involves a small random adjustment of the statistics and is considered the most satisfactory technique for avoiding the release of information that could identify individual survey respondents while maximising the range of information that can be released. These adjustments have a negligible impact on the underlying pattern of the statistics.

After perturbation, a given published cell value will be consistent across all tables. However, adding up cell values to derive a total will not necessarily give the same result as the published totals. As such, proportions may add to more or less than 100%. Users are advised to use the published totals rather than deriving totals based on the component cells.

Cells with relatively small values may be proportionally more affected by perturbation than large values. Users are advised against conducting analyses and drawing conclusions based on small values.

All data presented in the publications produced for the 2021-22 Personal Safety Survey have had perturbation applied to 2016 and 2021-22 estimates.