Patient Experiences in Australia: Summary of Findings methodology

This is not the latest release View the latest release

Reference period

2019-20 financial year

Released

16/11/2020

Overview

This publication contains results from the Patient Experience Survey, a topic on the Multipurpose Household Survey (MPHS) conducted throughout Australia from July 2019 to June 2020. The MPHS, undertaken each financial year by the Australian Bureau of Statistics (ABS), is a supplement to the monthly Labour Force Survey (LFS) and is designed to collect statistics for a number of small, self-contained topics.

The survey collected information from people about their experiences with selected aspects of the health system in the 12 months before their interview, including access and barriers to a range of health care services. Respondents were asked about their experiences with medical professionals, the frequency of their visits, waiting times, and barriers to accessing care, as well as their self-assessed health status, long term health conditions and private insurance. Data was also collected on aspects of communication between patients and health professionals. Labour force characteristics, education, income and other demographics was also collected.

How the data is collected

Scope

The scope of the survey was restricted to people aged 15 years and over who were usual residents of private dwellings and excludes:

members of the Australian permanent defence forces
certain diplomatic personnel of overseas governments, customarily excluded from Census and estimated resident population counts
overseas residents in Australia
members of non-Australian defence forces (and their dependants)
persons living in non-private dwellings such as hotels, university residences, boarding schools, hospitals, nursing homes, homes for people with disabilities, and prisons
persons resident in the Indigenous Community Strata (ICS).

The scope for MPHS included households residing in urban, rural, remote and very remote parts of Australia, except the ICS.

Coverage

In the LFS, rules are applied which aim to ensure that each person in coverage is associated with only one dwelling, and hence has only one chance of selection in the survey. See Labour Force, Australia for more detail.

Data collection

The survey is one of a number of small, self-contained topics on the MPHS.

Each month, one eighth of the dwellings in the LFS sample were rotated out of the survey and selected for the MPHS. After the LFS had been fully completed for each person in scope and coverage, a usual resident aged 15 years or over was selected at random (based on a computer algorithm) and asked the additional MPHS questions in a personal interview.

In the MPHS, if the randomly selected person was aged 15 to 17 years, permission was sought from a parent or guardian before conducting the interview. If permission was not given, the parent or guardian was asked the questions on behalf of the 15 to 17 year old (proxy interview).

Data were collected using Computer Assisted Interviewing (CAI), whereby responses were recorded directly onto an electronic questionnaire in a notebook computer, with interviews conducted either face-to-face or over the telephone. The majority of interviews were conducted over the telephone.

Sample size

After taking into account sample loss, the response rate for the 2019-20 survey was 76.4%. In total, information was collected from 29,793 fully responding persons. This includes 494 proxy interviews for people aged 15 to 17 years, where permission was not given by a parent or guardian for a personal interview.

How the data is processed

Weighting

Weighting is the process of adjusting results from a sample survey to infer results for the total 'in-scope' population. To do this, a 'weight' is allocated to each enumerated person. The weight is a value which indicates the number of persons in the population represented by the sample person.

The first step in calculating weights for each unit is to assign an initial weight, which is the inverse of the probability of being selected in the survey. For example, if the probability of a person being selected in the survey was 1 in 600, then the person would have an initial weight of 600 (that is, they represent 600 people).

Benchmarks

The initial weights were calibrated to align with independent estimates of the population of interest, referred to as 'benchmarks'. Weights calibrated against population benchmarks ensure that the survey estimates conform to the independently estimated distribution of the population rather than the distribution within the sample itself. Calibration to population benchmarks helps to compensate for over or under-enumeration of particular categories of persons/households which may occur due to either the random nature of sampling or non-response.

The survey was benchmarked to the Estimated Resident Population (ERP) living in private dwellings in each state and territory at December 2019. People living in Indigenous communities were excluded. These benchmarks are based on the 2016 Census.

While LFS benchmarks are revised every 5 years, to take into account the outcome of the 5-yearly rebasing of the ERP following the latest Census, the supplementary surveys and MPHS (from which the statistics in this publication are taken) are not. Small differences will therefore exist between the civilian population aged 15 years and over reflected in the LFS and other labour household surveys estimates, as well as over time. If comparisons are being made over time then proportions should be used rather than estimates of persons.

Estimation

Survey estimates of counts of persons are obtained by summing the weights of persons with the characteristic of interest.

Confidentiality

To minimise the risk of identifying individuals in aggregate statistics, a technique is used to randomly adjust cell values. This technique is called perturbation. Perturbation involves a small random adjustment of the statistics and is considered the most satisfactory technique for avoiding the release of identifiable statistics while maximising the range of information that can be released. These adjustments have a negligible impact on the underlying pattern of the statistics. After perturbation, a given published cell value will be consistent across all tables. However, adding up cell values to derive a total will not necessarily give the same result as published totals. The introduction of perturbation in publications ensures that these statistics are consistent with statistics released via services such as TableBuilder.

Reliability of estimates

All sample surveys are subject to error which can be broadly categorised as either sampling error or non-sampling error. For more information refer to the Accuracy section.

Data quality

Information recorded in this survey is 'as reported' by respondents, and may differ from that which might be obtained from other sources or via other methodologies. This factor should be considered when interpreting the estimates in this publication.

Information was collected on respondents' perception of their health status and experiences with services. Perceptions are influenced by a number of factors and can change quickly. Care should therefore be taken when analysing or interpreting the data.

The definition of 'need' (in questions where respondents were asked whether they needed to use a particular health service) was left to the respondents' interpretation.

For some questions which called for personal opinions, such as self-assessed health or whether waiting times were felt to be unacceptable, responses from proxy interviews were not collected.

Classifications

Country of birth

Education

Geography

Industry

Occupation

Socio-economic Indexes for Areas (SEIFA)

Comparing the data

Comparability of Time Series

When comparing data from different cycles of the survey, users are advised to consult the questionnaires (available from the Data downloads section), check whether question wording or sequencing has changed, and consider whether this may have had an impact on the way questions were answered by respondents.

All data items shown in time series tables are comparable between the survey cycles presented.

Comparability to monthly LFS Statistics

Since the survey is conducted as a supplement to the LFS, data items collected in the LFS are also available in this publication. However, there are some important differences between the two surveys. The LFS had a response rate of over 90% compared to the MPHS response rate of 76.4%. The scope of the Patient Experience Survey and the LFS also differ (refer to these sections above). Due to the differences between the samples, data from this survey and the LFS are weighted separately. Differences may therefore be found in the estimates for those data items collected in the LFS and published as part of the Patient Experience Survey.

Comparability with other ABS surveys

Caution should be taken when comparing across ABS surveys and with administrative by-product data that address the access and use of health services. Estimates from the Patient Experience Survey may differ from those obtained from other surveys (such as the National Aboriginal and Torres Strait Islander Health Survey, National Aboriginal and Torres Strait Islander Social Survey, National Health Survey, General Social Survey and Survey of Disability, Ageing and Carers) due to differences in survey mode, methodology and questionnaire design.

How the data is released

Datacubes/spreadsheets

Data Cubes containing all tables for this publication in Excel spreadsheet format are available from the Data downloads section of the main publication. The spreadsheets present tables of estimates and proportions, and their corresponding relative standard errors (RSEs) and/or Margins of Error (MOEs).

As well as the statistics included in this and related publications, the ABS may be able to provide other relevant data on request. Subject to confidentiality and sampling variability constraints, tables can be tailored to individual requirements for a fee. A list of data items from this survey is available from the Data downloads section. All enquiries should be made to the National Information and Referral Service on 1300 135 070, or email client.services@abs.gov.au

DataLab

Detailed microdata will be available on DataLab for approved users who are required to undertake interactive (real time) complex analysis of microdata in the secure ABS environment. For more details, refer to About the DataLab.

Accuracy

Show all

Reliability of the estimates

The estimates in this publication are based on information obtained from a sample survey. Any data collection may encounter factors, known as non-sampling error, which can impact on the reliability of the resulting statistics. In addition, the reliability of estimates based on sample surveys are also subject to sampling variability. That is, the estimates may differ from those that would have been produced had all persons in the population been included in the survey. This is known as sampling error.

Two types of error are possible in estimates based on a sample survey:

non-sampling error
sampling error

Non-sampling error

Non-sampling error is caused by factors other than those related to sample selection. It is any factor that results in the data values not accurately reflecting the true value of the population.

It can occur at any stage throughout the survey process. Examples include:

selected people that do not respond (e.g. refusals, non-contact)
questions being misunderstood
responses being incorrectly recorded
errors in coding or processing the survey data

Sampling error

Sampling error is the expected difference that can occur between the published estimates and the value that would have been produced if the whole population had been surveyed. Sampling error is the result of random variation and can be estimated using measures of variance in the data.

Standard error

One measure of sampling error is the standard error (SE). There are about two chances in three that an estimate will differ by less than one SE from the figure that would have been obtained if the whole population had been included. There are about 19 chances in 20 that an estimate will differ by less than two SEs.

Measures of error in this publication

This publication reports the relative standard error (RSE) for estimates of counts ('000) and the margin of error (MOE) for estimates of proportions (%).

Relative standard error

The relative standard error (RSE) is obtained by expressing the standard error as a percentage of the estimate.

\(R S E \%=\left(\frac{S E}{e s t i m a t e}\right) \times 100\)

Only estimates with RSEs less than 25% are considered reliable for most purposes. Estimates with larger RSEs, between 25% and less than 50% have been included in the publication, but are flagged to indicate that they should be used with caution. Estimates with RSEs of 50% or more have also been flagged and are considered unreliable for most purposes. RSEs for these estimates are not published.

Margin of error

The Margin of Error (MOE) shows the largest possible distance (due to sampling error) that could exist between the estimate and what would have been produced had all people been included in the survey, at a given level of confidence. It is useful for understanding and comparing the accuracy of proportion estimates. Confidence levels can vary (e.g. typically 90%, 95% or 99%), but in this publication, all MOEs are provided for estimates at the 95% confidence level. At this level, there are 19 chances in 20 that the estimate will differ from the population value by less than the provided MOE.

The 95% confidence level MOE is obtained by multiplying the standard error by 1.96.
\( M O E=S E \times 1.96\)

The RSE can also be used to directly calculate a 95% MOE by:

\(M O E=\Large\frac{R S E \% \times e s t i m a t e \times 1.96}{100}\)

These can be converted to a 90% confidence level by multiplying the MOE by:

\( \Large\frac{1.615}{1.96}\)

or to a 99% confidence level by multiplying the MOE by:

\(\Large\frac{2.576}{1.96}\)

Depending on how the estimate is to be used, a MOE of greater than 10% may be considered too large to inform decisions. For example, a proportion of 15% with a MOE of plus or minus 11% would mean the estimate could be anything from 4% to 26%. It is important to consider this range when using the estimates to make assertions about the population.

Confidence Intervals

The estimate combined with the MOE defines a range, known as a confidence interval. This range is likely to include the true population value with a given level of confidence. A confidence interval is calculated by taking the estimate plus or minus the MOE of that estimate. It is important to consider this range when using the estimates to make assertions about the population or to inform decisions. Because MOEs in this publication are provided at the 95% confidence level, a 95% confidence interval can be calculated around the estimate, as follows:

\( 95 \% \text { Confidence Interval }=(\text {estimate}-M O E, \text { estimate }+M O E)\)

Calculating measures of error

Proportions and percentages formed from the ratio of two estimates are also subject to sampling errors. The size of the error depends on the accuracy of both the numerator and the denominator. A formula to approximate the RSE of a proportion is given below. This formula is only valid when the numerator (x) is a subset of the denominator (y):

\( {RSE}\left(\frac{x}{y}\right) \approx \sqrt{[R S E(x)]^{2}-[R S E(y)]^{2}}\)

When calculating measures of error, it may be useful to convert RSE or MOE to SE. This allows the use of standard formulas involving the SE. The SE can be obtained from RSE or MOE using the following formulas:

\(S E(y)=\frac{R S E(y) \times y}{100}\)

\(S E=\Large\frac{M O E}{1.96}\)

Calculating differences

The difference between two survey estimates (counts or percentages) can also be calculated from published estimates. Such an estimate is also subject to sampling error. The sampling error of the difference between two estimates depends on their SEs and the relationship (correlation) between them. An approximate SE of the difference between two estimates (x-y) may be calculated by the following formula:

\(S E(x-y) \approx \sqrt{[S E(x)]^{2}+[S E(y)]^{2}}\)

While this formula will only be exact for differences between separate and uncorrelated characteristics or sub populations, it provides a good approximation for the differences likely to be of interest in this publication.

Significance testing

When comparing estimates between surveys or between populations within a survey, it is useful to determine whether apparent differences are 'real' differences or simply the product of differences between the survey samples.

One way to examine this is to determine whether the difference between the estimates is statistically significant. This is done by calculating the standard error of the difference between two estimates (x and y) and using that to calculate the test statistic using the formula below.

\(\left(\frac{|x-y|}{S E(x-y)}\right)\)

where:

\(S E(y)=\frac{R S E(y) \times y}{100}\)

If the value of this test statistic is greater than 1.96 we can say there is good evidence of a statistically significant difference at 95% confidence levels between the two populations with respect to that characteristic. Otherwise, it cannot be stated with confidence that there is a real difference between the populations.

Glossary

Show all

After hours GP

After hours means before 8am or after 1pm on a Saturday, any time on a Sunday or Public Holiday, or before 8am or after 8pm on any other day.

At least once delayed seeing or did not see .... when needed - cost a reason

To be placed in this category, respondents must have stated that cost was one of the reasons they delayed going or did not go to see a health professional when needed.

At least once delayed seeing or did not see .... when needed - reasons other than cost

To be placed in this category, respondents must have stated that the main reason they delayed going or did not go to see a health professional when needed, included:

Dislike or fear of the service
Waiting time was too long
Service was not available when required
Had an upcoming appointment
Was too busy, or
Other reason (besides cost)

Coordination of health care

Coordination of health care has been defined as the deliberate organisation of patient care activities between two or more participants involved in a patient’s care to facilitate the appropriate delivery of health care services.

Dental professional

Includes dentists, dental hygienists and dental specialists such as periodontists, orthodontists, and oral and maxillofacial surgeons.

Full-time status of work

Full-time work is 35 hours or more per week. The number of hours can be calculated based on the number usually worked or the number actually worked during the week before interview.

General Practitioners (GPs)

GPs are doctors who have completed a basic medical degree and internship, then do additional medical training in general practice. This qualifies them to provide continuing care for everyone from babies to the elderly. They have broad knowledge and skills and are usually the first point of contact for health issues and referrals to specialists or other health professionals.

Hospital admission

The formal acceptance by a hospital or other in-patient health care facility of a patient who is to be provided with a room and continuous nursing service. This includes respondents who have been to a hospital emergency department and have also been admitted to hospital.

Hospital emergency department visit

Any time a person went to an emergency department for their own health, whether it was within normal GP practising hours or after hours.

Imaging test

Imaging tests or diagnostic imaging include all tests that produce images or pictures of the inside of the body in order to diagnose diseases. Tests involve the use of radiant energy, including x-rays, sound waves, radio waves, and radioactive waves and particles that are recorded by photographic films or other types of detectors.

Index of relative socio-economic disadvantage

This is one of four Socio-Economic Indexes for Areas (SEIFAs) compiled by the ABS following each Census of Population and Housing. This index summarises attributes such as low income, low educational attainment, unemployment, jobs in relatively unskilled occupations and dwellings without motor vehicles. The first or lowest quintile refers to the most disadvantaged areas, while the fifth or highest quintile refers to the least disadvantaged areas.

Long term health condition

A condition that has lasted or is likely to last six months or more. Respondents were specifically asked whether they had any of the following conditions:

arthritis or osteoporosis
asthma
cancer
diabetes
a heart or circulatory condition
a mental health condition, including depression or anxiety
a long term injury
any other long term health condition.

If respondents sought clarification, interviewers were instructed to include:

conditions currently controlled by medication
cancer where the respondent reports having cancer without any explanation
cancer where the respondent was undergoing treatment such as chemotherapy or radiotherapy
cancer in partial remission
mental illness where the respondent was not currently experiencing an episode.

and to exclude pregnancy, and cancer where the respondent received a false positive test result.

Medical specialist

Medical specialists play a crucial role in the management and treatment of health conditions where they have specialist knowledge and skills. If respondents sought clarification on the definition of medical specialist, interviewers were instructed to advise that medical specialists provide services which are covered, at least in part, by Medicare (e.g. dermatologists, cardiologists, neurologists and gynaecologists).

Need

In the data presented in this publication, populations are sometimes based on those who needed to use a service. In most cases, this population is a combination of those who used the service and those who didn't but said they needed to use the service. The definition of need was left to the respondents' interpretation for this survey.

Other health professionals

Some people may receive health care from health professionals other than their General Practitioner (GP), dental professional or medical specialist for their physical and/or emotional or psychological health. Examples of selected other health professionals include:

Audiologist or Audiometrist
Chemist or Pharmacist for advice only
Chiropractor
Diabetes Educator
Dietician or Nutritionist
Occupational therapist
Naturopath or Acupuncturist
Osteopath
Podiatrist or Chiropodist
Physiotherapist or Hydrotherapist
Psychologist or Accredited counsellor
Radiographer or Sonographer
Social worker or Welfare officer
Speech therapist or Speech pathologist
Other

Part-time status of work

Part-time work is less than 35 hours per week. The number of hours can be calculated based on the amount usually worked or the number actually worked during the week before interview.

Pathology test

A laboratory test that includes analysis of specimens such as urine and blood in order to diagnose disease.

Personal income

Relates to gross income.

Private health insurance

Refers to voluntary coverage through a private insurer (e.g. Medibank Private, HCF and Bupa). Depending on the type of cover purchased, private health insurance provides cover against all or part of hospital theatre and accommodation costs in either a public or private hospital, medical costs in hospital and costs associated with a range of services not covered under Medicare, including ambulance services, private dental services, optical, chiropractic and physiotherapy.

Public dental care

Any public dental service that is partly or fully funded by the government, including public dental services provided at a private dental clinic.

Remoteness

The Australian Statistical Geography Standard (ASGS) was used to define remoteness. Remoteness Areas divide Australia into five classes of remoteness on the basis of a measure of relative access to services. The Remoteness Structure is described in detail in the Australian Statistical Geography Standard (ASGS): Volume 5 - Remoteness Structure, July 2016.

Self-assessed health

A person's impression of their own health against a five point scale from excellent through to poor.

Statistical significance

Differences between population estimates are said to be statistically significant when it can be stated with 95% confidence that there is a real difference between the populations (see the Accuracy section for more information).

APA

Patient Experiences in Australia: Summary of Findings methodology

APA

Overview

How the data is collected

Scope

Coverage

Data collection

Sample size

How the data is processed

Weighting

Benchmarks

Estimation

Confidentiality

Reliability of estimates

Data quality

Classifications

Country of birth

Education

Geography

Industry

Occupation

Socio-economic Indexes for Areas (SEIFA)

Comparing the data

Comparability of Time Series

Comparability to monthly LFS Statistics

Comparability with other ABS surveys

How the data is released

Datacubes/spreadsheets

DataLab

Accuracy

Show all

Reliability of the estimates

Non-sampling error

Sampling error

Standard error

Measures of error in this publication

Relative standard error

Margin of error

Confidence Intervals

Calculating measures of error

Calculating differences

Significance testing

Glossary

Show all

After hours GP

At least once delayed seeing or did not see .... when needed - cost a reason

At least once delayed seeing or did not see .... when needed - reasons other than cost

Coordination of health care

Dental professional

Full-time status of work

General Practitioners (GPs)

Hospital admission

Hospital emergency department visit

Imaging test

Index of relative socio-economic disadvantage

Long term health condition

Medical specialist

Need

Other health professionals

Part-time status of work

Pathology test

Personal income

Private health insurance

Public dental care

Remoteness

Self-assessed health

Statistical significance

Abbreviations

Show all