CHAPTER 2 SURVEY METHODOLOGY


SCOPE OF THE SURVEY

Inclusions

The NATSIHS included Indigenous persons who were usual residents of private dwellings in Australia. Private dwellings are houses, flats, home units and any other structures used as private places of residence at the time of the survey. Usual residents are those people who usually live in a particular dwelling and regard it as their own or main home.

The estimated resident Indigenous population at 31 December 2004, excluding those living in non-private dwellings, was 474,310.

The 2004-05 NATSIHS was conducted in non-remote and remote areas in all States and Territories.


Exclusions

The survey excluded visitors to private dwellings. Those visitors who were a usual resident of another dwelling in scope of the survey had a chance of being selected in the survey at that dwelling or, if not selected, would have been represented by similar persons who were selected in the survey.

Non-Indigenous people were not eligible for selection in the NATSIHS.

Indigenous persons usually resident in non-private dwellings such as hotels, motels, hostels, hospitals, short-stay caravan parks, prisons and other correctional facilities were also excluded.

The 2004-05 NHS inclusions and exclusions were comparable to the NATSIHS, except that non-Indigenous people were eligible for selection and it excluded very remote areas of Australia.


SURVEY DESIGN

The NATSIHS was designed to produce reliable estimates at the national level and for each State and Territory. In addition, the Torres Strait Islander population was over sampled in order to produce reliable estimates for the Torres Strait Area and the remainder of Queensland.

The design of the NATSIHS incorporated a sample of discrete Indigenous communities (including any outstations associated with them) in remote areas of Western Australia, South Australia, Queensland and the Northern Territory. The NATSIHS also incorporated a sample of dwellings in other areas of these state/territories not covered by the discrete Indigenous community sample and the remaining states/territory (referred to as 'non-community').

The community sample was obtained from a random selection of discrete Indigenous communities and outstations across Australia taken from a specially developed Indigenous Community Frame (ICF). The ICF was constructed using both 2001 Census information and information collected in the 2001 Community Housing and Infrastructure Needs Survey. Within selected communities and outstations a random selection of dwellings was made.

Dwellings in non-community areas were selected using a stratified multistage area sample. A sample of Census Collection Districts (CDs) was randomly selected with the likelihood of a CD's selection based on the number of dwellings containing Indigenous persons in the area as at the 2001 Census of Population and Housing. A random selection of dwellings within selected CDs were then screened to assess their usual residents' Indigenous status.

Within each household, a random sub-sample of usual residents was selected for inclusion in the survey as shown below.

Households selected in community areas selected:

• up to one adult (aged 18 years and over); and,
• up to one child aged 0-17 years.

Households selected in non-community areas selected:

• up to two adults (aged 18 years and over); and,
• up to two children aged 0-17 years.

Sample was also obtained from the 2004-05 NHS. Indigenous households selected in the NHS sample selected:

• one adult (aged 18 years and over) (Indigenous or non-Indigenous);
• up to one child aged 0-17 years (Indigenous or non-Indigenous);
• and, if applicable (other Indigenous adults/children in the household),
• an additional adult (Indigenous only); and,
• an additional child (Indigenous only).

The Indigenous adult and/or child selected as part of the NHS sample responded to both the questions required for the NHS and additional NATSIHS specific questions. The additional Indigenous adult and/or child responded to only the NATSIHS questions.

Non-Indigenous households in the NHS selected one adult and up to one child aged 0-17.


DATA COLLECTION

In non-community areas, households were screened for Indigenous residents by asking any responsible adult in the household whether any of the usual residents identified themselves as Indigenous. After screening about 180,000 households, approximately 2.1% were identified. Approximately 83% of in-scope households responded fully to the survey. In communities, the response rate was 85%. These response rates exclude the households that were unable to be contacted to establish the Indigenous status of the occupants, and do not take into account those households where partial response was received and incorporated into the sample.

A total of 10,439 Indigenous persons from 5234 households contributed to the final sample for the NATSIHS. This represents about 1 in 45 of the total Indigenous population from across Australia.

Data collection was undertaken by ABS interviewers. Persons aged 18 years or more were interviewed personally, with the exception of persons who were too sick or otherwise unable to respond personally. Persons aged 15 to 17 years were interviewed with the consent of a parent or guardian. If consent wasn't obtained a parent or guardian was interviewed on their behalf. For persons aged less than 15 years, information was obtained from a person responsible for the child. Information about the dwelling, the financial situation of the household, and income for those who had not been selected was collected from a nominated household spokesperson.

There were a number of differences between the data collection methods used in remote communities (in Western Australia, South Australia, Queensland and the Northern Territory) and those used in other geographic areas. In remote communities the standard household survey approaches were modified to take account of language and cultural issues. Interviews were conducted using a paper questionnaire. The survey content in these remote communities excluded topics for which acceptable quality could not be collected. Some questions were reworded to assist respondents in understanding the concepts. Information on substance use was not collected. Only a subset of the supplementary women's health topics was collected and this was done through personal interview with adult female respondents who were informed of the potential sensitivity and voluntary nature of these questions.

Interviews conducted in other geographical areas used a Computer Assisted Interviewing (CAI) questionnaire. CAI involves the use of a notebook computer to record, store, manipulate and transmit the data collected during interviews. In addition, there were two small paper questionnaires which covered substance use (for all persons aged 15 years and over) and specific supplementary women's health topics (for women aged 18 years and over). These additional questionnaires were voluntary and self-enumerated.

Because of the different collection methodologies described above not all data items are available for the total Indigenous population. The content for the NATSIHS in remote community areas is a subset (approximately 80%) of the content collected in other areas. Data items not collected in remote Indigenous communities are not released for the remote/very remote geographic area in general. For more details on geographic restrictions to data items see Geography in Chapter 3: Using the CURF Data


WEIGHTING, ESTIMATION AND BENCHMARKS

As the survey was conducted on a sample of Indigenous households in Australia, it is important to take account of the method of sample selection when deriving estimates from the CURF. This is particularly important as a person's chance of selection in the survey varied depending on the state or territory in which they lived.

Where estimates are derived from the CURF it is essential that they are calculated by adding the weights of persons in each category, and not just by counting the number of records falling into each category. If each person's 'weight' were to be ignored, then no account would be taken of a person's chance of selection or of different response rates across population groups, with the result that estimates produced could be seriously biased.

Weighting is the process of adjusting results from the sample survey to infer results for the total in-scope population. To do this, a weight is allocated to each sample unit i.e. each person or household. The weight effectively indicates how many population units are represented by the sample unit.

The first step in calculating weights for each sample unit is to assign an initial weight which is equal to the inverse of the probability of being selected in the survey. For example, if the probability of a person being selected in the survey was one in 600, then the selected person would have an initial weight of 600 (that is, they represent 600 persons in the population) The initial weights are then calibrated to align with independent estimates of the population of interest, referred to as 'benchmarks'. This is done to ensure that the survey estimates conform to the independently estimated distribution of the population, rather than the distribution within the sample itself.

Replicate weights, which can be used to calculate sampling error, have been included on the CURF. For more information, refer to the 'Standard Errors' section in Chapter 3: Using the CURF Data. Age standardised weights and age standardised replicate weights have also been included on the CURF. For more information refer to the 'Age standardisation' section in Chapter 3: Using the CURF Data.

The survey was benchmarked to the estimated Indigenous population living in each state and territory. The process of weighting ensures that the survey estimates conform to person benchmarks at broad age, sex, State/Territory and remoteness (major cities of Australia, inner regional Australia, outer regional Australian and other) levels.

The 2004-05 NATSIHS was benchmarked to the estimated Indigenous population (adjusted for the scope of the survey) as at 31 December 2004. These estimates were based on the 2001 Census of Population and Housing. For further information about the benchmarks and weighting, see Chapter 2 of the Users' Guide. For details regarding the benchmarks and weighting for the 2004-05 NHS, see Chapter 2 of the National Health Survey: Users' Guide (cat no. 4363.0.55.001).


Reliability of Estimates

Estimates derived from the CURF are subject to both sampling and non-sampling errors.

Sampling error arises because the estimates are based on a sample of units and so will differ from estimates that would have produced if all units in the population had been included in the survey. One measure of the likely difference is given by the standard error (SE), which indicates the extent to which an estimate might have varied because only a sample of dwellings was included. There are about two chances in three that the sample estimates will differ by less than one SE from the number that would have been obtained if all persons had been surveyed and about 19 chances in 20 that the difference will be less than two SEs. Instructions on how to calculate SEs can be found in Chapter 3: Using the CURF Data.

Another measure of the sampling error is the relative standard error (RSE) which is obtained by expressing the SE as a percentage of the estimate. Generally, only estimates (numbers, percentages, means and medians) with RSEs less than 25% are considered sufficiently reliable for most purposes. The formula for calculating the RSE of an estimate (y) is:

RSE(y)=SE(y)/y * 100%

Further information about the use of replicate weights to produce the SEs or RSEs is contained in Chapter 3: Using the CURF Data of this manual and in Appendix 6 to the Users' Guide. It should be noted that not all statistical computer packages may allow direct calculation of SEs using the Jackknife replicate weights. However, those packages that allow the direct use of Balanced Repeated Replication (BRR) methodology generally include the option of an adjustment factor. This factor can be incorporated to overcome the difference between the variable formulae.

Non-sampling errors are inaccuracies that occur because of imperfections in reporting by respondents and interviewers, and errors made in coding and processing data. These inaccuracies may occur in any enumeration whether it be a full count or a sample of the population. Every effort is made to reduce the non-sampling error to a minimum by careful design of questionnaires, intensive training and supervision of interviewers, and efficient processing procedures.