| Module 3: Interpreting Data
3. Questioning the data
Many important and costly policies can flow from the results of a study. So it is important to be sure that a study is conducted in such a way that it produced (as much as possible), unbiased data.
Eliminating bias can be harder than you think.
3.1 Was the variable well defined?
Consider the following scenario.
| 4102.0 - Australian Social Trends, 1995
PREVIOUS ISSUE Released at 11:30 AM (CANBERRA TIME) 20/06/1995 |
The extract below is from the Australian Bureau of Statistics (ABS) site, describing data collection in household crime. The ABS collected their data from a survey of 10,000 people.
Housing Stock: Safe as houses?
In 1993, over half a million households were victims of household crime. 62% of these households were not members of neighborhood or rural watch.
Since World War II most western countries have experienced increases in recorded crime as measured by police statistics. However, increases in official crime rates may merely indicate an increase in the number of offences recorded by the police rather than an actual increase in the number of offences committed. There is a public perception that crime, particularly violent crime, has increased in Australia in recent years. But 1993 data indicate that there has been only a marginal change in the level of violent crime compared to 1983. While the victimisation rate for robbery has doubled from 0.6% to 1.2%, the rate for sexual assault has remained virtually unchanged at 0.6% and the rate for other types of assault has decreased from 3.4% to 2.5%. It should be noted that these 1983 and 1993 data are 'snapshots' of the incidence of violent crime and that criminal activity may have fluctuated at times between these two years. |
(Australian Bureau of Statistics 1995) [2]
| Comments on the ABS crime surveys
While ABS Crime and Safety Surveys can be used to measure changes in patterns of crime, care must be used in using their results because of methodological and definitional differences between the surveys. Important differences between the 1983 and 1993 surveys include:
- in 1983 data were collected using face-to-face interviewing while in 1993 self-completed questionnaires were used;
- although both surveys had a 12-month reference period, the 1993 survey was conducted in April while the sample for the 1983 survey was spread over 12 months, from February 1983 to January 1984.
The victimisation rate is the number of people or households in a particular category who reported being victims of crime, expressed as a percentage of all people or households in that category. Victims were counted only once for each type of offence, regardless of the number of incidents of that type.
Household crime consists of break and enter, attempted break and enter and motor vehicle theft. The latter includes the theft of a motor vehicle, owned or used exclusively by a household member, which may have occurred away from the home
Comparisons with police statistics
Responses obtained in ABS Crime and Safety Surveys are based on the respondents' perceptions that they have been the victim of an offence. Data on crimes not reported to police are collected. The terms used summarise the wording of questions asked of respondents and may not correspond with legal or police definitions. |
This scenario shows the complexities involved in defining and using a variable, especially when there are two or more agencies using the same concept.
Problems in discussing data on crime include:
1. different collection methodologies within ABS - face to face and then unsupervised questionnaire
2. different time frames for ABS collection
3. police data is based only on reported crime (the ABS tries to identify such discrepancy in interviews)
4. Police definitions may differ from ABS definitions
5. Some household crime may not occur in the home (car theft)
6. Victimisation rates may be lower than the real rate of incidents because multiple counting is not used.
The above scenario also suggests how important it is to keep in mind that in the print media, on TV and on the internet you are possibly reading not just the researchers' reports but other people's ideas of how the data were produced, presented and interpreted.
In reading about the results of a study the first thing that you must think about is how the data were produced. This relates to identifying the population of interest for a study.
|  |
