4221.0 - Schools, Australia, 2009  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 28/09/2010  Reissue
   Page tools: Print Print Page Print all pages in this productPrint All

APPENDIX 3 ALTERNATIVE MEASURES OF ENGAGEMENT IN SECONDARY EDUCATION


ALTERNATIVE MEASURES OF ENGAGEMENT IN SECONDARY EDUCATION

This article briefly describes a suite of measures of school engagement based on aggregate data from the National Schools Statistics Collection (NSSC). Taken together, these measures can be used as an alternative to the standard Apparent Retention Rate (ARR). At the time of developing these measures the ABS anticipated replacement of the ARR as a measure of student engagement. It is clear, however, that until direct measurement of student transitions from unit record data is developed, ARRs provide a useful measure for performance monitoring of retention by Indigenous status. Accordingly, the ABS will continue to publish ARRs for Indigenous, non-Indigenous and total students by jurisdiction.

The alternative measures presented here nevertheless provide valuable insight into secondary school participation, continuation and progression for the total school student population. They also provide more accurate measures of progression than ARRs both within and across jurisdictions.

Successful completion of Year 12 is associated with positive outcomes in a wide variety of areas of social concern. It provides the springboard to future study and consequent gains for employment opportunities with associated benefits both for the individual's economic wellbeing and the prosperity of the nation. There are also positive associations between Year 12 attainment and other areas such as health and law and justice. Accordingly, government policies related to productivity are directed at boosting the level of Year 12 or equivalent attainment. Furthermore, current policy by jurisdictions to increase the compulsory age for participation in education to 17 years will increase the proportion of young people who remain in education. The broadening of education pathways means that students are able to choose alternative options to full-time school-based learning such as part-time study and Vocational Education and Training (VET).

Within the context of social inclusion, there is special interest in those groups in the population who are unable to complete their schooling and therefore are at risk of experiencing continued disadvantage. These groups include Indigenous students, students from low socioeconomic backgrounds and students with a disability.

There is currently no national dataset that would allow the direct monitoring of student transitions throughout the course of their education, whether in the school sector or elsewhere.


Current measures

In the absence of direct measurement, Apparent Retention Rates (ARRs) based on aggregate enrolment data were developed to provide an indicative measure of the retention and progress of students through secondary school. ARRs are the ratio between the number of students enrolled in a base year, such as the commencement year of secondary school, and the number enrolled in selected Year levels such as Year 10 or Year 12 in subsequent years.

The value of ARRs has been their capacity, albeit with some limitations, to provide information not only on the total population of students but also on the engagement of Indigenous students in the school system and facilitate comparisons between Indigenous and non-Indigenous students by jurisdiction.

There are, however, a number of limitations to ARRs which have been documented elsewhere. These include distortion due to interstate and, overseas migration, and mature-age students returning to complete their schooling. Specifically:

  • Jurisdictions that have net gains in migration will have consequent increases in ARRs due to the arrival of new students rather than improved retention of the base group. States with net losses experience corresponding reductions in ARRs irrespective of student behaviour.
  • Mature-age students are absent from their own commencement year cohort but inflate the ARR for the cohort they join.
  • A further limitation of ARRs is their narrow focus on retention of full-time students only over a time period determined by the 'usual' rate of student advancement of one Year level per calendar year. With alternative education pathways and the increase in some jurisdictions of part-time options for schooling, a growing minority of students are progressing through school at a different rate.


Alternative measures of engagement

In response to concerns about the accuracy and scope of ARRs the ABS developed a suite of alternative measures, first published in the research paper 'Deriving measures of engagement in secondary education from the National Schools Statistics Collection in 2006' (cat. no. 1351.0.55.016). Since that time, these measures have been progressively introduced into Schools Australia 2009 and associated web-based outputs.

The new measures are based on the proportion of the population enrolled in school rather than on counts of enrolments. The main benefit of the new approach is that year to year changes in enrolments are adjusted for underlying population change within the jurisdiction. Progression is also limited to the dominant age groups in the Year level thereby removing the effects of atypical age enrolments. Further, the measures are presented as a suite rather than a single measure to support a more complete picture of student transitions. The suite comprises the following measures:
  • School Participation Rate (SPR)
  • Apparent Continuation Rate (ACR)
  • Apparent Progression Rate (APR)

While the new Apparent Progression Rate is the closest counterpart to the Apparent Retention Rate, the Apparent Continuation Rate is a supplementary measure which includes part-time as well as full-time students and examines continuation by age not limited by the rate of progression.

The new measures also have a number of limitations. The ACR and APR are based on aggregate data and as such provide indirect or apparent measures of student engagement and progression like the ARR. Furthermore, since the new measures rely on population estimates, their ability to report on a sub population of interest is limited by the availability or non-availability of age and annual population estimates for that group. While there is ongoing work to improve estimation of the Indigenous population, a reliable series at the required disaggregation by single year of age and jurisdiction is not yet available. Population estimates by socioeconomic status or disability are not produced.


School Participation Rate (SPR)

What proportion of the population by age are enrolled at school?

Nationally, between 2006 and 2009, the proportion of young people aged 16 enrolled at school rose from 84% to 86%. In 2009, 66% of young people of this age in the Northern Territory were enrolled at school. Among the other jurisdictions, apart from the ACT, the School Participation Rate for young people aged 16 ranged from 81% in NSW and Western Australia to 96% in South Australia. The corresponding School Participation Rate for the ACT was 107% reflecting cross-border enrolments from neighbouring centres in regional NSW (Table 20, Schools Australia 2009).

The School Participation Rate (SPR) is the basic building block of the new suite of measures. It is calculated by dividing the number of enrolled school students for each single year of age by the Estimated Resident Population (ERP) for that age. SPRs are calculated for each jurisdiction and separately for males, females and persons. In the years where school attendance is compulsory, SPRs are generally close to 100%. They decrease in older ages as young people take up alternative educational pathways (e.g. VET outside school, or university) or leave education altogether. In South Australia, for example the SPR was 100% for young people aged 15, 96% for those aged 16 , 75% for those aged 17 and 16% for those aged 18 (Table 20, Schools Australia 2009).

SPRs have a number of advantages over simple counts of the number of students by age. First, since they are percentage measures, they quickly show the tapering of school participation with increasing age and allow for comparison among jurisdictions. Second, SPRs can be divided into components, such as full-time and part-time students, students at government schools and students at non-government schools, or students by Year level. SPRs therefore facilitate a detailed set of characteristics for comparison over time and across jurisdictions.

There are a number of quality issues associated with the SPR due to differences between the NSSC (source of school enrolment data - the numerator) and ERP (source of population data - the denominator). In particular, the NSSC counts students where they are enrolled, whereas ERP estimates the number of people where they usually reside. Thus students who reside in one jurisdiction but attend school in another may affect state/territory participation rates. For example, there are cross-border enrolments in ACT/ Regional NSW, Albury/Wodonga and Coolangatta/ Tweed Heads. The ACT, however, is viewed as the only jurisdiction where rates are significantly affected by cross-border students. In 2009, SPRs for ACT students aged 14, 15 and 16 ranged from about 105% to 115%, consistent with the pattern for previous years. This arises as a result of students from neighbouring centres in NSW attending ACT schools so that the number of school students in these age groups exceeds the ACT resident population estimate. NSW students in ACT schools may also inflate SPRs for older age groups even though the rates are less than 100%.

Nevertheless, SPRs by age for the ACT generally reflect relative changes in the proportion of the population enrolled at school as age increases. One exception is likely to be the change in enrolments between Year 10 and Year 11 which may reflect a jump in student numbers independent of population growth due to students moving from surrounding NSW centres to complete their final years of school at an ACT college. The measures of continuation and progression based on the ratio between SPRs for different time periods may still provide useful insight into engagement of students by age and Year level in the ACT school system.

The following differences between the NSSC and ERP do not generally have a large impact on SPRs at the juristiction level:
  • Jurisdiction boundaries for the NSSC are close to the ERP estimates but not exactly the same. The NSSC includes Jervis Bay with ACT and Norfolk Island with NSW, whereas ERP estimates do not. Christmas Island and the Cocos (Keeling) Islands are included in ERP figures but not in the NSSC.
  • There is little difference between the NSSC (1 July) and ERP (30 June) birth year reference dates, and at most a two-month only gap between the ERP enumeration date (30 June) and NSSC Census date (first Friday in August).
  • Students from overseas who enter Australia on a short-term visa (for less than 12 months) are counted in the NSSC, but not considered Australian residents for the calculation of ERP.

Since ERP figures are subject to periodic revisions, the ABS proposes to update SPRs every five years in line with ERP revisions.

SPRs by jurisdiction from 1999 are reported in Tables 19 and 20 (Schools Australia 2009) and SPRs from 1997 are published in NSSC Tables 61a and 61b as part of the Schools Australia 2009 suite of products on the ABS web site.

The table below illustrates the calculation of SPRs for young people aged 16 in South Australia. School enrolments by sex are divided by the corresponding population estimates. The SPR for all students is the sum of the SPR for full-time students and SPR for part-time students.

School Participation Rate, persons aged 16 - South Australia - 2009

Enrolments
School participation rate
Full-time
Part-time
Total
Estimated Resident Population(a)
Full-time
Part-time
Total
No.
No.
No.
No.
%
%
%

Males  
9 938
219
10 157
10 790
92.1
2.0
94.1
Females
9 662
324
9 986
10 292
93.9
3.1
97.0
Persons
19 600
543
20 143
21 082
93.0
2.6
95.5

(a) At 30 June 2009.
Source: NSSC Table 61b - Schools Australia 2009 (cat. no. 4221.0)



Continuation

What proportion of an age group have continued at school from one year to the next?

Based on Apparent Continuation Rates (ACRs), nationally, 90% of students aged 15 in 2008 continued their schooling in 2009. The corresponding rates were 88% for males and 92% for females. Among the jurisdictions, the Age 15/16 ACR for 2009 ranged from 83% in the Northern Territory to 97% in South Australia (NSSC Table 21, Schools Australia 2009).

Apparent Continuation Rate, students aged 15 in 2008 - Australia - 2008 to 2009


The Apparent Continuation Rate (ACR) measures the proportion of an age cohort who remain at school from one year to the next. It is measured by comparing SPRs for successive years. For example, to look at continuation from age 15 to 16, the SPR for students aged 15 in one calendar year is compared with the SPR for students aged 16 one year later.

The advantage of ACRs is that they report on the continued engagement in school of both full-time and part-time students including those who may not have progressed to the next Year level. They therefore provide a complementary measure to the Apparent Progression Rate (see below), which is focussed on full-time students only who progress at the usual rate from one Year level to the next over each calendar year.

Further, since the ACR is based on SPRs, the measure takes some account of migration. It does not simply compare the number of students enrolled in successive calendar years but the ratio between the proportion of the state/territory population enrolled at each time point. Changes in student numbers due to interstate or international migration are to some extent also reflected in the underlying population estimates. Nevertheless, the measure is an apparent rate and migration flows may have an effect on the results.

The way in which the ratio between SPRs takes account of population change can be seen more clearly if the ratio is expressed in a slightly different form. The ACR for an age group in calendar year y, that is measuring continuation from one year (y-1) to the next (y), can be written as follows:

Equation: Andrew W

In other words, the ACR is the ratio between enrolments at the different time points multiplied by an adjustment factor for population change. If a jurisdiction experiences population growth, then the enrolment ratio is reduced. If population falls, the ratio is adjusted upwards. This formulation shows the value of using SPRs even where they exceed 100% since the key issue is whether or not the adjustment factor of population change adds value to the simple ratio of enrolment numbers.

ACRs by jurisdiction for the current and previous reference year are reported in Table 21 (Schools Australia 2009) and ACRs from 1998 are published in NSSC Table 62a as part of the Schools Australia 2009 suite of products on the ABS web site.

The table below illustrates the calculation of ACRs for students aged 15 in Victoria in 2008 who continued at school from 2008 to 2009. The Age 15/16 ACR for 2009 is the ratio between the SPR for students aged 16 in 2009 and the SPR for students aged 15 in 2008. The ACR for persons (93.5%) is the weighted sum of the ACR for males (92.0%) and females (95.1%) (see footnote b. to the table).

A variation of the Apparent Continuation Rate (ACR) is the grade cohort ACR. This measure assesses the continuation from one year to the next of all students of a particular age up to a certain grade and can help identify the age and grade at which young people exit the school system. It is not currently reported in Schools Australia 2009. For further information see 'Deriving Measures of Engagement in Secondary Education from the National Schools Statistics Collection 2006' (cat. no. 1351.0.55.016).

Apparent Continuation Rate, students aged 15 - Victoria - 2008 to 2009

Enrolments (Full-time & part-time)
Estimated Resident Population(a)
School Participation Rate
Weight(b)
Apparent Continuation Rate
No.
No.
%
No.
%

Males
Aged 15 in 2008
34 029
35 245
96.56
0.5070
92.0
Aged 16 in 2009
31 852
35 872
88.80
-
-
Females
Aged 15 in 2008
33 088
33 779
97.97
0.4930
95.1
Aged 16 in 2009
32 092
34 450
93.15
-
-
Persons
Aged 15 in 2008
67 117
69 024
97.25
1.0000
93.5
Aged 16 in 2009
63 944
70 322
90.93
-
-

- nil or rounded to zero (including null cells)
(a) At 30 June 2009.
(b) The weight is the proportion of male/female students in the base year (2008). The ACR for persons is the weighted sum of the ACRs for males and females (i.e. ACR multiplied by weight for each group and then summed).
Source: NSSC Table 62a - Schools Australia 2009 (cat. no. 4221.0)



Progression

What proportion of full-time students have progressed through school at the usual rate?

Based on Apparent Progression Rates (APRs), nationally between 2007 and 2009, 72% of full-time students progressed from Year 10 to Year 12. The corresponding rate was 68% for male students and 77% for female students. Among the jurisdictions, the Year 10/12 APR for 2009 ranged from 57% in Tasmania to 75% in Queensland, 76% in Victoria and 80% in the ACT (NSSC Table 22, Schools Australia 2009).

The ratio of full-time SPRs from one Year level to the next over a 12-month period is used to provide a measure of school progression. Therefore, like the ACR, the APR removes much of the bias arising from migration. By focusing on the dominant age groups in each Year it also removes bias due to returning mature-age and repeating students.


Age-specific Apparent Progression Rate (APR)

The first step in deriving the overall progression measure is calculation of progression from Year level to Year level for each age separately. For example, using SPRs we look at the progression from Year 10 to Year 11 of students aged 14 in Year 10. Separately, we look at progression to Year 11 of students aged 15 in Year 10. And again, at progression for students aged 16 in Year 10. Each of these ratios results in a different age-specific Apparent Progression Rate (APR) (full-time students only are included in this measure and ungraded students are excluded).

Similar to the case for Apparent Continuation Rates above, the use of the SPR ratios in the calculation of Apparent Progression Rates means that progression based on enrolment numbers is adjusted for a factor to account for population change. Age-specific APRs may exceed 100% especially for the oldest age group in a Year level. In these cases, they are capped at 100% when used in subsequent calculations.

Age-specific APRs for each jurisdiction from 1997 are published in NSSC Table 60a as part of the Schools Australia 2009 suite of products on the ABS web site.

The table below illustrates the calculation of age-specific APRs for students aged 15 in Year 10 in NSW in 2008 who progressed to Year 11 in 2009. The age-specific APR is the ratio between the SPR for students aged 16 in Year 11 in 2009 and the SPR for students aged 15 in Year 10 in 2008. The age-specific APR for persons (82.4%) is the weighted sum of the age-specific APR for males (79.3%) and females (85.3%) (see footnote b. to the table).

Age-Specific Apparent Progression Rate, full-tme students aged 15 in Year 10 - New South Wales - 2008 to 2009

Full-time enrolments
Estimated Resident Population(a)
Full-time School Participation Rate
Weight(b)
Age-specific APR
No.  
No.
%
No.
%

Males
Aged 15 in Year 10 in 2008
31 132
47 449
65.61
0.4886
79.3
Aged 16 in Year 11 in 2009
25 019
48 061
52.06
-
-
Females
Aged 15 in Year 10 in 2008
32 580
45 246
72.01
0.5114
85.3
Aged 16 in Year 11 in 2009
28 109
45 763
61.42
-
-
Persons
Aged 15 in Year 10 in 2008
63 712
92 695
68.73
-
82.4
Aged 16 in Year 11 in 2009
53 128
93 824
56.63
-
-

- nil or rounded to zero (including null cells)
(a) At 30 June 2009.
(b) The weight is the proportion of male/female students in the base year (2008). The ACR for persons is the weighted sum of the ACRs for males and females (i.e. ACR multiplied by weight for each group and then summed).
Source: NSSC Table 60a - Schools Australia 2009 (cat. no. 4221.0)



Apparent Progression Rate over one Year

The second step looks at progression from one Year level to the next for the whole group of students. As illustrated in the example above, each Year level is comprised of students of different ages. Differences in enrolment policy further increases this variety across the jurisdictions. For example, students may be aged 14, 15 or 16 or other ages in Year 10. Depending on the jurisdiction, the majority may be in one or other of these single year age groups. As a consequence, at least three age-specific APRs would need to be calculated to measure progression from Year 10 to Year 11.

More generally, the APR measures the proportion of full-time students who progress from one Year level to the next over a 12-month period. For the purpose of calculation, the measure is restricted to the cohort of students based on the three dominant age groups in the Year level in the base year. At the national level the dominant ages for each Year are based on age at 1 July and run from 12, 13 and 14 in Year 8 to 16, 17 and 18 in Year 12. These ages capture the vast majority of students in respective Year levels across the jurisdictions.

APRs by jurisdiction for the current reference year are reported in Table 22 (Schools, Australia 2009). APRs for each jurisdiction from 1997 are published in NSSC Table 65a as part of the Schools Australia 2009 suite of products on the ABS web site.

The table below illustrates the calculation of the APR for female students who progressed from Year 10 to Year 11 in Queensland between 2008 and 2009. The Year 10/11 APR for 2009 (90.1%) is the weighted sum of age-specific APRs for female students who were aged 14 (90.4%), 15 (92.0%) and 16 (100%) in Year 10 in 2008 (see footnote b. of the table).

Apparent Progression Rate(a), female students in Year 10 to Year 11 - Queensland - 2008 to 2009

Full-time enrolments, 2008
Age-specific APR(b)
Weight(c)
Component rates and APR
No.
No.
%
%

Aged 14 in Year 10 in 2008
10 750
90.40
0.3801
34.4
Aged 15 in Year 10 in 2008
16 341
92.00
0.5778
53.2
Aged 16 in Year 10 in 2008
1 192
100.00
0.0421
4.2
Total female students
28 283
-
-
91.7

- nil or rounded to zero (including null cells)
(a) In the original research paper, this measure was referred to as the Apparent Grade Progression Rate.
(b) Age-specific APRs are capped at a maximum value of 100% for the purpose of calculation.
(c) The weight is the relative proportion of students in each of the three dominant age groups in the base year (2008). The APR for total female students is the weighted sum of the age-specific APRs for each age (i.e. age specific APR multiplied by weight for each group and then summed).
Source: NSSC Table 65a - Schools Australia 2009 (cat. no. 4221.0)



Apparent Progression Rate over more than one Year

The third and final step looks at progression over more than one Year level. The key intervals for APRs are Year 8 to Year 10, Year 8 to Year 12 and Year 10 to Year 12.

Whereas the Apparent Retention Rate (ARR) is the simple ratio between enrolments in the start Year and end Year, the APR is calculated by multiplying each APR from the start to end Year. For example, the APR from Year 8 to Year 12 is the product of APRs from Year 8 to Year 9, Year 9 to Year 10, Year 10 to Year 11 and Year 11 to Year 12.

APRs by jurisdiction for the current reference year are reported in Table 22 (Schools, Australia 2009). APRs for each jurisdiction from 1997 are published in NSSC Table 65a as part of the Schools Australia 2009 suite of products on the ABS web site.

The table below illustrates the calculation of APRs for students in Western Australia. The Year 8/10 APR for 2007 (97.7%) is the product of the Year 8/9 APR for 2006 (99.8%) and Year 9/10 APR for 2007 (97.9%). Similarly, the Year 8/12 APR for 2009 (67.5%) is the product of the four single grade APRs shown in the table.

Apparent Progression Rate(a) - Western Australia - 2005 to 2009

Single grade APR
Year 8/10 APR, 2007 (AxB)
Year 10/12 APR, 2009 (CxD)
Year 8/12 APR, 2009 (AxBxCxD)
%
%
%
%

Year 8 to Year 9 (A)
2005 to 2006
99.76
-
-
-
Year 9 to Year 10 (B)
2006 to 2007
97.91
97.7
-
-
Year 10 to Year 11 (C)
2007 to 2008
89.34
-
-
-
Year 11 to Year 12 (D)
2008 to 2009
77.32
-
69.1
67.5

- nil or rounded to zero (including null cells)
(a) In the original research paper, this measure was referred to as the Cumulative Apparent Grade Progression Rate.
Source: NSSC 65a - Schools Australia 2009 (cat. no. 4221.0)



Comparison of the Apparent Progression Rate and Apparent Retention Rate

In 2009, the Year 8/12 Apparent Progression Rate (APR) for Australia was 70% compared with the corresponding Apparent Retention Rate (ARR) of 76% a difference of 6 percentage points. In all jurisdictions the Year 8/12 APR was lower than the corresponding ARR with the difference ranging from 1 percentage point in the Northern Territory to 9 percentage points in the ACT.

Year 8-12 aprs and arrs by state and territory - 2009
Graph: Year 8-12 aprs and arrs by state and territory - 2009


Looking at APRs and ARRs for Year 8 to Year 12 over the past eight years we observe that for each series the rates for females are generally higher than the rates for males indicating higher rates of progression/retention among female than male students. The exception is the ACT, where male APRs are similar to female APRs and male ARRs are similar to female ARRS.

When comparing the two measures, we observe that the APR series is generally lower and smoother than the corresponding ARR series. APRs and ARRs are more closely aligned in NSW than in the other jurisdictions while in the Northern Territory there is no clearly defined relationship between them (see charts below).


Future Directions

The ABS will continue to work with stakeholders to improve Indigenous identification and the identification of parental characteristics useful for assessing socioeconomic status in the NSSC. While the APR has advantages over the ARR for the total student population, since it adjusts for migration and mature-age students, the ARR based on unadjusted enrolments is nevertheless a useful measure for Indigenous students. Comparisons between APRs for the total population and ARRs for Indigenous students could be considered.

The ABS views direct measures of student transitions as preferable to apparent measures. We will continue to work with jurisdictions and through advisory committees under MCEECDYA to transform the NSSC into a national longitudinal dataset of unit record data. Such a dataset would form the core of the proposed Australian Longitudinal Learning Database (ALLD) to be developed by the ABS as a student-centred information base covering early education, school, VET and higher education. The implementation of a national unique student identifier within schools would also facilitate data linkage and support direct measurement of student transitions.

The ALLD will not only comprise unit level pathways for each student but, with community support, could be linked to the Census of Population and Housing and other data sources to enrich the availability of contextual and outcome information. This would provide information on schooling for social inclusion research of at-risk subpopulations such as students from low socioeconomic backgrounds and students with disabilities.

An additional benefit of a national student-centred longitudinal learning database would be its potential to examine engagement in education within regions or by remoteness. Because of the limitations of aggregate data, neither ARRs nor the new suite of measures can be meaningfully calculated for geographic outputs other than jurisdiction.


Apparent Progression Rates (APR) and Apparent Retention Rates (ARR) 2001 - 2009
New South Wales
Graph: NEW SOUTH WALES


Victoria
Graph: Victoria


Queensland
Graph: Queensland


South Australia
Graph: South Australia


Western Australia
Graph: Western Australia


Tasmania
Graph: Tasmania


Northern Territory
Graph: Northern Territory


Australian Capital Territory
Graph: Australian Capital Territory