Work-Related Training and Adult Learning, Australia methodology

1 This publication contains results from the Survey of Work-Related Training and Adult Learning (WRTAL), a topic on the Multipurpose Household Survey (MPHS) conducted throughout Australia from July 2016 to June 2017. The MPHS, undertaken each financial year by the Australian Bureau of Statistics (ABS), is a supplement to the monthly Labour Force Survey (LFS) and is designed to collect statistics for a number of small, self-contained topics. In 2016-17, the topics were:

• Work-Related Training and Adult Learning
• Household Use of Information Technology
• Patient Experiences in Australia
• Barriers and Incentives to Labour Force Participation
• Retirement and Retirement Intentions
• Crime Victimisation
• Income (Personal, Partner's, Household)
   

2 The WRTAL survey collected information about the level of participation of Australia's population in formal and non-formal learning, with a particular focus on work-related training and personal interest learning. Information available from the survey includes participation rates in non-formal learning, the reasons for participation, the time spent, personal costs incurred and also data on the barriers that prevent some people from undertaking training. Information on labour force characteristics, education, income and other demographics was also collected.

3 The WRTAL survey was previously conducted in April 2013 as a supplement to the monthly LFS. Further details are outlined below in the Data Comparability section.

4 The scope of the WRTAL survey was restricted to persons aged 15-74 years who were usual residents of private dwellings excluding: 

• members of the Australian permanent defence forces
• certain diplomatic personnel of overseas governments, customarily excluded from the Census of Population and Housing and estimated resident population counts
• overseas residents in Australia
• members of non-Australian defence forces (and their dependants)
• persons living in non-private dwellings such as hotels, university residences, boarding schools, hospitals, nursing homes, homes for people with disabilities, and prisons
• persons resident in the Indigenous Community Strata (ICS).

5 The scope for MPHS included households residing in urban, rural, remote and very remote parts of Australia, except the ICS.

6 In the LFS, rules are applied which aim to ensure that each person in coverage is associated with only one dwelling, and hence has only one chance of selection in the survey. See Labour Force, Australia (cat. no. 6202.0) for more detail.

7 Each month, one eighth of the dwellings in the LFS sample were rotated out of the survey. These dwellings were selected for the MPHS. In these dwellings, after the LFS had been fully completed for each person in scope and coverage, a usual resident aged 15 years or over was selected at random (based on a computer algorithm) and asked the additional MPHS questions in a personal interview. The publication Labour Force, Australia (cat. no. 6202.0) contains information about survey and sample design, scope, coverage and population benchmarks relevant to the monthly LFS, and consequently the MPHS. This publication also contains definitions of demographic and labour force characteristics, and information about telephone interviewing.

8 In the MPHS, if the randomly selected person was aged 15 to 17 years, permission was sought from a parent or guardian before conducting the interview. If permission was not given, the parent or guardian was asked the questions on behalf of the 15 to 17 year old (proxy interview). 

9 Data were collected using Computer Assisted Interviewing (CAI), whereby responses were recorded directly onto an electronic questionnaire in a notebook computer, with interviews conducted either face-to-face or over the telephone. The majority of interviews were conducted over the telephone.

10 After taking into account sample loss, the response rate for the Survey of Work-Related Training and Adult Learning was 72.1%. In total, information was collected from 28,207 fully responding persons. Of these, 25,411 persons were within the scope of the WRTAL topic, that is, those people aged 15 to 74 years. This includes 463 proxy interviews with people aged 15 to 17 years, where permission was not given by a parent or guardian for a personal interview. 

11 All respondents who reported being permanently unable to work and those aged 65-74 years who were permanently not intending to work, were not asked questions about participation in work-related training. These people were all classified as having not participated. There were an estimated 1.4 million persons in Australia that made up these two groups, comprising around 330,000 people who are permanently unable to work and 1.1 million people aged 65-74 years who were permanently not intending to work. Note that information regarding participation in personal interest learning and formal learning was collected for these respondents. 

12 Weighting is the process of adjusting results from a sample survey to infer results for the total 'in-scope' population. To do this, a 'weight' is allocated to each enumerated person. The weight is a value which indicates how many persons in the population are represented by the sample person.

13 The first step in calculating weights for each unit is to assign an initial weight, which is the inverse of the probability of the unit being selected in the survey. For example, if the probability of a person being selected in the survey was 1 in 600, then the person would have an initial weight of 600 (that is, they represent 600 people).

14 The initial weights are calibrated to align with independent estimates of the population, referred to as benchmarks. The population included in the benchmarks is the survey scope. 
This calibration process ensures that the weighted data conform to the independently estimated distribution of the population rather than the distribution within the sample itself. 
Calibration to population benchmarks helps to compensate for over or under-enumeration of particular categories of persons which may occur due to either the random nature of sampling or non-response.

15 The survey was benchmarked to the Estimated Resident Population (ERP) aged 15-74 years living in private dwellings in each state and territory, at December 2016. People living in Indigenous communities were excluded. These benchmarks are based on the 2011 Census of Population and Housing.

16 While LFS benchmarks are revised every 5 years, to take into account the outcome of the 5-yearly rebasing of the ERP following the Census, the supplementary surveys and MPHS (from which the statistics in this publication are taken) are not. Small differences will therefore exist between the civilian population aged 15 years and over reflected in the LFS and other labour household surveys estimates, as well as over time.

17 Survey estimates of counts of persons are obtained by summing the weights of persons with the characteristics of interest.

18 To minimise the risk of identifying individuals in aggregate statistics, a technique is used to randomly adjust cell values. This technique is called perturbation. Perturbation involves a small random adjustment of the statistics and is considered the most satisfactory technique for avoiding the release of identifiable statistics while maximising the range of information that can be released. These adjustments have a negligible impact on the underlying pattern of the statistics. After perturbation, a given published cell value will be consistent across all tables. However, adding up cell values to derive a total will not necessarily give the same result as published totals. The introduction of perturbation in publications ensures that these statistics are consistent with statistics released via services such as TableBuilder.

19 Perturbation has been applied to the 2016-17 WRTAL data. Data from the 2013 WRTAL survey have not been perturbed.

20 All sample surveys are subject to error which can be broadly categorised as either: sampling error or non-sampling error. For more information refer to the Technical Note.

21 Sampling error is the difference between the published estimate, derived from a sample of dwellings, and the value that would have been produced if all dwellings in scope of the survey had been included. 

22 Non-sampling error may occur in any collection, whether it is based on a sample or a full count of the population such as a census. Sources of non-sampling error include: non-response; errors in reporting by respondents or recording of answers by interviewers; and errors in coding and processing data. Every effort was made to reduce the non-sampling error by: careful design and testing of the questionnaire; training and supervision of interviewers; follow-up of respondents; and extensive editing and quality control procedures at all stages of data processing.

23 Information recorded in this survey is ‘as reported’ by respondents, and may differ from that which might be obtained from other sources or via other methodologies. In addition, the labour force characteristics collected in the survey, such as employment status, industry and occupation, relate to the week before the survey interview and therefore may not reflect the respondent's actual labour force status at the time they participated in the training. This factor should be considered when interpreting some of the estimates in this publication. However, most tables with labour force characteristics, have been restricted to those employed persons who undertook their training as part of their current main job.

24 A small proportion of respondents were resident in areas with no Socio-Economic Indexes for Areas (SEIFA) scores allocated. For the purposes of the Survey of Work-Related Training and Adult Learning, these records have had a SEIFA decile imputed, based on the deciles of the surrounding areas. For information on SEIFA, see the Socio-Economic Indexes for Areas (SEIFA) section below.

25 The WRTAL survey was previously conducted in April 2013 as a supplement to the monthly LFS. Both survey vehicles use similar collection methodologies, i.e. both were primarily personal telephone interviews, conducted after the LFS, with one randomly selected person from the household. The questions were the same for both surveys.

26 The key difference between the 2013 and 2016-17 surveys relates to the length of enumeration time during which the data was collected. The 2013 WRTAL collected data during a one month period while the MPHS enumeration occurred over 12 months. This means that for the 2013 WRTAL, the reference period '...in the last 12 months' relates to learning undertaken in the same 12 month period (May 2012 to April 2013). However, for the 2016-17 MPHS, '...in the last 12 months' depends on which month the respondent was interviewed. For example, if a person was interviewed in July 2016, '...in the last 12 months' would refer to the period August 2015 to July 2016. If a person was interviewed in June 2017, '...in the last 12 months' would refer to the period July 2016 to June 2017. The difference in reference period is not expected to impact on the comparability of the data.

27 While there were no changes to wording for the work-related training and adult learning specific questions, the LFS survey questionnaire underwent a number changes in July 2014. For further information see Information Paper: Questionnaire Used in the Labour Force Survey, July 2014 (cat. no. 6232.0).

28 In 2013, a total of 19,976 people aged 15-74 years provided data for the WRTAL survey. Generally, increasing the sample size will reduce the error, allowing for more detailed data analysis. The increase in responding sample to 25,411 people should improve the reliability of the estimates for many of the data items.

29 Many of the data items included in the WRTAL survey are similar to those collected in the Survey of Education and Training (SET), (please refer to Education and Training Experience (cat. no. 6278.0). When comparing results from these collections, there are a number of differences in the collection methodology which impact on the final participation rates. These changes include mode effect, context effect, question wording and changes to the in-scope population. 

30 Mode effect refers to the impact of the survey delivery method on the responses to the survey. The SET was part of the Special Social Survey program which collected responses using a face-to-face interview with each person in the selected households. In particular, respondents were provided with prompt cards enabling them to read and then select the response categories for various questions. By comparison, the WRTAL survey was collected predominantly as a telephone interview with one randomly selected person in each household. As a telephone survey, prompt cards could not be used. Respondent recall bias is also likely to have more impact in telephone interviews than face-to-face interviewing, and may explain part of the difference in participation rates between 2005 and 2013 noted in the publication.

31 Context effects occur when the preceding questions influence responses to subsequent questions or when the order in which the questions are asked affects the correlation between the target and the context questions. The WRTAL survey focused on work-related training and therefore the majority of questions were framed around participation in this form of learning. SET was a large survey which focused mainly on participation in formal learning, with the questions about non-formal learning following the modules about formal learning. It is possible that the context effects may contribute to the differences in reporting of participation in work-related training.

32 Due to the change in mode of collection between SET and the WRTAL survey it was necessary for a number of the questions to be redesigned. In particular, questions about participation in work-related training were asked directly in the WRTAL survey, whereas in SET, participation in work-related training was derived from particular responses to questions relating to the reasons for undertaking all non-formal training.

33 The in-scope population for the WRTAL survey and SET were slightly different which may have contributed to differences in estimated participation rates for learning activities. While the in-scope population for the WRTAL survey included all people aged 15-74 years, the in-scope population for the latest iteration of SET was people aged 15-64 years and people aged 65-74 years who were in or marginally attached to the labour force.

34 Finally, due to the limited enumeration time available for supplementary surveys and the MPHS compared to Special Social Surveys, the WRTAL survey only collected information about the most recent work-related training and personal interest learning course for each respondent. SET, however, collected information on up to four most recent courses (either work-related training or personal interest learning) for each person in the household. This allowed for additional data to be presented in the SET publication at the specific course level that is not available in the WRTAL publications.

35 Since the WRTAL survey is conducted as a supplement to the LFS, data items collected in the LFS are also available in this publication. However, there are some important differences between the two surveys. The LFS had a response rate of over 90% compared to the MPHS response rate of 72.1%. The scope of the WRTAL survey and the LFS (refer to the Scope and Coverage section above) also differ. Due to the differences between the samples, data from the WRTAL survey and the LFS are weighted separately. Differences may therefore be found in the estimates for those data items collected in the LFS and published as part of the WRTAL survey. 

36 Estimates from the WRTAL survey may differ from the estimates for the same or similar data items produced from other ABS collections for several reasons. For example, all sample surveys are subject to different sampling errors so users should take account of the relative standard errors (RSEs) and margins of error (MOEs) on estimates where comparisons are made. Differences may also exist in scope and/or coverage, reference periods reflecting seasonal variations, non-seasonal events that may have impacted on one period but not another, or because of underlying trends in the phenomena being measured. 

37 The survey of Education and Work, Australia (SEW) (cat. no. 6227.0) has some similarities with the WRTAL survey. Conducted annually, SEW provides a range of indicators about educational participation and attainment, and data on people's transition between education and work. Comparison of SEW and WRTAL data should be undertaken with caution due to different collection methodologies, scope and sample size. SEW is based on a household interview with any responsible adult who responds on behalf of all persons aged 15-74 years in the household. Whereas WRTAL is conducted as a personal interview with one randomly selected person aged 15-74 years in the household.

38 Australian geographic data are classified according to the Australian Statistical Geography Standard (ASGS): Volume 1 - Main Structure and Greater Capital City Statistical Areas, July 2011 (cat. no. 1270.0.55.001). Remoteness areas are classified according to the Australian Statistical Geography Standard (ASGS): Volume 5 - Remoteness Structure, July 2011 (cat. no. 1270.0.55.005) 

39 Country of birth data are classified according to the Standard Australian Classification of Countries (SACC),2016 (cat. no. 1269.0). 

40 Industry data are classified according to the Australian and New Zealand Standard Industrial Classification (ANZSIC), 2006 (Revision 2.0) (cat. no. 1292.0). 

41 Occupation data are classified according to the Australian and New Zealand Standard Classifications of Occupations, 2013, Version 1.2 (cat. no. 1220.0).

42 Education data are classified according to the Australian Standard Classification of Education (ASCED), 2001 (cat. no. 1272.0). The ASCED is a national standard classification which can be applied to all sectors of the Australian education system including schools, vocational education and training and higher education. The ASCED comprises two classifications: Level of Education and Field of Education.

43 Level of Education is defined as a function of the quality and quantity of learning involved in an educational activity. There are nine broad levels, 15 narrow levels and 64 detailed levels. For definitions of these levels see the Australian Standard Classification of Education (ASCED), 2001 (cat. no. 1272.0).

44 Field of Education is defined as the subject matter of an educational activity. Fields of education are related to each other through the similarity of subject matter, through the broad purpose for which the education is undertaken, and through the theoretical content which underpins the subject matter. There are 12 broad fields, 71 narrow fields and 356 detailed fields. For definitions of these fields see the Australian Standard Classification of Education (ASCED), 2001 (cat. no. 1272.0).

45 Level of highest educational attainment was derived from information on highest year of school completed and level of highest non-school qualification. The derivation process determines which of the 'non-school' or 'school' attainments will be regarded as the highest. Usually the higher ranking attainment is self-evident, but in some cases some secondary education is regarded, for the purposes of obtaining a single measure, as higher than some certificate level attainments.

46 The following decision table is used to determine which of the responses to questions on highest year of school completed (coded to ASCED Broad Level 6) and level of highest non-school qualification (coded to ASCED Broad Level 5) is regarded as the highest. This table has been modified since the 2013 SEW. Note that this table was designed for the purpose of obtaining a single value for level of highest educational attainment and is not intended to convey any other hierarchy.

47 The decision table is also used to rank the information provided in a survey about the qualifications and attainments of a single individual. It does not represent any basis for comparison between differing qualifications. For example, a person whose highest year of school completed was Year 12, and whose level of highest non-school qualification was a Certificate III, would have those responses crosschecked on the decision table and would as a result have their level of highest educational attainment recorded as Certificate III. However, if the same person answered 'certificate' to the highest non-school qualification question, without any further detail, it would be crosschecked against Year 12 on the decision table as Level not determined. The decision table, therefore, does not necessarily imply that one qualification is 'higher' than the other. For further details, see Education Variables (cat. no. 1246.0)

48 SEIFA is a suite of four summary measures that have been created from 2011 Census information. Each index summarises a different aspect of the socio-economic conditions of people living in an area. The indexes provide more general measures of socio-economic status than is given by measures such as income or unemployment alone. 

49 Each index ranks geographic areas across Australia in terms of their relative socio-economic advantage and disadvantage. The four indexes each summarise a slightly different aspect of the socio-economic conditions in an area. It is important to note that the indexes are assigned to areas and not to individuals. They indicate the collective socio-economic characteristics of the people living in an area. The respondents in the WRTAL survey have been assigned the 2011 Census SEIFA for the area in which they live. Consequently, they may not necessarily have the same personal characteristics that describes the socio-economic status of their geographic area as a whole.

50 The index used in the Work-Related Training and Adult Learning publication is the Index of Relative Socio-Economic Disadvantage (IRSD), derived from Census variables such as income, educational attainment, employment/unemployment, occupation and some housing variables. The index ranks areas on a continuum from most disadvantaged to least disadvantaged. A low score on the index (i.e. lowest quintile or decile) indicates a high proportion of relatively disadvantaged people in an area. Such areas include many households with low income, people with no qualifications and many people in low skill occupations. It should be noted that it cannot be concluded that an area with a very high score has a large proportion of relatively advantaged (‘well off’) people, as there are no variables in the index to indicate this. It can only be concluded that such an area has a relatively low incidence of disadvantage.

51 The indexes and supporting material are found in the publication Census of Population and Housing: Socio-Economic Indexes for Areas (SEIFA), Australia (cat no 2033.0.55.001).

52 A Data Cube (spreadsheet) containing all tables produced for this publication is available from the Data downloads section. The Data Cube presents tables of estimates and their corresponding Relative Standard Errors (RSEs) as well as proportions and their associated Margins of Error (MOEs).

53 For users who wish to undertake more detailed analysis of the data, the survey microdata will be released through the TableBuilder product. For further details, refer to Microdata: Work-Related Training and Adult Learning (cat. no. 4234.0.30.001).

54 Special tabulations are available on request. Subject to confidentiality and sampling variability constraints, tabulations can be produced from the survey incorporating data items, populations and geographic areas selected to meet individual requirements. These can be provided in printed or electronic form. A list of data items which can be tabulated from the 2016-17 WRTAL survey is available from the Data downloads section in this publication. All enquiries should be made to the National Information and Referral Service on 1300 135 070.

55 For further information about these and related statistics, contact the National Information and Referral Service on 1300 135 070, or email client.services@abs.gov.au. The ABS Privacy Policy outlines how the ABS will handle any personal information that you provide to us.

56 ABS publications draw extensively on information provided freely by individuals, businesses, governments and other organisations. Their continued cooperation is very much appreciated; without it, the wide range of statistics published by the ABS would not be available. Information received by the ABS is treated in strict confidence as required by the Census and Statistics Act 1905. 

57 The ABS expects to conduct this survey again in 2020-21.

58 Current publications and other products released by the ABS are available from the ABS website. The ABS also issues a daily upcoming release advice on the websites that details products to be released in the week ahead.

1 The estimates in this publication are based on information obtained from a sample of the Australian population. Although care has been taken to ensure the survey results are as accurate as possible, there are certain factors that can affect their reliability for which no adequate adjustments can be made. These factors, known as non-sampling error and sampling error, should be kept in mind when interpreting the results of this survey.

2 Non-sampling error may occur in any collection, whether it is based on a sample of the population or a full count such as a census. Sources of non-sampling error include non-response as well as errors in reporting by respondents, recording of answers by interviewers, coding and processing data. Every effort is made to reduce non-sampling error by careful design and testing of questionnaires, training and supervision of interviewers, and extensive editing and quality control procedures at all stages of data processing. It is not possible to quantify the non-sampling error.

3 Sampling error is the difference between the published estimates, derived from a sample of persons, and the value that would have been produced if the total population (as defined by the scope of the survey) had been included in the survey. The size of the sampling error depends on the following factors:

• Sample design - the final design attempted to make the key survey results as representative as possible within cost and operational constraints
• Sample size - the larger the sample on which the estimate is based, the smaller the associated sampling error
• Population variability - the extent to which people differ on the particular characteristic being measured. This is referred to as the population variability of a particular characteristic. The smaller the population variability of a particular characteristic, the more likely it is that the population will be well represented by the sample, and, therefore the smaller the sampling error. Conversely, the more variable the characteristic, the greater the sampling error.

4 One measure of the sampling error is given by the standard error (SE), which indicates the extent to which an estimate might have varied by chance because only a sample of persons was included. There are about two chances in three (67%) that a sample estimate will differ by less than one SE from the number that would have been obtained if all persons had been surveyed, and about 19 chances in 20 (95%) that the difference will be less than two SEs. For estimates of population sizes, the size of the SE generally increases with the level of the estimate, so that the larger the estimate, the larger the SE. However, the larger the the sampling estimate that smaller the SE becomes in percentage terms. Therefore larger estimates will be relatively more stable than smaller estimates. See What is a Standard Error and Relative Standard Error, Reliability of estimates for Labour Force data for more details.

5 Another measure of the likely difference is the relative standard error (RSE), which is obtained by expressing the SE as a percentage of the estimate to which it is related. The RSE is a useful measure in that it provides an immediate indication of the percentage errors likely to have occurred due to sampling and thus avoids the need to refer also to the size of the estimate.

6 Only estimates (numbers or percentages) with RSEs less than 25% are considered sufficiently reliable for most analytical purposes. However, estimates with larger RSEs have been included. Estimates with an RSE in the range 25% to 50% should be used with caution while estimates with RSEs greater than 50% are considered too unreliable for general use. The Excel spreadsheets in the Data downloads section contain all the tables produced for this release and the calculated RSEs for each of the estimates.

7 All cells in the Excel spreadsheets with RSEs greater than 25% contain a comment indicating the size of the RSE. These cells can be identified by a red indicator in the corner of the cell. The comment appears when the mouse pointer hovers over the cell.

8 Another useful measure is the Margin of Error (MOE), which shows the largest possible difference that could be between the estimate due to sampling error and what would have been produced had all persons been included in the survey with a given level of confidence. It is useful for understanding and comparing the accuracy of proportion estimates. Confidence levels typically used are 90%, 95% and 99%. Where provided, MOEs for estimates in this publication are calculated at the 95% confidence level. At the 95% confidence level the MOE indicates that there are about 19 chances in 20 that the estimate will differ by less than the specified MOE from the population value (the figure obtained if all dwellings had been enumerated). The 95% MOE is obtained by multiplying the SE by 1.96.

9 The 95% MOE can also be calculated from the RSE by:

10 This can easily be converted to a 90% confidence level by multiplying the MOE by:

or to a 99% confidence level by multiplying by a factor of:

11 A confidence interval expresses the sampling error as a range in which the population value is expected to lie at a given level of confidence. The confidence interval can easily be constructed from the MOE of the same level of confidence by taking the estimate plus or minus the MOE of the estimate.

12 Estimates of proportions with an MOE more than 10% are annotated to indicate they are subject to high sample variability and particular consideration should be given to the MOE when using these estimates. Depending on how the estimate is to be used, an MOE greater than 10% may be considered too large to inform decisions. In addition, estimates with a corresponding standard 95% confidence interval that includes 0% or 100% are annotated to indicate they are usually considered unreliable for most purposes. 

13 The Excel spreadsheets in the Data downloads section contain all the tables produced for this release and the calculated MOEs for each of the proportions. All cells in the Excel spreadsheets with MOEs greater than 10% or where the 95% confidence interval includes 0% or 100% contain a comment regarding their reliability. These cells can be identified by a red indicator in the corner of the cell. The comment appears when the mouse pointer hovers over the cell.

14 This publication contains calculations of medians. The median value is the middle value of a set of values when the values are sorted in size order.

15 Standard errors can be calculated from the median and their corresponding RSE using the following formula:

16 The difference between two survey estimates (counts or percentages) can also be calculated from published estimates. Such an estimate is also subject to sampling error. The sampling error of the difference between two estimates depends on their SEs and the relationship (correlation) between them. An approximate SE of the difference between two estimates (x-y) may be calculated by the following formula:

17 While this formula will only be exact for differences between separate and uncorrelated characteristics or sub populations, it provides a good approximation for the differences likely to be of interest in this publication

18 When comparing estimates between surveys or between populations within a survey, it is useful to determine whether apparent differences are 'real' differences between corresponding population characteristics or simply the product of differences between survey samples. One way to examine this is to determine whether the difference between the estimates is statistically significant. A statistical significance test for a comparison between estimates can be performed to determine whether it is likely that there is a difference between the corresponding population characteristics. The SE of the difference between two corresponding estimates (x and y) can be calculated using the formula shown above in the standard error of a difference section (paragraph 16). This SE is then used to calculate the following test statistic:

19 If the value of this test statistic is greater than 1.96 then there is evidence, with a 95% level of confidence, of a statistically significant difference in the two populations with respect to that characteristic. Otherwise, it cannot be stated with confidence that there is a real difference between the populations with respect to that characteristic.