Australian Bureau of Statistics
4228.0.30.001 - Microdata: Programme for the International Assessment of Adult Competencies, Australia, 2011-2012 Quality Declaration
Latest ISSUE Released at 11:30 AM (CANBERRA TIME) 15/02/2013
|Page tools: Print Page Print All RSS Search this Product|
Replicate weights enable variances of estimates to be calculated relatively simply. They also enable unit record analyses such as chi-square and logistic regression to be conducted which take into account the sample design. Replicate weights for any variable of interest can be calculated from the 60 replicate groups, giving 60 replicate estimates. The distribution of this set of replicate estimates, in conjunction with the full sample estimate (based on the general weight) is then used to approximate the variance of the full sample.
To obtain the standard error (SE) of a weighted estimate y, the same estimate is calculated using each of the 60 replicate weights. The variability between these replicate estimates (denoted y(g) for group number g) is used to measure the standard error of the original weighted estimate y using the formula:
Research Paper: Weighting and Standard Error Estimation for ABS Household Surveys (Methodology Advisory Committee), July 1999 (cat. no. 1352.0.55.029).
Use of the 60 group Jack-knife method for complex estimates, such as regression parameters from a statistical model, is not straightforward and may not be appropriate. The method as described does not apply to investigations where survey weights are not used, such as in unweighted statistical modelling.
STANDARD ERRORS FOR ESTIMATES WITH PLAUSIBLE VALUES
In order to minimise respondent burden, the three skill domains of literacy, numeracy and problem solving in technology rich environments were not directly assessed for each respondent. PIAAC used a matrix-sampling design to assign the assessment exercises to individuals so that a comprehensive picture of achievements in each skill domain across the country could be assembled from the components completed by each individual. PIAAC relied on Item Response Theory scaling to combine the individual responses to provide accurate estimates of achievement in the population. With this approach, however, aggregations of individuals values can lead to biased estimates of population characteristics. To address this, the PIAAC scaling procedures also used a multiple imputation or "plausible values" methodology to obtain data for all individuals, even though each individual responded to only a part of the assessment item pool. By using all available data, ten "plausible values" were generated for each respondent for each of the three domains of literacy, numeracy and problem solving in technology rich environments.
For each domain, proficiency is measured on a scale ranging from 0 to 500 points. Each person's score denotes a point at which they have a 67 per cent chance of successfully completing tasks with a similar level of difficulty. To facilitate analysis, these continuous values have been grouped into 5 skill levels for Literacy and Numeracy with Level 1 being the lowest measured level. The levels indicate specific sets of abilities, and therefore, the thresholds for the levels are not equidistant. As a result, the ranges of values in each level are not identical. The relatively small proportions of respondents who actually reached Level 5 often resulted in unreliable estimates of the number of people at this level. For this reason, whenever results are presented in the main report by proficiency level, Levels 4 and 5 are combined. Further information about the Plausible Values and definitions of the three domains can be found in Scores and skill levels of the publication Programme for the International Assessment of Adult Competencies (PIAAC), 2011-2012 (cat. no. 4228.0).
Each record contains the ten plausible values for each of the three domains.
For simple point estimates in any of the domains, it is sufficient to use one of the corresponding ten plausible values (e.g. PVLIT1 for the literacy domain), chosen at random to derive population estimates. If this method is chosen the standard error of the plausible score can be calculated using the formula for standard error for estimates without plausible values, as shown above.
Furthermore, when producing estimates by other variables available on the file the process must be performed for each skill level by each category of the by variable(s). For example in order to report an estimate of the total number of males at Level 1 for literacy, first calculate the weighted estimate of the number of males at Level 1 for each of the ten plausible values for literacy (PVLIT1-PVLIT10) individually. Next sum the ten weighted estimates obtained. Then divide the result by ten to obtain the estimate of the total number of males at Level 1 for literacy. The process must then be repeated for each skill level.
All values presented in the publication Programme for the International Assessment of Adult Competencies (PIAAC), 2011-2012 (cat. no. 4228.0) are obtained by using all ten plausible values in combination, as described above.
Due to the use of multiple possible exercises and the application of plausible scoring methodology, the PIAAC plausible values also include significant imputation variability. The effect of the plausible values methodology on the estimation can be reliably estimated and is included in the calculated SEs. An accepted procedure for estimating the imputation variance using plausible values is to measure the variance of the plausible values (with an appropriate scaling factor) as follows:
The total SE can be then obtained as the square root of the total variance. This SE indicates the extent to which the estimate might have varied by chance because only a sample of persons was included, and because of the significant imputation used in the literacy scaling procedures.
The total Relative Standard Error (RSE), can then be obtained by expressing the total SE as a percentage of the estimate to which it relates:
NOT APPLICABLE CATEGORIES
Some data items included in the microdata include a 'Not applicable' category. The classification value of the 'Not applicable' category, where relevant, is shown in the data item list in the Downloads tab. In order to comply with the scheme used by other countries participating in PIAAC, the following classification scheme was used to describe 'Not applicable' categories:
The population relevant to each data item is identified in the data item list and should be borne in mind when extracting and analysing data from the CURF. The actual population count for each data item is equal to the total cumulative frequency minus the 'Valid skip' category.
Generally all populations, including very specific populations, can be 'filtered' using other relevant data items. For example, if the population of interest is 'Employed persons' the Labour Force status data item CD05 can be used by applying the filter CD05 = 1.
These documents will be presented in a new window.
This page last updated 15 February 2013