• Summary
• Downloads
• Explanatory Notes
• Related Information
• Past Releases

Measuring the Impact of Webforms in the Survey of Employee Earnings and Hours

The ABS is currently in the process of introducing an internet-based mode of collection (web forms) for its business and household surveys. Web forms were introduced for the May 2012 Employee Earnings and Hours (EEH) survey. EEH is conducted biennially and provides statistics on the composition and distribution of earnings of employees, the hours they are paid for and the methods used to set their pay (ie awards, collective agreements and individual arrangements).

An "opt-out" approach was taken for the introduction of the EEH web form, whereby businesses selected in the survey were initially given a link to an online web form. A paper or spreadsheet form was only sent out if a selected business subsequently requested one. This approach proved very successful, with about 90% of respondents providing data using the web form. The relatively small amount of data obtained directly from paper and spreadsheet forms limited our ability to detect a systematic impact on EEH responses due to the introduction of web forms. Nevertheless, an analysis was undertaken to provide reassurance that an obvious web form effect did not impact on EEH estimates.

The analysis comprised four parts:

1. Exploratory data analysis - Using the 2012 EEH data, a scatter plot matrix of the continuous variables of interest was produced. For each scatter plot, the distributions of the web and non-web responses were compared.

2. Comparison of EEH responses with other data - This involved comparing EEH web and non-web responses at the employer level with corresponding data provided by the same employer in other data sources. The values of common variables of interest for these units were compared using scatter plots, to examine if the distributions for the web and non-web responses differed significantly. A more formal analysis was then conducted using linear regression analysis.

3. Modelling earnings and number of employees - Data from the May 2010 and May 2012 Average Weekly Earnings (AWE) surveys (both paper form surveys) was used to estimate how units common to both the 2010 EEH and 2012 EEH surveys would have responded if these units were provided with a paper form in 2012. The relationship between the modelled and actual 2012 EEH values for the web and non-web businesses was then compared.

4. Propensity score sub-classification - A logistic regression model was created to estimate the probability that each business in the EEH sample would respond via a web form. The sample was then grouped into five categories based on these estimated probabilities. A web form impact was estimated within each category separately, and these were then combined to form an overall estimated impact.

Our analysis provided reassuring evidence that there were no systematic web form impacts for the 2012 EEH. The opt-out approach will continue to be the main method adopted by the ABS for migrating the rest of its business surveys to web forms. For surveys moving to web forms, conducting these analyses at the same time as the editing stage should help processing staff to identify particular responses that may have been impacted by the introduction of a web form. Further editing effort may then be directed towards those responses to determine whether a change is required.

Further Information
For more information, please contact Melanie Black (02 6252 7241, melanie.black@abs.gov.au) or Lyndon Ang (02 6252 5279, lyndon.ang@abs.gov.au)