Statistical Business Transformation Program achievements

The way the ABS collects, analyses and shares data is being modernised. Through the Statistical Business Transformation (SBT) Program a range of initiatives are being undertaken to digitally streamline processes, to deliver benefits and reduce statistical risk.

Improving the way we collect data

Our new data acquisition capabilities are delivering efficiencies in the way we collect data. Our use of innovative household approaches, in addition to increasing self-enumeration rates, is improving user experiences for ABS data providers and survey respondents.

This year a Data Acquisition Taskforce was established to drive this transformative work. Milestones achieved include:

• migration of a range of administrative data sourced from Commonwealth, state and territory government agencies to enhanced acquisition capabilities
• increasing the availability of online data collection. For the first time participants in ABS Special Social Surveys such as the Survey of Income and Housing (SIH) and the General Social Survey (GSS) were offered the option to complete their survey online. Across multiple collection campaigns we saw between 20 and 35 per cent of respondents take advantage of this new option.
• migration of selected annual and sub-annual (more than once a year) business surveys to new collection infrastructure.

Increased use of enabling infrastructure

Our use of the Next Generation Infrastructure platform increased during the year. A range of new and existing ABS information and communication (ICT) systems were migrated to take advantage of the increased automation and scalability of this new environment. As part of these migrations enhanced testing and deployment practices were introduced to support the increased use of agile development approaches across the ABS.

We delivered enterprise security and access management solutions to underpin technical capabilities introduced by the SBT Program. These capabilities deliver efficiencies in the management of data access through increased standardisation and automation.

The ABS Intelligent Coder is ready for use; it will improve the efficiency and accuracy of statistical coding and classification of data, through the use of machine learning capabilities.

Improving how we share data

Application Programming Interface

This year we introduced the ABS Indicator API. This new data sharing services releases data faster. Now, we can digitally provide key economic data to approved media partners, split seconds after the 11.30.00am embargo, enabling them to programmatically access selected market moving indicators instantly.

The API technology that underpins this new service will be extended in 2019-20, enabling other ABS customers to access our wide range of ABS data releases instantaneously.

ABS DataLab

Use of ABS DataLab continued to grow this year. We saw an increase in the number of approved researchers, as well as an increase in the range of data sets available through the DataLab.

New ABS website progresses

To enhance reliability and responsiveness, our existing ABS website moved to a new infrastructure platform. This enables faster publishing of ABS statistical releases and provides increased capacity in response to the increased usage of the website experienced over the year.

We also released ABS Beta, www.beta.abs.gov.au, to ABS customers and stakeholders for their feedback. ABS Beta enhances the accessibility of ABS content and showcases new content guidelines as well as publishing tools and standards.

Over the coming year and into the future

Work resulting from the SBT Program will continue to migrate into our business as usual processes for data acquisition and information dissemination. Adoption of these capabilities enhances the way we produce high quality data and deliver solutions, and enables us to inform Australia’s important decisions into the future.

Figure 3.1: SBTP Timeline and Achievements March 2016 - June 2019

2016

2017

2018

2019

March 2016 Refreshed ABS website released. May 2016 Updated ABS Stats mobile app released. December 2016 ABS DataLab introduced. DataLab is now widely used to securely provide access to confidentialised ABS microdata across government.

June 2017 Next Generation Infrastructure (NGI) delivered allowing efficient deployment and management of ICT processing, storage and applications. Successful trial of new ABS Indicator API data services with Commonwealth Treasury and other key stakeholders. September 2017 Enterprise Data Management Environment (EDME) and Metadata Registry and Repository (MRR) delivered. October 2017 Statistical Workflow Management System and Statisticians Workbench delivered, providing centralised access and management of statistical workflows and processing tools.

January 2018 Health Literacy Survey (HLS) becomes first collection to use the new data acquisition solution, creating a more integrated and robust process for collecting statistical data. February 2018 Security Identity and Access Management (SIAM) capabilities delivered. Household Survey Innovation panel established. This panel allows trials of new collection approaches to be undertaken using a representative sample of households, providing insights into the impact of changes in approach. May 2018 New data acquisition capabilities to collect Special Social Survey (SSS) data. For the first time respondents in SSS collections have the option to respond online. Take-up rates exceed 25% with successful follow-up undertaken by ABS field interviewers.

June 2018 Successful trial of ABS Indicator API data services undertaken with financial media outlets. This new technology allows for the instantaneous delivery of ABS statistics. August 2018 Constrained optimisation introduced for supply use tables in National Accounts. This reduces the effort required to balance supply use tables and reduces the statistical risk associated with this function. September 2018 SIGBOX, a secure dropbox, is delivered providing a revised and more secure approach to management of pre-embargo briefings with Commonwealth agencies. Updated provider materials for Monthly Population Survey trialled with Survey innovation panel. Increases take-up rate of web collection from 18% to 25%.

September 2018 Refined approaches to sample design and intensive follow-up agreed. This change will reduce collection effort and costs with minimal impact on statistical quality. November 2018 Production release of ABS Indicator API data services to approved media partners. Delivers market sensitive indicators to partners immediately upon lifting of embargo. ABS Intelligent Coder delivered. Intelligent Coder uses machine learning to improve the accuracy and efficiency of statistical coding. December 2018 New ABS Provider Portal released, allowing providers to see all ABS obligations in a single place. Beta of ABS website released for external feedback. Beta of new Internal Publishing Environment released for user Testing.

January 2019 First use of Data Acquisition to capture business data. Business Expenditure on Research and Development survey dispatched with enhanced collection tool utilising in-form edits. First use of Data Acquisition to capture administrative data. Data transfer from ATO uses new secure data transfer facilities. February 2019 Collections from Queensland Registry and Department of Home Affairs on boarded to new data infrastructure. March 2019 Trial Process and Analyse tools supported pre-processing of GSS Dress Rehearsal responses including statistical coding of occupation, industry and education data. ABS mobile app updated to reflect new Commonwealth electoral boundaries.

April 2019 All ABS market sensitive statistics migrated to Indicator API, allowing the instant delivery of ABS statistics. May 2019 Release of the HLS publication, the first collection to use new Data Acquisition capabilities. Data made available in TableBuilder and ABS DataLab, providing real time access for users. The GSS achieves online completion rate of over 30% in the first production use of online collection for ABS SSS. ABS DataLab further utilised to support inclusion of 2015-16 Agriculture Census data integrated into the Business Longitudinal Analysis Data Environment (BLADE). Job Vacancy Survey (JVS) is the first sub-annual business survey to use new data acquisition capabilities. Published data maintains its quality.

June 2019 New ABS Internal Publishing Environment (IPE) goes live and migration of existing web content begins.