ABS use of web scraping

Web scraping is the process of using software to extract publicly available information from websites using an automated data collection process. This page describes how the ABS uses web scraping to extract publicly available data from websites.

The ABS uses web scraping to extract pricing data, product and other information from specific websites for statistical purposes consistent with the Census and Statistics Act 1905. 

Web scraping enables the ABS to efficiently and effectively collect data and increases the quantity and accuracy of data collected. 

To support the web scraping process the ABS will: 

• contact data providers to advise them that the ABS intends to undertake web scraping of their websites
• make reasonable efforts to avoid impeding the performance of the providers’ website.

Many businesses provide significant amounts of data to the ABS. By using a variety of indirect data collection methods, including web scraping, the ABS minimises provider burden. Web scraping enables accurate data to be collected frequently, reducing the need for manual and direct data collection.

Personal information is not collected through the web scraping process.

The ABS and its staff are legally bound to protect your data. It is an offence to reveal any information collected under the Census and Statistics Act 1905. Breaching this requirement can result in heavy penalties, including significant fines and imprisonment. 

The ABS complies with the Australian Privacy Principles and all relevant federal legislation including the: 

• Privacy Act 1988
• Australian Bureau of Statistics Act 1975
• Archives Act 1983

For further details on how the ABS uses web scraping see Web Scraping in the Australian CPI 

If you have any queries about web scraping or would like more information on anything covered in this page please contact the ABS: 

• Phone: 1800 179 691 during business hours
• Online: Business Surveys Contact Form