DataLab

DataLab is the analysis solution for high-end users who want to undertake real time complex analysis of detailed microdata. Compare data services to see if detailed microdata in the DataLab is the right service for you.

Note: Government organisations seeking to leverage the DataLab cloud infrastructure to host their own data and manage their own end-users should consider the Secure Environment for Analysing Data (SEAD) service. 

• View and analyse unit record information
• Recent versions of analytical software, including R, SAS, Stata and Python
• Virtual access to files that remain in the secure ABS environment
• All analytical output that you want to use outside DataLab are checked by the ABS before release

Detailed survey and integrated microdata are available for approved projects, organisational users must be:

• government employees
• government contractors and individuals sponsored by government
• academics
• researchers from public policy research institutes
• sponsored by government

All users need to also meet ABS safe people criteria, including researchers who:

• belong to an Australian organisation (international researchers and organisations will be considered on a case by case basis)
• belong to an organisation with a Responsible Officer Undertaking (ROU) in place with the ABS
• are located in Australia when accessing the microdata
• have completed all relevant undertakings and declarations
• have the ability to use at least one of the statistical analytical languages available in the DataLab
• have at least three years of either quantitative research experience or university study with a significant component working with quantitative data, or have a referral from an experienced researcher working on the same project
• have an approved safe project that is for statistical and/or research purposes and demonstrates public value
• have completed ABS safe researcher training and refresher training as per ABS refresher policy
• meet additional criteria that apply to specific microdata.

The ABS is committed to keeping the ABS DataLab safe and secure. We have a strong data protection culture and extensive experience in keeping data secure as Australia’s national statistical organisation and as an Accredited Data Service Provider. The ABS DataLab is hosted in Microsoft Azure and meets PROTECTED level security standards as prescribed in the Australian Government Information Security Manual (ISM). It is subject to Independent Security Registered Assessors Program (IRAP) certification, ongoing security audits and robust IT security testing and patching delivering the Safe Settings aspect of the Five Safes Framework.

The technology underpinning the ABS DataLab includes:

• data encryption at rest to mitigate against unauthorised access to microdata
• Azure Storage Accounts to securely hold individual research products and allow querying from authorised users
• cloud servers (including backup servers) hosted exclusively onshore, with access only authorised for use in Australia unless approved by the ABS
• closed network virtual machines to provide secure, isolated research spaces for the analysis of microdata
• guarded access through multi-factor authentication and workspace segmentation inhibiting data sharing between projects
• a DataLab Product Storage Account protected with Microsoft Defender providing threat detection against malicious/unusual behaviour.

The ABS employs the above with a focus on industry standard security posture management to provide a safe and secure platform for policy and program delivery work.

• Designed specifically for use within the DataLab environment
• Direct identifiers (such as names and addresses) removed
• Further appropriate confidentiality applied within the context of the other security features of the DataLab
• Topics include Census, health, education, labour force, Aboriginal and Torres Strait Islander peoples, migrants, crime, business, disabilities, ageing and carers
• Datasets include ABS survey results, administrative data collected by other organisations and integrated datasets
• Data item lists are linked in detailed microdata topics in the DataLab

Approved users can access standard detailed microdata in DataLab for approved projects. This includes:

• ABS survey and census collections
• data ABS has collected from other organisations (with custodian approval)
• integrated microdata such as:
  • Person Level Integrated Data Asset (PLIDA)
  • Business Longitudinal Analysis Data Environment (BLADE) Core plus BLADE standard module (various ABS surveys), Intellectual Property Longitudinal Research Data (IPLORD) and Merchandise Imports and Exports
  • PLIDA/BLADE linked data

Additional charges apply for customised data integration services.  

DataLab access incurs an annual charge. This charge is based on the number of analysts with virtual machine access in a project. The charge covers the annual costs of:  

• project establishment and ongoing administration and support 
• researcher onboarding (including training and refresher training) 
• changes to analysts and discussants within a project 
• standard virtual machine access for analysts 
• standard software access for analysts (e.g., R, Python, Stata) 
• project storage up to one terabyte 
• standard output and input clearance 

The ABS is committed to supporting the DataLab service and subsidising DataLab users. In 2023-24, increasing costs and budget constraints require the ABS to move towards a more sustainable partial cost recovery arrangement with all our clients. The below updated charges will allow the ABS to maintain its service levels as well as deliver critical system and infrastructure enhancements.   

The ABS is adopting a staggered approach to price increases to minimise the impacts on existing projects. Existing projects are projects established before 1 July 2023. Existing and new projects will be subject to different pricing models for 2023-24 and 2024-25. Please see the below for more details.  

Please note: 

• Individual quotes will be prepared for projects with over 25 analysts. 
• An increase in the number of analysts to the next tier will incur an additional charge equivalent to the next tier.
• Significant changes in project scope may result in the establishment of a new project.
• Project extensions after 1 July 2023 will be charged at the new project rates.
• Access to non-standard services are subject to additional charges, please see ‘Additional non-standard access and services charges’ table below.
• All charges are calculated quarterly, based on the month of request. For example, a project commencing in October will be charged the annual fee and any non-standard DataLab access based on three quarters of the financial year.
• Goods and Services Tax (GST) will be applied to charges unless the organisation is exempt.

Existing projects are projects established before 1 July 2023.  

Tier 1 and 2 projects (projects with under 10 analysts) 

• Tier 1 and 2 projects will continue to be charged the same annual rates for financial years 2023-24 and 2024-25.
• Prices will increase from 2025-26 for all Tier 1 and 2 projects.

Tier 3 projects (projects with 11 to 25 analysts) 

• Tier 3 projects will be charged half the 2022-23 annual charge of $10,000 ($5,000 excluding GST) for 1 July 2023 to 31 December 2023, with prices increasing from 1 January 2024.
• From 1 January 2024, Tier 3 projects will be subject to a new charge rate. 
• Tier 3 projects will be charged half the new 2023-24 annual charge of $30,000 ($15,000 excluding GST) for 1 January 2024 to 30 June 2024. 
• The delayed roll out of the new charging model provides project leads the opportunity to decrease usage or close their project before prices increase. 

Tier 4 projects (projects with over 25 analysts) 

• Tier 4 projects will be charged custom pricing from 1 July 2023 under the new pricing model.

New projects are projects established after 1 July 2023.  

For projects and data services covered by the current ABS/Universities Australia Agreement, students, staff and researchers are not individually invoiced for their use of the charged ABS data services. 

For more information refer to the ABS/Universities Australia Agreement.

New projects may progress their access requests outside of the ABS/Universities Australia Agreement, and will be subject to access fees and charges as outlined on the DataLab page.

ABS International Merchandise Trade (IMT) statistics are subject to additional disclosure controls and consistency checks with published ABS data when releasing IMT data from the DataLab. These additional activities incur additional costs. From 2024-25 projects requiring access to IMT data are subject to the following charges:

• $500 (GST exclusive) – per project requesting access to merchandise imports or exports data.
• $250 (GST exclusive) – per IMT output request submitted.

Non-standard charges apply to all projects for services that are not within the scope of the annual charge. Non-standard charges include access to SAS, Databricks, non-standard virtual machines and storage above one terabyte. Non-standard charges also apply to increased service levels, such as priority clearance and high service level. More information on non-standard access and services are in the tables below. 

For criteria, refer to Who can access the DataLab

Organisation approval

• Your organisation must have a verified Responsible Officer Undertaking (ROU) in place with the ABS. If one does not exist, your organisation will not be available for selection in the myDATA online project proposal.
• To check if there is an active ROU for your organisation, go to the myDATA user portal homepage and select Dashboard/Organisation.

Researcher approval

• You must have a commitment to protect the confidentiality of data.
• Every member of your project team who will see or discuss uncleared outputs (whether or not they will be using DataLab) needs to be approved.

Project approval

• Projects must be for statistical and/or research purposes and provide public benefit.
• Projects must not be for compliance or regulatory purposes.
• Every project needs to be approved by the ABS.
• Projects for or about Aboriginal and/or Torres Strait Islander peoples may be subject to a Cultural Review by the Centre of Aboriginal and Torres Strait Islander Statistics at the ABS.
• Some projects also require consideration and approval by data custodians.

Refer to What is DataLab and Using DataLab responsibly for more information.

Registration of an account will allow you to:

• create a project proposal within the myDATA user portal
• enrol in DataLab safe researcher training
• collect forms for your onboarding process after training has been completed
• draft and review projects you are participating in.

Register in the myDATA Portal and agree to the Conditions of use

• Use your organisation email address. If you are a user in more than one organisation, you will need to register separately using the email address for each organisation.
• Authenticate your account - myDATA will automatically email your registered account with steps to authenticate.
• If you encounter errors in myDATA, please submit a System support query.

You can complete your training while your project proposal is underway.

Access will not be granted until DataLab safe onboarding is completed. For the onboarding process, the following documents are required to be submitted:

• Individual Undertaking and DataLab Declaration of Compliance
• DataLab quiz
• additional documents relevant to particular datasets, if requested by ABS
• if you are not an experienced researcher, you may need to submit a DataLab Referral, please email data.services@abs.gov.au to request this form.

For further information, please refer to the myDATA user guide.

DataLab is enabled by cloud infrastructure, which may be blocked by some organisations’ firewall settings. 

ABS cannot make changes to external organisations' infrastructure. Project Leads need to supply the information below to each organisation participating on this project. 

Network/IT Security sections in each organisation need to review and make changes to authenticate access. This only needs to be done once per organisation.

1. Enable authentication to the tenant

Users need to authenticate to one of ABS Azure Active tenants, which may be strictly controlled by government agencies and academic workplaces. Authentication must be enabled to the tenant:

• absmydata.onmicrosoft.com

This tenant is in the Azure Australia East and Azure Australia Central regions. 

2. Allow user access to URLs

Users will need to access the following URLs:

• DataLab production portal: datalab.abs.gov.au and sead.abs.gov.au

3. Configure your organisation's network to allow outbound connections to the following addresses required for Azure Virtual Desktop (AVD):

• login.microsoftonline.com 
• *.wvd.microsoft.com 
• *.servicebus.windows.net 
• go.microsoft.com 
• aka.ms 
• learn.microsoft.com 
• privacy.microsoft.com 
• query.prod.cms.rt.microsoft.com 

These addresses all utilise the TCP protocol and outbound port 443 for communication. 

Contact data.services@abs.gov.au for further assistance.

The ABS is actively working to comply with Web Content Accessibility Guidelines (WCAG) 2.1 at Level AA, which describe how to make web content more usable and accessible for everyone. Our goal is to provide information that is accessible to all, including people with disabilities.

Recent accessibility reviews confirm that the DataLab/SEAD cloud platform meets the necessary organisational standards as it:

• runs on Windows 11, which enables the utilisation of existing accessibility features and functions
• utilises virtual machines that are subject to Microsoft compliance obligations
• contains statistical tools that are either WCAG compliant or provide integration with Windows Accessibility features

Note: Any Windows accessibility features not enabled by default can be requested, and will be considered in line with relevant data sensitivities.

The ABS privacy policy and DataLab privacy notice outline how the ABS handles any personal information that you provide to us.