Confidentiality in ABS microdata

The competing obligations of providing useful statistical information while protecting the privacy and secrecy of data providers are key considerations for many National Statistical Organisations (NSOs). Meeting these obligations involves balancing data utility and disclosure risk of data products. However, decisions about access and use of data are subjective and context-dependent. The Australian Bureau of Statistics (ABS) meets these obligations with an approach based on the internationally recognised Five Safes framework. Our approach enables us to make informed decisions guided by continual methodological research and extensive experience. This paper uses ABS cross-sectional microdata as an example to share our practices in using the Five Safes to design different data access solutions for different data products. This paper describes different unit risk measures for microdata. It argues that different unit risk measures are useful tools to assist decisions on releasing datasets. The decisions on releasing datasets should be considered with the context and mode of data access in mind, to ensure that data utility and disclosure risk are appropriately balanced.

Edwin Lu, Rian Jenkins

methodology@abs.gov.au

There is growing recognition that data is a strategic resource, and that realising the full potential of data will unlock insights that benefit Australian society. The ABS enables this by making ABS data as accurate and accessible as possible. Further, we are committed to protecting the privacy and secrecy of data providers. This is to meet our legal obligation under the Census and Statistics Act 1905 and to maintain public trust so that data providers are happy to continue supplying us with quality data.

Providing useful statistical information while protecting the privacy and secrecy of data providers is challenging. The more detailed the statistical information, the greater the risk that the information will disclose private information about a data provider. This is the risk-utility trade-off. To meet our competing obligations, the ABS uses the Five Safes framework to make informed decisions about access and use of data, with the aim of maximising data utility without compromising the privacy and secrecy of data providers (Australian Bureau of Statistics, 2017a). The internationally recognised framework was developed by the United Kingdom Office of National Statistics. Statistics New Zealand, Statistics Canada, the Australian Institute of Health and Welfare, and many other National Statistical Organisations and research organisations also use the framework.

Decisions about data access are context-dependent and need to be made after careful consideration of a number of dimensions. This paper shares the ABS’s practices in using the Five Safes to guide decisions about access and use of microdata, and to optimally balance data utility with disclosure risk. Our decisions are informed by continual methodological research and extensive experience. We illustrate our data confidentiality approach with ABS cross-sectional microdata, beginning with an overview of controls we apply under the Five Safes. We then elaborate on the Safe Data dimension of the Five Safes by discussing our choice of unit risk measures. Finally, we introduce differential privacy and synthetic data methods as potential additions to our suite of methods under the Five Safes.

We structure the paper as follows: Section 2 describes the overarching considerations when enabling access to ABS microdata. Section 3 explains why and how we apply the Five Safes to ABS microdata. Section 4 elaborates on the Safe Data dimension of the Five Safes by describing several unit risk measures and discussing our choice of unit risk measures for ABS microdata. Section 5 describes differential privacy and synthetic data methods as potential additions to our suite of methods under the Five Safes. Section 6 concludes by summarising our key points.

The overarching consideration when enabling access to public sector microdata is providing useful microdata while protecting the privacy and secrecy of data providers. The ABS aims to enhance data utility while ensuring low disclosure risk.

Statistical agencies must protect the privacy and secrecy of data providers and also ensure that microdata is accessible so that it can be used to inform decisions that have significant impact on the public interest (Chien, Welsh, Moore, 2020). The fact that utility and disclosure risk are dependent complicates this. For example, microdata with high utility would be very detailed and therefore potentially have high disclosure risk if not subjected to controls on the user, access environment, etc. Further, reducing disclosure risk through statistical disclosure control (SDC) could also reduce the utility of the microdata. Thus, it is not possible to simultaneously maximise utility and minimise disclosure risk. A data confidentiality approach that focuses solely on mitigating disclosure risk without concern for preserving utility will diminish the value of the microdata to users.

To provide good data utility, we must address the question of what utility means. A simple definition is that utility is the value of a dataset for analytical and research purposes, referring to both the completeness of the dataset and the accuracy of the values within. However, not all variables and values in a microdata set are equally important – it depends on the context. The relative importance of the variables and values in the microdata set ultimately depends on the intended purpose and intended users of the data. For example, research aimed at understanding the relationship between education and employment outcomes would place more weight on preserving the completeness and accuracy of education and employment variables. This causes difficulty in developing a satisfactory measure of utility that is adequate for all contexts.

To ensure low disclosure risk, we must determine what level of risk is acceptable. This inevitably depends on legal and business contexts and is subjective. For example, the legislation might describe requirements in non-quantitative terms which are subject to interpretation. The business environment in which a data custodian operates might influence the risk appetite. These contextual factors vary geographically and differ from one data custodian to another. This causes difficulty in developing a satisfactory measure of disclosure risk that is adequate for all contexts.

Therefore, decisions about access and use of data are context-dependent and need to be made with careful consideration of a number of dimensions, not just based on objective mathematical calculations. Even seemingly objective measures of utility and disclosure risks are based on subjective assumptions and, sometimes, hypothetical risk scenarios (Desai et al., 2016; Halfner et al., 2015; Skinner, 2012). It is partly for this reason that the ABS uses the Five Safes framework. As Desai et al. (2016) point out, the real value of the Five Safes framework is in identifying different solutions based on what matters most in different contexts. It is not designed to deliver a confidentiality approach that fits all contexts.

The ABS uses the Five Safes framework to guide decisions about access and use of data, and to expand the limits of what we can achieve in terms of enhancing utility while ensuring low disclosure risk. The Five Safes framework introduces a holistic approach to managing disclosure risk that incorporates context in the decision-making, especially decisions about what and how much SDC to apply to the data.

Section 3.1 introduces the Five Safes framework and gives two key reasons why the ABS uses it. Section 3.2 describes how the ABS applies the Five Safes framework to ABS microdata.

The ABS views the Five Safes framework as a multi-dimensional approach for managing disclosure risk and for making informed decisions about access and use of data (Australian Bureau of Statistics, 2017a). Table 1 (adapted from Ritchie, 2017) describes the five dimensions of the framework. For an in-depth discussion, we refer readers to Desai et al. (2016) and Ritchie & Green (2020). The ABS uses the Five Safes framework for two key reasons.

Reason 1: to guide decisions regarding data access and use

Decisions about access and use of data are inherently subjective as they depend on the context, including intended purpose of the data, intended users of the data, and the legal and business contexts. Even objective measures of utility and disclosure risks are based on subjective assumptions and, sometimes, hypothetical risk scenarios (Desai et al., 2016; Hafner et al., 2015; Skinner, 2012). Relying solely on measures that are based on arbitrary statistical models leads to spurious objectivity (Hafner et al., 2015). Recognising the limitations of a data-only approach to data confidentiality, which relies only on utility and risk measures and SDC, was the original motivation for the ABS to use the Five Safes framework.

The Five Safes framework explicitly acknowledges that decisions about access and use of data are subjective (Ritchie & Green, 2020). With additional dimensions that reflect managerial controls (see Table 1), the framework ensures that contextual considerations are front of mind when making decisions. The framework does not prescribe a method or a solution. Rather, it allows flexibility for users of the framework to choose the types and levels of control to apply in each dimension, and they can be tailored to the particular context of a data access activity. The controls under the five dimensions are jointly evaluated to decide whether a particular data access activity is acceptable and, if not, what controls should be strengthened or introduced to enable the activity. While decisions are ultimately subjective, the ABS makes informed decisions based on continual methodological research and extensive experience.

Reason 2: to expand the limits of what is achievable in terms of risk-utility trade-off

The Five Safes framework expands the limits of what we can achieve in terms of enhancing utility while ensuring low disclosure risk. This allows us to produce data products with higher utility than is possible with a data-only approach which relies only on SDC. To explain this, we contrast a data-only approach against a Five Safes approach using a conceptual risk-utility (R-U) map (Figures 1 and 2). It is ‘conceptual’ because although its axes are ‘disclosure risk’ and ‘utility’, our view is that there is no definitive measure of disclosure risk or utility. The axes are presented to illustrate a concept.

Consider the case where we use a data-only approach. A simple way to eliminate all disclosure risk from a data product would be to suppress every unit so that we have an empty dataset. However, such a product would have no utility. Conversely, to produce a detailed dataset with no protections would mean high utility, but also unconscionably high disclosure risk. There is a trade-off between these two extremes: greater level of SDC tends to reduce both disclosure risk and utility. Figure 1 depicts the trade-off in the form of a R-U trade-off frontier, which is a production possibilities frontier in economic terms. Each data product corresponds to a point in the map, either on or to the left of the frontier. The frontier represents the theoretical limits of what is achievable with a data-only approach, in the following sense: even if we exhaustively try all methods of SDC available with our current knowledge and technology, we are unable to produce a product to the right of the frontier. If we can specify a maximum acceptable disclosure risk, then a sensible choice is to produce products on the frontier at that level of risk.

Now consider the case where we use the Five Safes approach. Managerial controls from Safe Projects, Safe People and Safe Settings reduce disclosure risk without reducing the utility of the data that users can access, although there are costs from reduced accessibility, fixed set up costs for restricted facilities, and ongoing costs from people, project and output assessment (Desai et al., 2016). In Figure 2, managerial controls shift a data product vertically downwards (e.g. shifts point A down), resulting in a product with lower disclosure risk but with utility unchanged. These controls have a similar downward-shifting effect on any product in the R-U map. This ‘expansion’ of the frontier allows us to produce products in the desirable lower right region of the map, which is unreachable with the data-only approach. Consequently, we can produce products with higher utility using the Five Safes approach.

Figure 1. The R-U trade-off frontier represents the theoretical limits of what is achievable, given all methods of SDC currently available. Data products correspond to points on or to the left of the frontier. If a maximum acceptable disclosure risk can be specified, we aim to create products on the frontier at that level of risk to enhance utility.

Figure 2. The Five Safes expands the R-U trade-off frontier. For each level of utility, we can achieve lower disclosure risk. E.g. we cannot create product A with the data-only approach as it is above the maximum acceptable disclosure risk, but we can create it if we apply managerial controls under the Five Safes approach.

ABS microdata products include basic and detailed microdata, in which each row corresponds to a population unit and each column corresponds to a variable (categorical or numeric). Microdata products may contain data for all units in a population, or only for units that are selected in a survey. Basic microdata is for simple modelling and analysis, and is accessible by a wide range of users. Detailed microdata is for complex modelling and analysis, and is aimed at facilitating research that cannot be done using basic microdata alone. Detailed microdata offers higher utility to safe users for vetted research projects but may have higher disclosure risk due to the level of detail in the data.

Detailed microdata and basic microdata differ mainly in the level of detail in the data and are consequently subject to different controls on users, projects, outputs and access environment. Table 2 describes controls placed on ABS microdata products through the Five Safes framework (Australian Bureau of Statistics, 2017b, 2019).

The Five Safes framework allows the ABS to balance disclosure controls across five dimensions, rather than only focusing on making data safe. Designing controls based on context allows the ABS to offer detailed microdata with high utility to safe users for vetted research projects, while still providing basic microdata with good utility for more general users.

Under the Safe Data dimension of the Five Safes, we apply unit risk measures to assist in managing disclosure risks. Unit risk measures measure the disclosure risk of records or units in a microdata set. However, we emphasise that all risk measures are based on subjective assumptions and hypothetical intruder scenarios, even if they are based on objective mathematical calculations. Thus, it is important to avoid using unit risk measures as the sole driver of all decisions about access and use of data. Further, we choose unit risk measures and apply SDC with controls under Safe Project, Safe People and Safe Settings in mind. For example, if we plan to enable access to a detailed microdata set through the DataLab, which is subject to strong managerial controls (see Table 2), we choose unit risk measures that are appropriate for that context and apply less SDC compared to basic microdata sets, which are subject to more relaxed managerial controls (see Table 2). This ensures we apply appropriate SDC to preserve utility while reducing residual disclosure risk that has not been mitigated by the other three Safes.

In this section, we focus on unit risk measures for assessing re-identification risk in cross-sectional microdata, and on measuring rather than treating risks. These unit risk measures inform the SDC methods described in the last row of Table 2. The risk measures in this section are chosen to illustrate the thinking used by the ABS to choose the measures we currently use. We are not attempting to give an overview of all risk measures in the literature, nor are we attempting to formally describe the risk measures. For the sake of clarity and brevity, we may use terminology that is different from the terminology in other research papers.

Section 4.1 describes some unit risk measures for cross-sectional microdata in both plain and mathematical language, with an example for illustration. Section 4.2 compares the risk measures against a set of criteria that is important to the ABS and explains our choice of risk measures for ABS microdata.

ABS microdata products are subject to controls under the Five Safes framework to manage disclosure risk (see section 3). This includes controls placed on users, the access environment, the data, the projects for which the data are used, and outputs produced from the projects. These correspond to the five dimensions of the framework. Controls under Safe Data and Safe Outputs are applied last and are aimed at reducing residual disclosure risk that has not been mitigated by the other three Safes.

We carefully design controls under Safe Data (i.e. unit risk measures, SDC) with managerial controls under the Five Safes in mind, to avoid reducing the utility of the data more than is necessary. Unit risk measures should aim to detect units in the microdata set with potentially high disclosure risk. We manually review those units to decide which units have high risk and require SDC. The review process requires unit risk measures to be easy to interpret. This motivates a set of criteria for assessing the usefulness of unit risk measures for ABS microdata. Table 6 compares the unit risk measures in section 4.1 against this set of criteria.

Cross-tabulation analysis, SUDA and the NN algorithm are strictly better than the PIF method when assessed using the set of criteria in Table 6. The three methods each have their relative advantages which make them useful for assessing disclosure risk in ABS microdata. The ABS considers these three methods as part of a suite of methods under Safe Data for managing residual disclosure risk.

Detailed microdata products which are accessible only through the DataLab are subject to strong managerial controls and strong controls under Safe Outputs (see Table 2). These controls significantly reduce a data user’s ability and likelihood to match detailed microdata with external data to re-identify records. This means DataLab users can analyse detailed microdata with high utility for complex modelling and analysis. In contrast, basic microdata products are also subject to strong managerial controls but these are more relaxed than those for detailed microdata (see Table 2). Thus, stronger controls are needed under Safe Data for basic microdata than for detailed microdata. We might apply multiple unit risk measures and more SDC to a basic microdata set than to a detailed microdata set (see Table 2). This allows users of basic microdata to perform simple modelling and analysis. The ABS tailors controls under the Five Safes to provide different data access solutions.

Many approaches to maintaining data confidentiality can be viewed through the lens of the Five Safes framework. The ABS is currently exploring differential privacy methods and synthetic data methods as potential additions to our suite of methods under Safe Data and Safe Outputs dimensions of the Five Safes.

Differential privacy aims to ensure that the presence or absence of an individual record in the microdata does not significantly affect statistical outputs produced from the microdata (Dwork & Roth, 2014). Since statistical outputs are insensitive to the presence or absence of individual records, differential privacy limits how much information data users can learn from the statistical outputs about any individual record. Variations of the concept of differential privacy exist with similar aims. In some researchers’ views (e.g. Culnane, Rubinstein, Watts, 2020), differential privacy removes the need for the Five Safes. However, the value of differential privacy relies on implicit assumptions about the data generation process (Kifer & Machanavajjhala, 2011). In light of this, differential privacy can benefit from additional controls applied to data users, to the projects the data is used for and to the data access environment. We view differential privacy methods as part of a suite of methods we can apply under Safe Data and Safe Outputs.

We are exploring one such method (Australian Bureau of Statistics, 2020) that is based on the Pufferfish framework for differential privacy (Kifer & Machanavajjhala, 2014). Differential privacy methods need to be applied with care because they implicitly assume some form of worst-case intruder scenario, which could result in protections that destroy or significantly reduce data utility (Bambauer, Muralidhar, Sarathy, 2013). In contexts where such scenarios are very unlikely, for example, due to other controls under the Five Safes, we currently do not recommend differential privacy methods.

Synthetic data methods aim to create synthetic versions of a microdata set that preserve important aggregate level information about the data. While doing so, they reduce disclosure risk by replacing real records in the data with synthetic ones. Researchers can use synthetic microdata for their research without ever accessing the real microdata. We view such methods as potential additions to our suite of methods under Safe Data and plan to explore them more in the near future.

The goal of enhancing data utility while protecting privacy and secrecy of data providers cannot be achieved by any attempts at developing a prescriptive standardised data confidentiality approach. While the pursuit of quantitative measures for disclosure risk assessment is useful, decisions about data access and use are inherently subjective and necessarily context-dependent. The ABS enhances utility by using the Five Safes framework to design different data access solutions for different data products. Data-only methods are just one of five dimensions. Our approach considers context when making decisions about access and use of data, and allows us to expand the limits of what we can achieve in terms of enhancing utility while ensuring low disclosure risk. Continual methodological research and extensive experience ensures that our decisions, while subjective, are well-informed. We also continue to research new methods such as differential privacy and synthetic data to incorporate into the Five Safes. Through diligent development and implementation of our approach, the ABS provides access to useful data, with confidence that the privacy and secrecy of data providers is protected.

We thank our colleagues at the ABS for providing input to this paper, including colleagues in the following sections: Data Access and Confidentiality Methodology Unit, Microdata Resources, National Data Commissioner Support, Data Services, Data Integration Delivery Assurance and Strategy, Census Futures, Strategic Partnership Managers, Strategic Communications, and Census Futures. In particular, we thank Michelle Gifford (ABS) and Lachlan MacRae (ABS) for formatting this paper for publishing, and our director, Joseph Chien (ABS), for reviewing this paper many times throughout the multiple rounds of edits. We also thank Felix Ritchie from UWE Bristol for providing helpful comments.