Can I firstly thank IPAA for taking the initiative to schedule a session around the 2016 Census and Cyber Security.

You are getting two presentations for the price of one today, as Alastair MacGibbon and I will speak about some related but different aspects – I will focus more on the 2016 Census and ABS and Alastair will have more emphasis on cyber security.

Some background always helps to place these events in context:

• The ABS is one of the public sector’s most significant public facing organisations, with around 500 statistical releases a year, all potentially subject to scrutiny and comment, all critical for informed decision making. Our statistical processes are transparently reported and our key stakeholders expect us to get these key official statistics right first time every time.
• The ABS has a complex business model, with around 2500 staff across nine locations doing national lines of business, and a further 400 or so regular field interviewers.

The recent Census experience is still pretty raw, so today you have the advantage of getting a very contemporary perspective that draws on our real battle scars as well as our innovations and successes.

As I have said at recent Senate Estimates hearings and the Senate Inquiry into the 2016 Census, the ABS stuffed up some things in the conduct of the 2016 Census. The ABS should have done better. We understand this reflected not only on the ABS but on the reputation of government and the public sector more generally. We may have made your job more difficult, but hope also that you can learn and benefit from our mistakes.

Our time today is short, so I will concentrate on a few of our learnings and provide some high level perspectives. These are already feeding into the preliminary considerations for the possible nature of the 2021 Census.

The key learnings relate to implementing a major change program, providing a public service, responding to a disruptive environment, managing risk, privacy, and managing staff through such an event. I then have a few comments about the implications for our broader ABS transformation and an update on the Census process.

1. Designing and managing a major change program

The Census is a major exercise, with planning and implementation spanning at least 5 years. The 2016 Census cost around $500 million, involved up to 38,000 field staff, and complex delivery models for different populations, such as indigenous, remote, homeless, and elderly.

The ABS decision in 2011-12 to change from the traditional drop and collect paper forms model to a digital first Census, was a bold change, particularly for a cautious organisation such as the ABS. The ABS was changing the overall Census approach to one where there was no Australian reference history or limited information on which to make judgments.

It was a justifiable and reasonable change, given Australia’s adoption of technology, secure apartment buildings and gated communities were making it more difficult to physically access some dwellings, and it was becoming more difficult and costly to secure 50,000 temporary staff as Census collectors to operate. The change was consistent with the Government’s digital transformation agenda.

The overall strategy, and its ambition, was not the problem – this new Census approach needs to be continued.

The ABS had a detailed plan and had properly envisaged and developed the 2016 Census as a major process transformation. It was not just digitising a paper form.

Managing a major event over an extended period does bring its own challenges. During the five years of Census preparation there were 3 Australian Statisticians (one of these temporarily in the role for around 12 months), 7 Ministers and even 4 Prime Ministers over the five years that the 2016 Census was planned and implemented. However, it is not unusual in the public sector to have senior people changing roles, or changes in Ministers - this is part of our operating environment.

So what went wrong?

The ABS underestimated the nature, complexity and risk of the change process.

We were expecting the majority of the public to engage with the Census in a new manner.

The ABS applied behavioural economics to develop our new Census letter and then used traditional methods of testing processes and estimating public behaviour – specifically conducting large annual field tests of up to a staggering 100,000 households. These tests proved to be unrepresentative of public behaviour. In hindsight, these kinds of tests were never going to accurately represent this new model of Census – where public understanding is driven more through advertising, media and social media – rather than doorstep contact.

Further, benchmarks of public behaviour in August 2014 (or 2011) will not necessarily represent behaviour in 2016 and can never account for the actual event and the context surrounding it. This lead to a significant underestimation of the number and timing of phone calls that would be received – with over 3.5m phone call attempts.

We also attempted to deliver the 2016 Census largely independently. Whilst the conduct of the Census is clearly the ABS’ responsibility – we failed to recognise that our successes and failures reflect on all of government. With the size, scale and complexity of this project and the environment in which we operate – the ABS needs to be working in partnership with other agencies – central, service delivery and security in the design and delivery of the service - and them working in partnership with us..

Plans for major events or programs need to include considerable preparedness around things that might go wrong. We should ensure that planning does not only seek to ensure a successful outcome, but also extend to potential strategies that can be used (and more than likely modified) in the event of a major failing or disaster. Outside of the cauldron of the particular real-time challenge, planning around key roles and key actions can help increase the likelihood that effective, comprehensive and timely responses are put in place when stuff inevitably happens. We could have certainly improved in this area.

2. Providing a public service

Another learning from this year’s Census relates to the service delivery expectations of the public. The Census seeks to be all-encompassing in its coverage of the community over a very short time period. This means that any call centre facility, mail service and e-Census needs to be designed with an understanding of consumer expectations and behaviours.

Public-facing government agencies, as part of the public face of government, are expected to provide a good consumer experience for citizens. This does lead to the desirability of having an open dialogue with central agencies and government about funding envelopes that would provide different levels of consumer service and responsiveness.

We also didn’t think about the load that would fall on the call centre if something didn’t go to plan. Indeed, things didn’t go to plan before the Census night outage. Media coverage about fines and a number of other coinciding events drove people to our call centre earlier than we expected and at volumes much higher than our tests predicted. Our automated form request system was not scaled to deal with a surge of requests.

Our call centre didn’t cope with the volumes. There are two elements to this. Firstly, we had a level of acceptance about some calls being blocked. We also under estimated the enthusiasm of the Australian people to want their census form before Census night. While the call centre and automated form request system were both scaled up, there were limits.

Many people were markedly inconvenienced by the unavailability of the e-Census after 730pm on Census night, 9 August, and some tried multiple times to access the e-Census system on the night, with considerable frustration.

4.9 million households did complete the Census on line – 2.2 million before the outage and a further 2.7 million after it was restored. Those that used the e-Census facility found it a quick and easy way to provide their Census information – in fact the online form reduced the time taken by households by 70% compared to paper.

The Census responses, however, also suggest a number also gave up on the e-Census and reverted to a paper based response. The Preliminary Census overall participation rate of over 96% suggests that few gave up on completing the Census.

Around the time of the e-Census outage, the ABS did not do an adequate job to provide Ministers and the Government with sufficient information to allow them to help explain these aspects to the public. We were focussed on trying to more fully understand what was happening and seeking to resolve it as soon as possible. Our emphasis was on bringing the right people together quickly to deal with a problem. This is necessary but far from sufficient. We needed pre-prepared crisis communications plans and checklists. ABS accepts there was a communications deficit to the public and to the government. In this instant communications world (now encompassing traditional media and social media) we need to do much better.

The ABS needs to better understand we are seen by the public as part of government. Public sector insiders are well aware of the statutory independence of the ABS, to safeguard the integrity of Australia’s official statistics, as enshrined in our legislation and the Statement of Expectations from the Government to the ABS.

However, as recognised by the 2013 Capability Review, the ABS had taken this important dimension of “independence” to an art form, and become isolated from our key stakeholders. The ABS can improve our public service, at the same time as improving our key stakeholder relationships, in a manner that does not compromise the integrity of our official statistics. This is a key focus of our current transformation, and one that needs further attention following this Census experience.

3. Be alert to a disruptive, changing environment

Continuing the theme from earlier, this year’s Census was being conducted in 2016, not 2011 or earlier.

We cannot pre set our planning and implementation. We need adaptive responses within such a major program over an extended program duration.

We expected the public, media and politicians to respond like they had in the previous Census – and they didn’t.

In the build up to 2016 Census, there was a significant public discourse about privacy and the security of Census information, and considerable dialogue about the capacity of the on-line system to cope with the anticipated scale of responses on Census night. The extent and nature of this was not anticipated by the ABS. Our past experience was that 33% of Australians had completed the Census online in 2011, Australians had been required to provide their names and addresses in all previous Censuses and data integration had been a feature of Australian Censuses since 2006.

A number of factors affect how Australians respond to the public administration in general and ABS specifically in 2016 – the 2016 election campaign, national security concerns, metadata retention, experiences of other digital services, our own decision to retain names and addresses for a longer period, and the list goes on.

We obviously cannot jump at every shadow, and respond to every alternative perspective. Not everything warrants a strong response and our budgets do not stretch that far.

But we do need to be sensitised to the potential slow burn of dimensions that might look innocuous at first, but then accelerate into a full blown issue – bit like the Canberra bushfires.

The ABS should have communicated more clearly at an early stage why retaining names and addresses was important. We clearly didn’t persuade people about the security of their information.

Part of this new disruptive environment recognises the changing nature of traditional media. There is little you can do to stop the media and social media reporting claims, even blatantly false ones. ABS staff, including myself, did 380 media interviews in the weeks leading up to the Census – this may have contributed to the strong level of public participation in the Census, however could not extinguish the fear and concern. I make these points not to blame the media – but to recognise that we have a different media environment than in the past and simply point out it’s a complex challenge which the ABS clearly failed to address effectively.

I am now more wary about use of past experience as a guide to future performance, and would provide this advice to anyone who is procuring a significant service from an external provider. IBM was seen as a low risk option because they had successfully delivered the e-Census facility in 2006 and 2011.

However, in a world where the environment was changing significantly, such as we are seeing with cyber security, past suppliers who had a previous approach may turn out to be a high risk choice. For example, the environment in which the service is being offered may have changed so significantly (such as the cyber security environment) that re-use of essentially the same approach/strategy in place some years previously is not sufficient to deal with contemporary risks.

4. Managing risk in a contemporary environment

Key lesson here: you cannot outsource risk!

In mid-2014, IBM, one of the biggest global ICT companies was contracted to provide a relatively simple e-Census application, ensure the application was available to the public over the 8-week Census response period, provide sufficient capacity to deal with the peak response load expected on Census night of August 9 and ensure effective security for the e-Census system. The contract to IBM was worth around $10 million, within the total Census budget of around $500 million.

IBM failed to deliver the contracted service to the necessary standards, and they have provided financial recompense to cover the additional costs we and the Government have had, and will still incur as a result of this incident.

But the reputation of the ABS also took a big hit, because ultimately ABS is responsible for a good Census process as well as delivering quality Census data.

For public facing agencies and for agencies delivering critical services, such as the ABS, we need to operate with a low risk appetite. There is little public or government tolerance for error of any kind. I recognise the impact of this risk was also not just on the ABS but on the broader public administration and Government.

There are two dimensions related to risk management that are pertinent here: accurate assessment of risk appetite and active management of key risks.

We had a misplaced sense of confidence, indeed complacency, about the e-Census and its security. We worried about the element that we knew would change: the increase in the number of users. We didn’t adequately test and review the things that we thought would not change – particularly the DDOS security. A more thorough, independent, review of the DDOS defences would have identified key weakness in the architecture – a reliance on a single layer of protection called ‘Island Australia”.

On the surface, we had a regime for risk management in place – the risk of DDOS was identified, the impact of a successful attack was assessed as extreme and we considered an attack to be likely. A set of risk mitigations was documented and the Census board was given a report indicating that the residual risk was acceptable. However, the mitigations were not adequate. More independent assurance was needed but we also need to foster a culture that sees active risk management as an integral and valuable component of our approach, beyond the form filling and administrative compliance.

5. Privacy and use of data

Australian Censuses have a long history of debates about privacy and use of Census data – 1971, 1976, 1996 and 2006 are other examples.

The ABS submission to the recent Senate Inquiry into the 2016 Census provides a good summary of some of the pertinent issues, such as the need to collect names in the Census for sound statistical reasons. This is international best practice for Census taking, and Australia has been doing it for over 100 years.

The ABS has world class data security arrangements, encompassing our legislation, our training and practices around data security, our confidentialisation methodologies and expertise as well as our physical security.

This has been enhanced over the past decade through the development of our specialised and accredited data integration facility. Many other national statistical offices across the world have similarly developed their data linkage capability, as it provides an opportunity to provide more key statistics from existing information using techniques that are privacy-preserving.

Over the last decade, the ABS has used data linkage of the Census with other administrative data to provide more accurate statistics on a range of dimensions such as Indigenous life expectancy, outcomes for migrants, educational transitions and outcomes for students participating in school based apprenticeships.

The decision by the ABS to retain names/addresses for up to 4 years, rather than the previous 18 months, was to meet the needs of public administration and the community to produce more valuable statistics for policy and/or evaluation purposes. This would enable us to further improve the quality of some of our statistics as well as produce new statistics around issues such as industry restructuring, changes in local communities and changing family arrangements.

A number of other countries keep names and addresses from their Census indefinitely (eg, Canada, New Zealand, Ireland, etc) because of the additional statistical value that can be derived for public good from this major investment in a Census.

The ABS has accepted that the Privacy Impact Assessment undertaken by the ABS was deficient in a number of ways. The lack of independent, external or rigorous enough assessments provided a platform for negative and critical views to project from, regardless of the reality or the facts.

One further insight from the recent Census experience for me was that many of those who saw the value of greater use of public data did not give voice to their desire for greater use of public data – some did, but many did not. Our community can achieve greater value from more effective use of our public data resources in ways that do not jeopardise the privacy of sensitive personal or business information. The challenge is to help encourage and mobilise those who see value in better use of our public data resource so they can be vocal advocates for use of public data assets, and provide greater balance to the more vocal privacy advocates. This is a message the ABS can’t provide on our own.

The other key learning from the recent Census is that the general population probably has a different view of the privacy of their personal information and the capacity of the ABS to secure this effectively compared to the fears of some privacy advocates. As part of our communication campaign, we were monitoring community sentiment towards the Census through the Census lead up, and this remained strong with around 97% of the population intending to complete the Census right through the privacy debate. The Census response rate looks like it will be consistent with these expressed intentions.

6. Managing your people

ABS staff have considerable pride in what we do and what we achieve. There is a shared purpose to deliver quality, timely, relevant statistics.

Those working on the 2016 Census, and across the entire organisation, were hurt professionally and personally by this experience.

After the initial shock of what had happened, the organisation did come together spectacularly to ensure we could still deliver a quality Census outcome in 2016. The key dimensions of the 2016 Census design – a stable e-Census facility from 11 August, a reminder letter in late August, field force activity from early September, monitoring of household responses and targeted deployment of field staff to low response areas – were critical to achieving a good outcome – but so were the efforts of our managers and staff to achieve this outcome.

Senior managers ensured we were available and visible, walking the floors in the subsequent days and weeks, supporting people, ensuring good decisions were taken in a timely fashion and trying to ensure staff received rest breaks during an intensive effort. Staff put in many additional hours to ensure to ensure the 2016 Census would deliver quality information. Employee Assistance arrangements already available were amplified, and staff encouraged to make use of these.

People from across the organisation helped out with Census related activities, at the same time as we kept to our original schedule of still releasing other essential economic, population, social and environmental statistics. Our field interviewers conducted an extensive expanded post enumeration survey covering over 50,000 dwellings, at the same time as we were conducting our usual monthly household survey of around 26,000 households (that delivers our labour force estimates).

Some other things also happened. We were inundated with FOI requests and other correspondence. This was a significant impost on the organisation and our resources at the very time we were working hard to ensure a quality Census outcome. We also received some vile emails, letters and phone calls that no-one should ever receive.

We also received many kind wishes of support, from across the public sector, from key users of ABS data as well as family and friends. At times like these, personal and professional networks really matter and for those of you here who reached out – thank you.

A number of staff have deep scars from this experience, and as an organisation we need to be alert to impacts that might continue for some people for some time.

However, there has already been an upside. Professionally, a number of ABS staff have learnt much from this Census that will help the organisation plan, implement and manage our critical processes better in the future. ABS staff have broader real-world understandings and expertise following this experience. I have already seen some improvement in our culture change towards greater internal collaboration and greater recognition of the need for us, as a mid-sized organisation, to access external expertise and perspective.

I hope that you can learn from our Census experience without having such a lived and public experience as we have had over recent months.

7. ABS transformation 2015-20 – the implications of Census 2016

After I was appointed Australian Statistician in December 2014, I gave two main messages to senior staff:

• The ABS needed to transform itself, responding to the very clear messages out of the 2013 Capability Review and the 2014 Parkinson Review, and
• For the ABS to succeed, we needed to be ambidextrous – to continue to deliver quality, timely, relevant statistics (our core purpose) and at the same we also needed to be more innovative.

Some of you might be aware of the major and ambitious transformation underway at the ABS since mid-2015.

This will markedly change the way we work as an organisation.

The comprehensive ABS transformation program encompasses six dimensions of:

• Our environment – we collaborate with stakeholders to understand and respond better to the current and future external environment
• Our strategy – our strategies enable rigorous statistics, strong partnerships and effective use of resources
• Our governance – our governance supports responsive decision making, prioritisation and management of enterprise risk
• Our people - we have a diverse, expert, motivated and agile workforce
• Our culture – we are high performing, aligned, engaged, innovative and accountable
• Our infrastructure – our infrastructure is effective, efficient and adaptable.

A number of lessons from the 2016 Census experience will reshape our transformation program:

• We recognise that the Census experience has damaged some of our key stakeholder relationships and we need to rebuild those relationships. We have damaged the brand and reputation of the public service and have made our collective task harder;
• our strategy needs to have greater regard to cyber security as a key contemporary risk of operating in a digital world and where ongoing security of the sensitive information we receive is paramount;
• our governance reforms are not as well advanced as they need to be, with a need to focus on building our risk management maturity and bringing in more external expertise;
• continued action is required to diversify our workforce with skills/experiences from outside ABS, building on steps already taken;
• further action is required to continue transforming the ABS culture, drawing upon greater external expertise and engagement and building more internal collaboration; and
• continuation of the ambitious transformation of the ABS IT and business processes for the statistical systems (we are 18 months into a 5 year modernisation program).

  8. Census 2016 – latest information

I want to end with a bit of a post-script of the latest information on the 2016 Census.

Our preliminary estimate is that over 96% of households in occupied dwellings completed the Census – a figure that is comparable with previous Censuses and better than a number of our international counterparts. We also collect information from persons outside of their usual residence, from hotels, maritime vessels, airports, national parks, etc.

Over 80% more households participated online in 2016 than 2011. Whilst we didn’t achieve our target of a 100% increase, this increase of 2.2 million more online forms and smart innovation in our online form, should contribute to more accurate data.

The targeted enumeration strategies for hard to count populations and people, such as homeless people and Indigenous communities, were not affected by the e-Census outage.

We have completed an expanded post enumeration survey, drawing information from over 40,000 households, which is essential to produce high quality population estimates.

We have formed an Independent Panel, with prominent Australian and international members, to provide advice and assurance around the quality of the 2016 Census data.

The first Census data will be available on 11 April 2017, 2 ½ months earlier than previously. Further data releases will be in June and October 2017. A further instalment of the Australian Census Longitudinal Dataset, adding a 5% sample from 2016 to the 2006 and 2011 samples, is expected to be available in December 2017.

We look forward to working with many of you to unleash the power of this 2016 Census data, providing benefits back to the Australian community that have supplied us with their information.