PLACEMENT


"Placement" is the first component of a quality gate. It refers to the placement of quality gates throughout a statistical process (also known as a business process cycle, or statistical process cycle). Placement of a quality gate is determined by the level of risk associated with given points in the production process. Specifically, the placement of a quality gate should occur where a risk assessment of the process reveals that there is a need for a quality gate due to the impact on the process and statistical outputs that would occur if the risk was realised.

In determining where to place quality gates it is important to ask the questions:

• What can go wrong?
• When can it occur?
• What impact can it have?

Along with thinking about the inherent risks, it is helpful to draw a basic map of the production process that is to be monitored which can also help in determining placement of a quality gate. Mapping processes helps users understand how a system works and identifies how a system interacts with other systems and processes. For example, it will help users determine interdependencies with one another in the production of statistics. Mapping the statistical process provides a simple conceptual framework to identify logical and key areas where quality gates can be used to monitor quality.

The ABS uses the Generic Statistical Business Process Model (GSBPM) as a guide to map the activities of statistical processes against. This is done to ensure all aspects of the statistical process are included for monitoring purposes. For example the "Collect" phase of the GSBPM would include activities related to obtaining data. This could include interviewing respondents, obtaining data from an administrative source, interviewer training, sample monitoring etc., depending on the nature of the statistical process being mapped.

Figure 1 - Taken from the Generic Statistical Business Process Model (Vale, 2009)

The above diagram could be simplified further to "Input, Process, Output" for mapping against. This simplification may be beneficial to agencies where statistical operations are not their primary function.

Figure 2 - Simplified Business Process Model

By identifying the key activities associated with each step of the statistical process, an assessment of whether there are any risks in those steps can be made up front. This assists with determining where best to place quality gates.

Some common risky areas in a process include:

• Hand-over or integration of data between multiple areas;
• Data transformation; and
• Changes to processes, methods and systems.

It is important when considering placement of the quality gates that each gate is placed as early in the process as possible to allow early detection of issues. This promotes greater efficiencies as errors are picked up earlier in the process and resolved in a timely fashion. However, when positioning quality gates the impact of time constraints on the overall production of the statistics needs to be taken into account. This means that there is a limit to the number of quality gates that can be effectively implemented in any given process due to deadlines.

The ABS has an overarching Risk Management Framework, based on the International Risk Management Standard ISO 31000:2009, which details the ABS approach to risk management. The ABS has adapted this Risk Management Framework to suit the business needs of the organisation. One such adaptation of this Risk Management Framework is the ABS' Statistical Risk Management Framework (see below). It cross classifies the levels of "Likelihood" (chance of the risk occurring) and "Consequence" (effect on the immediate process or statistical outputs of the risk occurring) to reveal an overall assessment of the statistical risk which could be either Low (L), Moderate (M), High (H) or Extreme (E).


ABS Statistical Risk Management Framework

Where a statistical risk assessment reveals that the risk rating is extreme or high it is recommended that a quality gate be utilised to mitigate the statistical risk. For medium risk ratings it may be useful to utilise additional quality measures in existing quality gates that assist in monitoring the aspects which will highlight if the process isn't working correctly. Routine procedures are generally sufficient for the monitoring of low risk ratings. More information on the ABS' Statistical Risk Assessment Framework can be found in the Appendix. It is worth noting that each organisation will have their own risk matrix based on their tolerance for risk and that the ABS' Statistical Risk Assessment Framework may not be suitable for use by other organisations depending on their needs.

It is also important to note that the placement of a quality gate may be different for each production process and that the impact on the immediate process and overall quality of the statistical outputs are key pieces of information to assist in the placement of quality gates.

Along with these considerations it is worth keeping in mind that all quality gates used to monitor one statistical process cycle may not be controlled by one area alone. It may be that there are several areas (such as in the case of a hand-over situation) that have responsibility for the development, maintenance and assessment of quality gates for a particular part of the statistical process.