QUALITY MEASURES
The second component of a quality gate is "quality measures". Quality measures are a set of indicators that provide information about potential problems at a given point in the process. When determining what quality measures should be included in a specific quality gate it is important to consider the risks and what information would be required in order to make an assessment about fitness for purpose at that point in time. That is, what quality measures are going to reveal if there is a problem with the process. Along with thinking about the immediate needs of the process it is important to keep in mind the outcomes that are required from the process to ensure that these can be met through use of appropriate quality measures.
A quality gate may have multiple quality measures which assist in determining problems in the process. Observing the quality measure as a time series may be of benefit to ensure that business as usual is being observed. It is also recommended that each quality measure within a particular quality gate is mutually exclusive (see ABS Experience section).
Using common or shared definitions where possible for quality measures is encouraged. Where a common definition is not appropriate it is important to ensure that the quality measure is thoroughly documented so that no misunderstandings occur. Often simple errors occur in processes because common knowledge of that particular process is assumed to be held by all parties involved. Documenting the definitions used for the quality measures is an important part of knowledge management for a quality gate.
It is important to specify the level of detail required for a quality measure. For example, instead of examining the response rate at the Australia level only the quality measure may require the response rate for a particular State or Territory. Drilling down to more detailed information may reveal issues with the process that are not apparent at higher levels. Historical information of the process can be used to determine if there are any known issues that should be monitored along with the initial risk assessment. Please note that a quality measure may not itself identify a specific problem in the process but it will report on symptoms which may indicate that something is not right and further investigation is required. It is also important to note that the source of a quality measure may be different to the source of the problem. When thinking about quality measures it is useful to also consider what it is about the process that they are revealing as it may not be a straight forward one to one link.
Not every detailed check that is undertaken in a process will constitute an individual quality measure, however they may be utilised by quality measures. An example of this is a check list of the different ways a data set is validated (e.g. internal consistency checks, non-zero values, number of records in is equal to number of records out) which may not in itself be a quality measure but combined with other detailed checks it may form a part of a quality measure.
As with any monitoring activity it is useful to prioritise the quality measures in terms of their level of importance in ensuring the quality of the process. This is because quality gates are labour intensive and there is a cost to benefit trade off assessment that needs to be made in order to achieve high quality outputs in a timely manner. That is why it is important to choose good indicators (quality measures) of potential problems. The identification of good quality measures will become more apparent with experience and practice in using quality gates over time.
Print Page
Print All