The fifth dimension of quality in the ABS DQF is Coherence. Coherence refers to the internal consistency of a statistical collection, product or release, as well as its comparability with other sources of information, within a broad analytical framework and over time. The use of standard concepts, classifications and target populations promotes coherence, as does the use of common methodology across surveys. Coherence is an important component of quality as it provides an indication of whether the dataset can be usefully compared with other sources to enable data compilation and comparison. It is important to note that coherence does not necessarily imply full numerical consistency, rather consistency in methods and collection standards. Quality statements of statistical measures must include a discussion of any factors which would affect the comparability of the data over time.
The Coherence of a statistical collection, product or release can be evaluated by considering a number of key aspects:
- Changes to data items: to what extent a long time series of particular data items might be available, or whether significant changes have occurred to the way that data are collected.
- Comparison across data items: this refers to the capacity to be able to make meaningful comparisons across multiple data items within the same collection. The ability to make comparisons may be affected if there have been significant changes in collection, processing or estimation methodology which might have occurred across multiple items within a collection.
- Comparison with previous releases: the extent to which there have been significant changes in collection, processing or estimation methodology in this release compared with previous releases, or any 'real world' events which have impacted on the data since the previous release.
- Comparison with other products available: this refers to whether there are any other data sources with which a particular series has been compared, and whether these two sources tell the same story. This aspect may also include identification of any other key data sources with which the data cannot be compared, and the reasons for this, such as differences in scope or definitions.
To assist in evaluating the Coherence dimension of a dataset or a statistical product, we provide some suggestions of questions which might be asked below.
Suggested questions to assess Coherence
- Is it possible to compile a consistent time series of a particular data item of interest over a number of years?
- To what extent can a user meaningfully compare several data items within this collection?
- Could any natural disasters or significant economic events have influenced the data since the previous release?
- Have these data been confronted with other data sources, and are the messages consistent from all data sources?