Julian Whiting, Mark Zhang, Methodology Division, Australian Bureau of Statistics
Seasonal adjustment throughout periods of significant disruption and uncertainty
This research paper includes some experimental methods, and was first presented to the ABS Methodology Advisory Committee in October 2020.
Released
7/04/2021
Methodology Advisory Committee
The ABS Methodology Advisory Committee (MAC) is an expert advisory group of statisticians and data scientists drawn mainly from, but not restricted to, universities across Australia and New Zealand.
The function of the MAC is to provide expert advice to the Chief Methodologist on selected methodological issues relevant to the production of official statistics, including questionnaire development, survey design, data collection, data linkage, estimation, time series analysis, confidentialisation and the use of new and emerging data sources. The MAC is a key mechanism used by the ABS to ensure outputs are based on sound and objective statistical principles.
The Committee generally meets two or three times each year to discuss papers on specific issues. The papers can relate to work at different stages of the methodology solution cycle:
- during initial thinking around a statistical and/or data science method to meet new or emerging needs;
- as investigations proceed or develop the more complex aspects of a proposed method;
- as methods are finalised and circulated for public comment; or
- as part of ongoing reviews of methods already implemented in ABS work.
In providing advice the Committee considers aspects such as:
- the statistical validity of the approach proposed;
- similar statistical or data science problems, arising in other fields, where work done may provide the basis for improved solutions to the given problem; and
- the implications of the statistical or data science methods proposed for the valid use of the outputs, and in particular the inferences that might be drawn from the resulting data.
Purpose of this paper
- Describe the principles guiding decision making about how the ABS intends to treat the impacts of COVID-19 on time series outputs over the next few years.
- Seek feedback from MAC on the utility of seasonally adjusted estimates when there is mounting evidence of structural change in seasonal behaviour.
- Propose a new method which aims to improve the standard approach for identifying and estimating a structural break in seasonal behaviour.
ABS context
- The impacts of COVID-19 may reduce the utility of time series analytical outputs published by the ABS to the extent that these outputs will not be useful for decision making.
- The strategies for managing the impacts on time series outputs need to be practical to implement within the current seasonal adjustment framework and systems.
Problem specification
- The filter-based algorithm used for ABS seasonal adjustment will produce distorted seasonal factor estimates when the input time series contains abrupt structural changes or outliers which are not accounted for. Accounting for these impacts requires the application of corrections in order to create a prior-adjusted input series which is free of structural changes and outliers.
- Clear principles are needed for decision making about prior corrections and the dissemination of seasonally adjusted and trend outputs. The COVID-19 disruption may diminish the utility of these outputs irrespective of the corrections applied, and the principles seek to minimise loss of utility.
- Treatment of impacts should be based on the nature of disruption observed and confidence about future series behaviour. Corrections and decisions about dissemination need to be continually reassessed as we progress from treatment of initial impacts, to managing the impacts during the pandemic, and then as recovery from the pandemic evolves.
- Estimation of the corrections to treat the COVID-related abrupt changes involves comparing the observed series with modelled estimates of the ‘counterfactual’ which would have been observed in the absence of COVID-19 impacts. It may not be sufficient to estimate the counterfactual based solely on the historical dynamic of the target series.
Questions for MAC
- Does MAC endorse the principles outlined in Section 4 for how the standard methods and tools are applied to series significantly disrupted by COVID-19?
- Does MAC have any views on the potential of the method described in Section 5 for improving the identification and estimation of structural breaks in seasonality?
- What series could possibly be used to improve estimation of structural breaks for series measuring levels of consumption and production, avoiding spurious correlations?
- Can MAC suggest alternative methods which could enable earlier identification and improved quantification of structural breaks in seasonality?
1. Introduction
There will be ongoing intense analysis of ABS economic time series to understand the nature and magnitude of the disruption caused by COVID-19. It is vital that signals in the time series are extracted as effectively as possible and that users appropriately interpret the time series analytical outputs.
The fundamental challenge of seasonal adjustment is accurately decomposing the series into the trend, seasonal and irregular components in a timely fashion. The final estimates of the seasonal factors and the trend for a segment of a time series take account of series behaviour before and after the segment, and so the initial estimates depend on expectations and assumptions about future series behaviour.
The challenge of seasonal adjustment is intensified when there is a major disruption like COVID-19 which can have wide-ranging impacts on time series. COVID-19 could affect the trend, seasonal or irregular components, the changes could happen abruptly or more slowly, and the duration of disruption will vary substantially across series. Van den Brakel et al. (2017) discuss the challenges for estimating seasonal factors in the context of the most recent period of significant global economic disruption, the Global Financial Crisis in 2008-2010. Not only does the increased series volatility make it more difficult to identify any evolving changes in seasonal influences, there is the possibility of the disruption abruptly triggering a permanent change in the seasonal pattern.
The problem addressed in this paper is the strategy for applying prior correction treatments as part of the existing ABS process for producing time series outputs in an environment of major disruption and heightened uncertainty. These treatments attempt to remove the disruptive effects from the observed series so that the disruption does not distort seasonal factor estimates calculated using the X-11 algorithm. The choice between the outlier, trend break and seasonal break treatments determines how the disruption is assigned to the trend, seasonal and irregular components. This paper is not concerned with alternatives to the X-11 framework for estimating seasonal factors, so the strategies explored are in the context of estimating prior corrections.
This paper is structured into two parts. The first part, comprising Sections 2-4, describes the current methods, processes and issues relating to seasonal adjustment in the ABS, the impacts of COVID-19, and recommendations for applying the standard methods to treat these impacts in the short and medium term. The second part of the paper, Section 5, considers the possibility of improving estimation of structural change in seasonal behaviour by using data external to the series of interest. This innovative approach promises to improve the power to detect structural breaks in seasonality with more confidence and earlier than our usual methods, thereby increasing the utility of the series to our users. Concluding remarks are presented in Section 6.
2. ABS Seasonal Adjustment
2.1 Estimating the decomposition with X-12-ARIMA
The ABS publishes seasonally adjusted and trend series for most of its time series. The seasonally adjusted outputs support macroeconomic analysis and decision making by removing systematic calendar-related variation in order to more clearly reveal underlying movements. The trend captures the medium to long-term movements in a time series, and is a reflection of the underlying level.
The seasonal adjustment method used is based on RegARIMA models with X-12-ARIMA (Findley et al., 1998). The method decomposes the time series in a filter-based iterative algorithm (X-11), so there is not an explicit model describing the seasonal, trend and irregular components. A National Statistical Office produces thousands of time series, and an important feature of the X-12-ARIMA approach is that appropriate parameter settings for the X-11 algorithm (e.g. choice of trend and seasonal filters) can be quickly established for each series and seldom require updating.
For over a decade it has been standard practice in the ABS to use concurrent seasonal adjustment, whereby seasonal factor estimates are updated as each observation is added to the series. Concurrent adjustment replaced the 'forward factors' method which involves an annual process to estimate the seasonal factors for the forthcoming year and update the seasonal factors applied to previously-observed points.
2.2 Prior corrections and intervention analysis
Prior corrections are applied in the standard seasonal adjustment process to prevent unusual effects from distorting the seasonal factor estimates. The corrections considered in this paper are additive outliers, trend breaks and seasonal breaks. The unusual effects are removed prior to running the X-11 algorithm, and after estimating the intermediate trend, seasonal and irregular components the additive outliers and trend breaks are put back into the series by assigning them to the irregular and trend components respectively.
This paper considers prior corrections to treat abrupt changes in series behaviour which have been caused by COVID-19. It is not necessary to apply corrections to treat changes which evolve after the initial COVID-19 impacts because slow changes in behaviour will be appropriately estimated by the X-11 algorithm. This paper focuses on corrections to treat structural changes, which are the abrupt changes in behaviour which persist for an extended number of time periods. At least three time points would be considered an extended period of change for a change in level, and at least three years is an extended period of change in the seasonal characteristics.
Estimation of the corrections to treat the abrupt changes attributable to COVID-19 involves comparing the observed values with modelled estimates of the series which has all impacts of COVID-19 removed. This modelled series is referred to in this paper as a “pseudo control” of the counterfactual scenario. The standard approach for intervention analysis is RegARIMA modelling, which involves fitting an autoregressive integrated moving average (ARIMA) model with regression. The “pseudo control” derived from RegARIMA is predicted from the past dynamics of the series, so no information about irregular variation unrelated to COVID-19 is involved in the assessment of structural changes attributable to COVID-19. Section 5 considers the possibility of constructing the “pseudo control” using predictors from contemporaneous time series in order to represent variations in the target series which are unrelated to COVID-19 effects. The purpose of the time series models used in this process is estimating prior corrections before applying the X-11 algorithm to calculate the seasonal factors. Considering the additional modelling effort required, this approach would be reserved for series which have higher economic importance and for which suitable predictor series can be identified.
2.3 Assessment of quality of seasonal adjustment
The quality of seasonally adjusted and trend outputs is difficult to precisely quantify. Seasonally adjusted and trend series are analytical constructs, and there is no 'true' value of the decomposition for a particular data point. Users with different analysis questions in mind can have varying perspectives about trend smoothness and how the seasonal factor for a data point should relate to the seasonal influences apparent in previous and subsequent years.
The initial estimates for the seasonally adjusted and trend series at a time point will be revised as subsequent data points are observed. The size of revision is an aspect of the quality of the initial estimate, as well as being a measure of improvement over time. Revisions are a concern if initial estimates are modified to the extent that users would have made different decisions if the initial estimates were closer to the revised estimates.
Revisions can be expected to be larger for series which involve prior correction treatments because the initial assessment of treatment required can be refined considerably as subsequent data points are observed. An extreme example is when a prior correction is removed entirely because the subsequent data points reveal the correction was unnecessary.
3. Potential impacts of COVID-19 on series and adjustment
COVID-19 could have significant impacts on each of the trend, seasonal and irregular components of the time series decomposition. The magnitude and duration of these impacts will vary a lot across ABS economic time series, and for each series analysts will be intensely interested in the nature of impacts.
3.1 Impacts on level
A significant drop in the activity measured by many series has already been observed as a consequence of lockdown measures. Some series will return to pre-COVID levels within two or three periods, but for the remaining series there are many potential paths for the future direction of the series. A ‘recovery phase’ is expected in most cases, though some series may never return to their pre-COVID level and there would be much interest in signals for a plateau in the level and end of the recovery. Continuation of disruptive influences and uncertainty over the next few years will result in above-average volatility of series during the recovery phase.
3.2 Impacts on seasonal influences
In addition to the changes to the underlying series level and more frequent outliers, the pandemic could have a temporary or longer-lasting effect on the seasonal influences. Perhaps the most likely type of change to the seasonal influences is a temporary general weakening, which means the relative size of the usual seasonal peaks and troughs become less pronounced. For example, the pandemic may cause the December peak in Retail trade to be smaller in relative terms in just the next couple of years. Disruption to seasonal effects besides a general dampening effect are also possible. Dagum and Morry (1985) discuss the example of temporary disruption to the usual seasonal pattern in series measuring unemployment. They observed disruption to the seasonal influences during recession periods because the type of work sought by the larger population of jobseekers is different during a recession compared with non-recession periods.
3.3 Illustration of impacts
Table 1 presents four hypothetical example scenarios of combinations of impacts on the trend, seasonal and irregular components, and Figure 1 depicts examples. In practice, only the combined effects on each component is observed, but appropriate removal of COVID-related impacts requires understanding how each of the individual components has been affected. This understanding can be informed by not only impacts to the series already observed, but also ‘real-world’ deterministic information (e.g. Government announcements on international travel restrictions). The levels of confidence about the nature of impact on each component and their expected future behaviours should determine the type of treatment and when the treatment is applied. Section 4.5 describes the application of recommended treatments for some of these hypothetical scenarios.
| Scenario | Length of ‘crisis’ period | Length of recovery period | Impact on irregular volatility | Impact on level | Impact on seasonal during recovery | Impact on seasonal in the longer term |
|---|---|---|---|---|---|---|
| A | At least 12 months | 2 years | Minor | Extreme | Existing pattern moderately subdued | Similar to pre-crisis |
| B | 9 months | 3+ years | Increased volatility during crisis period | Large | Existing pattern considerably subdued | Similar to pre-crisis, but has evolved somewhat |
| C | 3 quarters | 1.5 years | Increased fluctuations during crisis and recovery | Large | Similar to pre-crisis | Similar to pre-crisis |
| D | 1 quarter | 2 quarters | Minor | Moderate | Significant change (e.g. Q3 replaces Q4 as the low period) | Significant change (e.g. Q3 replaces Q4 as the low period) |
Figure 1 (Scenario A - D): Illustration of combined effects of impacts for scenarios in Table 1
3.4 Effect of disruption on utility of seasonally adjusted series
The nature of disruption to the existing seasonal pattern will typically be difficult to discern when there has been just one or two observations for each quarter or month affected by the pandemic. The duration of the disruption could also be highly uncertain. The impact to the seasonal component and associated uncertainty can reduce the utility of the seasonally adjusted series. Although prior corrections which attempt to account for the disruption can reduce utility loss, in some cases the utility could be affected to the extent that suspending publication of the seasonally adjusted series should be considered.
Within the first two years of the pandemic there will typically be insufficient information to estimate changes to the seasonal, and the standard action is to assume no significant seasonal change. The estimates of seasonally adjusted series at the current end will therefore essentially describe current behaviour in the context of the pre-COVID seasonal pattern.
If the pre-COVID seasonal pattern differs from the seasonal influences actually impacting the activity measured in the series, the seasonally adjusted estimates will be more volatile. The irregular component contained in the seasonally adjusted series will reflect the disruption to the existing seasonal pattern. Therefore, the seasonally adjusted estimates of period-to-period change will be less useful to users seeking to understand what recent movements indicate about underlying short-term behaviour.
The ability to account for a change in seasonal influences and the impact on revisions are dependent on the duration of disruption. If the disruption is not a structural change it is challenging to estimate – or even describe – the change to the seasonal component. Quite drastic disruptions to the usual seasonal influences may have a relatively small impact on the seasonal factors if the disruption does not persist beyond around three years. This also means revisions will be relatively small when the existing seasonal behaviour is restored within three years. Elliott, Kirchner and McLaren (2018) provide an empirical example of this, and this is also illustrated in the hypothetical examples presented in Section 4.5. This highlights that revisions are only one aspect to consider when assessing utility.
Estimating a structural break in seasonality
Accounting for a structural change is less challenging than adjusting for short disruptions to the seasonal because there will be (eventually) more data available to estimate the change. Seasonally adjusted estimates will be more reliable to users if the structural break can be identified as early as possible, and so the estimation problem is to estimate the structural change in a timely manner and not confound the structural change with other unrelated variation.
The confusion matrix in Table 2 shows the consequences of taking a conservative or aggressive approach to treating a potential structural break. The matrix illustrates that higher uncertainty of a structural break is associated with higher likelihood of revisions, and that making a correction for a structural break to the seasonal impacts the interpretation of the seasonally adjusted series.
For the task of estimating an appropriate seasonal break, Section 4 discusses the principle of waiting for sufficient information from the observations of the affected series behaviour. Section 5 considers the possibility of an alternative approach to the estimation problem which involves using predictor variables from other time series.
| Real-world effect on series | |||
|---|---|---|---|
| Pandemic causes a structural change (persistence of the disruption observed initially) | Pandemic does not cause a structural change (disruption observed initially does not persist) | ||
| Decision on treatment based on limited observed data | Determine a structural change | Minimise the delay in settling towards the ultimate decomposition, so: | The adjustment for an apparent structural break is later removed, so: |
| • users sooner interpret behaviour in the context which provides clearer signals about future series behaviour | • reduced clarity of short-term direction of series (particularly for those points which reveal the break was not warranted) | ||
| • reduce size of revisions and number of points with significant revision | • Some data points have an earlier revision reversed (initial revision arises when the break is inserted, but is reversed when the break is removed) | ||
| Determine no structural change | Delay identifying the structural change, so: | The correct decision to not react to the early indication of a break means: | |
| • longer period of higher volatility in seasonally adjusted series, reducing clarity about the short-term direction of the series | • avoiding increased volatility across the periods which reveal the disruption was only temporary | ||
| • more points get revised when structural break adjustment is eventually implemented | • avoiding some points being revised in opposite directions | ||
4. Application of treatments at different stages of disruption
4.1 Principles for treating disrupted series
Clear principles are needed to guide decision making about applying corrections and the dissemination of seasonally adjusted and trend outputs. The principles aim to minimise loss of utility and avoid dissemination of outputs which are not informative for the decision making of users.
The principles need to address how to reassess as new information becomes available. Each additional data point in the series can add clarity on the nature of impacts already observed, and as time progresses there may be external information which will change the confidence about expectations of future behaviour. For example, after the first year of the pandemic the government could announce that policies which were initially intended as temporary measures will persist, and this could potentially change the likelihood of an ongoing change to the seasonal pattern.
Table 3 summarises the proposed application of additive outliers, trend breaks and seasonal breaks at different stages of the pandemic. Once the nature of impacts on level are known, there may be scope to improve the quality of seasonal adjustment by applying customised prior corrections which model the trajectory of the series level. For example, Lytras and Bell (2013) describe fitting ‘ramp’ models to describe recession-caused declines lasting at least five months during the GFC. In their analysis such models provided noticeable improvements to the fit of the RegARIMA model but these improvements resulted in only small changes to seasonally adjusted estimates.
| Year number from start of pandemic | Additive outlier* | Trend break* | Seasonal break |
|---|---|---|---|
| First two years | Apply outlier corrections as required, using RegARIMA to help identify and estimate. Subsequent insertion of trend breaks may require removing existing outlier corrections. | Apply after three observations confirm sustained change in level | Do not apply a seasonal break unless there is a strong deterministic reason for ongoing impact to the existing seasonal pattern |
| Third year | Apply outlier corrections as required, using RegARIMA to help identify and estimate. Subsequent insertion of trend breaks may require removing existing outlier corrections. | Apply after three observations confirm sustained change in level | Do not apply a seasonal break unless there is evidence of an abrupt change in seasonal and there is extra supporting information (assessed on a case-by-case basis) |
| Beyond three years | If a seasonal break is added, some historical outlier corrections may no longer be required | Apply after three observations confirm sustained change in level | Apply seasonal break if intervention analysis confirms a significant change in the seasonal pattern |
* Forward factors may be used in first three years. If so, these corrections would be applied at the annual review points
The key principle for prior corrections targeted at the current end of the series is to not assume a structural break until there is a significant amount of supporting evidence from the time series itself or there is external information which provides near certainty that there will be an ongoing change (e.g. an ongoing policy change affecting the timing of government payments). This principle is consistent with the general practice in official statistics of not prognosticating about future events, and avoids the risk of having to reverse a revision previously applied.
In the absence of external information which provides near certainty that a change will be ongoing, the principle means:
- waiting for three periods before implementing a trend break to adjust for an abrupt and sustained level shift, and
- waiting for at least three years before implementing a seasonal break to adjust for an abrupt change in the seasonal pattern.
As noted in Table 3, outlier prior corrections and/or forward factors will be required in the interim before structural breaks can be established and confidently estimated.
The next principle is to apply forward factors while there is heightened uncertainty about the series. Using forward factors instead of concurrent adjustment avoids the requirement to assess corrections every period while still preventing seasonal factor estimates from being distorted by COVID-related disruption. During ‘normal’ times concurrent adjustment has the advantage of providing more immediate seasonal factor improvements, however during periods of disruption the irregular component will be larger and concurrent adjustment has increased likelihood of misinterpreting evolving change in the seasonal pattern (IMF, 2018).
After each year of using forward factors a decision is required about whether to continue with the method for another year. At this review point prior corrections must be applied to treat COVID-affected points in the preceding year, and existing prior corrections need to be reviewed as well. The latest year of data may provide the required evidence to implement a structural break which occurred prior to the preceding year, and in this case the added trend break or seasonal break correction would replace some existing additive outlier corrections. A new set of forward factors is estimated at the review point if it is decided the forward factors method will continue for another year. Beyond three years from the start of the pandemic it is unlikely the forward factors method will be advantageous because by this time COVID-related prior corrections should rarely need to be added or reassessed.
The other two principles concern decisions about suspending publication of the trend and seasonally adjusted series, and are discussed further in Sections 4.2-4.4 in the context of the interpretation and utility at different stages of the pandemic impacts. The principles are summarised below:
- Do not assume a structural break until there is a significant amount of supporting evidence from the time series data or there is external information which provides near certainty there will be structural change.
- Apply forward factors for up to three years until the trajectory of the series has stabilised and there is unlikely to be future major disruption to the series level.
- Suspend publication of the trend series while abrupt changes in level are likely and there are insufficient data points to confirm and measure the size of any trend breaks caused by COVID-19.
- Suspend publication of the seasonally adjusted series during the second and third year of impacts if there is evidence that COVID-19 has triggered a substantial change to the seasonal pattern and there is a real-world reason for this change.
4.2 Interpretation and utility during first year of impacts
In the first year of impacts the utility of the trend series may be significantly diminished, which is why the ABS suspended publication of the trend for most series after the first impacts of the pandemic were observed. The ABS publishes the trend series in addition to the original and seasonally adjusted to convey the ‘underlying level’ of the series. This ‘underlying level’ is unlikely to be estimated with confidence over short periods with abrupt changes, and uncertainty about future changes in series level means there is high likelihood of large revisions in the trend estimates at the current end.
During the first year of impacts the seasonally adjusted estimates at the current end of the series will be in the context of the pre-COVID seasonal pattern. Adjusting for the historical pattern should be useful in the first year of impacts for users seeking to understand the immediate effects of the pandemic. These impacts would be assessed by comparison with pre-COVID observations, and it is desirable for such comparisons to use similar seasonal factors.
4.3 Interpretation and utility during the second and third year of impacts
Publication of the trend series should recommence once there is a stable trajectory for the series and there is confidence that further significant abrupt changes are unlikely.
Disruption to the seasonal influences into the second and third year could significantly reduce the utility of seasonally adjusted estimates produced at this time. Assuming there is no strong external information to justify a seasonal break correction, no seasonal break would be applied and seasonally adjusted estimates produced during the second and third years would be essentially in the context of the pre-COVID seasonal pattern.
When there is disruption to the seasonal influences the utility of the seasonally adjusted estimates will depend on the proportion of the usual seasonal variation which currently affects the series. If there is still a high proportion of the usual seasonal variation, the seasonal adjustment will remove most of the variation and the seasonally adjusted series will assist analysts understand short-term behaviour. However, if a high proportion of the usual seasonal variation is not evident, the current end of the seasonally adjusted series will be noticeably more volatile because the seasonal factors do not reflect the disrupted seasonal influences. If the estimates of period-to-period change are likely to mislead about the short-term direction of the series, suspending publication of the seasonally adjusted series may be the best option during the second and third year of impacts.
4.4 Intervention analysis after three years
After three years there may be sufficient data to estimate a change in the seasonal pattern that has been triggered by the pandemic. The nature of a long-lasting seasonal pattern may be quite different to the seasonal influences during the initial phase of disruption, and in this case more than three years of evidence would be required to reliably estimate the ongoing seasonal pattern. The ability to accurately measure a seasonal break caused by the pandemic will be improved if the “pseudo control” can include the variations which can be predicted from other series.
4.5 Example application of principles
This section discusses treatment of Scenarios A-C presented in Section 3.3. The distinctive feature of Scenario D is a structural break in the seasonal behaviour, and managing structural breaks in the seasonal is considered in detail in Section 5.
Figure 2.1 presents Scenario A, where activity falls to almost zero and then a year of almost zero activity is followed by a two-year recovery to a new level. This scenario mimics the impact on series measuring overseas travel up to mid-2020.
All 12 months of observations between March 2020 and February 2021 are treated by corrections: a trend break adjustment is applied at May 2020 and all other points are treated as additive outliers. (If forward factors are used from March 2020, these corrections do not need to be implemented until February 2021.) Another trend break is required when activity resumes from the extremely low base. Suspension of publication of the trend series should continue for as long as there is uncertainty about the trajectory of the recovery. Publication of the trend could recommence by mid-to-late 2021, if significant future shocks are considered unlikely at this time.
Disruption to the seasonal influences extends into the recovery phase, but the extent of disruption is minor relative to the changes in level during the recovery. For example, the seasonally adjusted movements in January 2022 and January 2023 are weaker than the growth in the trend level, indicating that during the recovery the usual January peaks are relatively less pronounced than usual. During the second and third year of impacts the seasonal adjustment removes a significant proportion of calendar-related variation, and the seasonally adjusted series should continue to be published (with explanation given to users about interpretation). The red and orange lines in Figure 2.2 show the seasonally adjusted estimates produced using data up to February 2022 and February 2023 respectively. Revisions are not particularly large because the seasonal pattern is essentially restored to the pre-pandemic pattern by 2024.
In Figure 3 (Scenario B), the initial impact on the series level is less severe compared with Scenario A, but the disruption to the usual seasonal pattern is more extreme. The SI charts in Figures 3.2 and 3.3, which plot the combined seasonal and irregular components, depict estimation of the December and January seasonal factors. The pink dots are initial SI values which the automatic outlier-correction algorithm in X-11 has replaced with less extreme SI values for seasonal factor calculation purposes. The red lines are the seasonal factor estimates, and are derived as a moving average of the blue SI dots. The appearance of similar adjacent pink dots indicates strong disruption to the usual seasonal influences.
Figure 3.2: SI chart - data up to Feb 2022
Image
Description
Data for Figure 3.2: SI chart - data up to Feb 2022
Figure 3.3: SI chart - data up to Dec 2024
Image
Description
Data for Figure 3.3: SI chart - data up to Dec 2024
In the seasonally adjusted chart the ‘over-correction’ in the adjustment of the December peaks in the original series indicates a significant weakening of the usual seasonal influences between 2020 and 2022. In the SI charts this weakening is represented by the outlying pink dots in December between 2020 and 2022 which are much closer to 1 than usual. Between mid-2020 and the end of 2022 there are many months in which the seasonally adjusted estimate of month-to-month change have been significantly affected by the weakening of the existing seasonal pattern. In real time this would be viewed as evidence of a potential seasonal break. Serious consideration should be given to suspending publication of the seasonally adjusted series if there were no indications during 2021 of a quick return of the previous strong seasonal effects.
To illustrate the revisions to the seasonal factor estimates, the two SI charts compare the seasonal factor estimates produced at February 2022 and December 2024. Despite three consecutive Decembers with lower-than-usual seasonal peaks, the December seasonal factors produced in February 2022 are not noticeably revised. This is because firstly, the automatic outlier-correction algorithm in X-11 replaces the initial SI values with alternative values much closer to the existing December seasonal factor. The other reason is that the existing December seasonal effect gets essentially restored by 2024.
The seasonally adjusted series for Scenario C, a quarterly series, is shown in Figure 4.1. In addition to the large drop at the second quarter of 2020, during 2020 and 2021 the series is considerably more volatile compared with previous years. The SI chart in Figure 4.2 shows the estimation of the seasonal factors when the only corrections are trend breaks at Q2 2020, Q1 2021 and Q3 2022. There are no post-2020 sequences of pink dots above or below the seasonal factor line, which indicates the increased volatility of the seasonally adjusted series is not associated with a systematic change to the seasonal influences. The seasonally adjusted series therefore retains its utility.
The SI chart in Figure 4.3 depicts seasonal factor estimation when all observations in 2020 and 2021 are treated as additive outliers. Comparison of the SI charts shows how the seasonal factor estimates can be affected by the choice of prior corrections, though overall the discrepancies in the seasonal factor estimates are fairly small. Figure 4.2 shows that when no points are treated as additive outliers the automatic outlier correction in X-11 does not treat some points distant from the seasonal factor line and that some replaced SI values are not brought particularly close to the line.
Publication of the trend series should be suspended at the start of the disruption. If there were confidence by mid-2022 that further abrupt changes in the series are unlikely, publication of the trend could recommence at this time.
Figure 4.1: Seasonally adjusted for Scenario C
Image
Description
Data for Figure 4.1: Seasonally adjusted for Scenario C
Figure 4.2: SI chart - only TB corrections
Image
Description
Data for Figure 4.2: SI chart - only TB corrections
Figure 4.3: SI chart - AOs in 2020 & 2021
Image
Description
Data for Figure 4.3: SI chart - AOs in 2020 & 2021
5. Bayesian Structural Time Series Model (BSTS) for intervention analysis
The second part of this paper focuses on identifying possible candidate variables that have predictive power for the target series, and are not affected by COVID-19 in order to improve constructing the “pseudo control” and intervention analysis.
The irregular component contains both measurement errors and real-world short-term shocks and variations. However, it has no contribution to the “pseudo control” because of its zero-expectation assumption. Therefore, the “pseudo control” derived from modelling the time series alone may not be sufficient to explain some predictable variations. For example, an unusual cold winter or hot summer increases additional electricity consumption for heating and cooling. Therefore, a predictive model including other variables could improve the accuracy of the “pseudo control”, and therefore, provide a better intervention measurement.
The contributions from other contemporaneous time series are often modelled as a regression component for the target time series. Autoregressive integrated moving average model with regression (ARIMAX) or RegARIMA are examples of such models in the literature.
Our motivation is to construct a “pseudo control” based on a large set of candidate predictor variables which are not affected by the COVID-19 pandemic. In practice, there are often many such series available, and the challenge is to choose the relevant subset to use as contemporaneous control variables. The classical ARIMAX or RegARIMA is usually not capable for such tasks. Our focus is on new techniques that work well for regression variable feature selection problems.
We propose in this paper using a Bayesian structural time series model (BSTS) introduced by Scott and Varian (2014) for predictor feature selection and time series forecasting. One main building block of the BSTS model is that the target time series dynamic aspect is handled through the Kalman filter (see Harvey, 1990; Durbin and Koopman, 2002; Petris et al., 2009) while taking into account the trend, seasonality, regression, etc.. The second building block is the “spike and slab” variable selection, which was developed by George and McCulloch (1997) and Madigan and Raftery (1994), by which the most important regression predictors are selected. The third building block is the Bayesian model averaging (see Hoeting et al., 1999), which combines the feature selection results and prediction calculation.
Sub-section 5.1 presents a general intervention analysis workflow for intervention measurement, focusing attention on estimating structural seasonal breaks. The seasonal breaks estimated from the structural time series model in this workflow are applied as prior correction factors for the X-11-based approach. The structural time series model is not used to provide the seasonally adjusted estimates, but rather the seasonally adjusted estimates are derived according to the current ABS seasonal adjustment practice.
An empirical case study and simulations are carried out in sub-section 5.2 to examine how our approach performs under various conditions. A summary and further work are presented in sub-section 5.3.
5.1 Model Specification and intervention measurement workflow
The common univariate structural time series model for the observed variable \(\large {y_t}\) without abrupt intervention is defined as
\(\Large {y_t} = {u_t} + {\gamma_t} + {\varphi _t} + {\varepsilon _t},\,\,\,\,\,{\varepsilon _t} \cong NID(0,\sigma _\varepsilon ^2)\) --------- (5.1)
where \(\large {u_t}\) and \(\large {\gamma _t}\) are the unobserved trend and seasonal respectively, \(\large {\varepsilon _t}\,\) is an irregular disturbance term that is often normally and independently distributed with zero mean and variance \(\large \sigma _\varepsilon ^2\). The unobserved trend and seasonal components in the structural time series model can be time-varying and therefore are treated as stochastic variables. The components are modelled using the state equations below.
We propose a semi-local linear trend model for the trend component from the observation equation (5.1)
\(\Large {\mu _t} = {\mu _{t - 1}} + {\nu _{t - 1}} + {\xi _t},\,\,\,\,\,{\xi _t} \cong NID(0,\sigma _\xi ^2),\) --------- (5.2)
\(\Large {\nu _t} = d + \alpha ({\nu _{t - 1}} - d) + {\eta _t}, \,\,\,\,\,{\zeta _t} \cong NID(0,\sigma _\eta ^2).\) --------- (5.3)
This model differs from the local linear trend model (where \(\large d=0\) , and \(\large \alpha = 1 \)) which assumes the slope \(\large \nu_t\) follows a random walk. A stationary AR(1) process in (5.3) is proposed since it is less variable than a random walk when making projections far into the future, and so this model often gives more reasonable uncertainty estimates when making long-term forecasts.
The seasonal component is modelled using the following simple autoregressive model
\(\Large {\gamma_t} = - \sum {_{j = 1}^{s - 1}} {\gamma_{t - j}} + {\omega _t},\,\,\,\,\,{\omega _t} \cong NID(0,\sigma _\omega ^2),\) --------- (5.4)
The regression component \(\Large\;{\varphi _t}\) is specified as a linear model
\(\Large \;{\varphi _t} = {X_t}\beta \) --------- (5.5)
where \(\large {X_t}\) is a time series matrix of predictor variables and/or designed regression matrix, and \(\large \beta \) is the regression coefficient vector which usually needs to be estimated.
We propose a two-phase approach for an intervention measurement process in Figure 5. The purpose of the first phase is to produce the “pseudo control” for identifying the nature of intervention, and this is achieved by a diffusion-regression state-space model which predicts the counterfactual that would have been observed had no intervention taken place. The R packages bsts (Scott and Varian, 2014) and CausalImpact (Brodersen et al., 2015) were utilised to perform Bayesian structural time series model tasks¹.
Figure 5: Workflow of impact and structural change measurement for seasonal adjustment
Image
Description
This workflow depicts a sequence of eight steps for the estimation and application of interventions, and comprises two phases.
The first phase derives the "pseudo control" series and identifies the required interventions. At Step 1, a "pseudo-control" is calculated by predicting the counterfactual result. Step 2 asks the question of whether there are any related time series not affected by the impact of interest, and at Step 3 BSTS is applied. Step 4 concerns the detection and estimation of outlier points and trend breaks. If divergence of evolving trends is detected at Step 4, the check for a seasonal break, at Step 5, requires prior application of a trend filter.
The workflow moves to Phase 2, where at Step 6 BSTS is applied using all of the interventions identified at the first phase. The second part of Phase 2 is performing seasonal adjustment: At Step 7a the interventions estimated from BSTS are applied as prior correction factors, and at Step 7b an X-11 type seasonal adjustment is applied to the prior-corrected time series. Step 8, the final step, is a quality assessment of the seasonal adjustment.
In Phase 1, from STEP 1 to STEP 3a and 3b, the objective is to construct the “pseudo control” for the post intervention period. The purpose of STEP 4a to STEP 5a and 5b is to identify the nature and combination of different types of interventions by exploring the difference between the real observations (treatment) and the “pseudo control”. The focus of this paper is structural change in the seasonal component. In Phase 2, the emphasis is on the estimation of intervention factors after the nature and combination of the interventions are determined (ie. regression matrix of a designed intervention dummy with the inclusion probability value set to 1 as a part of spike-and-slab regression), and X-11 seasonal adjustment after the estimated interventions are applied as prior corrections. To illustrate application of the proposed BSTS model and workflow, ABS Retail turnover survey time series are used as an empirical case study in the next sub-section.
5.2 Empirical case study and simulations
5.2.1 Retail survey data
ABS Retail survey (ABS, cat 8015.0) is a monthly survey as a part of consumption and expenditure activity measurement. Temperatures (ie. seasons) obviously affect retail activity among other factors of seasonal effects such as holidays. Although the regular temperature variation effect is largely captured by the (slowly evolving) seasonal factors, an unusual average temperature in a month can affect retail sales for the month, and the resulting impact is likely to be represented in the irregular component. For the purpose of illustrating the proposed method², we use monthly average maximum and minimum temperatures as potential predictive candidates for the Retail turnover in constructing the “pseudo control” because temperature is not affected by the pandemic. Table 4 shows the locations of monthly average minimum and maximum temperatures sourced from the Bureau of Meteorology.
| Geographic site | Sydney Airport AMO | Melbourne Airport | Brisbane | Adelaide Airport | Perth Airport | Hobart Airport | Darwin Airport | Tuggeranong, ACT |
|---|---|---|---|---|---|---|---|---|
| Station # | 66037 | 86282 | 40913 | 23034 | 9021 | 94008 | 14015 | 70339 |
A monthly temperature series is decomposed into 12 contemporaneous predictor series, one for each month, in order to determine whether temperature in any particular month has predictive power, ie.
\(\Large {\tau _t} {= (\tau _{m,t}) =} \{_0^{\zeta_t}\) \(\Large_{others}^{\,\,\,t\in m}\)
where \(\large{t}\) is the date index, \(\large{m}\) (1,2,3 …, 12) is the month indicator, \(\large {\zeta _t}\) is the temperature recorded at date \(\large{t}\). 192 (=2x12x8) temperature predictor variable series are generated for the monthly average maximum and minimum temperature record from the eight sites. They are labelled as month.type.site. For example, 6m.min.066037 is interpreted as the June average minimum temperature series at the Sydney Airport site.
We transformed the national Retail total turnover series to the logarithmic scale³ first, standardized all the variables involved and fitted the model with spike-and-slab regression using Markov Chain Monte Carlo (MCMC) with 10000 iterations to obtain the posterior distribution of the model hyper-parameters and the regression coefficients. The potentially useful temperature variables are identified on the basis of a high inclusion probability from the MCMC simulation⁴. Figure 6 shows the Retail total turnover series in blue and the two most prominent temperature predictor variables (6m.min.066037 and 7m.max.023034) with inclusion probabilities 0.91, 0.13, and mean coefficients, -1.521624e-02, -2.038996e-03 respectively. The result can be interpreted as an unusual low temperature in June and July (negative values of the scale value) have a positive impact (negative coefficient) to the Retail turnover.
Figure 7.1 shows the “pseudo control” against the real observations after starting COVID-19 intervention (February 2020). The orange shading depicts the 95% posterior probability intervals. Figure 7.2 is the result of subtracting the “pseudo control” from the actual observations during the post-intervention period with a semi-parametric Bayesian posterior distribution for the COVID-19 causal effect. Obviously, with only five observations in the post-intervention period it is too early to make judgement whether COVID-19 has caused a seasonal break. However, the result could be valuable for the assessment of outliers and trend breaks.
The treatment of structural seasonal breaks is the focus of this paper, and to study our proposed method to estimate the seasonal break factors, we need to create a seasonal break in a simulated time series.
5.2.2 Simulation study setup
Australia has suffered multiple interventions since the start of 2020 which could impact Retail turnover, such as drought, bushfires, and the COVID-19 pandemic, and estimating their individual effects is complicated. For the purpose of our simulation study, we used the data only up to December 2019 in this section.
A seasonal break vector was randomly generated as
| Jan | Feb | Mar | Apr | May | Jun | Jul | Aug | Sep | Oct | Nov | Dec |
|---|---|---|---|---|---|---|---|---|---|---|---|
| -0.037 | -0.0417 | 0.105 | 0.2254 | 0.0722 | -0.0731 | -0.1493 | 0.1602 | -0.0309 | -0.1168 | -0.0046 | -0.1094 |
and applied from January 2017 to December 2019 ie. the seasonal patterns for these three years are very different to the previous years. Figure 8 shows the simulated data in dark blue against the 'real-life' observed series on the logarithmic scale, where the vertical black line indicates the start of seasonal break.
For the purpose of the simulation study, we label the three series to distinguish their differences for the rest of this section.
- “Benchmark”: the original observed series without seasonal break
- “Real observations”: the simulated series based on the “Benchmark” series with seasonal break starting from Jan 2017
- “Pseudo control”: the predicted series commencing from Jan 2017, modelled from the “Real observations” prior to Jan 2017
Figures 9.1 and 9.2 show the consequences of failing to account for the seasonal break. Figure 9.1 compares the seasonal factor estimates of the “Real observations” (in dark blue) with the actual seasonal factor (in light blue). The seasonal break not only affects the quality of seasonal factor estimates after the seasonal break (denoted by the black vertical line), but also creates revisions and residual seasonality prior to the seasonal break. Figure 9.2 shows the differences between the seasonal factors. We can easily conclude that the utility of seasonally adjusted estimates is in doubt if the seasonal break is not accounted for.
5.2.3 Identifying seasonal breaks by “pseudo control” (STEP 4, 5a and 5b)
Pretending the “Benchmark” were unknown, we constructed a “pseudo control” using BSTS with spike-and-slab regression on the contemporaneous temperature predictor variables. Figure10.1 presents the “pseudo control” as the result of BSTS model prediction, with its posterior 95% confidence range shown in red. Figure 10.2 compares the “Benchmark” and “Pseudo control”, and Figure 10.3 plots the difference between the two series.
It appears that the “Pseudo control” is reasonably accurate for short run, but diverges from the “Benchmark” after 6 periods (months). This result is not a surprise because the accuracy of most time series model forecasts deteriorates over longer ranges (see Figure 10.1). In this case, applying a semi-local linear trend and contemporaneous regression did not prevent deterioration of the longer term forecast accuracy, and this would cause distortion for identifying a seasonal break pattern from the difference between the “Benchmark” and “Pseudo control”. Detrending both the “Real observations” and “Pseudo control” can mitigate the long-range forecast problem. There are different filters and ways of applying them to perform de-trending. Various detrending filters including the Direct Filter Approach (Wildi, 2007) have been explored⁵. We found the Hodrick-Prescott (HP) filter has the desired frequency response property, is practically robust and is reliable subject to a seasonal break.
Figures 11.1 and 11.2 show the result from applying an HP filter with the standard smoothing parameter 14400 for monthly time series to both the “Pseudo control” and “Real observations”.
Figure 11.1 presents the trends of the “Real observations” and “Pseudo control” after HP filtering. Figure 11.2 presents the detrended series. It can be seen that the respective seasonal patterns are persevered well during intervention periods. Figure 11.3 overlays the difference between the detrended “Real observations” and “Pseudo control” and the known structural seasonal break. This direct comparison enables assessment of the extent of the structural seasonal break. The close alignment demonstrates that the differences of the detrended series are better in revealing the true seasonal break than its counterpart presented in Figure 10.3.
This simulation study demonstrated the improved identification of a structural break from applying the HP filter to both the “Pseudo control” and “Real observations”. Once the seasonal break is identified, a more sophisticated model can be specified with the suitably-designed seasonal break dummy matrix and other designed regression variables to estimate the structural changes along with the contemporaneous predictor variables. Furthermore, the prior inclusion probabilities for the structural change regression matrix were set as 1 because they have been firmly identified from STEP 4 to STEP 5b. We use a simulated series of Retail turnover for Department stores to illustrate how the seasonal break can be estimated by the BSTS method.
5.2.4 Simulated Department store example (for STEP 6)
Retail turnover for Department store data from January 2000 to December 2019 were used for this simulation study. A simulated “Real observations” was generated by applying the same seasonal factor structural change as in the previous section.
A structural seasonal break dummy matrix with 12 columns was created as
\(\Large {\chi_t} = ({\chi_{m,t}}) = \left\{ {_0^1} \,\,\,{_{t < 2017.01\,\,or\,\,t \notin m}^{\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,t\in m}} \right.\)
where \(\large {m}\) (=1,2,3…,12) is the month index, and \(\large {t}\) is the date index.
The whole regression matrix is constructed by concatenating the seasonal structural change dummy matrix \(\large {\chi _t}\) and the temperature predictor variables \(\large{\tau _t}\) ie. \(\large {X_t} = ({\chi _t}:{\tau _t}).\)
The prior inclusion probability vector for the regressor matrix \(\large{X_t}\) is set as \(\left( {{1_{12}}:{k_{192}}} \right)\) where \(\large {k}\) is derived from the expected size of selected temperature predictor variables divided by the number of temperature predictor variables. We used a default expected size of 3. Therefore, \(\large {k}\) = 3/192 = 0.015625⁶. We set the inclusion probability to 1 for the structural seasonal break dummy matrix. The spike-and-slab regression algorithm estimates the seasonal break coefficient and inclusion probability for each temperature predictor variable by MCMC.
The chain stability of the MCMC were evaluated based on the Geweke, Heidelberger-Welch and Raftery-Lewis tests (Tsai and Gill, 2012) for the model hyper-parameters⁷ in equations 5.1 – 5.4 (variance of observation \(\large (\sigma _\varepsilon ^2)\), variance of trend level \(\large (\sigma _\xi ^2)\), mean of trend slope \(\large (d)\), trend AR coefficient \(\large (\alpha )\), variance of trend slope \(\large (\sigma _\eta ^2)\), and variance of seasonal component \(\large (\sigma _\omega ^2)\)), and visual examination of their traces and densities. The Raftery-Lewis diagnostic also suggests a 5000 MCMC sample is sufficient to produce stable posteriors for the hyper-parameters because there are multiple hyper-parameters.
Figures 12.1 and 12.2 show the result of the BSTS with spike-and-slab regression on the Department store data with the above prior inclusion probability vector, a default cut-off of 10% posterior inclusion probability and 10000 MCMC iterations. The structural seasonal break dummies are labelled as S.month.# for month #.
The Figure 12.1 shows the posterior inclusion probabilities of the regression variables. The temperature predictor variable inclusion probabilities are listed in descending order. The two top temperature variables are M07m.max.066037 (July average maximum temperature at Sydney Airport, and M04m.max.070339 (April average maximum temperature at ACT). The colours of the horizontal bar indicates the positive (orange) and negative (light blue) coefficients (the details are presented in Table 5).
The Figure 12.2 illustrates the frequency of the selected regression variables in the 10000 MCMC iterations. It appears that model size less than 17 (ie. less than 6 temperature predictor variables) took about 75% cases.
Figures 13.1 and 13.2 present the posterior distributions of the modelled seasonal and the combined regression components from the BSTS with spike-and-slab regression. The yellow shading range illustrates the 95% posterior distribution.
| Quality assessment | |||||
|---|---|---|---|---|---|
| Residual SD | Prediction SD | R-Square | Relative GOF | ||
| 2.01E-02 | 6.35E-02 | 0.993291 | 0.946443 | ||
| Regression Size | |||||
| Min. | 1st Qu. | Median | Mean | 3rd Qu. | Max. |
| 14 | 15 | 16 | 16.04 | 17 | 25 |
| Regression Coefficients | |||||
| mean | sd | mean.inc | sd.inc | Inc.prob | |
| S.month.12 | -1.37E-01 | 3.98E-02 | -1.37E-01 | 3.98E-02 | 1.00E+00 |
| S.month.11 | 1.39E-02 | 4.13E-02 | 1.39E-02 | 4.13E-02 | 1.00E+00 |
| S.month.10 | -1.26E-01 | 4.22E-02 | -1.26E-01 | 4.22E-02 | 1.00E+00 |
| S.month.09 | -4.19E-02 | 4.07E-02 | -4.19E-02 | 4.07E-02 | 1.00E+00 |
| S.month.08 | 1.58E-01 | 4.15E-02 | 1.58E-01 | 4.15E-02 | 1.00E+00 |
| S.month.07 | -1.70E-01 | 4.48E-02 | -1.70E-01 | 4.48E-02 | 1.00E+00 |
| S.month.06 | -7.43E-02 | 4.31E-02 | -7.43E-02 | 4.31E-02 | 1.00E+00 |
| S.month.05 | 9.57E-02 | 4.04E-02 | 9.57E-02 | 4.04E-02 | 1.00E+00 |
| S.month.04 | 2.61E-01 | 4.26E-02 | 2.61E-01 | 4.26E-02 | 1.00E+00 |
| S.month.03 | 9.02E-02 | 3.97E-02 | 9.02E-02 | 3.97E-02 | 1.00E+00 |
| S.month.02 | -5.73E-02 | 3.97E-02 | -5.73E-02 | 3.97E-02 | 1.00E+00 |
| S.month.01 | -7.71E-02 | 4.01E-02 | -7.71E-02 | 4.01E-02 | 1.00E+00 |
| M07m.max.066037 | -1.86E-02 | 1.22E-02 | -2.53E-02 | 5.75E-03 | 7.35E-01 |
| M04m.max.070339 | -1.56E-02 | 1.10E-02 | -2.14E-02 | 6.37E-03 | 7.29E-01 |
| M03m.min.066037 | -3.41E-03 | 6.81E-03 | -1.56E-02 | 4.68E-03 | 2.18E-01 |
| M04m.max.086282 | -5.61E-03 | 1.19E-02 | -2.72E-02 | 9.90E-03 | 2.06E-01 |
| M03m.max.066037 | -1.86E-03 | 4.52E-03 | -1.14E-02 | 4.02E-03 | 1.63E-01 |
| … | … | … | … | … | … |
Columns ‘mean.inc’ and ‘sd.inc’ in Table 5 are the average and standard deviation of the regressor’s coefficient when it is included in the MCMC. Column ‘Inc.prob’ is the posterior inclusion probability of the variable. Column ‘mean’ and ‘sd’ are the posterior Bayesian model average coefficient and its standard deviation respectively, and are used for predictions and associated inferences.
The structural seasonal break factors can then be derived from the estimated coefficients by re-centring them at their average to satisfy the seasonal constraints (ie. sum of them equal to zero). Figure 14 compares the estimated structural seasonal break factor (labelled as bsts.bs.dummy and est.sb.dummy_TM for the two different models including (1) seasonal dummy only and (2) seasonal dummy with temperatures respectively) and the real seasonal break factors (labelled as sb).
From visual examination of Figure 14, the estimated seasonal breaks from the model with seasonal dummy and temperature out-performs the model with seasonal dummy only. Their errors against the true seasonal break factor measured by the standard deviation against the 12-month true seasonal break factors are 0.075 and 0.023 respectively. It appears that the estimates of the structural seasonal break factors from the model with seasonal dummy and temperature are of reasonable quality with only three years of post-intervention periods, which is equivalent to only three observations for each month.
We now consider whether the extra temperature predictor variables improve the model fitting as well as the accuracy of the estimates of the structural seasonal break factors. We conducted the same simulations as the Department store example for other retail classifications with regression on
- only the structural seasonal break dummy (SB only);
- the structural seasonal break dummy and temperature predictor variables (SB+TM).
Table 6 presents the five quality measures from the BSTS with spike-and-slab regression models.
| Retail classification | Regression | Residual SD | Predict SD | R-Square | Relative GOF⁸ |
|---|---|---|---|---|---|
| Food | SB only | 1.30E-02 | 1.91E-02 | 9.98E-01 | 9.56E-01 |
| SB+TM | 1.30E-02 | 2.51E-02 | 9.98E-01 | 9.24E-01 | |
| Household goods | SB only | 1.74E-02 | 3.65E-02 | 9.95E-01 | 9.14E-01 |
| SB+TM | 1.69E-02 | 4.09E-02 | 9.96E-01 | 8.93E-01 | |
| Clothing, footwear and personal accessory | SB only | 2.02E-02 | 4.87E-02 | 9.95E-01 | 9.39E-01 |
| SB+TM | 1.91E-02 | 5.93E-02 | 9.95E-01 | 9.10E-01 | |
| Department store | SB only | 2.44E-02 | 6.79E-02 | 9.90E-01 | 9.39E-01 |
| SB+TM | 2.01E-02 | 6.35E-02 | 9.93E-01 | 9.46E-01 | |
| Other retailing | SB only | 1.48E-02 | 3.41E-02 | 9.97E-01 | 9.47E-01 |
| SB+TM | 1.44E-02 | 3.83E-02 | 9.98E-01 | 9.33E-01 | |
| Cafes, restaurants and takeaway food services | SB only | 1.50E-02 | 2.46E-02 | 9.98E-01 | 9.16E-01 |
| SB+TM | 1.50E-02 | 2.57E-02 | 9.98E-01 | 9.08E-01 | |
| Total (Industry) | SB only | 1.38E-02 | 2.69E-02 | 9.98E-01 | 9.52E-01 |
| SB+TM | 1.34E-02 | 3.03E-02 | 9.98E-01 | 9.38E-01 |
One of the common features is that BSTS with “SB+TM” has a better in-sample fitting reflected in smaller “Residual SD” and larger “R-Square” against its counterpart BSTS with “SB only”. However, in terms of prediction reliability (measured by “Predict SD”) and predictive power (measured by “Relative GOF”), the results are not consistent. Only for Department store does BSTS with “SB+TM” improve both prediction quality measures compared with its counterpart. The prediction qualities are somewhat reduced for the rest of retail classifications. It indicates that even the temperature predictor variables selected by the spike-and-slab regression may not be reliable predictors for the rest of retail classifications.
The above results do not suggest deficiencies with the BSTS with spike-and-slab method itself, but rather that temperature variables are probably not good predictors except for the Department store series. In addition, the results also suggest that the prediction quality measures (“Predict SD” and “Relative GOF”) can be utilised to determine whether any related predictor variables should be included for constructing the “pseudo control” and structural change factor estimation in general.
5.3 Summary
With the present paper we established the link between intervention analysis and a counterfactual prediction as a "pseudo control". We proposed a Bayesian Structural Time Series (BSTS in state-space form) model for the construction of the "pseudo control" to exploit the predictive signals in a large set of potential predictor variables by spike-and-slab regression.
We also established a practical workflow for (1) measuring the COVID-19 impact (or any intervention) as a causal impact inference problem, and (2) estimating the associated structural changes.
One of the practical problems in revealing a structural seasonal break is the reliance on long-range prediction accuracy which is usually hard to achieve beyond a few periods. We proposed to use the HP filter for detrending, which appears effective for removing the effect of the diverging trends between the real observations and "pseudo control".
From the simulation study on the Retail turnover and the temperature data, our empirical application shows that the BSTS with spike-and-slab regression model can improve "pseudo control" accuracy and is capable of estimating (seasonal) structural change. For simplicity, we did not include trading day, non-regular holidays (e.g. Easter, Ramadam), survey related measurement error, or another type of structural change (e.g. increasing volatility due to heteroscedasticity) in the current model, but hope the results could be improved after their inclusion.
6. Conclusion
Managing the impacts of COVID-19 on time series will require ongoing monitoring to ensure seasonal factors are estimated appropriately. Identifying a structural break in the seasonal behaviour will be difficult in the first couple of years from the onset of the pandemic, and we recommend a conservative approach to wait for conclusive evidence before estimating and inserting seasonal breaks. Users can expect increased volatility at the current end of the seasonally adjusted series until a seasonal break can be applied to treat a structural break or the pre-COVID seasonal pattern is largely restored.
The increased volatility arising from changed seasonal influences can reduce the utility of the seasonally adjusted series. Users should be warned about suspected impacts on the seasonal influences and about how the disruption can impact series volatility. Suspending publication of seasonally adjusted estimates may need to be considered in some cases. For each particular series, decisions about prior corrections and suspending publication of the seasonally adjusted series will need to consider decisions for related series in the interests of coherence. Similar treatments should be applied to related series impacted in a similar fashion.
We have illustrated the BSTS method for incorporating predictive series which are not affected by the pandemic. The method is used to assist identifying and estimating any structural change in the seasonal component by constructing a better quality “pseudo control” used for intervention analysis. BSTS with the spike-and-slab regression method provides a very efficient approach to select the most powerful predictive variables.
Initial simulation work suggests the BSTS method provides improvement in prediction accuracy for estimation of seasonal break factors, where the example used a temperature-related predictor for a series on Department store Retail sales. The use of temperature data was only for illustrative purposes, and it may be possible to improve results using additional series with prediction potential. Further improvement in the results may also be possible if the BSTS modelling also accounts for other calendar-related effects such as trading day, and for increased heteroscedasticity. The main challenge for the application of this innovative method will be identifying the potential predictive variables not affected by COVID-19 in the first place.
Feedback
Feedback on this paper is welcomed and can be provided to methodology@abs.gov.au.