Methodological News, June Quarter 2024

This issue contains two articles:

• Optimising online and phone pathways for completing the Monthly Population Survey
• Preliminary investigations into determining causal relationships between variables using convergent cross mapping

The ABS encourages households selected for the Monthly Population Survey (MPS) to complete their survey either online or over the phone with an ABS interviewer. Currently, both the online and phone pathways for completing the survey are not an optimal experience for providers.

Known pain points include:

• Limited self-service options to assist with online survey completion. Data providers need to call the ABS during business hours for assistance if they encounter technical issues. 
• Data providers who call the ABS expecting to complete their survey on the spot  often need to wait for a call back to provide their data. This call will often be at an unspecified date and time. 
• Unclear transitional pathway for those who completed over the phone in the first month and wish to complete online in subsequent months (during the 8 months they are in the survey). 

For this project, the team designed new strategies to improve the online and phone pathways for completing the MPS. We trialled these strategies with a sub-group of providers in the MPS to gauge their effectiveness. 

Strategies designed and trialled included: 

1. A new online support page on the ABS website, which included video instructions on how to complete online, as well as an online survey help form to request technical assistance. 
2. Offering data providers who call up the option to complete the survey on the spot with an ABS interviewer where possible. Where this wasn’t possible (because the survey was not yet open), providers were able to set an appointment at their preferred date/time with an ABS interviewer. We then sent confirmation and reminder notifications for their upcoming appointment, in line with industry standards. 
3. Updated scripting at the end of the phone survey to promote the benefits of completing the survey online in subsequent months, and updated support materials for providers transitioning from phone to online survey completion.

Key findings from the trial of the new strategies included:

• Most providers (around 65%) who sought information on how to complete their survey online opted to access this information through the newly designed online support webpage. This finding confirms the importance of optimising an online channel for technical support.
• Most providers (around 84%) who called up when the survey was open opted to complete the survey on the spot with an interviewer. When survey completion on the spot wasn’t possible (i.e. survey was not yet open), more than half of the providers (around 53%) opted to set an appointment for a convenient later date with an interviewer. 
• Updating the scripting at the end of the phone survey to promote the benefits of completing the survey online in subsequent months resulted in more providers opting online for the following month (updated scripting = around 21% opted online; standard scripting = around 14% opted online). Furthermore, the improvement to the support materials meant more providers followed through on their intention to complete online (updated support materials = around 48% completion; standard support materials = around 43% completion).

The project team is now working with operational areas within the ABS to implement these new strategies for future MPS cycles. Furthermore, the findings from the trial will provide critical historical benchmarks for continuing to iteratively improve the online and phone pathways.

For further information, please contact Yvette Kezilas and Katherine Birrer at  methodology@abs.gov.au.

We have begun exploring a relatively new analysis method called convergent cross mapping (CCM).  Unlike traditional statistical analysis methods that can only determine correlation or association between variables, CCM can identify causal relationships between variables within a non-linear dynamic system using rich time-series data.    

 While it is a method mostly used in ecological studies, we believe the inclusion of CCM into our analytical toolkit could help uncover deeper insights into the causal mechanisms driving socio-economic trends, leading to more evidence-based decision-making and policy development.  CCM could also be useful for a more robust program or impact evaluation within complex, dynamic systems. 

 As its name suggests, CCM relies on observing convergence in the accuracy of predicted values of one variable made through cross mapping from another variable to determine a causal relationship between those two variables.  It does this by leveraging the Takens’ embedding theorem to reconstruct the state space of a non-linear system from singly observed time-series.  This reconstruction preserves the intrinsic properties of the system such that, if X causes Y within the system, then the reconstructed states using historical values of Y will be able to predict values of X (cross mapping).  The accuracy of the predictions measured by the cross-mapping skill (that is, the correlation between observed and predicted values of X) should also increase as more data (that is, a longer time series) is used to reconstruct Y (convergence).   The method can identify both uni-directional (X causes Y or Y causes X) and bi-directional (X causes Y and Y causes X) causal relationships. 

 In our initial experiments, we applied CCM to four variables related to the labour market:  the consumer price index, the cash rate, the unemployment rate, and the participation rate.  We viewed the labour market as a potentially suitable non-linear dynamic system for this exploration because it contains an abundance of interacting entities, such as employers and employees, and various economic and social factors and variables available for analysis. 

 Our preliminary investigations are yielding some important lessons, as follows:  

• Data requirements 

CCM does not require time-series data to comply with specific distributional assumptions, but careful consideration must be given to the relative scaling, non-linearity, and the assumption that the variables belong to the same dynamic system.  The efficacy of CCM to accurately infer causality between two variables improves with longer time-series data.  While literature often uses data with 300-500 time points, we found that high-quality data with around 150 monthly time points can be sufficient to identify causal relationships. 

• Sensitivity to embedding parameters 

Reconstructing state spaces from time-series data requires selecting appropriate embedding parameters.  The performance of the CCM method can be sensitive to these values.  We are currently investigating optimal parameter selection techniques and their impact on identifying causal relationships between variables.  

• Types of causality 

We are undertaking further work to better understand the effect of causal phenomenon such as direct and indirect causality, and strong forcing, where one variable can saturate the information in a system and lead to incorrect interpretations of causality.  Some recent applications in the literature extend the bivariate CCM method to multivariate cases and results indicate some improvement in identifying direct and indirect causality.  We are also examining the application of the CCM method to specific subsets of the labour market and different economic contexts to reduce complexity and improve the interpretability of identified causal relationships. 

 For further information, please contact Anderson Yang and Susan Fletcher at  methodology@abs.gov.au.

Please email methodology@abs.gov.au to:

• contact authors for further information
• provide comments or feedback
• be added to or removed from our electronic mailing list.

Alternatively, you can post to:

Methodological News Editor
Methodology Division
Australian Bureau of Statistics
Locked Bag No. 10
Belconnen ACT 2617

The ABS Privacy Policy outlines how the ABS will handle any personal information

Releases from June 2021 onwards can be accessed under research.

Releases up to March 2021 can be accessed under past releases.