3401.0 - Overseas Arrivals and Departures, Australia, May 2017 Quality Declaration 
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 14/07/2017   
   Page tools: Print Print Page Print all pages in this productPrint All

APPENDIX 2 DATA QUALITY ISSUES


1. INTRODUCTION

Administrative information on persons arriving in, or departing from, Australia is collected via various processing systems, passport documents, visa information, and incoming and outgoing passenger cards (see Appendix 1). Incoming persons provide information in visa applications except those travelling as Australian or New Zealand citizens. These administrative data are collected by the Australian Government Department of Immigration and Border Protection (DIBP) under the authority of the Migration Regulations (Migration Act, 1958).

ABS statistics on overseas arrivals and departures (OAD) are compiled using information from DIBP sources. Overseas movements are collected and matched (where possible) by DIBP and then stored with movement records on the Travel and Immigration Processing System (TRIPS). Each month the matched OAD movement records are supplied to the ABS and then processed. From October 2014, due to an increase in missing passenger cards, both matched and unmatched OAD movement records are now supplied to the ABS for processing within the OAD system.

The purpose of this appendix is to provide a reference of the various changes that occur from time to time that may impact on the quality of OAD statistics. These changes can be due to any part of the end to end processing, from passenger data collection to the output of OAD statistics. It can range from the design, provision and collection of the passenger cards through to the administrative systems and updates at DIBP. The changes can also result from better capture of passenger data, methodological improvements or improved processing systems.

In particular, this appendix provides information on changes to the OAD processing system including the imputations undertaken. It also provides a reference to any historical changes that have impacted the data over time.

An example of the current Australian passenger card (Passenger Card Appendix 1) is provided with the monthly Overseas Arrivals and Departures (cat. no. 3401.0) release under the Explanatory Notes tab.

2. REBUILD OF THE OAD SYSTEM

In 2013, the ABS completed a rebuild of the system which creates OAD data. The primary aim of this project was to improve the quality of OAD data, given its importance as an input to a broad range of statistical outputs. The new system has been thoroughly tested by processing over ten years of data. Imputations for missing traveller data have been improved, in particular the duration of stay and the country of birth variables.

The rebuild of the system has resulted in a break in series, necessitating a revised time series for OAD data based on the improved methodology from July 2004 to December 2013. The break in series is from July 2004.

3. UPGRADE TO THE OAD SYSTEM DUE TO INCREASES IN MISSING PASSENGER CARDS

The ABS has upgraded the OAD system due to two changes at DIBP that have resulted in increases in missing passenger cards over time.

1. From October 2014 to May 2015, DIBP trialled a different provider for the processing of passenger cards. During this period there was a higher number of cards missing than expected.
2. From August 2015 onwards, DIBP progressively rolled out Departure SmartGates at the international airports. An increase in missing departure cards was observed throughout the roll-out.

To accommodate the increases in missing passenger cards the ABS completed an upgrade to the OAD system in March 2016 to introduce new methods and maintain data quality. The upgrade includes the use of additional sources of data. These include: an individual's movement information from TRIPS where no card was received (i.e. an unmatched record); retrieving and using information from an individual's most recent known future movement (movement after the reference movement) or historical movement (movement before the reference movement) where appropriate; and flight information to ascertain an individual's country of embarkation or disembarkation. This will maintain the quality of most variables and has reduced reliance on the use of imputations. For further information on each variable and the changes applied see Section 5, 'Overview of Variables Imputed' below.

Based on the new methods, a revision of OAD data has been undertaken from October 2014 onwards to accommodate for the increases in missing passenger card data. From October 2014 the ABS has adjusted for missing passenger card data each month - see Table A4 below.

4. DATA IMPUTATIONS

Improved Imputations from July 2004

Through the process of the rebuild of the OAD system, all derivations, logical edits and imputations were re-designed based on the best information, practices and methodology available.

There are a number of imputations undertaken that improve the quality of variables with missing responses. The variables which are subject to imputation for missing values are: age, country of birth, country of citizenship, country of embarkation/disembarkation, country of residence/stay, duration of stay, passenger card box type, reason for journey, sex and state of residence/stay, and a specific one for the country of birth of New Zealand (NZ) citizens. All missing values for these variables are fully imputed except country of citizenship and the country of residence/stay for permanent arrivals. All imputations within the OAD system have been undertaken using a 'hot deck' imputation method.

For hot deck imputation, a record with missing responses (called the recipient), receives those of another similar record (called the donor) which has a full set of responses before the imputation process began. The recipient record keeps all of its original responses and only has the missing responses imputed, thereby keeping as much of the collected information for that record as possible.

The hot deck imputation method uses a set of characteristics that choose the donor and recipient records which are as similar as possible. The characteristics used within the OAD system to align a recipient with a suitable donor, vary between the different imputations. A combination of different characteristics was tested for each of the imputations to ascertain which would give the best results. The characteristics used include: age, country of birth, country of citizenship, country of residence/stay, direction of traveller, category of movement, passenger card box type, reason for journey, sampled or non-sampled data, sex, state of clearance and visa group.

For details on each variable being imputed see the next section 5. Overview of Variables Imputed.

Missing responses

Table A1 below shows the missing response rates for the current month. The rate is a percentage of the total relevant records. Total relevant records are a combination of all permanent and long-term movements plus the sample of short-term movements. From January 2005, approximately 5% of all short-term movements have been selected for sample, see Explanatory Note 9.


A1 MISSING RESPONSE RATES PRIOR TO, AND AFTER IMPUTATION(a) - May 2017

Pre-imputation
Post-imputation
OAD Variable
%
%

Age
0.04
-
Country of birth
(b)5.18
-
Country of Citizenship
0.26
(c)0.22
Country of embarkation/disembarkation
6.03
-
Country of residence/stay
14.11
(d)3.66
Duration of stay
6.58
-
Passenger card box type
(e)0.99
-
Reason for journey
7.20
-
Sex
0.26
-
State or territory of residence/stay
7.08
-

- nil or rounded to zero (including null cells)
(a) Not all variables are imputed, including marital status, occupation, port of clearance, state of clearance and visa type.
(b) For country of birth, the majority of imputations are for New Zealand citizens. See 'Specific imputation for country of birth of New Zealand citizens' in this Appendix.

(c) Country of citizenship is imputed only if partial information is available. For example if Europe was supplied as the nationality on the passenger card it is then imputed to a country in Europe.
(d) Country of residence/stay is not imputed for permanent arrivals. All other movements are fully imputed.
(e) The rate for the passenger card box type is based on all movements not just the relevant records.

Missing responses for country of residence/stay

A further breakdown in Table A2 shows the proportion of responses missing for each passenger card box type.

A2 COUNTRY OR RESIDENCE/STAY — PROPORTION OF MISSING RESPONSES FOR EACH PASSENGER CARD BOX TYPE

May 2017
Box type
%

A: Migrating permanently to Australia(a)
76.23
B: Visitor or temporary entrant
9.41
C: Resident returning to Australia
9.25
D: Visitor or temporary entrant departing
9.39
E: Australian resident departing temporarily
19.14
F: Australian resident departing permanently
19.48

(a) Country of residence/stay is not collected from the passenger card for permanent arrivals. However, some information is collected from some permanent arrival visas.


5. OVERVIEW OF VARIABLES IMPUTED

Age

The primary source for data on this variable is passport or visa information. An alternate source used is the passenger card. Age is calculated using date of birth.

For imputation, the variables used to align the recipient with a suitable donor are: the passenger card box type and reason for journey. Generally, little imputation is required for age (less than 1% of relevant records). All relevant records are fully imputed for this variable. See section 4. 'Data Imputations' above.

Country of birth

The primary source for data on this variable is passport or visa information if available. It is not available from the passenger card. The majority of imputations are for NZ citizens.

There are two separate parts to the imputation for country of birth. A specific imputation is in place for the country of birth of New Zealand (NZ) citizens, as data for this variable is not directly available from the passport or visa of NZ citizens. For details see 'Specific imputation for country of birth of New Zealand citizens' in Section 6 below. The second part is only used for non-NZ citizens. For this imputation, the variables used to align the recipient with a suitable donor are: category of movement and country of citizenship. Generally, imputation required for country of birth is less than 7.5% of relevant records. All relevant records are fully imputed for this variable. See section 4. 'Data Imputations' above.

Country of Citizenship

The primary source for data on this variable is passport or visa information. An alternate source used is the passenger card.

Country of citizenship is not imputed unless partial information is supplied on the passenger card. For example, if no official nationality is available and Europe was supplied as the nationality on the passenger card it would then be imputed to a country in Europe. The specific region or country grouping provided as the recipient's citizenship is used to align the recipient with a suitable donor for imputation. For the example noted above the donor would be from Europe. Generally, little imputation is required for country of citizenship (less than 1.7% of relevant records). See section 4. 'Data Imputations' above.

Country of embarkation/disembarkation

The primary source for data on this variable is the passenger card. When the passenger card is missing, the alternate source is the flight schedule information provided by DIBP. Prior to October 2014 the only available source for data on this variable was the passenger card.

There are two separate parts to the imputation for country of embarkation/disembarkation. The first part is used if a response is missing. For this imputation the variables used to align the recipient with a suitable donor are: sampled/non-sampled data and country of residence/stay. The second part is only used if partial information is supplied on the passenger card. For example, if Europe was supplied as the country of embarkation/disembarkation on the passenger card it is imputed to a country in Europe. The specific region or country grouping provided as the recipient's country of embarkation/disembarkation is used to align the recipient with a suitable donor for imputation. For the example noted above the donor would be from Europe. Generally, imputation required for country of embarkation/disembarkation is less than 8% of relevant records. All relevant records are fully imputed for this variable. See section 4. 'Data Imputations' above.

Country of residence/stay

The primary source for data on this variable is the original passenger card. When a departure card is missing an alternate source used is an individual's secondary passenger card. For example, the vast majority of travellers (excluding permanent migrants) have two legs to their journey, either an arrival followed by a departure or, vice versa, a departure followed by an arrival. Retrieving an individual's most recent other movement (where possible) allows for the country of residence/stay to be collected from that same individual's secondary passenger card. Another alternative data source is visa information which may be used for some travellers when available. Prior to October 2014, the only source for data on this variable was the original passenger card (from the reference movement), and alternately visa information for some travellers.

There are two separate parts to the imputation for country of residence/stay. The first part is used if a response is missing. For this imputation the variables used to align the recipient with a suitable donor are: sampled/non-sampled data, country of citizenship, category of movement and reason for journey. The second part is only used if partial information is supplied on the passenger card. For example, if Europe was supplied as the country of residence/stay on the passenger card then it is imputed to a country in Europe. The specific region or country grouping provided as the recipient's country of residence/stay is used to align the recipient with a suitable donor for imputation. For the example noted above the donor would be from Europe. Generally, imputation required for country of residence/stay is less than 16% of relevant records. Country of residence/stay is not imputed for permanent arrivals but all other relevant records are fully imputed. See section 4. 'Data Imputations' above.

Prior to the rebuild of the OAD system and the revision back to July 2004, the ABS imputed this data item in two stages. In the first stage, records with country of residence/stay missing were set to country of disembarkation/embarkation if a response was available. In the second stage, for remaining records where country of stay/residence was missing, values were imputed at the category of movement, reason for journey and country of citizenship level based on responses to other cards within each subgroup. For permanent arrivals, imputation was undertaken using a combination of country of embarkation and the stated responses of other permanent arrivals.

Duration of stay - current

Data on this variable is from two separate sources depending on whether the reference movement is the first leg, or the second leg of a travellers' journey.
  • For the first leg of a journey the primary source available is the passenger card and is based on a travellers' intended duration of stay. For example, long-term visitor arrival (LTVA), short-term visitor arrival (STVA), long-term resident departure (LTRD), and short-term resident departure (STRD) are all the first leg of a journey and the duration of stay is based on intention. There is no alternate source used for duration of stay from the first leg of a journey.
  • For the second leg of a journey the primary source available is administrative systems at DIBP which measures the exact date a traveller crosses the Australian border and therefore accurately records a travellers' duration of stay on the second leg of their journey. For example, long-term visitor departures (LTVD), short-term visitor departures (STVD), long-term resident returns (LTRR), and short-term resident returns (STRR) are all the second leg of a journey and the duration of stay is recorded by DIBP. From October 2014, if a departure card data was missing and a box D (visitor or temporary entrant departing) had been identified, the ABS processing systems would replicate the same procedure as in DIBP administrative systems and calculate the exact duration of stay based on an individual's previous border crossing.

    The quality for actual duration of stay recorded at the second leg of a journey is better than that based on a traveller's intended duration of stay from the first leg of the journey. For example, analysis shows that the majority of those with an intention of permanently departing, return to Australia within the following year (which is captured at the second leg of the journey). In the calendar year 2011, of the 84,240 Australian residents who stated they were departing permanently, only 15,890 (19%) spent 12 months or more overseas. Similarly, around 75% of all visitors who stated they intended to stay for one year exactly, actually stayed less than one year. A similar pattern is true for resident departures, although the proportion is lower at around 65%.

There are two separate parts to the imputation for duration of stay. The first part is used if a response is missing. For this imputation, the variables used to align the recipient with a suitable donor are: passenger card box type, country of birth and country of citizenship. From July 2004 to April 2015 the variable stay-intent based on 'intention to live in Australia for next 12 months (for arrivals only)' was also used until its quality declined.

The second imputation is only used when a traveller has put one year exactly as their intended duration of stay. It will therefore only apply to those who have completed box B or box E on the passenger card. This imputation reflects historical patterns that clearly show the majority stay less than one year. The imputation first involves creating an historical data set based on information from two years earlier. It then calculates the actual recorded duration of stay for those travellers who had originally put one year exactly as their intended duration of stay. For this imputation the variables used to align the recipient with a suitable donor are: corresponding months in the historical data set, those who also stated exactly one year, passenger card box type, and country of citizenship. For the proportions imputed to either a long-term stay and short-term stay, for each passenger card box type for the current months, see Table A3 below.

Generally, imputation required for missing duration of stay is less than 10% of relevant records. All relevant records are fully imputed for this variable. See section 4. 'Data Imputations' above.


A3 DURATION OF STAY — IMPUTATION FOR 1 YEAR EXACTLY - May 2017

Box type
no.
%

B: Visitor or temporary entrant who stated 1 year exactly
18,927
100.00
those imputed to long-term visitor arrivals
4,596
24.28
those imputed to short-term visitor arrivals
14,331
75.72
E: Australian resident departing temporarily who stated 1 year exactly
3,217
100.00
those imputed to long-term resident departures
1,480
46.01
those imputed to short-term resident departures
1,737
53.99

Classification of duration of stay by category of movement is as follows:
  • Permanent arrivals: Duration of stay not applicable - set to zero.
  • Permanent departures:
      • Australia-born residents: Duration of stay not applicable - set to zero.
      • Overseas-born Australian residents: Based on actual duration of stay as calculated by TRIPS using the initial permanent arrival date. However, there are some records that cannot be matched with the initial permanent arrival date. Duration of stay for these records is calculated using 1 July 1990, as no earlier data is available for matching. For these cases the migration arrival date is imputed to be 1 July 1990 which is the introduction date of TRIPS. An assumption is therefore being made that the migration of those overseas-born Australian residents was prior to this date.
  • Visitor departures and returning Australian residents - second leg of journey: Based on actual duration of stay/absence as calculated by TRIPS using the most recent arrival/departure date. However, when the previous movement cannot be found in TRIPS the duration of stay/absence is supplied by DIBP to ABS as missing. These are then imputed by the ABS as noted earlier.
  • Visitor arrivals and Australian residents departing temporarily - first leg of journey: Based on the intended duration of stay/absence as stated by travellers on the incoming and outgoing passenger cards.

For a complete list of the categories of movement see the Glossary under the Explanatory Notes tab.

There is evidence to suggest that when completing the intended duration of stay question on the incoming passenger card (Box B), some passengers are entering their arrival/departure date or their birth date rather than their intended duration of stay. From September 2003, a rule has been implemented to the data processing system stating that if all three elements are complete (years, months and days), then the intended duration of stay is to be coded to a non-response. Prior to July 2004, the ABS assigned a 'not stated' duration of stay as 10 days and therefore as a short-term movement.

Duration of stay - historical

Prior to the rebuild of the OAD system, a simple assumption was in place that set any traveller with a missing duration of stay to 10 days and therefore to a short-term movement. Missing duration of stay using the improved hot deck imputation methodology, as noted in 'Duration of stay - current' section above, has been revised back to July 2004.

Over time, there have been a number of changes to information collected on duration of stay. Initially, the intended duration of stay was only collected from information provided by all travellers on incoming and outgoing passenger cards in the intended length of stay fields. Therefore historically, the first leg and second leg of a journey both collected duration of stay based on intention.

With the introduction of TRIPS by DIBP in July 1990, the new system made possible the calculation of the actual length of stay/absence for travellers on the second leg of their journey (i.e. departing overseas visitors and returning Australian residents). This calculation based on TRIPS data commenced in July 1998. This change resulted in an improvement in data quality for duration of stay. In particular, the distribution of the number of passengers staying for one year exactly declining significantly for this group of travellers.

The introduction of a new passenger card processing system from July 2001 provided further evidence of travellers rounding to one year exactly for their intended duration of stay in Australia or overseas. To reflect the historical movement patterns, the records with a reported duration of one year exactly were allocated to short-term or long-term. For visitors arriving in Australia, 75% of such records were allocated to short-term and 25% to long-term. For residents departing Australia, the distribution was 67% short-term and 33% long-term. With the rebuild of the OAD system, these proportional splits were able to be based on the behaviour of travellers from two years earlier - see Table A3 and the section above 'Duration of stay - current'. Data back to July 2004 has been revised using the new methodology based on these dynamic proportional splits.

Missing response rates for the duration of stay are only available since November 1998. Prior to this, imputation carried out as part of processing by DIBP prevented reliable estimation for missing duration of stay.

Passenger card box type

The primary source for data on this variable is the passenger card. Administrative systems at DIBP are also used as an alternate source for some travellers. During the edit process some items are corrected where they conflict with other known information. For example, all travellers with a permanent arrival visa arriving for the first time would be converted to box A (migrating permanently to Australia) or all visitors on a temporary visa would be converted to a box B (for arrivals) and box D (for departures).

From October 2014, when a departure card is missing and if box type is still unknown, an additional source used is an individual's secondary passenger card. For example, the vast majority of travellers (excluding permanent migrants) have two legs to their journey, either an arrival followed by a departure or vice versa a departure followed by an arrival. By retrieving an individual's most recent other movement (where possible) allows for the corresponding box type to be collected from that same individual's secondary passenger card. Therefore, if a traveller is box B (visitor or temporary entrant) on arrival then conversely they would have to be a box D (visitor or temporary entrant departing) on departure and vice versa. If a traveller is box C (resident returning to Australia) on arrival then conversely they would have to be a box E (Australian resident departing temporarily) on departure and vice versa.

For this imputation, the variables used to align the recipient with a suitable donor are: direction of traveller, country of citizenship and visa group data. Prior to October 2014 sampled or non-sampled data were used instead of visa group data. From July 2004 to October 2014 the variable stay-intent based on 'intention to live in Australia for next 12 months (for arrivals only)' was also used until its quality declined. Historically, very little imputation was required for passenger card box type (less than 1.6% of all movement records). All relevant records are fully imputed for this variable. See section 4. 'Data Imputations' above.

Reason for journey

The only source available for data on this variable is the passenger card.

For this imputation the variables used to align the recipient with a suitable donor are: category of movement, sex and age. Prior to October 2014 donors used were: sampled or non-sampled data, passenger card box type, category of movement, and age. Generally, imputation required for reason for journey is less than 10% of relevant records. All relevant records are fully imputed for this variable. See section 4. 'Data Imputations' above.

Sex

The only sources available for data on this variable are passport or visa information.

For this imputation the variables used to align the recipient with a suitable donor are: the passenger card box type and reason for journey. Generally, little imputation is required for sex (less than 2.4% of relevant records). All relevant records are fully imputed for this variable. See section 4. 'Data Imputations' above.

State or territory of stay/residence

The primary source for data on this variable is the original passenger card (from the reference movement). When a departure card is missing the alternate source used is an individual's secondary passenger card. For example, the vast majority of travellers (excluding permanent migrants) have two legs to their journey, either an arrival followed by a departure or vice versa a departure followed by an arrival. By retrieving an individual's most recent other movement (where possible) allows for the state or territory of residence/stay to be collected from that same individuals secondary passenger card. Prior to October 2014 the only source for data on this variable was the original passenger card.

For this imputation the variables used to align the recipient with a suitable donor are: sampled or non-sampled data, state of clearance, and country of citizenship. Generally, imputation required for state or territory of stay/residence is less than 8% of relevant records. All relevant records are fully imputed for this variable. See section 4. 'Data Imputations' above.

If a correction to the passenger card box type marked by a long-term visitor departure is made (e.g. a visitor incorrectly marks a resident box of E (Aust. resident departing temporarily) or F (Aust. resident departing permanently)), then the state of stay recorded in the incorrect box is applied.


6. SPECIFIC ISSUES FOR NEW ZEALAND PASSPORT HOLDERS

Allocating passenger card box type

Under the Trans-Tasman Agreement, New Zealand (NZ) citizens are not required to have a visa to travel to Australia. As a result, on their arrival in Australia visa documentation cannot be used to determine whether they are either a permanent migrant or a temporary visitor, or an Australian resident returning from NZ. Analysis undertaken by DIBP suggests that a substantial proportion of holders of NZ passports tick Box A (migrating permanently to Australia) each time they arrive in the country, causing an over count of NZ migrants entering Australia. The following edits were applied to correct the over-counting of NZ migrants.

From July 2001 to June 2002, DIBP coded all NZ citizen arrivals who had ticked Box A (migrating permanently to Australia) and had been to Australia previously (based on DIBP records) to resident returning (Box C). If these people were visitors previously, this recoding had the effect of incorrectly reducing the number of NZ migrants whilst at the same time incorrectly increasing the number of NZ citizen who were returning residents. This problem was overcome by moving the NZ citizens who had been changed by DIBP from Box A to Box C back to Box A.

Since July 2002, DIBP has utilised a new edit system to ensure accurate measurement of permanent arrivals of NZ citizens. Where a person ticks Box A on his/her passenger card (migrating permanently to Australia), the record is verified by checking previous entries and related passenger card records, and if the person is previously recorded as a permanent migrant or resident then they will be counted as returning residents. This resulted in more accurate recording of NZ citizens who were migrating permanently to Australia and those who were residents returning.

In 2007, to better measure the changes in traveller behaviour and more accurately capture and measure temporary migration, the ABS introduced improved methods for calculating net overseas migration. This is now the most appropriate source for statistics on migration into, and out of, Australia. Data is available from December quarter 2003. See Explanatory Note 77 in Migration, Australia (cat. no. 3412.0).

Specific imputation for country of birth of New Zealand citizens

With the introduction of biometric passports for New Zealand (NZ) citizens in April 2005, the country of birth information was removed from the passport and replaced with a place of birth, for example Wellington, Auckland, Christchurch or Melbourne. The passport was the only source of information on the country of birth of NZ citizens travelling to, or from, Australia. For other travellers who are not NZ citizens, country of birth information can be obtained from their passport or visa information. However, visa information for most NZ citizens is not available as, under the trans-Tasman agreement, they do not need to hold a visa prior to travel to Australia.

Therefore, with the increased numbers of travellers holding NZ biometric passports, the proportion of movement records with a missing country of birth has increased substantially. For example, by April 2013 the matched OAD data showed the number of records missing a country of birth was 251,176. NZ passport holders represented 95% of these records. Earlier, for April 2005, NZ passport holders represented only 6% of the missing country of birth records. By April 2007 this had increased to 79%. As a temporary fix to alleviate this growing issue a basic imputation was introduced by the ABS in late 2007. From August 2007 the imputation used donors with a similar category of movement and country of citizenship.

From 2013, a special imputation for country of birth of NZ citizens has been introduced within the rebuilt OAD system, with data revised back to July 2004. It has improved country of birth statistics in OAD, and also outputs on net overseas migration (NOM), and the Estimated Resident Population by country of birth.

There are five steps to the process to generate country of birth when missing:

1. Prior to imputation, if country of birth is missing for a NZ citizen the system will scan historical records of NZ citizens back to 2003 to see if there is an earlier record of the individual's country of birth. This is made possible through the use of a unique personal identifier provided to each traveller who crosses Australia's international border. This step will look for a record with a matching personal identifier and if one is found will use the country of birth of the matched record. In 2015 approximately 77% of records with a missing country of birth are being matched with an historical record.

2. If country of birth is still unknown after Step 1, the system will scan all previous imputations for country of birth for NZ citizens to see if there is an existing record for that individual. This ensures an individual's country of birth is only ever imputed once although they may cross Australia's international borders many times.

3. If country of birth is still unknown after Step 2, but there is a place of birth supplied on the NZ biometric passport, then a place to country of birth concordance is used. This concordance is dynamic and is updated each month from the historical time series, which is also updated monthly with additional data supplied by DIBP. The number of records for each place of birth, separately within each country of birth, is then determined cumulatively from the historical time series. That is, if the name of a place of birth is used in more than one country, for example - 'Wellington' can be found in Australia, Canada, NZ, South Africa, UK and the USA, then the method adds up the number of instances within each of those countries from the historical series. Where a record is missing country of birth, the imputation will consider all possible donors with a matching place of birth. It will then choose a random donor based on its probability of occurring from the concordance, and copy across the donors corresponding country of birth.

By the end of Step 3, up to 98% of NZ citizens with a missing value have been provided a country of birth.

4. If country of birth is still unknown after Step 3, but there is a place of birth supplied, then a search is done on all NZ towns and place names. If a match is found it is assumed the country of birth of that record is New Zealand. Very few records are imputed using this step.

5. Lastly, if country of birth is still unknown for any NZ citizen after all other steps are taken, then the standard hot deck imputation is applied but only for non-New Zealand born as it is assumed any New Zealand born will have been picked up in the previous four steps. Currently, less than 1% of records are imputed using this step.


7. HISTORY OF PROCESSING CHANGES

July 1998, Permanent Departures

Prior to July 1998, the number of overseas-born (excluding NZ) permanent departures of Australian residents was overstated.

In July 1998, DIBP introduced a Box type validation edit to the processing system. This edit checks and corrects the Box type according to the Visa Class/subclass. With the exception of Australian and NZ citizens, only Australian residents departing permanently (Box F) who hold permanent visas are retained in this Box type. For temporary visa holders who incorrectly ticked Box F, their Box type was changed to visitor or temporary entrant departing (Box D).

July to December 1998, Reason for Journey

Before the introduction of the redesigned passenger card in July 1998, 5% of short-term visitor arrivals, on average, were recorded as having a reason for journey of 'Other' or 'Not Stated'. This percentage rose to 14% for July, 16% in August and 29% in September 1998 as a result of processing problems. These problems were addressed by DIBP, with the percentage of 'Other' and 'Not Stated' dropping to 8% and 7% in October and November respectively.

From January 1999, OAD statistics referencing these three months have been revised. The revised data were calculated by estimating the number of persons responding 'Other/Not Stated' using past trends for each country of citizenship and proportionally allocating any persons in excess of the estimated 'Other/Not Stated' total amongst the remaining categories.

July to December 1998, State or territory of residence/stay

For the months of August 1998, September 1998 and October 1998, data entry problems experienced by DIBP caused an overstatement of the Northern Territory as the main state of stay with a corresponding understatement for the remaining states and territories. In November 1998 these numbers returned to levels more comparable with previous years, with DIBP indicating that they had instigated data quality procedures to address this issue.

From January 1999, OAD statistics referencing these months have been revised. The revised data were calculated by estimating the number of persons indicating the Northern Territory as their main state of residence/stay using past trends and proportionally allocating any persons in excess of these estimates amongst the remaining states and territories.

With the introduction of the new processing system from July 2001, DIBP provided the ABS with data on all missing values for state or territory of residence/stay. From July 2001 to Jun 2004, any missing state or territory of residence/stay were imputed using category of movement and state of clearance.

September 1998, Age, Country of Birth, Citizenship and Sex

A problem was experienced in the processing of OAD data for movement dates between 6 September 1998 and 16 September 1998, following the introduction of changes to DIBP's input processing system. This problem may affect in the order of 10% of all September 1998 records used in estimation and result in incorrect details for citizenship, date of birth, sex and country of birth.

September 1999, China and Hong Kong

September 1999 overseas arrivals and departures data were revised for movements from, and to, China and Hong Kong in respect of three variables: country of birth, country of citizenship and country of residence/stay. Changes to 'country of birth' and 'country of citizenship' have been made from data supplied by DIBP. Changes to 'country of residence/stay' have been made by assuming the average proportion of country of birth to country of residence/stay for migrants from China and Hong Kong in September 1995 to September 1998.

July 2004, All Data

In 2013, the ABS completed a rebuild of the system which creates OAD data. All OAD data have been revised back to July 2004 based on the improved methodology.

January 2013, Duration of Stay and Reason for Journey

Investigations by the ABS and DIBP uncovered a high non-response rate for both duration of stay and reason for journey for the month of January 2013. This was mainly due to changes to the collection and processing of passenger cards, which were introduced in that month. January is the only month that has been affected and the non-response rates for subsequent months are at an acceptable level. The ABS and DIBP reprocessed January 2013 data. All associated time series spreadsheets and data files were revised.

October 2013 to July 2014, Duration of Stay

In the August 2014 release, OAD data from October 2013 to July 2014 was revised. Investigations by the ABS uncovered a quality issue with the input data used for imputation of duration of stay, leading to a higher count of long-term visitor arrivals and a lower count of short-term visitor arrivals. For more information about this imputation see the section "Duration of stay - current" in this appendix.

The ABS reprocessed data from October 2013 to July 2014. All associated time series spreadsheets and data files were revised. The specific categories of movement that changed due to the changes of the duration of stay are shown in a table in the Data Quality Issues Appendix of the August 2014 release.

October 2014 to May 2015, Delay in release of overseas arrivals and departures statistics and missing passenger cards

A number of releases of Overseas Arrivals and Departures, Australia (cat. no. 3401.0) were delayed due to passenger card processing issues announced by the Department of Immigration and Border Protection. The ABS increased quality assurance processes for October 2014 to May 2015.

October 2014 onwards, Increases in Missing Passenger Cards

From October 2014 to May 2015, the Department of Immigration and Border Protection (DIBP) trialled a transition to a new provider of passenger card processing. During this period there was a higher number of cards missing than expected.

DIBP has rolled out new technologies to streamline and strengthen border operations. Departure SmartGates are currently in use at all international airports in Australia for processing passengers leaving the country. Passengers using Departure SmartGates are required to place their outgoing passenger card in drop boxes located near the Departure SmartGate rather than handing them to Australian Border Force officers.

Coinciding with the roll-out of Departure SmartGates, there has been an increase in outgoing passenger cards not being collected due to passengers failing to place their cards into the drop boxes. The ABS and DIBP are working together to mitigate this issue and to minimise the impact on data quality.

For May 2017, the ABS added 235,965 records to account for missing outgoing passenger cards. The ABS also added 119 records to account for a higher level of missing incoming passenger cards. A total of 236,084 missing records were added to the overseas arrivals and departures data for May 2017. For further information, see Section 3 above 'Upgrade to the OAD system due to increases in missing passenger cards'. Table A4 shows the number of records that the ABS has added to adjust for missing passenger card data over time.

A4 ADJUSTMENTS FOR MISSING PASSENGER CARDS BY AIRPORT(a)


International Airport


Movement month
and year
Adelaide
Airport
Cairns
Airport
Darwin
Airport
Eagle Farm
(Brisbane)
Airport
Gold Coast
Airport
Kingsford
Smith
(Sydney)
Airport
Perth
Airport
Tullamarine
(Melbourne)
Airport
Total

May 2016
Arrivals
6
8
81
224
-
-
-
-
319
Departures(b)
-
-
1,084
13,890
-
57,931
169
68,871
141,945
June 2016
Arrivals
-
31
117
201
166
511
199
-
1,225
Departures(b)
-
635
1,670
21,476
5,488
60,796
16,658
55,389
162,112
July 2016
Arrivals
11
132
183
-
3,073
-
-
-
3,399
Departures(b)
3,921
3,390
736
21,141
7,061
59,894
22,555
44,559
163,257
August 2016
Arrivals
-
-
296
1,256
-
628
-
-
2,180
Departures(b)
4,126
3,204
468
22,839
6,119
68,094
22,554
43,417
170,821
September 2016
Arrivals
-
-
176
282
-
-
-
211
669
Departures(b)
3,657
3,252
285
19,802
4,581
66,061
23,021
47,552
168,211
October 2016
Arrivals
-
-
76
-
-
-
-
-
76
Departures(b)
3,525
2,903
184
25,367
3,944
72,623
17,003
49,962
175,511
November 2016
Arrivals
19
7
168
-
-
-
-
-
194
Departures(b)
3,575
2,782
81
24,559
3,572
86,911
18,444
52,802
192,726
December 2016
Arrivals
-
19
165
-
-
-
-
326
510
Departures(b)
4,653
3,749
182
30,038
4,003
102,478
28,976
77,434
251,513

January 2017
Arrivals
-
41
262
-
297
-
-
1,680
2,280
Departures(b)
3,789
3,687
229
30,528
3,377
111,681
26,006
70,315
249,612
February 2017
Arrivals
348
27
89
320
-
-
-
428
1,212
Departures(b)
2,517
3,283
92
26,099
3,132
104,522
19,726
71,008
230,379
March 2017
Arrivals
-
-
160
-
19
193
-
-
372
Departures(b)
3,503
3,192
319
27,743
4,295
107,351
25,391
60,817
232,611
April 2017
Arrivals
-
95
-
94
-
1,810
180
-
2,179
Departures(b)
4,251
3,255
167
29,700
3,896
100,465
24,384
68,214
234,333
May 2017
Arrivals
18
-
-
-
18
-
83
-
119
Departures(b)
3,464
3,079
108
31,277
3,712
101,655
21,630
71,040
235,965

- nil or rounded to zero (including null cells)
(a) To maintain quality due to higher rates of missing passenger card data, the ABS is using additional sources of data. For further information, see Section 3 above 'Upgrade to the OAD system due to increases in missing passenger cards'.

(b) From August 2015 onwards, DIBP progressively rolled out Departure SmartGates at the international airports. An increase in missing departure cards was observed throughout the roll-out. The ABS now adjusts each month for missing passenger card data.


July 2016, Norfolk Island

From 1 July 2016, Norfolk island was integrated into Australia for administrative purposes. Travel between mainland Australia and Norfolk Island is no longer considered an international movement and has been excluded from the Overseas Arrivals and Departures statistics.

February 2017, New Zealand Citizen resident returns to Australia

In February 2017, the ABS became aware of an increase of resident returns to Australia by New Zealand citizens with an overseas stay of 1 year or more. This increase is not fully supported by real world explanations and has been traced to the data ABS receives from DIBP. This issue has been flagged as a data quality concern and is being investigated by the ABS and DIBP.