Imputation theory and methodology
Across all indexes, missing price observations occur on a regular basis and this could be due to factors such as:
- temporary out of stock items,
- discontinued items,
- or seasonal elements.
In any price reference period, these factors can make it impossible to obtain a price measure for a particular product.
The ABS employs a number of imputation methods to address temporarily missing observations within price indexes. These include:
- the imputation a movement for the product based on the price movement for all other products in the sample
- the use the price movement from another price sample, or
- repeat the previous period’s price of the product (also called carry forward method).
These options are known as imputation.
Their purpose is to calculate a price for the temporarily missing product. The aim of imputation is to provide prices such that the resulting movement in the price index is the same as would have been calculated had all prices been observed. In achieving such a result, it is necessary to make an assumption regarding the price behaviour of the temporarily missing product.
Imputation from price sample
The rationale for imputing a price movement from other products in the sample is that products are bought and sold in a competitive marketplace and in those cases where an individual product has not been observed in the current period, it is assumed that its price behaviour is reflected by similar products in the sample. The design of elementary aggregates to contain products that are homogeneous in terms of price behaviour (as noted above) ensures that the assumption underlying this method of imputation is generally robust.
Imputing from other products in the sample is also mathematically equivalent to excluding the product, for which a price is unavailable in one period, from both periods involved in the index calculation. It strictly maintains the ‘matched sample’ concept.
In order to impute a movement resulting from excluding the product it is necessary to construct a measure of price change from the previous period to the current period for those products common to both periods. This calculation is dependent upon the price index formula used for the elementary aggregate.
When the elementary aggregate is compiled using a Laspeyres formula, it is first necessary to derive the implicit quantity shares underlying the weights of the matched products. This can be achieved by dividing the weight for each product by its reference period price.
The resulting quantity shares for the matched products are then used to calculate the price change from the previous period to the current period.
\(\Large s_{q,i}=\frac{\frac{w_i}{p^0_i}}{\sum_\limits {MATCHED} \frac{w_i}{p^0_i}}\)
\(\large M^t_{t-1} = \frac{\sum_\limits {MATCHED}s_{q,i}p^t_i}{\sum_\limits {MATCHED}s_{q,i}p^{t-1}_i}\)
\(\large \hat{p}_j^t=M^t_{t-1}\times p^{t-1}_j\)
where \(S_{q, i}\) is the implicit quantity share in the reference period for matched product \(i\), \(w_i\) is the weight for matched product \(i\), \(p^0_i\), \(p_i^{t-1}\),\( p^t_i\) are respectively the reference period price, previous period price, and current period price for matched product \(i\) (at time t), \(M^t_{t-1}\) is the price movement between the previous and current period for the matched products, and \(\hat{p}^{t-1}_j\) is the imputed price for missing product \(j\) at time \(t\).
An example of this calculation is shown in Table 3.7 below.
| Reference Period Value Share | Reference Period Price ($) | Previous Period Price | Current Period Price | |
|---|---|---|---|---|
| Product A | 30 | 5 | 8 | 12 |
| Product B | 60 | 10 | 16 | 20 |
| Product C | 10 | 2 | 4 | n.a. |
| Implicit quantities | Implicit quantity share | Share x Previous Period Price | Share x Current Period Price | |
| Product A | 6 | 0.5 | 4.0 | 6.0 |
| Product B | 6 | 0.5 | 8.0 | 10.0 |
| Total | 12.0 | 16.0 | ||
| Movement | 1 | |||
| Current period Price after impute | Price relative after impute | Weight x relative | ||
| Product A | 12 | 2.4 | 72 | |
| Product B | 20 | 2 | 120 | |
| Product C | 5.333 | 2.666667 | 26.66667 | |
| Laspeyres price index | 218.6667 |
Imputation from another price sample
The second approach to imputation for the Producer and International Trade Price Indexes is to use the price movement from another related sample or comparable product. This approach is used in cases where price changes from a comparable product (or products) from a similar type of provider can be expected to be similar to the missing product.
Carry forward imputation
The rationale for adopting a carry forward imputation is that failure to observe a price for a product reflects no transactions for the product, and hence there can be no price change. However, each product in the price sample represents similar products purchased and sold elsewhere in the marketplace, and such an assumption does not hold in most cases. Application of this method of imputation when transactions are actually occurring in a marketplace (but not observed by the sample) consistently biases the index towards zero (that is, biased downward when prices are rising and biased upward when prices are falling).
It is for these reasons that the price statisticians apply this imputation mechanism only under specific conditions where it is known that failure to observe a transaction means that no transactions are occurring (such as where there is only one sale per year of a type of agricultural crop, for example, or where the price changes only once per year during annual price setting).