Data Collection
CPI Data Collection 2020
Methodology
Field collection
Statistics NZ price collectors gather prices through a variety of means eg; directly from retail outlets; through surveys, on-line data collection, web-scraping, and administrative data.
Sample size
We collected about 100,000 prices each quarter and send out about 1,700 surveys.
##General information
Imputation
For a number of years, the main method of imputation for missing items in the CPI was to carry the previous price forward. An item can only be carried forward for two periods before it will be deemed permanently out of stock and replaced with a new item. From the June 2020 quarter, during the COVID-19 lock-down, we updated our default method of imputation to overall mean imputation, that is, within the lowest level sub-item index, the missing prices were imputed with the average movement from the other collected prices in that category. This method is recommended by the CPI manual and is consistent with how other countries treat missing prices.
We applied this imputation method to all monthly and quarterly items that were available and collected in the June quarter. For items collected annually in other periods we will continue to use carry forward imputation. More information on the treatment of items that were unavailable during and after the lock-down see https://www.stats.govt.nz/methods/impacts-of-covid-19-on-the-methodology-used-for-the-june-2020-quarter-cpi.
Review of the CPI
Reviews of the CPI are undertaken every three years. We implemented the latest review when the September 2020 CPI was published. The review involved reselecting the basket of representative goods and services, updating the new national expenditure weights, and updating regional expenditure weights. Consumers price index review: 2020 has more information.
Impacts of COVID-19 on the 2020 CPI re-weight
Ordinarily, a three-yearly re-weight of the CPI is sufficient to pick up changing consumer expenditure patterns. Due to COVID-19, supply and demand factors are likely to speed up this rate of change, with some items more affected than others.
At the time the 2020 review was implemented, New Zealand’s borders were still closed. As the borders re-open, this is likely to be in a limited way, and prices could be higher – especially if social distancing is still in force. Taking this into consideration, along with guidance from international bodies, talking to other national statistical offices and key stakeholders, and undertaking some sensitivity analysis, we have adjusted the weights for domestic and international airfares and overseas accommodation cost pre-paid in New Zealand. Impacts of COVID-19 on the 2020 CPI re-weight has more information.
We intend to adjust the weights for these three items annually. This means that the relative weight of all other CPI basket items will scale in association with the annual reweight of these three items. We aim to keep an eye on shifting expenditure patterns for other expenditure items over the next three years’, but would only consider changing weights for other items where there is a clear-cut case for doing so. This would be signalled well in advance of any change.
Care required when using CPI to adjust monetary values
The CPI is used to adjust monetary values, such as those in legislation and contracts.
Special care is required when the monetary values being adjusted exclude GST during periods which include GST adjustments. For example, a common use of the CPI is to adjust housing rentals, which are not subject to GST. Using the CPI to adjust rental values during a period in which GST increased (or decreased) would mean the adjustment would include the overall effect of the GST increase (or decrease) on the CPI.
Reference population
The reference population of the CPI covers approximately 98 percent of the usually-resident New Zealand population living in permanent dwellings.
Expenditure weights
Expenditure weights give the relative importance of the goods and services in the CPI basket.
The current set of weights are derived from the 2018/19 Household Economic Survey (HES) and other sources.
CPI weights are based on household spending for the year to June 2019 (the ‘weight reference period’), expressed in June 2020 quarter prices (the ‘price reference period’).
The relative importance of the CPI groups shows that $28.00 of every $100 households spend on goods and services in the CPI is spent on housing and household utilities, $18.72 is spent on food, and $11.93 is spent on transport.
The CPI Review tables Consumers price index review: 2020 show the relative importance of CPI groups, subgroups, and classes.
Collection methods
We send out about 1,700 different surveys each quarter and our price collectors personally visit over 2,800 different shops in 12 pricing centres throughout the country: Whangarei, Auckland, Hamilton, Tauranga, Napier-Hastings, New Plymouth, Palmerston North, Wellington, Nelson, Christchurch, Dunedin, and Invercargill.
The types of outlets visited include department stores, appliance stores and speciality stores. We collect prices weekly for motor fuels and for fresh fruit and vegetables; monthly for food, non-food groceries, alcoholic beverages, and newspapers; and quarterly for other goods and services.
Increasingly we are trialling and implementing new price collection techniques from alternative data sources. By adopting collection from secondary data sources, we are able to increase the scope and accuracy of our pricing while reducing respondent burden.
We have expanded the collection of prices for certain items such as golf green fees and cinema admission tickets, using web-scraping and API (application programming interface) querying. This reduces manual processing time and the chance of human error. In general, we are working towards replacing more surveys and field-collected prices with web-scraped data and online collection where feasible.
The June 2020 quarter was the first time we incorporated point of sale (checkout scanner) price data supplied directly by some supermarkets as an alternative data source. This data replaced all the items that would have previously been manually priced in-store from these supermarkets. This represents a great improvement over in-store price collection, and we intend to continue this method in future.
Pricing frequency
Whether we collect prices weekly, monthly, quarterly, or annually, depends on the expected frequency of price changes the goods or service exhibit.
Accuracy of the data
Elementary aggregate formula
Average prices in the CPI are called elementary aggregates. These elementary aggregates are the first level of the index aggregation. We calculate regional elementary aggregates for each of the 12 pricing centres where price collection supports regional estimation. In other cases, we calculate regional elementary aggregates for five CPI broad regions (Auckland, Wellington, rest of North Island, Canterbury, rest of South Island) or, where prices do not support regional estimation, directly to a national elementary aggregate. Since the 2006 review of the CPI, we have used the geometric mean, or Jevons, formula to calculate the elementary aggregate indexes for items where outlet substitution is possible (eg for groceries and appliances).
The geometric mean (or Jevons) formula is used to calculate elementary aggregate indexes for goods and services that are considered to be subject to outlet substitution. The Jevons formula, which is recommended by the International Labour Office (ILO), implicitly assumes that consumers increasingly favour outlets showing lower relative price change, whereas the formula used up to this point, the ratio of arithmetic mean prices (or Dutot), assumes that consumers do not substitute between outlets.
The Dutot formula ('ratio of arithmetic mean prices') is used for items where:
• Outlet substitution is not possible (eg local authority rates);
• Prices are subsidised and may fall to zero (eg GPs' fees);
• Fresh fruit and vegetables (as the first stage of aggregation is across both outlets within each region, and across weeks within each month); and
• It is not currently practical to adopt the Jevons formula (eg when prices are aggregated directly to a national elementary aggregate, rather than aggregated to a regional level).
Method of aggregating monthly collected prices from monthly to quarterly level
Prices are collected monthly for the food group and a number of non-food items in the CPI, including electricity, cigarettes and tobacco, alcoholic drinks, and air travel. To include them in the CPI, we average these prices over the quarter.
The method we use to calculate these averages is to obtain monthly regional average prices for the item – by outlet-weighting the prices collected at different outlets within each region. We use the monthly regional average prices to calculate quarterly regional average prices – by weighting each monthly regional average price by the number of days in the month in which it was collected. This is called day weighting. We aggregate all the regions to obtain the national quarterly index by weighting together regional price movements from the base (ie June 2017) quarter to the current quarter, using the regional expenditure weights.
We collect petrol and diesel prices weekly, usually on Fridays. The CPI petrol price index measures price changes of 91 octane petrol and 95/98 octane petrol. Within each CPI region, we calculate an average price per 10 litres of each fuel from the prices surveyed each week at individual service stations. Monthly regional average prices for each fuel are then calculated as simple averages of the averages for the weeks in each month. We calculate quarterly regional average prices for each fuel as the day-weighted averages of the averages for the three months in the quarter. We then weight regional price movements from the base (ie June 2017) quarter to the current quarter by the regional population-weighted share of the national expenditure weight, to calculate the national petrol and diesel price indexes for the current quarter.
Since we collect petrol and diesel prices either 12 or 13 times within each quarter, a price change during the quarter is only partly reflected in that quarter, with the remainder being reflected in the following quarter. This also occurs for commodities that are priced monthly, such as cigarettes and tobacco.
Regional expenditure weights
From the September 2014 quarter CPI onwards, we weight regional price change using regional expenditure weights for the five broad regions (Auckland, Wellington, rest of North Island, Canterbury, and rest of South Island). Regional expenditure weights use expenditure in each region to weight regional price change. This ensures that price change in regions where households spend more per person on a particular item relative to regions (eg Auckland has 33.41 percent of the population and a CPI regional expenditure weight of 35.98 percent) has more influence on the combined national price change for that item.
For broad regions with multiple pricing centres (ie rest of North Island and rest of South Island), we use population shares to allocate the regional expenditure weight to the pricing centres.
We publish CPIs for the five broad regions based on regional council area boundaries. These indexes are available from Infoshare.
For the 2020 regional expenditure weights for the five broad regions, see the CPI review tables Consumers price index review: 2020.
Outlet weights
We give outlets appropriate weights to reflect their relative importance in terms of household spending.
'On special' prices
We include items that are 'on special' in the CPI at the price levels we observe when collecting prices. Quantity specials (eg 15-pack of beer at a cheaper shelf price than the 12-pack) are also considered where appropriate (as the price per bottle for the special is lower).
Key concepts
Standard and non-standard series
CPI series that contribute to the hierarchical structure of the overall CPI are known as standard series. For example, the clothing index, combined with the footwear index, contributes to the clothing and footwear index, which in turn contributes to the all groups index. Components of this pyramid-like structure are known as standard index series. The CPI is published at the following levels: group, subgroup, and class – all at the national level. We also publish selected sections within the food group.
In addition, we publish a selection of non-standard series in the information release tables; additional series are available from Infoshare. Examples of these non-standard series include:
– all groups CPI less each of the 11 CPI groups
– all groups CPI plus interest
– interest.
Tradable and non-tradable non-standard series
The tradable and non-tradable component series that appear in Consumers price index: September 2020 quarter. Table 1 decomposes CPI goods and services into two components: one contains goods and services that are imported or in competition with foreign goods, either in domestic or foreign markets tradables); the other contains goods and services that do not face foreign competition (non-tradables).
Movements in the tradables component (tradable inflation) demonstrate how international price movements and exchange rates are affecting consumer prices. The non-tradables component shows how domestic demand and supply conditions are affecting consumer prices. The June 2020 quarter expenditure weight of the tradables component is 39.92 percent, compared with 43.58 percent in September 2017. The June 2020 quarter weight of non-tradables is 60.08 percent, compared with 56.42 percent in September 2017.
Trend measures of price-level change
Over the long term, the CPI captures the broad pattern of price change, but it can be influenced by one-off events when analysing price change over shorter timeframes (eg a supply disturbance affecting petrol prices). To remove such influences, we calculate analytical measures of price change in an attempt to isolate the more persistent – or underlying – component of general price-level changes. We construct several analytical measures to give a good guide to underlying price-level change.
There is a range of 'trimmed means' series and a range of 'weighted percentiles' series, including a weighted median.
Trimmed means exclude the influence of the largest increases and decreases in the CPI. We do this at the item level of about 650 goods and services in the CPI basket (eg 91 octane petrol or strawberries). The trimmed means progressively remove the influence of the largest increases and decreases.
Weighted percentiles order price change, during the reference period, from lowest to highest, then show the price change at various percentile points (10th, 25th, 50th/median. 75th and 90th) along this distribution.
These show the level of inflation at various percentile levels. For example, the weighted median or 50th percentile is what the basket item in the middle of that period’s distribution (by weight) increased/decreased by.
Weighted percentiles highlight the price change of basket items at various percentile points in the distribution (specifically the 10th, 25th, 50th /median, 75th and 90th percentiles) for the reference month or quarter.
Weighted percentiles order price change, during the reference period, from lowest to highest, then show the price change at various percentile points (10th, 25th, 50th /median, 75th and 90th) along this distribution.
The central and local government charges index, made up just 8.78 percent of the CPI at the June 2020 quarter. This non-standard series includes items such as:
– Housing New Zealand and local authority rentals
– land transfer registration fees
– local authority rates
– water supply and part of refuse disposal, electricity
– prescription charges and oral contraception, general practitioner fees
– vehicle relicensing fees, road user charges, driver licensing fees
– postage
– State and integrated schools, tertiary education, other education
– official passports, licences and certificates.
The goods and services component series that appear in tables 3.01, 3.02, and 3.03 allow customers to decompose the CPI into its goods and services components, respectively. The goods component made up 62.60 percent, and the services component 37.40 percent at the June 2020 quarter.
The goods component comprises:
– the food group (except restaurant meals)
– alcoholic beverages and tobacco group
– clothing and footwear group (except clothing services)
– purchase of new housing, property maintenance materials, water supply, and household energy
– household contents and services group (except repair and hire of household appliances, hire of major tools and equipment, and other household services)
– medical products, appliances and equipment; dentures
– purchase of vehicles, vehicle parts and accessories, petrol, other vehicle fuels and lubricants
– telecommunication equipment
– recreation and culture group (except recreational and cultural services, accommodation services)
– miscellaneous goods and services group (except hairdressing and personal grooming services, jewellery and watch repair, insurance, credit services, and other miscellaneous services).
The services component comprises all items not included in the goods component.
Household Living Cost Price Indexes
While the CPI is an aggregate measure that represents the price change experienced on average by households, the Household Living Cost Price Indexes (HLPIs) provide a better insight into the inflation experienced by 13 different household groups:
• beneficiaries
• Māori
• income quintiles (five groups)
• expenditure quintiles (five groups)
• superannuitants.
More information about the HLPIs can be found in Household living costs price indexes: Background.
Consistency with other periods or datasets
Index reference period
All CPI indexes have an index reference period of the June 2017 quarter (=1000), except where we added additional indexes in subsequent CPI reviews.
Reconciling the FPI and food group of the CPI
When comparing the FPI and the food group of the CPI over a review period, note that the quarterly food group index number is not the average of the relevant three-monthly FPI numbers. Where there are changes to food basket items; prices for new CPI basket items would be collected in April, May and June (to apply the price change between the June and September quarters), whereas prices for new FPI items are collected for June (to apply the price change between June and July months).
The FPI has a monthly price reference period, and the prices for the June month can differ from the average of the April, May and June months. As a result, the monthly and quarterly base weights can differ, even though the same annual quantities were used.
Treatment of fresh fruit and fresh vegetables – removal of seasonal adjustment
Until the June 2006 quarter, we adjusted fresh fruit and fresh vegetable items that exhibited a seasonal pattern – to remove the effect of normal seasonal change. This treatment reduced the influence of normal seasonal price fluctuations, but did not completely eliminate the effects of seasonal fluctuations if shifts in seasonal patterns occurred.
From the September 2006 quarter onwards, the CPI incorporates seasonally unadjusted prices for fresh fruit and fresh vegetables. This is in line with a recommendation made by the 2004 CPI Revision Advisory Committee.
The ongoing, fully unadjusted CPI is linked at the June 2006 quarter to the previously published CPI, which is partly seasonally adjusted. As such, annual movements calculated over the annual period encompassing the June 2006 quarter are based on fully unadjusted index numbers for the latest quarter, compared with partly adjusted index numbers for the same quarter of the previous year. However, analytical time series provided annual movements on a fully unadjusted basis during the year-long transition of the official CPI. During this time, we based annual movements on fully unadjusted index numbers for the latest quarter, compared with partly adjusted index numbers for the same quarter of the previous year.
The 2013 CPI Advisory Committee recommended we add analytical seasonally adjusted series to our publications. We are seasonally adjusting the CPI and FPI at the all groups, group, subgroup, and class levels. The headline CPI remains unadjusted.
We use direct adjustment (rather than indirect) to seasonally adjust the data as this produced better quality statistics. Indirect seasonal adjustment occurs when individual component series of the main aggregate series are seasonally adjusted, then aggregated to derive totals. For example, an indirect seasonally adjusted fruit series would be compiled by adding all the seasonally adjusted series (for apples, pears, kiwifruit, etc) together. Direct seasonal adjustment occurs when seasonally adjustment is done at the aggregate level, independently of seasonally adjusting the components. A direct seasonally adjusted fruit series would be made up by adjusting the aggregate of all the unadjusted series (for apples, pears, kiwifruit, etc).
We use the x13 ARIMA-SEATS package to run our seasonal adjustment.
Interpreting the data
Rounding index numbers and calculating percentage changes
We use movements based on unrounded index numbers to determine whether items have increased, showed no change, or decreased in price. Previous period expenditure weights are used to indicate the proportion of the expenditure weight that has increased, showed no change, or decreased.
We publish percentage changes to one decimal place and calculate them from index numbers rounded to the nearest index point. For comparisons that cross the index reference period, customers should compare rounded index numbers (for the later period) with unrounded index numbers (for the earlier period).
Distribution of item-level index movements table
The distribution of item-level index movements (table 12 in the main CPI release) gives additional information on the distribution of price movements for the current quarter's CPI. The analytical statistics in the table indicate how widespread price changes are, and their relative magnitude when compared with previous quarters.
Detailed contribution information tables
Tables 7.01 and 7.02 provide a broader perspective of the categories contributing to movement in the all groups CPI. Where there is only one class within a subgroup, we omit the class to avoid unnecessary duplication. We give the contribution information as index points, percentage points, and percentage contributions from the previous quarter and from the same quarter of the previous year.
Information is calculated from unrounded index numbers. Percentage changes are calculated from index numbers rounded to the nearest index point (see 'Rounding of index numbers and calculation of percentage changes', above). As such, the sum of each of the group, subgroup, or class percentage point contributions may differ from the overall percentage change in the CPI all groups.
Weighted average retail prices of selected food items
We include a selection of average retail prices for the current and previous quarter in table 5 of CPI releases. The weighted average prices are calculated by applying index movements to weighted average prices for the June 2017 quarter CPI. They are not statistically accurate measures of average transaction price levels, but do provide a reliable indicator of percentage changes in prices.
Determining the effect of a specified change in a lower-level index
The index points effect and percentage contribution on a higher-level index of a specified percentage change in a lower-level index that contributes to the higher-level index can be determined by:
• Adjusting the lower-level index for the previous period by the specified percentage change to derive the index number for the current period.
• Calculating the index points effect on the higher-level index of the specified change in the lower-level index.
• Calculating the percentage change in the higher-level index that would be caused by the specified change in the lower-level index.
Example: The effect of a 5.0 percent increase in the petrol index (weight is 5.03 percent in the CPI) from the June 2014 quarter to the September 2014 quarter on the all groups CPI index is calculated by:
• Increasing the petrol index for the June 2014 quarter by 5.0 percent to derive the index number for the September 2014 quarter
• Calculating the index points effect on the all groups CPI index of the 5.0 percent increase in the petrol index:
• Calculating the percentage change in the all groups CPI index that would be caused by a 5.0 percent change in the petrol index.
Timing of published data
We generally publish the CPI 12 working days after the reference quarter, with the December quarter typically being an exception due to the holiday break.
For more information on the review, please contact:
Fiona Smillie or James Griffin 04 931 4600 info@stats.govt.nz
en-NZ
