Data Collection
Labour Market Statistics: March 2017 quarter
#Period-specific information
Update on the investigation into the change in employment over the June 2016 quarter
This note provides an update on our investigation into the changes that were potentially introduced into our employment indicators, as a result of the redeveloped Household Labour Force Survey (HLFS).
In the June 2016 quarter, the seasonally adjusted number of employed people increased by 2.4 percent, by far the largest quarterly increase ever recorded in the HLFS. While other labour market indicators supported the direction of employment growth, the magnitude of this increase needed to be explored in terms of real world changes versus changes to the survey.
In the project to redevelop the HLFS, options were explored around how the new survey would be introduced into the field, including whether a dual run could be done for a specific period of time. The logistics and cost involved meant that there were no feasible options for doing so. We agreed that our approach would be to do an assessment after several quarters of new HLFS data to determine if there had been any structural changes in key time series as a result of the redevelopment, and to estimate this effect to the best of our ability before investigating options for adjusting the time series (if possible). Unfortunately we had no information prior to the collection of the June 2016 quarter that suggested we might see such a large change in employment.
Some of the increase in employment over the March to June 2016 quarter could be attributed to changes in the HLFS collection. The inclusion of armed forces personnel in the survey population contributed to this increase. Specifically, people living in households and employed in the armed forces were no longer considered ineligible to participate in the survey. The other component of the increase that carries a much higher degree of uncertainty, is the observation that the new HLFS appears to be more likely to identify people as employed, whereas in the old survey they may have been identified as ‘not in the labour force’. Our analysis to date suggests this was particularly apparent with self-employed people. As part of our analysis, we were able to confront the new ‘job tenure’ information in the HLFS to confirm that some people previously identified as not working had in fact been running their own business for a number of years.
However, finding a statistically sound method of disentangling survey changes from real world changes is a challenge. Particularly when taking into account the level of uncertainty that already exists in our employment series – the absolute sampling error is typically around 22,000 people. In order to ‘correct’ for the possible effects of the new survey, there are essentially three things that need to be done:
- producing an estimate of the size of the ‘level shift’ that occurs between the March and June 2016 quarters,
- determining a method for estimating the size of this shift at various points along the HLFS time series
- determining how estimates of the level shift would be implemented; whether this would be done at the unit record level or only to selected published series.
This summary focuses specifically on the first component of this, and what we concluded from our investigation.
Estimating the magnitude of the level shift
For this analysis we prioritised the following seasonally adjusted series:
- Male employed
- Female employed
- Male Not in the labour force (NILF)
- Female NILF
- Total actual hours worked
- Total usual hours worked
- Employed full-time
- Employed part-time
Here we will focus specifically on male and female employed but similar results are expected for the other six series.The male, female and total employed series were studied by fitting a time series model (i.e. the underlying data generating process) with a variable to estimate the possible change in the level of the series. Two different methods were used, which presented similar results. The analyses were done independently and the results presented in Table 1. The two models used are described below:
- An ARIMA model. This is how we would allow for the effect if we wished to add it to our seasonal adjustment system.
- An Unobserved Components Model (UCM). This is similar to the model we use when seasonally adjusting.
Table 1. Table presenting the level shift estimates for each time series with 95% Confidence Intervals (values in thousands of employed).
Level Shift Estimates | |||||
---|---|---|---|---|---|
Series | Method | Lower limit | Estimate | Upper limit | Employed as at June 2016 (000) |
Male employed | ARIMA | 7 | 28 | 50 | 1296 |
UCM | 15 | 33 | 51 | ||
Female employed | ARIMA | 0 | 21 | 42 | 1158 |
UCM | 0 | 20 | 40 | ||
Total employed | ARIMA | 7 | 49 | 92 | 2454 |
UCM | 29 | 53 | 77 |
Our unadjusted published estimate for total employed in the June 2016 quarter was 2,454,000 people, suggesting that there had been an increase of 45,000 people when compared to March 2016. The UCM method estimated that the shift in employment due to the 2016 redevelopment was between 29,000 and 77,000. Whilst the ARIMA method estimates this effect as being between 7,000 and 92,000 people. The results highlight the large degree of uncertainty in estimating the level shift, and in turn, the uncertainty in whether applying an adjustment would definitively improve the accuracy and comparability of the time series.
The investigation into the impact of the redevelopment on the employed series revealed the following:
- There is evidence of a level shift in the employed series between March and June 2016.
- The trend and seasonally adjusted series are unlikely to be impacted by the level shift. The use of moving averages to estimate the trend and seasonal components by the seasonal adjustment program, X-13-ARIMA-SEATS, effectively mitigates any impact the level shift would have on seasonally adjusted outputs.
- The uncertainty associated with the level shift estimates presents a challenge in making inferences about how much of the changes between the March and June 2016 quarters can be attributed to the re-development
- Applying any adjustments to the time series prior to June 2016 would require further investigation into how the changes from the re-development may have impacted the labour market series at different time points. The time and resources required for this work, along with the risk appetite for implications on the accuracy and comparability of the time series should be weighed against the potential value added by this intervention.
Actual Hours Imputation
Due to a routing problem in the questionnaire this quarter, people who were self-employed/owned a business or worked for a family business were not asked their actual hours worked even though they indicated that they were different from their usual hours. Normally, donor imputation is used to handle non-response of actual hours worked but there was a lack of suitable donors in the current quarter since those most similar to them also required imputation.
The routing discrepancy has been corrected for future quarters and in the current quarter we were able to create methodology to impute actual hours using a revised donor pool. This quarter, donors from the same quarter in the previous year (March 2016) were used. Those that were present in the survey in March 2016 were used as donors to impute their own hours for March 2017. Otherwise, a suitable donor was found using the normal imputation method, but from the March 2016 quarter.
We tested this modified method of imputation on past quarters and found good agreement between original and imputed values. Therefore, we are confident that we have minimized the impact of this issue to an acceptable level.
##Response Rates
Survey | Reference period | Response rate | Achieved Sample rate |
---|---|---|---|
HLFS | Each week during the quarter (8 Jan 2017 – 8 April 2017) | Target: 90 percent Achieved: 82.4 percent |
Target: 76 percent Achieved: 74.5 percent |
QES | The pay week ending on, or before, 17 Feb 2017 | Target: 89 percent Achieved: 87.7 percent |
N/A |
LCI | Pay rates at 15 Feb 2017 | Target: 94 percent Achieved: 95.0 percent |
N/A |
See New quality measures for the Household Labour Force Survey for more information on the sample rate and response rates.
##HLFS ###Outliers During the seasonal adjustment process, X-13-ARIMA-SEATS can give less weight to the irregular component. Specifically, if the estimated irregular component at a point in time is sufficiently large compared with the standard deviation of the irregular component as a whole, then the irregular component at that point can be downweighted or removed completely and re-estimated. We refer to such observations as partial- and zero-outliers, respectively. In practice, the downweighting of outliers does little to seasonally adjusted data, but the impact of the outliers on the trend series will generally be reduced. However, if an outlier ceases to be an outlier as more data becomes available, then significant revisions to the trend series become possible.
The table below shows any partial (P) and zero (Z) outliers for the last year of each time series
Outliers | ||||||
---|---|---|---|---|---|---|
Quarters | Male employed | Female employed | Male unemployed | Female unemployed | Male not in the labour force |
Female not in the labour force |
Jun 2016 | ||||||
Sep 2016 | P | P | ||||
Dec 2016 | P | |||||
Mar 2017 |
###HLFS pre- and post-calibration weight The following figure shows that while the distribution of the pre- and post-calibration weights differs within a quarter, the difference between the weights typically does not change from quarter to quarter.
The undercoverage rate indicates how representative the pre-calibrated sample is. The higher the undercoverage rate, the less representative the pre-calibrated sample.
Usually the undercoverage rate in the HLFS is around 20 percent. The overall undercoverage rate for the HLFS in the March 2017 quarter was 17.8 percent. This compares with 17.6 percent in the December 2016 quarter and 19.2 percent in the March 2016 quarter.
##General information and methodology
For general information and methodology on the specific surveys within the labour market statistics release, please see the following Datainfo+ pages:
en-NZ