2001 Post-enumeration Survey (Published)en-NZ
2001 Post-Enumeration Survey; 2001 Census Post-enumeration survey; 2001 PESen-NZ
Statistics New Zealanden-NZ
The 2001 Post Enumeration survey was the second survey run by Statistics New Zealand. In principle, a post-enumeration survey is a special survey undertaken shortly after the census to evaluate the completeness of census coverage. It involves an independent re-enumeration of a statistically designed sample of all dwellings covered by the national census. Post-enumeration surveys are an essential feature of census-taking in many countries. Post-enumeration Survey (PES) was conducted shortly after the 2001 Census of Population and Dwellings. Like the first PES in 1996, its main aim was to provide information on the completeness of census coverage – to gauge how many New Zealand residents were missed or counted more than once in the census. Statistics New Zealand intends to draw upon the PES results to adjust the population base for deriving post-censal population estimates and demographic projections. Users of census and population data will find this report informative and useful.
It is important to note that a post-enumeration survey is one aspect of examining the quality of census output and processes. As such, the 2001 PES will assist Statistics New Zealand in developing an enumeration strategy for the 2006 Census of Population and Dwellings.en-NZ
The 2001 PES was a sample survey of individuals in permanent private dwellings and was undertaken two weeks after the 2001 Census of Population and Dwellings (held on 6 March). The objective of the PES was to measure the level of coverage (undercount/overcount) in the 2001 Census. The survey did not aim to check the general accuracy or quality of the responses to specific questions in the census.en-NZ
The 2001 Post-enumeration Survey (PES) was the second undertaken by Statistics New Zealand, with the first undertaken five years previous to measure the coverage of the 1996 Census. The 2001 survey adopted a similar methodology to the 1996 survey.
While analysing the 2001 data, two methodological issues were uncovered in the 1996 survey which led to a revision of the 1996 PES results:
- A small number of individuals were omitted from the 1996 PES calculations. This occurred where individuals matched to dwellings classed as unoccupied in the census were given a post-stratification factor of zero.
- The 1996 eligibility criteria were overly restrictive, as they excluded all individuals in dwellings where there were some inadequate responses. The use of more conservative eligibility criteria used in deriving results from the 1996 PES partly reflected the fact that this was the first such survey conducted in New Zealand.
The revised 1996 net under coverage estimate was 60,000 people or 1.6 percent (+ 0.2 percent), whereas the original estimate had been 43,000 or 1.2 percent (+ 0.2 percent).
Because of the small sample size in the PES, the survey cannot provide reliable results below the published levels.
The data collection captures the methodological, collection and analysis information for the 2001 Post Enumeration Survey.en-NZ
Data collection The survey was carried out during 21 March–3 April 2001, following the completion of census fieldwork. The survey period was chosen to avoid overlap of census enumerators and PES interviewers in the field, and to avoid a clash with the Easter holidays, while being close enough to census date (6 March) to assist respondent recall. Data was collected by 159 specially trained interviewers using a household questionnaire. Information on occupants of the dwelling who satisfied the scope and coverage criteria was collected through a face-to-face interview wherever practicable. Alternatively, a proxy interview was conducted (ie details were obtained from another adult in the dwelling) and a follow-up interview was done over the telephone, unless the respondent insisted on a face-to-face interview. The actual number of responding dwellings was about 9,500 (or about 25,000 persons). Personal details sought on the PES questionnaire included: name, sex, date of birth or age, ethnicity and address. Besides usual address and census night address, the survey also collected information on any other addresses where the person might have been included on any other census form. This was to help increase the chances of finding and matching any individual census forms for a particular person, and to help identify multiple counts. A copy of the PES questionnaire is included in Appendix 1. In order for the PES to achieve its objectives, the processes needed to be independent of the census. To ensure this independence, the PES:
- used no census field staff
- was conducted after the census fieldwork was completed to avoid contact between census enumerators and PES interviewers
- used interviewers to gather information from occupants of the dwelling, whereas the census relied on individuals to fill in the census forms. While it is possible that people who were missed in the census may also have been missed in the PES, it is generally accepted that the coverage of the PES was more complete. The 2001 PES used more tightly controlled collection procedures and more experienced and better trained field staff than the census. Adjusting the base population The PES results are integral to the derivation of reliable national and subnational post-censal population estimates and demographic projections. Following a careful appraisal of the 1996 PES results, the Government Statistician decided to adjust the population base for net undercount for the purpose of deriving post-censal population estimates (national and subnational) as well as demographic projections (for background information and details on methodology, see Statistics New Zealand, 1998b). This was an integral part of a broader change in estimation methodology introduced for post-1996 demographic series. It also included an adjustment of the base population for New Zealand residents temporarily overseas on census night, and a shift from the traditional 'de facto' concept to a 'resident' population concept (Statistics New Zealand, 1996). Census counts were not adjusted as such. Following the approach in 1996, Statistics New Zealand intends to use an adjusted base population for deriving the post-2001 population estimates and projections. The base population will be adjusted for the census undercount, as estimated by the 2001 Post-enumeration Survey, and for the estimated number of New Zealand residents temporarily away overseas at census. Once again, the ‘adjustment methodology’ is perceived as a restricted statistical initiative. The 2001 Census figures will not be adjusted as such. Post-censal demographic series will use as the base the estimated resident population at 30 June 2001.
The assumption implicit in this approach is that the adjusted base yields a more realistic estimate of the people who, for example, normally reside in an area, pay rates, etc, and therefore should be regarded as the target population for planning, administrative and decision-making purposes.
Miscount and its sources In such a large and complex exercise as a census, it is inevitable that some people and dwellings will be missed. Reasons for people being missed are many and varied and include:
- failure of census field procedures
- dwellings entirely missed by enumerators
- occupied dwellings misclassified as vacant
- people deliberately avoiding the census – refusing or unwilling to respond (for fear that information given will be used against their interests)
- people being reluctant to open their door to strangers
- people shifting from one house to another around the time of the census
- multiple households living at the same address
- people being away temporarily (eg work, school)
- people having no usual residence (eg transients, street kids)
- newborn babies being overlooked.
On the other hand, a person may be counted more than once. Such ‘double counting’ may involve:
- students living away at school or university (and also being counted at the home of their parents)
- children under joint custody
- people living away from home while working
- people shifting from one house to another around the time of the census
- people living in institutions
- people with dual residences
- erroneous enumeration of deceased persons, babies born after census night, emigrants, etc
- vacant dwellings classified as occupied (non-respondent) dwellings, leading to the creation of substitute forms. It should be noted that an error in recording the correct geographic location of a person does not constitute a coverage error. For example, a person who is enumerated in the wrong area does not represent overcoverage for the area in which the person was enumerated nor undercoverage for the area in which the person should have been enumerated.
Why measure the undercount? In New Zealand there is growing interest in accurately determining the level of coverage of the census. This helps improve both census processes and the general quality of post-censal population estimates. Population estimates are used for a variety of purposes:
- allocation of funds to organisations using population-based weightings
- denominators for the calculation of rates (eg birth and death rates) and per capita time series
- determination of population weights for various surveys
- administration, policy-making and planning, by both central and local government
- demographic, social and economic studies. The experience of other countries is that those groups who are mobile or difficult to enumerate are most likely to be missed in the census. In New Zealand, indirect evidence drawn from demographic analysis, birth registrations and school enrolments suggests that the census does miss some people and that this under-enumeration varies across different groups.
Sample design The 2001 PES adopted the sample design of Statistics New Zealand’s Household Labour Force Survey (HLFS). The main reasons for choosing the HLFS design included:
- reduced costs for enumeration and field collection:
- interviewers were already working in and familiar with the geographic areas used in the sample
- fieldwork made use of existing maps and street listings
- minimisation of respondent burden by controlling the overlap between the PES and other household surveys. The sampling process was rather complex (Figure 1). The geographical framework of New Zealand consists of 36,946 meshblocks (a meshblock in urban areas is usually a block of residential area containing about 40 dwellings surrounded by streets; in rural areas a meshblock covers a much wider area because dwellings are sparsely spread). For the purposes of the HLFS, these meshblocks are aggregated into 19,102 Primary Sampling Units (PSUs). To improve the sampling efficiency these PSUs are stratified into 120 groups (or strata) based on region, urban/rural mix, Māori population, and other socio-economic variables (income, employment status, age 65+ population). Each stratum consists of about 160 PSUs on average. Across the 120 strata, 1,760 PSUs have been randomly selected for the HLFS. The PES randomly selected PSUs from among these, using sampling factions dependent on the stratum characteristics:
- 7/8 of PSUs from strata with high numbers of Māori, Pacific and Asian residents
- 5/8 of PSUs from other strata in the Auckland Region or with high numbers of Māori residents
- 1/2 of PSUs for all other strata. Overseas studies suggest that ethnic minorities and young persons are more likely to be missed by the census (Australian Bureau of Statistics 1997, US Bureau of the Census 1996). The higher sample ratios for ethnically diverse areas and for Auckland were thus designed to help increase the accuracy of the undercount estimates for subgroups of the population by reducing their sample errors. Each PSU in the HLFS comprises at least seven panels, and each panel consists of about nine randomly selected private dwellings. Most of the panels within a PSU are used by the HLFS on a rotational basis with one panel being used for each survey quarter in a year. A spare panel from each of the PSUs was used to make up the PES sample frame for sample selection. The 2001 PES sample comprised 999 PSU panels containing about 11,000 dwellings (or about 0.7 percent of total permanent private dwellings in New Zealand).
Sample selection The 2001 sample selected was subject to a small error, in that a "deathed" indicator on the household frame was overlooked. This meant that the probability of selection of a dwelling was in doubt. An adequate approximation was made from the frame, by including the deathed units. The effect is expected to be negligible, and the ratio estimator used to estimate the true population will correct any net over or under weighting.
Final Estimates of Person Coverage The last step in the estimation is to produce estimates of the level of undercount and coverage. The level of net undercount is defined as the net difference between gross overcount and gross undercount. The level of coverage is expressed as a percentage of the total population. The total population is the PES estimate of true census population.en-NZ