Research and Development Survey

The Research and Development Survey is run by Stats NZ jointly with the Ministry of Business, Innovation and Employment (MBIE). This survey measures the level of research and development activity, employment and expenditure for the business, government and higher education (university) sectors in New Zealand. It is a biennial survey, posted in August to approximately 3500 respondents. The survey has been run since 1989, with a number of changes in methodology over that time.

The Research and Development Survey was designed to collect data as defined by the Frascati Manual (2002). The definition had been reviewed in 2016 by using the latest version of the Frascati Manual (2015). Research and development comprises creative work undertaken on a systematic basis in order to increase the stock of knowledge. Any activity classified as research and development is characterised by originality. Investigation is a primary objective. Research differs from studies, in that studies involve collecting, processing and analysing data but are usually not characterised by novelty and innovation. The purpose of this survey is to collect data which will be used to produce summarised statistics of research and development activities for release to government, business and other users in the community. The statistics will be used in the development of science policy areas. This survey is partly sponsored by the Ministry of Business, Innovation and Employment (MBIE).

General information

Survey background

The Research and Development Survey (R&D Survey) was jointly developed by Stats NZ and the Ministry of Business, Innovation, and Employment (MBIE).

The R&D Survey measures the level of R&D activity, employment, and expenditure by businesses, government departments, government-owned trading entities, and higher education (universities). The R&D Survey is conducted every two (even) years by Stats NZ.

Since 2019, special business sector only iterations of the survey have been run every two (odd) years, outside of the normal two yearly all sectors cycle. The purpose of these special iterations was to gain accurate yearly estimates of R&D expenditure, to be able to contribute to the review of the IRD’s R&D tax incentive.

Data collection

The R&D Survey is a postal survey consisting of four questionnaires: a business questionnaire, a government questionnaire, a CRI questionnaire, and a higher education (universities) questionnaire. These questionnaires are specifically designed to capture data on R&D from these different organisation types.

Information collected included:

• the number of personnel within an enterprise working on R&D
• current and capital expenditure on R&D
• expenditure by type of R&D
• source of funds for R&D carried out
• the area of application of the R&D.

We requested information for the last financial year within the 12 months ending 30 September.

Target population

The target population is all economically significant enterprises we have pre-identified as performing or funding R&D activities in New Zealand.

A range of information sources are available that allow us to identify R&D performers. These, combined with the last few iterations of this survey, have allowed us to build a consistent picture of the types of firms carrying out R&D in New Zealand. These changes were first made to the survey in 2012 and have allowed us to more intensively survey these businesses and create a better picture of individual R&D performers and their characteristics.

Enterprises (business, government, and CRI) are included in the R&D Survey population if they:

• are economically significant and active on the Stats NZ Business Register
• are not classified to ANZSIC06 codes 'G', 'H', 'I', or 'P'
• are a university

• fulfil one or more of the following indicators of R&D activity:

• enterprises indicating they undertook R&D in the Annual Frame Update Survey
• enterprises applying for funding from the Ministry of Business, Innovation and Employment (and its predecessor agencies)
• enterprises applying for patents in the last two years
• enterprises recording R&D activity in the last 2 Business Operations Surveys (BOS) or the previous three R&D surveys

Excluding ANZSIC division codes ‘H’ (accommodation and food services), ‘G’ (retail trade), and ‘I’ (transport, postal and warehousing) is due to the previous equivalents of these industries showing little or no contribution to the total reported expenditure on R&D in the 2002 survey. We considered such contributions too small to justify their inclusion in the survey population so the equivalent industries have been excluded since the R&D Survey: 2004. ANZSIC division ‘P’ (education and training) is excluded, with the exception of universities, who perform the vast majority of R&D in this industry.

We also sent the higher education (universities) questionnaire in August. Data was collected for the year ended 31 December. The higher education (universities) questionnaire was designed to allow universities to use financial information that is generally produced for annual reporting purposes. This means that a number of data items for universities' R&D were produced using modelled information. Universities New Zealand – Te Pōkai Tara assisted us to determine these modelling specifications. Information collected included university discretionary income, internal and external research funding, academic staff salaries, university operating expenditure by faculty, and R&D personnel data.

There are usually around 4,000 to 5,000 enterprises in the population.

Statistical unit

Enterprises are selected from the Stats NZ Business Register, which is a comprehensive database of all known individual, private, and public sector businesses and organisations engaged in producing goods and services in New Zealand that meet significance criteria. Its primary purpose is as a statistical register, or frame, for the various business surveys operated by Stats NZ. The Business Register provides a consistent reference to standard classifications, which helps to integrate statistical outputs. The register also provides links to all economic and financial survey data, and the tax system, which means a more effective use of tax data that reduces respondent load.

We use external data sources to identify potential businesses for the population, but we need to match to our Business Register in order to be able to identify whether or not they are in scope for the survey or not. This is not always feasible, meaning some businesses may not be included in the population as we could not find an equivalent entry for them in our Business Register.

Sampling Procedure

The R&D Survey uses a stratified sample in its sample design. We developed strata based on industries defined by their sector (ie business, government, or higher education (universities)) and ANZSIC06 (ie industry), and key units.

We made some of these strata full coverage, meaning we selected all enterprises in the substratum for the survey. Keys are enterprises that had significant R&D expenditure in the previous three iterations of the survey. We identified 'keys' for intensive attention in the data collection phase,and are full coverage strata.

The government sector is also full coverage, as well as the universities portion of the higher education sector.

The sample size is usually around 4,000 to 5,000.

Sample maintenance

Sample maintenance is the process that maintains the sample over time, to reflect 'births', 'deaths' and other structural changes identified on the Business Register. Business Register changes can be from a variety of sources, including GST registrations and respondent contact.

We identify new enterprises when they register for GST. Once a year, the new enterprises are selected into the sample using the same criteria as for the original sample. These are referred to as births. When an enterprise ceases trading, we remove them from the survey. These are referred to as deaths.

Enterprises can also enter or leave the survey sample if they are reclassified to a different industry. Reclassifications occur when an enterprise changes its main form of activity (eg from ‘Corporate Head Office Management Services’ to ‘Computer Systems Design and Related Services’). We usually identify these in the Business Register Update Survey conducted in February of each year.

Sample reselection

We select the sample for the R&D Survey each year to ensure the sample reflects changes occurring in the New Zealand economy.

Questionnaire

The questionnaire for R&D Survey is a postal form that is sent to respondents around August of the collection year. Electronic pdf versions of the questionnaire are available to respondents on request.

Measurement errors

The R&D Survey results are subject to measurement errors. Customers need to consider these when analysing the results from the survey.

Non-sampling errors

Measurement errors include mistakes by respondents when completing the questionnaire, variation in respondents' interpretation of the questions asked, and errors made during data processing.

Sampling errors

The estimates in this release are based on a sample of business. Somewhat different figures might have been obtained if a complete census of the entire business population had been taken using the same questionnaire and processing methods etc. Because the estimates are based on a sample of businesses, all estimates have a sampling error associated with them. The variability of a survey estimate, due to the random nature of the sample selection process, is measured by its sampling error.

Sampling errors vary from estimate to estimate, and with population breakdown and population size.

Imputation methodology

In addition, the survey applies imputation methodologies to cope with non-respondents and item non-response (see Imputation methodology for more detail). These methods are not without error.

We adopt procedures to minimise these types of errors, but they may still occur and are not quantifiable.

Given the nature of the data collected, there are limitations on the level of accuracy that can be expected from the R&D Survey. Many respondents do not keep a separate account of their R&D expenditure, or they may include R&D with other scientific and technological services, such as consulting.

R&D expenditure estimates from other data sources

There are other data sources that collect R&D expenditure estimates from businesses. These differ from each other in terms of scope of expenditure inclusions, sample selection, definition etc. More information on the similarities and differences between these data sources, and a comparison of the R&D Survey expenditure estimates, can be found here:

An exploration into research and development (R&D) expenditure – definitions and comparative data | Stats NZ

Imputation methodology

This section outlines the imputation methodology we use in the R&D Survey (business, government, and CRIs). Unit non-response treatment is usually not required for the R&D higher education (universities) survey, since to usually achieving a 100 percent response rate.

Unit non-response

Unit (or complete) non-response occurs where units in the population do not return the questionnaire, or an invalid questionnaire is received. We use a weight adjustment method to rate up the responding firms to compensate for the non-responding firms within the same estimation cell. The data from responding firms are multiplied by the inverse of the response rate for the estimation cell.

We remove any enterprises that cease operation during the survey period from the weight calculations. If unit non-response is required for university units, then we use a combination of other data sources (such as annual reports), and forward adjustment movements on the previous years data, to impute any missing responses.

Item non-response

Item (or partial) non-response is where units return the questionnaire but fail to provide data for selected aggregates.

We apply item non-response imputation to breakdowns where a total can be sourced from another question and personnel questions where data is not provided. The item non-response imputation method then uses the mean proportion of all responding linked units (excluding outliers) within the item non-response estimation cell, and we apply these proportions to the sourced total. For personnel questions the totals are imputed using a similar method. In some cases, we may impute the total R&D expenditure from either the Business Operations Survey, or a businesses annual report.

Published sector and industry breakdowns

The published sector and industry breakdowns provided in this release are created using recommendations from the OECD's Frascati Manual 2002 to allow for greater international comparability.

This manual recommends that state-owned enterprises (Business Type 1996 classification) be classified to the business sector. In addition, the Frascati Manual 2002 recommends that the industrial classification code for significant research organisations be changed to the industry they predominantly serve. We apply the industry breakdowns using the Australian and New Zealand Standard Industrial Classification 2006 (ANZSIC06).

Business sector

Includes central and local government-owned trading enterprises and all other enterprises with the following New Zealand Institutional Sector (NZISC)1996 codes.

Government sector (excluding universities)

The government sector excludes the eight universities, and central and local government trading enterprises, and includes all enterprises with the NZISC96 codes included in the table below.

1. Central and local government trading enterprises are including in business sector.
2. All District Health Boards were included in R&D 2016 survey population.

Higher education (universities)

The higher education (universities) sector includes the eight New Zealand universities that are members of Universities New Zealand – Te Pōkai Tara. These are classified to NZISC96 code 3111 (central government excluding funded social security), with an ANZSIC06 code of P810200 (higher education).

Included within the higher education sector are universities' commercial arms. Before 2010, they were represented in the business sector.

Published industries

The published industries within the business sector are based on ANZSIC06 classification, apart from the reclassification of significant scientific research organisations (M691) to the industry they predominately serve and the inclusion of local and state-owned trading enterprises. Published industries are listed below, followed by their ANZSIC06 codes.

Business sector

• Primary industries – A and B
• Food product manufacturing – C11
• Beverage and tobacco manufacturing – C12
• Textile, clothing, footwear, and leather manufacturing – C13
• Petroleum, coal, chemical, and associated product manufacturing – C17, C18, and C19
• Non-metallic mineral product manufacturing – C20
• Metal product manufacturing – C21 and C22
• Machinery and equipment manufacturing – C23 and C24
• Other manufacturing – C14, C15, C16, and C25
• Wholesale trade – F
• Scientific research and technical services – M691 and M692 (excluding M6924)
• Computer services – M70
• Other services – D to S excluding (F, M691, M6921, M6922, M6923, M6925, M70)

Government sector

• Scientific research – M691
• Other government research – All ANZSIC codes except M691

Higher education (universities) sector

Total universities, including their commercial arms.

Data presentation

In 2016, we started presenting data in Infoshare. These tables show data available back to 2008, to allow easier time-series comparison.

Confidentiality

Stats NZ has a legal obligation to protect survey respondents' privacy and industry-sensitive information. Confidentiality rules are applied to all data released – to protect the information supplied by an individual company. Once all confidential financial items are identified, suppression of further items is done to complete the protection of the confidential value.

Liability

While all care and diligence has been used in processing, analysing, and extracting data and information in this publication, Statistics NZ gives no warranty it is error-free and will not be liable for any loss or damage suffered by the use directly, or indirectly, of the information in this publication.