The Research and Development Survey is joint collection between Stats NZ and the Ministry of Business, Innovation and Employment (MBIE). This survey measures the level of research and development activity, employment and expenditure for the business, government and higher education (university) sectors in New Zealand. It currently is an annual survey, that alternates between business only sectors in odd years, and all three sectors (business, government and higher education) in even years. It is posted in August to approximately 4000 respondents. The survey has been run since 1989, with a number of changes in methodology over that time.

The Research and Development Survey was designed to collect data as defined by the Frascati Manual (2002). The definition had been reviewed in 2016 by using the latest version of the Frascati Manual (2015). Research and development comprises creative work undertaken on a systematic basis in order to increase the stock of knowledge. Any activity classified as research and development is characterised by originality. Investigation is a primary objective. Research differs from studies, in that studies involve collecting, processing and analysing data but are usually not characterised by novelty and innovation. The purpose of this survey is to collect data which will be used to produce summarised statistics of research and development activities for release to government, business and other users in the community. The statistics may be used in the development of science policy areas. This survey is partly sponsored by the Ministry of Business, Innovation and Employment (MBIE).

Survey background

The Research and Development Survey (R&D Survey) was jointly developed by Stats NZ and the Ministry of Business, Innovation, and Employment (MBIE).

The R&D Survey measures the level of R&D activity, employment, and expenditure by businesses, government departments, government-owned trading entities, and higher education (universities). The R&D Survey is conducted every two (even) years by Stats NZ.

Since 2019, special business sector only iterations of the survey have been run every two (odd) years, outside of the normal two yearly all sectors cycle. The purpose of these special iterations was to gain accurate yearly estimates of R&D expenditure, to be able to contribute to the review of the IRD’s R&D tax incentive.

Data collection

The R&D Survey is a postal survey consisting of four questionnaires: a business questionnaire, a government questionnaire, a CRI questionnaire, and a higher education (universities) questionnaire. These questionnaires are specifically designed to capture data on R&D from these different organisation types.

Information collected included:

• the number of personnel within an enterprise working on R&D
• current and capital expenditure on R&D
• expenditure by type of R&D
• source of funds for R&D carried out
• the area of application of the R&D.

We requested information for the last financial year within the 12 months ending 30 September.

Target population

The target population is all economically significant enterprises we have pre-identified as performing or funding R&D activities in New Zealand. There are usually around 4,000 to 5,000 enterprises in the R&D survey population.

A range of information sources are available that allow us to identify R&D performers. These, combined with the last few iterations of this survey, have allowed us to build a consistent picture of the types of firms carrying out R&D in New Zealand. These changes were first made to the survey in 2012 and have allowed us to more intensively survey these businesses and create a better picture of individual R&D performers and their characteristics.

Enterprises (business, government, and CRI) are included in the R&D Survey population if they:

• are economically significant and active on the Stats NZ Business Register
• are not classified to ANZSIC06 codes 'G', 'H', 'I', or 'P'
• are a university

• fulfil one or more of the following indicators of R&D activity:

• enterprises indicating they undertook R&D in the Annual Frame Update Survey
• enterprises applying for funding from the Ministry of Business, Innovation and Employment (and its predecessor agencies)
• enterprises applying for patents in the last two years
• enterprises recording R&D activity in the last 2 Business Operations Surveys (BOS) or the previous three R&D surveys

Excluding ANZSIC division codes ‘H’ (accommodation and food services), ‘G’ (retail trade), and ‘I’ (transport, postal and warehousing) is due to the previous equivalents of these industries showing little or no contribution to the total reported expenditure on R&D in the 2002 survey. We considered such contributions too small to justify their inclusion in the survey population so the equivalent industries have been excluded since the R&D Survey: 2004. ANZSIC division ‘P’ (education and training) is excluded, with the exception of universities, who perform the vast majority of R&D in this industry.

We also sent the higher education (universities) questionnaire in August. Data was collected for the year ended 31 December. The higher education (universities) questionnaire was designed to allow universities to use financial information that is generally produced for annual reporting purposes. This means that a number of data items for universities' R&D were produced using modelled information. Universities New Zealand – Te Pōkai Tara assisted us to determine these modelling specifications. Information collected included university discretionary income, internal and external research funding, academic staff salaries, university operating expenditure by faculty, and R&D personnel data. The R&D survey higher education sector data only includes information from universities. Institutes of technology and polytechnics (ITP’s) and Wānanga are not included in the R&D survey due to their relatively small contribution to R&D expenditure, and they tend to have very low response rates.

Statistical unit

Enterprises are selected from the Stats NZ Business Register, which is a comprehensive database of all known individual, private, and public sector businesses and organisations engaged in producing goods and services in New Zealand that meet significance criteria. Its primary purpose is as a statistical register, or frame, for the various business surveys operated by Stats NZ. The Business Register provides a consistent reference to standard classifications, which helps to integrate statistical outputs. The register also provides links to all economic and financial survey data, and the tax system, which means a more effective use of tax data that reduces respondent load.

We use external data sources to identify potential businesses for the population, but we need to match to our Business Register in order to be able to identify whether or not they are in scope for the survey or not. This is not always feasible, meaning some businesses may not be included in the population as we could not find an equivalent entry for them in our Business Register.

Sampling Procedure

The R&D Survey uses a stratified sample in its sample design. We developed strata based on industries defined by their sector (ie business, government, or higher education (universities)) and ANZSIC06 (ie industry), and key units. Due to the fact that R&D can vary greatly between similar businesses, and across time, the R&D survey requires a high sampling fraction to be able to obtain accurate and representative results for the entire NZ economy. This means there is significant overlap of the businesses between years.

We made some of these strata full coverage, meaning we selected all enterprises in the substratum for the survey. Keys are enterprises that had significant R&D expenditure in any of the previous three iterations of the survey. We identified 'keys' for intensive attention in the data collection phase,and are full coverage strata.

The government sector is also full coverage, as well as the universities portion of the higher education sector.

The sample size is usually around 4,000 to 5,000.

Sample maintenance

Sample maintenance is the process that maintains the sample over time, to reflect 'births', 'deaths' and other structural changes identified on the Business Register. Business Register changes can be from a variety of sources, including GST registrations and respondent contact.

We identify new enterprises when they register for GST. Once a year, the new enterprises are selected into the sample using the same criteria as for the original sample. These are referred to as births. When an enterprise ceases trading, we remove them from the survey. These are referred to as deaths.

Enterprises can also enter or leave the survey sample if they are reclassified to a different industry. Reclassifications occur when an enterprise changes its main form of activity (eg from ‘Corporate Head Office Management Services’ to ‘Computer Systems Design and Related Services’). We usually identify these in the Business Register Update Survey conducted in February of each year.

Sample reselection

We select the sample for the R&D Survey each year to ensure the sample reflects changes occurring in the New Zealand economy. It also allows us to include businesses who have recently started carrying out or funding R&D, as well as removing businesses who have recently stopped.

Questionnaire

The questionnaire for R&D Survey is a postal or electronic pdf form that is sent to respondents around August of the collection year. Electronic pdf versions of the questionnaire are available to respondents on request.

Sample questionnaires are available here:

• Business sector

• Crown Research Institutes

• Government sector

• Higher Education sector

Measurement errors

The R&D Survey results are subject to measurement errors. Customers need to consider these when analysing the results from the survey.

Non-sampling errors

Measurement errors include mistakes by respondents when completing the questionnaire, variation in respondents' interpretation of the questions asked, and errors made during data processing.

Stats NZ adopts procedures to minimise these types of errors, but they may still occur and are not quantifiable.  Given the nature of the data collected, there are limitations on the level of accuracy that can be expected from the R&D Survey. Many respondents do not keep separate account of their R&D expenditure, or they may include R&D with other scientific and technological services, such as consulting.

Sampling errors

The estimates in this release are based on a sample of business. Somewhat different figures might have been obtained if a complete census of the entire business population had been taken using the same questionnaire and processing methods etc. Because the estimates are based on a sample of businesses, all estimates have a sampling error associated with them. The variability of a survey estimate, due to the random nature of the sample selection process, is measured by its sampling error.

Sampling errors vary from estimate to estimate, and with population breakdown and population size.

Imputation methodology

In addition, the survey applies imputation methodologies to cope with non-respondents and item non-response (see Imputation methodology for more detail).

Imputation methodology

This section outlines the imputation methodology we use in the R&D Survey (business, government, and CRIs). Unit non-response treatment is usually not required for the R&D higher education (universities) sector, since usually achieving a 100 percent response rate.

Unit non-response

Unit (or complete) non-response occurs where units in the population do not return the questionnaire, or an invalid questionnaire is received. We use a weight adjustment method to rate up the responding firms to compensate for the non-responding firms within the same estimation cell. The data from responding firms are multiplied by the inverse of the response rate for the estimation cell.

We remove any enterprises that cease operation during the survey period from the weight calculations. If unit non-response is required for university units, then we use a combination of other data sources (such as annual reports), and forward adjustment movements on the previous years data, to impute any missing responses.

Item non-response

Item (or partial) non-response is where units return the questionnaire but fail to provide data for selected aggregates.

We apply item non-response imputation to breakdowns where a total can be sourced from another question and personnel questions where data is not provided. The item non-response imputation method then uses the mean proportion of all responding linked units (excluding outliers) within the item non-response estimation cell, and we apply these proportions to the sourced total. For personnel questions the totals are imputed using a similar method. In some cases, we may impute the total R&D expenditure from either the Business Operations Survey, or a businesses annual report.

Higher Education sector

Higher Education sector R&D can be difficult to quantify, due to the fact that R&D activity, expenditure and staff time can be hard to split out from the usual day-to-day activities. Therefore, the R&D survey does not directly ask for R&D expenditure from the higher education sector, butsector but instead models it from various other pieces of data sources that are easily obtained from universities. For example, we assume that 15% of total academic costs is spent on research. Similarly, personnel is also modelled, based on the assumption that 30% of total staff time is spend doing research. These assumptions were in consultation with Universities NZ, and by reviewing international papers on the nature of R&D etc.

R&D expenditure estimates from other data sources

There are other data sources that collect R&D expenditure estimates from businesses. These differ from each other in terms of scope of expenditure inclusions, sample selection, definition etc. More information on the similarities and differences between these data sources, and a comparison of the R&D Survey expenditure estimates, can be found here: An exploration into research and development (R&D) expenditure – definitions and comparative data | Stats NZ

Published sector breakdowns

The published sector breakdowns provided in this release are created using recommendations from the OECD's Frascati Manual 2002 to allow for greater international comparability.

Business sector

Includes central and local government-owned trading enterprises and all other enterprises with the following New Zealand Institutional Sector (NZISC)1996 codes.

Government sector (excluding universities)

The government sector excludes the eight universities, and central and local government trading enterprises, and includes all enterprises with the NZISC96 codes included in the table below.

1. Central and local government trading enterprises are including in business sector.

2. All District Health Boards were included in R&D 2016 survey population.

Higher education (universities)

The higher education (universities) sector includes the eight New Zealand universities that are members of Universities New Zealand – Te Pōkai Tara. These are classified to NZISC96 code 3111 (central government excluding funded social security), with an ANZSIC06 code of P810200 (higher education).

Included within the higher education sector are universities' commercial arms. Before 2010, they were represented in the business sector.

Published industries breakdown

The published industries within the business and government sectors are based on ANZSIC06 classification, apart from the reclassification of significant scientific research organisations (M691) to the industry they predominately serve and the inclusion of local and state-owned trading enterprises. Published industries are listed below, followed by their ANZSIC06 codes.

Business sector

• Primary industries – A and B
• Food product manufacturing – C11
• Beverage and tobacco manufacturing – C12
• Textile, clothing, footwear, and leather manufacturing – C13
• Petroleum, coal, chemical, and associated product manufacturing – C17, C18, and C19
• Non-metallic mineral product manufacturing – C20
• Metal product manufacturing – C21 and C22
• Machinery and equipment manufacturing – C23 and C24
• Other manufacturing – C14, C15, C16, and C25
• Wholesale trade – F
• Scientific research and technical services – M691 and M692 (excluding M6924)
• Computer services – M70
• Other services – D to S excluding (F, M691, M6921, M6922, M6923, M6925, M70)

Government sector

• Scientific research – M691
• Other government research – All ANZSIC codes except M691

Higher education (universities) sector

Total universities, including their commercial arms.

Data presentation

In 2016, we started presenting data in Infoshare. These tables show data available back to 2008, to allow easier time-series comparison.

Confidentiality

Stats NZ has a legal obligation to protect survey respondents' privacy and industry-sensitive information. Confidentiality rules are applied to all data released – to protect the information supplied by an individual company. Once all confidential financial items are identified, suppression of further items is done to complete the protection of the confidential value.

More information about confidentiality is available here: https://www.data.govt.nz/toolkit/privacy-and-security/understanding-data-confidentiality/data-confidentiality-principles-and-methods-report

Each value in a table is rounded independently, including the totals. This means that the marginal totals can differ slightly from the corresponding sum of the rows or columns, ie if the columns or rows in a table are added, they will not always equal the total given.

Due to the random element of the rounding applied to published figures, some figures may have differed across different products. Previously rounding was applied to the excel tables, the downloadable csv and the Infoshare data, independently from each other. In 2024, the 2022 and 2023 data in Infoshare was updated to ensure consistent rounding.

Future releases will have consistent rounding across different products.

Random rounding to base three

For counts of businesses or entities, we apply random rounding to base three. Random rounding to base 3 involves randomly rounding every unsuppressed count in a table to a number divisible by three.

Statistics NZ applies random rounding to base 3 to outputs by rounding values to:

• the nearest multiple of three with a probability of two-thirds (applied approximately two-thirds of the time)

• the second closest multiple of three with a probability of one-third (applied approximately one-third of the time).

Values that are already multiples of three are left unchanged.

The reasons for randomly rounding all counts, large and small, to base 3 include:

• disguising the small counts of 0, 1, 2, and 3, which have the highest disclosure risk

• protecting against the recalculation of small counts from differencing large counts

• retaining almost all of the statistical properties of the table by adding only a little noise to the larger counts.

Graduated random rounding

For counts of personnel, we apply graduated random rounding. With graduated random rounding (GRR), cells of different sizes are rounded to different bases. The rounding base gradually increases as the cell size does. This means the ratio of the rounding base to the cell size is reasonably constant.

Liability

While all care and diligence has been used in processing, analysing, and extracting data and information in this publication, Statistics Stats NZ gives no warranty it is error-free and will not be liable for any loss or damage suffered by the use directly, or indirectly, of the information in this publication.