Research and Development Survey

The Research and Development Survey is run by Stats NZ jointly with the Ministry of Business, Innovation and Employment (MBIE). This survey measures the level of research and development activity, employment and expenditure for the business, government and higher education (university) sectors in New Zealand. It is a biennial survey, posted in August to approximately 3500 respondents. The survey has been run since 1989, with a number of changes in methodology over that time.

The Research and Development Survey was designed to collect data as defined by the Frascati Manual (2002). The definition had been reviewed in 2016 by using the latest version of the Frascati Manual (2015). Research and development comprises creative work undertaken on a systematic basis in order to increase the stock of knowledge. Any activity classified as research and development is characterised by originality. Investigation is a primary objective. Research differs from studies, in that studies involve collecting, processing and analysing data but are usually not characterised by novelty and innovation. The purpose of this survey is to collect data which will be used to produce summarised statistics of research and development activities for release to government, business and other users in the community. The statistics will be used in the development of science policy areas. This survey is partly sponsored by the Ministry of Business, Innovation and Employment (MBIE).

General information

Survey background

The Research and Development Survey (R&D Survey) was jointly developed by Stats NZ and the Ministry of Business, Innovation, and Employment (MBIE).

The R&D Survey measures the level of R&D activity, employment, and expenditure by businesses, government departments, government-owned trading entities, and higher education (universities). The R&D Survey is conducted every two years by Stats NZ.

Data collection

The R&D Survey is a postal survey consisting of four questionnaires: a business questionnaire, a government questionnaire, a CRI questionnaire, and a higher education (universities) questionnaire. These questionnaires are specifically designed to capture data on R&D from these different organisation types.

Information collected included:

• the number of personnel within an enterprise working on R&D
• current and capital expenditure on R&D
• expenditure by type of R&D
• source of funds for R&D carried out
• the area of application of the R&D.

We requested information for the last financial year within the 12 months ending 30 September.

Target population

The target population is all economically significant enterprises we have pre-identified as performing or funding R&D activities in New Zealand.

A range of information sources are available that allow us to identify R&D performers. These, combined with the last few iterations of this survey, have allowed us to build a consistent picture of the types of firms carrying out R&D in New Zealand. These changes were first made to the survey in 2012 and have allowed us to more intensively survey these businesses and create a better picture of individual R&D performers and their characteristics.

Enterprises (business, government, and CRI) are included in the R&D Survey population if they:

• are economically significant and active on the Stats NZ Business Register
• are not classified to ANZSIC06 codes 'G', 'H', 'I', or 'P'
• are a university
• fulfil one or more of the following indicators of R&D activity:

• enterprises indicating they undertook R&D in the Annual Frame Update Survey
• enterprises applying for funding from the Ministry of Business, Innovation and Employment (and its predecessor agencies)
• enterprises applying for patents in the last two years
• enterprises recording R&D activity in the last BOS or the previous two R&D surveys (only from full coverage strata)

Excluding ANZSIC division codes ‘H’ (accommodation and food services), ‘G’ (retail trade), and ‘I’ (transport, postal and warehousing) is due to the previous equivalents of these industries showing little or no contribution to the total reported expenditure on R&D in the 2002 survey. We considered such contributions too small to justify their inclusion in the survey population so the equivalent industries have been excluded since the R&D Survey: 2004. ANZSIC division ‘P’ (education and training) is excluded, with the exception of universities, who perform the vast majority of R&D in this industry.

We also sent the higher education (universities) questionnaire in August. Data was collected for the year ended 31 December. The higher education (universities) questionnaire was designed to allow universities to use financial information that is generally produced for annual reporting purposes. This means that a number of data items for universities' R&D were produced using modelled information. Universities New Zealand – Te Pōkai Tara assisted us to determine these modelling specifications. Information collected included university discretionary income, internal and external research funding, academic staff salaries, university operating expenditure by faculty, and R&D personnel data.

Measurement errors

The R&D Survey results are subject to measurement errors. Customers need to consider these when analysing the results from the survey.

Measurement errors include mistakes by respondents when completing the questionnaire, variation in respondents' interpretation of the questions asked, and errors made during data processing. In addition, the survey applies imputation methodologies to cope with non-respondents and item non-response (see Imputation methodology for more detail). These methods are not without error.

We adopt procedures to minimise these types of errors, but they may still occur and are not quantifiable.

Given the nature of the data collected, there are limitations on the level of accuracy that can be expected from the R&D Survey. Many respondents do not keep a separate account of their R&D expenditure, or they may include R&D with other scientific and technological services, such as consulting.

Analysis of results

We compare the R&D Survey results with annual reports and other indicators we publish. Where the survey results differ substantially, more detailed study of the data is made.

Imputation methodology

This section outlines the imputation methodology we use in the R&D Survey (business, government, and CRIs). No unit non-response was required for the R&D higher education (universities) survey, as a 100 percent response rate was achieved.

Unit non-response

Unit (or complete) non-response occurs where units in the population do not return the questionnaire, or an invalid questionnaire is received. We use a weight adjustment method to rate up the responding firms to compensate for the non-responding firms within the same estimation cell. The data from responding firms are multiplied by the inverse of the response rate for the estimation cell.

We remove any enterprises that cease operation during the survey period from the weight calculations.

Item non-response

Item (or partial) non-response is where units return the questionnaire but fail to provide data for selected aggregates.

We apply item non-response imputation to breakdowns where a total can be sourced from another question and personnel questions where data is not provided. The item non-response imputation method then uses the mean proportion of all responding linked units (excluding outliers) within the item non-response estimation cell, and we apply these proportions to the sourced total. For personnel questions the totals are imputed using a similar method.

Published sector and industry breakdowns

The published sector and industry breakdowns provided in this release are created using recommendations from the OECD's Frascati Manual 2002 to allow for greater international comparability.

This manual recommends that state-owned enterprises (Business Type 1996 classification) be classified to the business sector. In addition, the Frascati Manual 2002 recommends that the industrial classification code for significant research organisations be changed to the industry they predominantly serve. We apply the industry breakdowns using the Australian and New Zealand Standard Industrial Classification 2006 (ANZSIC06).

Business sector

Includes central and local government-owned trading enterprises and all other enterprises with the following New Zealand Institutional Sector (NZISC)1996 codes.

Government sector (excluding universities)

The government sector excludes the eight universities, and central and local government trading enterprises, and includes all enterprises with the NZISC96 codes included in the table below.

1. Central and local government trading enterprises are including in business sector.
2. All District Health Boards were included in R&D 2016 survey population.

Higher education (universities)

The higher education (universities) sector includes the eight New Zealand universities that are members of Universities New Zealand – Te Pōkai Tara. These are classified to NZISC96 code 3111 (central government excluding funded social security), with an ANZSIC06 code of P810200 (higher education).

Included within the higher education sector are universities' commercial arms. Before 2010, they were represented in the business sector.

Published industries

The published industries within the business sector are based on ANZSIC06 classification, apart from the reclassification of significant scientific research organisations (M691) to the industry they predominately serve and the inclusion of local and state-owned trading enterprises. Published industries are listed below, followed by their ANZSIC06 codes.

Business sector

• Primary industries – A and B
• Food product manufacturing – C11
• Beverage and tobacco manufacturing – C12
• Textile, clothing, footwear, and leather manufacturing – C13
• Petroleum, coal, chemical, and associated product manufacturing – C17, C18, and C19
• Non-metallic mineral product manufacturing – C20
• Metal product manufacturing – C21 and C22
• Machinery and equipment manufacturing – C23 and C24
• Other manufacturing – C14, C15, C16, and C25
• Wholesale trade – F
• Scientific research and technical services – M691 and M692 (excluding M6924)
• Computer services – M70
• Other services – D to S excluding (F, M691, M6921, M6922, M6923, M6925, M70)

Government sector

• Scientific research – M691
• Other government research – All ANZSIC codes except M691

Higher education (universities) sector

Total universities, including their commercial arms.

Data presentation

In 2016, we started presenting data in Infoshare. These tables show data available back to 2008, to allow easier time-series comparison.

The R&D Survey uses a stratified sample in its sample design. We developed strata based on industries defined by their sector (ie business, government, or higher education (universities)) and ANZSIC06.

Substrata were then developed using RME and annual GST from the Statistics NZ Business Frame. These are both captured from tax data.

We made some of these substrata full coverage, meaning we selected all enterprises in the substratum for the survey. Within the full coverage substrata, we identified 'keys' for intensive attention in the data collection phase. Keys are enterprises that made significant R&D expenditure in 2014.