Integrated data tools — Stats NZ
New Zealand is leading the world with its use of data integration for research and analysis, and Stats NZ is at the forefront.
Stats NZ's Integrated Data is a critical tool in the government's efforts to reduce poverty, improve health, and support a growing, inclusive economy. It enables evidence-based insights, and measurement of the effectiveness of government initiatives and services.
The Integrated Data Infrastructure (IDI) and Longitudinal Business Database (LBD) are powerful integrated data tools that bring together information from government agencies, Stats NZ surveys, and non-government organisations.
Stats NZ, as steward of the IDI and LBD, looks after the technology of these databases and ensures researchers meet the access conditions that keep the data safe.
Stats NZ's world-leading Integrated Data tools
The IDI contains information about people and households and aims to include all people living in New Zealand. This allows researchers to compare outcomes across the population. It can shine a light on the impact events have in people's lives in areas such as education, income, benefits, migration, justice and health.
The LBD contains information about businesses, enabling researchers to look at groups of businesses across their lifetime to see how they develop and change. The LBD is linked to the IDI through tax data on employers and employees, allowing further analysis of the impact of events on people and businesses over time.
> Detailed description of diagram
This diagram gives a basic overview of what kind of data is supplied to the Integrated Data Infrastructure (IDI) and Longitudinal Business Database (LBD) databases. Government agencies and non-government organisations provide data such as education data to Stats NZ to be linked to a central dataset called the 'spine'. The IDI spine aims to include all people living in New Zealand. The LBD spine aims to include all employees and employers so outcomes can be compared across the population. The diagram also shows how these two databases are linked to each other, providing further insights to researchers.
The diagram has two parts. On the left is a rectangle representing the spine of the IDI. Eight lines radiate out from the rectangle. At the end of each line is a circle that represents a category of data in the IDI. These circles are: people and communities' data, education and training data, income and work data, benefits and social services data, population data, health data, justice data and housing data.
On the right of the diagram is a circle representing the spine of the LBD. Six lines radiate out from the circle. At the end of each line is a circle that represents a category of data in the LBD. These circles are: innovation data, business financials data, agriculture data, international trade data, business practices data and employment data.
There is a dotted line running from the income and work data circle on the IDI side to the employment data circle on the LBD side. The line is labelled 'The IDI and LBD are linked through tax data'.
Responsible data custodians
The IDI and LBD are created by linking data originally collected for different purposes, through common variables. Their profiles and collation of a variety of data sources about individuals and businesses mean they pose particular security challenges.
Stats NZ is committed to safe access to integrated data, protection of individual privacy, and maintaining the public's trust and confidence. Stats NZ has in place a 'five safes' framework to keep data safe and ensure access to integrated data is only provided if all conditions in the framework are met. Before approved researchers access the IDI or LBD, they must prove their project is in the interest of the public good. Information that could identify individual people or businesses is removed or replaced. This keeps the data and privacy of people and businesses safe.
How Integrated Data is changing New Zealanders' lives
Researchers generate insights from integrated data which is used by policy and decision makers to improve the social and economic outcomes of New Zealanders.
Case study: Otago youth not in employment, education or training (NEET)
Stats NZ worked in partnership with the non-government organisation Methodist Mission Southern (MMS) to provide information about youth who were NEET. MMS uses data to understand the characteristics of NEET youth. This helps shape targeted, effective services to support young people in Otago to successfully manage their transition into employment or further education.
The future of Integrated Data
We are only just beginning to scratch the surface of what is possible with integrated data and Stats NZ is readying itself for the opportunities the future will bring.
The IDI has expanded beyond what was initially envisioned, both in terms of the data it contains and the number of users. Stats NZ is working on an expansion programme which will make the IDI and LBD easier to use, quicker and even more robust.
We are starting with simplifying and improving the way our customers access the IDI. This will increase the use and usability of the IDI, making more data available to a wider range of customers. This includes developing a tool that transforms a number of the most commonly used variables (like region most lived in and income) in the IDI into a more user friendly, visual format. Our goal is to build infrastructure that will work in a big data environment and increase usage of Stats NZ data for the good of New Zealand.
The IDI is part of a wider data ecosystem both within and beyond government that will deliver more efficient and effective services through better data-based evidence about what works for New Zealanders.
For more information about Stats NZ Integrated Data
Integrated Data featured in the Digital Government Showcase, which was part of Digital 5 2018, the 4th annual gathering of the world's most advanced digital nations that NZ hosted in February 2018.