The Commission is an independent research and advisory body that does not have a mandate to implement any policies or programmes. One of the ways we assess the impact of our work is through responses by government to recommendations made in our inquiry reports, and independent evaluations of the inquiry process and delivery we undertook to generate our recommendations.
What we do
Our work focuses on undertaking inquiries, conducting research, and promoting understanding on productivity-related matters. By carrying out high quality, innovative research, evidence-based inquiries and promoting understanding of productivity-related topics, we want to:
- explore the contributing factors to the productivity performance of Aotearoa New Zealand
- improve New Zealand’s understanding of what drives higher productivity and wellbeing, and the connection between them
- recommend policies to address those contributing factors.
We also contribute to the work of other organisations by supporting requests for insight and empirical support from the private and public sectors in relation to our past inquiries and research. This can happen years after completion of inquiries and publication of our final reports. We also provide input to government responses to our past inquiries following release of the final report.
Undertaking inquiries and reporting to referring Ministers
Inquiries are significant pieces of analysis and require a deep understanding of a topic. They are typically undertaken over 12 to 15 months. This timeframe recognises the importance of engaging extensively with interested parties and experts to ensure we consider all points of view, obtain the best available information, understand different perspectives, and test ideas. The Government chooses inquiry topics to ensure our work is relevant, and our advice relates to issues that Ministers have an interest in addressing. We are required to act independently in undertaking the inquiries set by the Government.
Conducting and publishing self-directed research
The Commission conducts research and publishes papers to provide an evidence base to offer advice to improve the productivity of Aotearoa New Zealand. This includes benchmarking New Zealand’s productivity performance over time, which is presented in our publication Productivity by the numbers. We collaborate closely with agencies actively working in productivity research. This allows us to access subject/sector specialists and benefit from the latest research and the cross-promotion of ideas and insights.
Promoting public understanding
We undertake a range of communications activities around our inquiries and research work to educate and promote understanding of productivity-related matters. We aim to reach diverse audiences through outreach activities, opinion journalism, and media articles using a range of communication channels including our website and social media.
|
Outcome(s) |
Lift the wellbeing of New Zealand; and, lift New Zealand’s productivity. |
|
Measure |
Volume and quality of formal and informal research output (e.g., inquiry reports, research working paper series, and broader commentary on trends in New Zealand productivity and wellbeing such as our regular Productivity by the numbers report) |
| Assessment method | Location of reporting |
|---|---|
|
A summary of performance evaluation material relevant to the year under review (e.g., expert reviews, surveys, and focus group reports of formal and informal outputs) |
See pages 26-36 |
|
Monitoring and review of Government responses to inquiries and Cabinet minutes, and ongoing follow-up with implementation agencies |
See page 28 |
|
Monitoring of media (including social media) commentary on formal and informal outputs |
See pages 3, 23–25, 29 |
|
Monitoring of Hansard, Select Committee reports and citation tools on formal and informal outputs |
See pages 3, 29 |
As part of our usual evaluation process, we undertook a post-inquiry evaluation on the final report and processes for our A Fair Chance for All inquiry. The results from this review have been included in our performance measures for this reporting period. We have included a comparison with the last year’s report, which covered the Immigration inquiry.
Independent expert review consultant, Dr Ruth Fischer-Smith conducted and reviewed the responses from focus groups, interviews with expert reviewers and survey participants.
The survey participants represented government officials, industry groups, businesses, iwi, academics, NGOs, social service providers and individuals, who had either engaged with the inquiry process through submissions, meetings, webinars or reading the final report. It was pleasing to see that 66% of survey participants had not engaged with the Commission previously. This shows the progress we have made in broadening the reach of our work.
- 146 survey participants
- 66% had not engaged with the Commission previously.
Impact indicators of our work
In this reporting period we received interim responses from the Government to our Immigration: Fit for the future inquiry report (April 2023), and our A Fair Chance for All: Breaking the cycle of persistent disadvantage inquiry report (September 2023). Formal responses for both reports are expected in 2024. To appropriately measure the impact of these inquiries on policy and behaviour change, we need to await full responses from the Government. Although we received a full response from the Government to our Frontier Firms follow-on review, this is not captured in the measures for this reporting period as it was not a full inquiry.
Impact indicator: Policies and behaviours change as a result of the Commission’s work
|
Measure |
Assessment 2021/22 |
Assessment 2022/23 |
|---|---|---|
|
Commission recommendations are explored, agreed, and implemented: |
Awaiting full response from Government. |
Awaiting full response from Government. |
|
How fully were the recommendations implemented or actively explored by the relevant policy makers? |
Awaiting full response from Government. |
Awaiting full response from Government. |
|
Understanding of productivity-related matters increases: |
46% by a little 35% by a lot |
50% by a little 37% by a lot |
1 The Commission can only inform and influence, we do not have the mandate, nor the ability to implement the recommendations we make following our inquiries.
Impact indicator: Generating discussion and debate2
| Measure | Assessment 2021/22 (All Commission work) | Assessment 2022/23 (All Commission work) |
|---|---|---|
|
Third party commentary on reports in the media: |
||
|
Generally positive sentiment.3 |
See Figure 3 below. |
|
817 mentions in the media. |
528 mentions in the media.4 |
|
Top five sources were:
|
Top five sources were:
|
|
Citing of the Commission’s work in Parliament, Select Committees, or in academic or other literature. |
The Commission was mentioned 13 times in Hansard reporting during this reporting period. We have no record of the Commission’s work cited in Select Committees or in academic or other literature during this period. |
The Commission was mentioned 21 times in Hansard reporting during this reporting period. We have no record of the Commission’s work cited in Select Committees or in academic or other literature during this period.5 |
2 We are reporting engagement for the Commission across the full year, including mentions of specific reports and inquiries.
3 In 2021/22 we did not capture the nature of the sentiment for each mention, sentiment was assessed at an aggregated level.
4 The perceived discrepancy in comparison for this number is due to the timing of the release of A Fair Chance for All inquiry report (just prior to the end of the 2022/23 reporting period), and the way the mentions were ‘filtered’ for inclusion this reporting period ensured they were all genuine mentions of NZPC and our work.
5 We are aware of places where our work is cited but we currently don’t have a tool to accurately capture this.
Figure 3 Nature of third party commentary on reports in the media, Assessment 2022/23
Impact indicator: Levels of engagement and response
| Measure | Assessment 2021/22 (Immigration) |
Assessment 2022/23 (A Fair Chance for All) |
|---|---|---|
|
Productivity analysis and advice improves |
||
|
Inquiry participants surveyed who agreed or strongly agreed that the inquiry helped to set or lift the standard in New Zealand for high-quality analysis and advice on [the topic]. |
6% strongly agreed 55% agreed |
25% strongly agreed 32% agreed |
|
Inquiry participants surveyed who agreed or strongly agreed that they will use the inquiry report as a resource and reference in the future. |
25% strongly agreed 39% agreed |
32% strongly agreed 44% agreed |
|
Measure |
Assessment 2022/23 |
|---|---|
|
Expert reviewer and focus group commentary on the quality of analysis and advice in the inquiry and if they will use the inquiry report as a resource and reference in the future. |
A Fair Chance for All inquiry: Expert review The value of the combined findings in the final report, created a valuable reference document to inform policy making and social change on the drivers behind persistent disadvantage and the public sector mechanisms that can be considered for reducing it. The expert reviewer commented that the A Fair Chance for All “inquiry was viewed positively, welcomed by many for the new information it brought to light, and largely regarded as high quality and analytically sound”, while also noting that the extensive research programme “reduced the knowledge gap in understanding persistent disadvantage”.6 |
|
Expert review commentary on the extent to which the research work:7
|
A Fair Chance for All inquiry: Expert review The breadth and depth of research commissioned to inform the inquiry was named by many as extremely valuable. Not only did this directly enhance the quality of inquiry reports (interim, final and quantitative), but it also provided a public resource to inform future thinking and changes to policy to lift productivity and improve wellbeing. |
6 See pages 23 and 32 of the evaluation report.
7 This will normally be assessed through the biennial evaluation of our economics and research function, which will be reported in the 2023/24 Annual Report. For this report we have interpreted this as relating to research that contributed to the A Fair Chance for All inquiry.
Output Measures
The following measures show the results of a survey and expert review carried out after the completion of our A Fair Chance for All inquiry during the 2022/23 period. We have included a comparison to our previous years’ measures relating to our Immigration inquiry reported in our 2021/22 Annual Report.
Measures for our economics and research function are assessed through a biennial evaluation and out of scope for this reporting period. This will be reported in the 2023/24 Annual Report.
Right focus
Relevance and materiality of inquiry report
|
Inquiry participants surveyed who agreed or strongly agreed that: |
Assessment 2021/22 (Immigration) |
Assessment 2022/23 (A Fair Chance for All) |
|---|---|---|
|
18% strongly agreed 48% agreed |
14% strongly agreed 39% agreed |
|
12% strongly agreed 40% agreed |
14% strongly agreed 38% agreed |
|
22% strongly agreed 43% agreed |
26% strongly agreed 43% agreed |
|
17% strongly agreed 43% agreed |
19% strongly agreed 45% agreed |
Relevance and materiality of paper(s) within the research work reviewed
| Assessment 2022/23 Summary comments from expert review for A Fair Chance for All inquiry |
|---|
|
The review acknowledged that this was quite a different type of inquiry for the Commission – different scope, different processes due to the nature of the topic resulting in a wide range of views. The evaluation broadly supported the inquiry’s ambition to cover system-level change, although views on execution of this differed. The key dynamics (short-termism and power imbalance within the system) were captured well, although some thought this was too broad, and some thought it was too narrow. The intention to cover both economic/longitudinal data and the public management system, while ambitious, was met with questions from all sides, which according to the review “probably meant the balance was about right.” |
Good process management
The extent to which inquiry issues papers, draft reports and final reports, and paper(s) within the research work reviewed was delivered to schedule
|
|
Assessment 2021/22 (Immigration) |
Assessment 2022/23 (A Fair Chance for All) |
|---|---|---|
|
All external milestones communicated in the Commission’s process planning are achieved:
|
All milestones achieved |
All milestones achieved |
Participant satisfaction with the inquiry process
|
Assessment 2021/22 (Immigration) |
Assessment 2022/23 (A Fair Chance for All) |
|
|---|---|---|
|
Inquiry participants surveyed who agreed or strongly agreed that overall, they were satisfied with the Commission’s inquiry process. |
17% strongly agreed 65% agreed |
25% strongly agreed 42% agreed |
|
Assessment 2022/23 Summary comments from expert review and focus group for A Fair Chance for All inquiry |
|---|
|
The inquiry team was seen to have tried a range of new things in its inquiry process due to the different nature of the inquiry topic. Overall, the reviewer was satisfied with the processes used to deliver the inquiry. They felt the team had pulled together well despite major challenges across several dimensions, including staff turnover, technology issues within the Commission and within partner agencies, resulting in time and resource constraints. These factors impacted capability and capacity of the Commission, and the overall inquiry timing resulted in release of the inquiry report separately from the supporting quantitative report.8 |
Satisfaction with the Commission’s management of research processes
|
Assessment 2021/22 (Biennial evaluation of our economics and research function) |
Assessment 2022/23 (Biennial evaluation of our economics and research function) |
|
|---|---|---|
|
Participants in the Commission’s research processes surveyed, and reviewer commentary, who agreed and strongly agreed that overall, they were satisfied with the Commission’s approach. |
17% strongly agreed 66% agreed |
Out of scope for this reporting period and will be included in the 2023/24 Annual Report. |
8 Please refer to page 14 of the evaluation report.
High quality work
Participant confidence in the Commission’s inquiry findings and recommendations
|
Assessment 2021/22 (Immigration) |
Assessment 2022/23 (A Fair Chance for All) |
|
|---|---|---|
|
Inquiry participants surveyed who considered the following aspects to be of good or excellent quality:
|
18% excellent 44% good |
38% excellent 39% good |
|
11% excellent 45% good |
37% excellent 33% good |
|
Inquiry participants surveyed who agreed or strongly agreed that:
|
19% strongly agreed 58% agreed |
25% strongly agreed 49% agreed |
|
19% strongly agreed 42% agreed |
Not included in survey for the inquiry. |
The degree of reviewer confidence in research findings and conclusions
|
Reviewer commentary indicates the following aspects to be of good or excellent quality:
|
Assessed through biennial evaluation of our economics and research function. Out of scope for this reporting period and will be included in the 2023/24 Annual Report. |
|
Reviewer agreed or strongly agreed that:
|
Assessed through biennial evaluation of our economics and research function. Out of scope for this reporting period and will be included in the 2023/24 Annual Report. |
Effective engagement
Participant perception of the quality of engagement by the Commission
|
Inquiry participants surveyed who agreed or strongly agreed that: |
Assessment 2021/22(Immigration) |
Assessment 2022/23(A Fair Chance for All) |
|---|---|---|
|
36% strongly agreed 46% agreed |
27% strongly agreed 41% agreed |
|
32% strongly agreed 45% agreed |
34% strongly agreed 35% agreed |
|
28% strongly agreed 52% agreed |
34% strongly agreed 45% agreed |
|
20% strongly agreed 42% agreed |
22% strongly agreed 36% agreed |
|
Summary comments from expert review for A Fair Chance for All inquiry Participants felt the engagement was robust and thorough throughout the inquiry. They considered the Commission’s engagement to be authentic, and “not a tick box exercise.” The Commission’s Chair’s approach in public engagements was considered authentic and of high value. Most participants considered there was an impressive spread of engagement types and reach. Consultation on the Terms of Reference was really valued and worked well for stakeholders. However, it may have created an ongoing expectation for high levels of engagement that couldn’t be sustained by the Commission over the life of the inquiry. Pasifika and Māori engagement was seen as positive and key partners saw their voices represented throughout the process and in the final report. The time and money invested in engaging with these communities to inform the inquiry was considered excellent and had lifted the bar for public sector engagement. |
|
|
Participants in Commission research processes surveyed who agreed or strongly agreed that:
|
Assessed through biennial evaluation of our economics and research function. Out of scope for this reporting period and will be included in the 2023/24 Annual Report. |
Engagement meetings held and submissions received
|
Assessment 2021/22 (Immigration) |
Assessment 2022/23 (A Fair Chance for All) |
|
|---|---|---|
|
Number of parties the Commission engaged with during the inquiry, as noted in the final report appendix. |
74 engagement meetings |
149 engagement meetings and wānanga |
|
Number of parties who made submissions during the inquiry, as noted in the final report appendix. |
181 submissions received |
154 submissions received |
Clear delivery of message
Participant perceptions of the effectiveness of the Commission’s communication of inquiry and research findings and recommendations
| Inquiry participants surveyed who agreed or strongly agreed that: |
Assessment 2021/22 (Immigration) |
Assessment 2022/23 (A Fair Chance for All) |
|---|---|---|
|
28% strongly agreed 57% agreed |
23% strongly agreed 59% agreed |
|
36% strongly agreed 55% agreed |
27% strongly agreed 56% agreed |
|
27% strongly agreed 47% agreed |
29% strongly agreed 52% agreed |
|
Summary comments from expert review for A Fair Chance for All inquiry The report was considered coherent, clear, and well-articulated with a good, logical flow. “Given that this was a new area of focus for the Commission, it is rewarding to see the Commission delivered the research findings and recommendations so well.” “The analytical frames used, and the breadth and depth of research commissioned to support the inquiry, were overall seen as high-quality, robust and meaningful.” |
|
|
Reviewer commentary on research papers indicate that:
|
Assessed through biennial evaluation of our economics and research function. Out of scope for this reporting period and will be included in the 2023/24 Annual Report. |
Overall quality
Independent expert evaluation of the overall quality of the inquiry
|
A report evaluating the overall performance of the inquiry from the final inquiry report (considering the focus of the report, process, analysis, engagement, and delivery of message) with recommendations for future improvements. |
Summary comments from expert review for A Fair Chance for All inquiry The reviewer saw the A Fair Chance for All inquiry was a different type of work for the Commission, acknowledging the need for different tools, approaches, and capabilities to really do it justice. Overall, the inquiry was viewed positively, welcomed by many for the new information it brought to light, and largely regarded as high quality and analytically sound. |
Independent expert evaluation of research work
|
A report evaluating the overall quality of the package of research work (taking into account the focus of the research work, process, analysis, engagement and delivery of message) with recommendations for future improvements. |
Assessed through biennial evaluation of our economics and research function. Out of scope for this reporting period and will be included in the 2023/24 Annual Report. |
Focus group evaluation of inquiry9
|
Report from a focus group representative of inquiry participants, facilitated by an independent person with significant experience in inquiry- type work with feedback on the inquiry and recommendations for future improvements (taking into account the focus of the report, process, analysis, engagement, and delivery of message). |
Summary comments from expert review for A Fair Chance for All inquiry Participants saw the nature of this piece of work as an important part of a mindset change in the public sector. They considered the Commission accomplished a good result with a broad topic, took actions to narrow it down (although not everyone agreed with the choices made in descoping). Some comments from survey respondents on the overall quality of the report: “I heartily support the findings and recommendations. I’m not confident about how easily these will find their way into policy, but they certainly set out very key issues and clear recommendations. Thank you for this important work.” “With such a huge issue, scoping the problem down to focus on the Public Management System was always going to create the issue of compartmentalising a massively complex set of problems.” |
Participant evaluation of inquiry
|
Percentage of inquiry participants surveyed who rated the overall quality of the inquiry as good or excellent (considering the focus of the report, process, analysis, engagement, and delivery of message). |
2021/22 (Immigration) 18% excellent 42% good |
2022/23 (A Fair Chance for All) 42% excellent 28% good |
9 Focus groups were carried out as part of the expert review for the A Fair Chance for All inquiry