NEW ZEALANDERS' SATISFACTION WITH PUBLIC SERVICES

Transcription

1 NEW ZEALANDERS' SATISFACTION WITH PUBLIC SERVICES Kiwis Count DECEMBER 2016 ANNUAL REPORT

2 Published by the State Services Commission PO Box 329, Wellington, New Zealand July 2017 CROWN COPYRIGHT ISBN This copyright work is licensed under the Creative Commons Attribution 4.0 International licence. In essence, you are free to copy, distribute and adapt the work, as long as you attribute the work to the New Zealand Government and abide by the other licence terms. To view a copy of this licence, visit Please note that neither the New Zealand Government emblem nor the New Zealand Government logo may be used in any way which infringes any provision of the Flags, Emblems, and Names Protection Act 1981 or would infringe such provision if the relevant use occurred within New Zealand. Attribution to the State Services Commission should be in written form and not by reproduction of the State Services Commission emblem or New Zealand Government logo.

3 Contents Executive Summary... 3 Trust... 3 Satisfaction... 4 Change to calendar year reporting... 5 Chapter 1: Trust in Public Services... 6 Trust in the public sector compared to trust in the private sector... 8 Demographic Breakdown... 8 Chapter 2: Service Satisfaction...11 Overall Service Quality Trend...11 Private Sector Service Quality...12 Demographic Breakdown...12 Individual Services Quality Trend...14 Current performance (2016 results)...16 Chapter 3: Ways to improve performance...21 Channel Use and Drivers of Satisfaction...21 Improving Satisfaction...21 Appendix 1: Background and Technical Information...24 Background...24 Survey Approach...25 New Survey Provider...25 Changes to Reporting Cycle...26 Questionnaire Content...26 Encouraging Online Participation...27 Response Rate...27 Sample Size...27 Service Quality Scores...27 Kiwis Count

4 2 Kiwis Count 2016

5 Executive Summary For 10 years the State Services Commission has been asking New Zealanders 1 about their experiences and views of public services. This provides rich information about how New Zealand s public services are performing in the eyes of the people who use them. Kiwis Count gives us three headline results for the Public Service: Trust in public services based on personal experience Trust in the public sector brand, and Satisfaction with the quality of services received. Headline results show that New Zealanders have high trust in, and satisfaction with, their public services. However, while results have increased steadily from 2007 to 2015, results for both trust based on personal experience and service quality have dropped slightly from 2015 to The 2016 results are still higher than 2014, and 2015 may prove to be an outlier. It is also not clear to what extent the results over has been influenced by wider events over this period. How the Headline results are measured Trust in public services based on personal experience is measured by asking respondents to think about their most recent public service interaction and to answer: Thinking about your most recent service contact, can you trust them [public servants] to do what is right? Trust in the public sector brand is measured by asking respondents: Thinking about your overall impressions and from what you know or have heard from family, friends or the media, to what extent do you trust the public service? Satisfaction with service quality is measured by asking respondents whether they have used any of 42 commonly used public services and, if so, to rate their satisfaction with that experience. The results for all 42 services are then aggregated to give the overall service quality score (SQS) or headline result for satisfaction. Trust In 2016, 79% of New Zealanders trust public services based on their personal experience. This is 12 percentage points higher than 2007 but 2 percentage points lower than the 2015 result. The reduction over 2016 was driven by a fall back from record 2015 trust levels in the health and tertiary education sectors. Trust in the public sector brand is 45% in Trust in brand increased 15 percentage points from 2007 to 2013 and has held steady since 2013, oscillating between 44% and 45%. Demographic analysis shows: New Zealanders in urban areas have greater trust in the public sector brand than other areas. 1 New Zealanders invited to participate in the Kiwis Count survey are randomly selected from the Electoral Roll. Kiwis Count

6 Men report higher levels of trust than women for both trust based on personal experience and trust in the public sector brand.. All ethnicities have high trust based on personal experience. Results for Māori for this measure used to be lower than other ethnicities but have improved since There are large differences by ethnicity for trust in the public sector brand. Those of Asian ethnicity have the highest trust in the public sector brand with Māori having the lowest. Trust within the 65+ age group is highest of any age group, based on both personal experience and trust in the public sector brand. Respondents with a disability had less trust based on personal experience and less trust in the public sector brand than respondents without a disability. Trust appears to increase as household income increases. New Zealanders with a degree or a higher qualification have much higher trust in the public sector brand than those with lower level qualifications. Satisfaction The overall Service Quality Score (SQS) for 2016 is 74. This is 6 points higher than 2007 but one point lower than The improvement in New Zealanders satisfaction with the quality of public services over the past decade is reflected in the fact that 23 out of the 42 services measured have had statistically significant increases since they were first measured and none of the services have had statistically significant declines over the period. The fall in the overall SQS over 2016 was driven by SQS decreases for high use services in the health and local government sector, as well as increasing usage of local government services. Local government services tend to have below average SQSs so increasing usage of these services tends to lower the overall score. Demographic analysis shows: Women report higher satisfaction with public sector services than men. All ethnic groups report high levels of satisfaction with public sector services. Service satisfaction with public sector services has remained relatively stable for NZ Europeans since 2012 while satisfaction of other ethnic groups has steadily improved. There was a 5 point difference in satisfaction between NZ Europeans and non-nz Europeans in In 2016, there was a 1 point difference. Satisfaction with public services among those aged 65+ is higher than for younger people. Satisfaction with public services is lower for respondents with a disability than those without. Since 2007 overall satisfaction with public sector services has been higher than for the private sector services. 4 Kiwis Count 2016

7 Change to calendar year reporting With this report Kiwis Count has moved to reporting on a calendar (December) year basis, whereas in previous reports June years were used. To maintain consistency with this new approach, all results in this report for have been recalculated based on calendar year data. This means some results are slightly different to those published in previous reports. See Appendix 1 for more information. Kiwis Count

8 Chapter 1: Trust Kiwis Count measures trust in public services in two ways: trust of New Zealanders based on personal experience of using public services and trust in the public service brand (perception) see figure 1 below. By both measures, trust (the percentage of people who answered a 4 or a 5 on a five point scale) has increased markedly since In 2016 trust in public services based on personal experience was high at 79% 2, 12 percentage points higher than when first measured in The 2016 result is two percentage points lower than in 2015, but still higher than in It may be that the 2015 result is an outlier. The fall in 2016 was driven by a fall back from record 2015 trust levels in the health and tertiary education sectors. Trust in the public sector brand was 45% in This result increased 15 percentage points from 2007 to It has held steady since 2013, oscillating between 44% and 45%. Figure 1: Trust in public sector services based on personal experience and trust in the public sector brand 2 With this report Kiwis Count moves to reporting on a calendar (December) year basis, from June years as in previous reports. To maintain consistency with this new approach, all results in this report for have been recalculated based on calendar year data. This means some results are slightly different to those published in previous reports. See Appendix 1 for more information. 6 Kiwis Count 2016

9 Trust in the public service brand is measured by asking respondents: Thinking about your overall impressions and from what you know or have heard from family, friends or the media, to what extent do you trust the public service? Trust in public services based on personal experience is measured by asking respondents to think about their most recent public service interaction. One of the subsequent questions is: Thinking about your most recent service contact, can you trust them [public servants] to do what is right? Consistently, New Zealanders trust in public services by recent experience has measured much higher than trust in brand. The same result is found in Canadian research 3 : when citizens evaluate services they have used recently, they draw on particular memories of actual experiences. When citizens rate government services in general, they draw on opinions and possibly stereotypes of government, and these tend to be negative. Distrust of the public sector (the percentage of people who answered a 1 or 2 on a five point scale) has reduced since In 2016, distrust in the public sector brand, at 14%, was eight percentage points lower than in 2007 and distrust based on personal experience, at 10%, had dropped two percentage points since Figure 2: Distrust in public sector services based on personal experience and distrust in the public sector brand 3 Erin Research (1998) Citizens First, and subsequent Citizens First research by the Institute for Citizen Centred Service. Kiwis Count

10 Trust in the public sector compared to trust in the private sector Since 2012, Kiwis Count has measured New Zealanders trust in the private sector ( to what extent do you trust the private sector? ), to help benchmark the results for public services. In 2012, New Zealanders trust in the private sector (40%) was very similar to the result for the public sector (41%). Figure 3 below, shows that since 2013, trust in the private sector has not improved (39% in 2016) and is six percentage points below trust in the public sector (45% in 2016). Figure 3: Trust in brand, public sector and private sector services Demographic Breakdown Region There is no clear difference in trust based on personal experience between regions although the variation between regions has reduced since There does seem to be a pattern that New Zealanders living in more urban areas (Auckland, Wellington and Canterbury) have greater trust in the public sector brand than those in other areas. Gender From the first survey in 2007 to 2015 women reported higher trust in public services based on their personal experience than men but in 2016, for the first time, men have rated their trust (80%) higher than women (79%). For trust in the public sector brand, men have consistently scored trust higher (48% in 2016) than women (43%). 8 Kiwis Count 2016

11 Ethnicity In 2016, people of all ethnicities had relatively similar, and high, results for trust based on personal experience (78%-80%). Before 2016, trust based on personal experience for those of Māori ethnicity was lower than other ethnicities this has shown a statistically significant improvement since However, there are large differences in the results for trust in the public sector brand: in 2016, those of Asian ethnicity have the highest trust in the public sector brand (59%), followed by NZ Europeans (44%), then Pasifika (40%) with Māori much lower at 32%. Unlike trust based on personal experience, Māori trust in the public sector brand has not improved and the gap between Māori and non-māori is now the highest (15 percentage points) it has been since the Kiwis Count survey began in Age Those aged 65 years and older have much higher levels of trust (based on personal experience and in brand) than other age groups. Figure 4: Trust by age group (2016) Disability In 2016, 76% of respondents with a disability scored a 4 or a 5 for trust based on personal experience compared to 80% of those without disabilities (a 4 percent difference). Respondents with a disability also had lower trust in the public sector brand compared to other respondents. However, these differences are not statistically significant. Kiwis Count

12 Household income Trust appears to increase as household income increases. It is too early to say whether this is a long-term pattern, as household income has only been collected since Highest qualification Those with a degree or higher qualification have much higher trust in the public sector brand (51%) than with those lower level qualifications (42%-45%). However there is no real difference when it comes to trust based on personal experience of public services. 10 Kiwis Count 2016

13 Chapter 2: Service Satisfaction Overall Service Quality Trend Kiwis Count measures New Zealanders satisfaction with 42 commonly used services. The results are used to calculate a Service Quality Score (SQS) for each service and an overall SQS across all services (headline result) is also calculated. New Zealanders satisfaction with the quality of public services has increased over the past decade. The overall SQS has increased six points from 68 in 2007 to 74 in However, the 2016 result shows a levelling off, with a decrease of one point on The 2016 result is still higher than 2014, and it may be that the 2015 result is an outlier. The decline over 2016 reflects that more individual services experienced declines in their SQSs than increases, and it is hard to get continuous improvement on a high baseline. Figure 5: Overall service quality score, public services Over half of the decrease in overall SQS score since 2015 is due to: Decreases in satisfaction for services in the health sector. This fall comes after record highs for health sector services in These services have high usage among New Zealanders, so they also have a high contribution to the overall SQS score. Decreases in satisfaction for local government services, combined with increasing usage of these services by New Zealanders. Local government services tend to have below average SQS scores, so increasing usage of these services tends to lower the overall SQS score. Kiwis Count

14 Private Sector Service Quality Kiwis Count also asks New Zealanders about their use of, and satisfaction with, seven types of private sector companies (banks or finance companies, insurance companies, internet service providers, postal or courier companies, telephone companies, credit card companies and electricity or gas companies). These scores are also aggregated to give an overall private sector SQS. Figure 6 shows that since 2007, satisfaction with private sector services has been lower than satisfaction with public services. While satisfaction with private sector services has increased since 2007, the rate of increase has been slower than the increase in satisfaction with public services with no increase in satisfaction with private sector services since Figure 6: Overall service quality score, public services and private sector services Demographic Breakdown Region There does not appear to be a systematic difference in satisfaction across regions for either public sector or private sector services. The variation between regions has reduced since 2012 for public sector services, but increased for private sector services. Gender Women report higher satisfaction with both public and private sector services than men. In 2016, the public sector SQS for women was 75, and for men it was Kiwis Count 2016

15 Ethnicity In 2016, all ethnic groups report high satisfaction with the quality of public services they used. There were similar scores for those of Asian (75), NZ European (74) and Pacific ethnicities (73), but lower for Māori (71). Service satisfaction has remained relatively stable since 2012 for NZ Europeans, while other ethnic groups have seen large increases in service satisfaction over the same period. Looking at a direct comparison of NZ Europeans vs non NZ Europeans, the SQS gap has steadily narrowed from a 5 point gap in 2012 to a 1 point gap in Services appear to be more equitably delivered to all ethnicities. Figure 7: Public Services Service Quality Score - NZ European vs non-nz European Age 65+ year olds report higher satisfaction with both public and private sector services than other age groups. Disability Satisfaction with the quality of public services for those with disabilities (72 in 2016) is slightly lower than for those without disabilities (74 in 2016). In contrast, satisfaction with private sector services for those with disabilities (69 in 2016) is slightly higher than for those without disabilities (67). Household Income and Highest Qualification There do not appear to be differences in satisfaction for those in different income or qualification groups. Kiwis Count

16 Individual Services Quality Trend Kiwis Count measures New Zealanders satisfaction with 42 commonly used public services 4. In this section, we discuss changes in satisfaction ratings for individual services. The trend over time is important as it highlights whether services are, or are not, continuing to meet or exceed individuals expectations. We highlight those changes in satisfaction with the quality of individual services that have been statistically significant 5 since services were first measured and in the latest year. Since first measured 23 Significantly 0 Significantly The improvement in New Zealanders satisfaction with the quality of public services over the past decade is reflected in the fact that 23 out of 42 services in the Kiwis Count basket have had statistically significant increases since they were first measured. None have had statistically significant decreases. Figure 8 shows the level of improvement since first measured of all 42 services (significant change since first measured is shown in green). Level of improvement since first measured ranges from +14 for A childcare subsidy to -3 for Living in a Housing NZ home and Visited sorted.org.nz for information to help manage your personal finance or retirement income (although these decreases are not statistically significant). The median level of improvement since first measured is +5 points. 4 See Appendix One for a description of the Kiwis Count methodology. 5 Significance is calculated at the 95% confidence level using a weighted two-sample t-test. This means that changes in SQS score are highlighted where there is less than a 5% chance that the change was due to sampling error. Sampling error is due to sample of the population being surveyed rather than the entire population. 14 Kiwis Count 2016

17 Figure 8: Service quality scores for individual services (2016), significant change since first measured Kiwis Count

18 Current performance (2016 results) There is a wide range in the satisfaction levels across the 42 services. At the top there are four services that have a rating around 85. This means that the average rating on a five point scale is between four and five. Visited a public library A passport New Zealand Superannuation Licensed or registered a vehicle At the other end, there are three services with rating less than 50. This means the average rating is just below three on a five point scale. Your local council about a building permit National environmental issues or the Resources Management Act Your local council about road maintenance Table 1 shows all annual results for each service. It is ordered from highest to lowest score in Significant changes from year to year are bolded in red. Table 1: Annual SQS for individual services Service Visited a public library A passport New Zealand Superannuation Licensed or registered a vehicle A kindergarten, day-care, crèche, preschool, home-based service, playcentre, Kōhanga Reo, Aoga Amata, Puna Reo or playgroup etc that your child attends or may attend in the future Emergency services i.e A hunting or fishing license Visited a national park Registering a birth, death, marriage or civil union The arrival process after landing at a New Zealand international airport from anywhere except Australia The arrival process after landing at a New Zealand international airport from Australia Obtain, renewed, change or replace a driver licence A state or state integrated (public) school that your child attends or may attend in the future Registering a new company or filing an annual return for a registered company A rental property bond lodgement, refund or transfer Visited sorted.org.nz for information to help manage your personal finances or retirement income Used an 0800 number for health information Kiwis Count 2016

19 Service ERO (Education Review Office) school or early childhood reports Stayed in a public hospital The Community Services card Registered a business entity for tax purposes or filed a tax return Accident compensation for injuries Received outpatient services from a public hospital (includes A & E) A university, polytechnic or wananga about a course you are attending or may attend in the future A childcare subsidy Contact with Statistics New Zealand for information or about taking part in a survey The Police (for a non-emergency situation) Obtaining family services or counselling Your local council about rubbish or recycling (excluding the actual collection of rubbish and recycling from your household each week) Your local council about property rates Employment or retraining opportunities Paying fines or getting information about fines A housing subsidy or accommodation supplement Enquired about tax, receiving tax credits (such as Working for Families), Student loan repayments or KiwiSaver Receiving a benefit such as Jobseeker Support, Sole Parent Support or a Supported Living Payment Importing goods into New Zealand or customs duties Applying for or receiving a student loan or student allowance A court, about a case you were involved with Living in a Housing New Zealand home * Your local council about a building permit National environmental issues or the Resources Management Act Your local council about road maintenance *This result should be treated with caution as sample size is low. Kiwis Count

20 Statistically Significant changes Over 2016, three services have improved and five services have declined. 3 Significantly 5 Significantly Statistically Significant Increases The levelling off in overall satisfaction in the 2016 results is seen in the fact that more services had statistically significant falls in the most recent year, than had increases. There were three services that had statistically significant increases over the year and each of these services recorded their highest ever SQS score: Receiving a benefit such as Jobseeker Support, Sole Parent Support or a Supported Living Payment Contact with Statistics NZ for information or about taking part in a survey NZ Superannuation The statistically significant increases for receiving a benefit such as Jobseeker Support, Sole Parent Support or a Supported Living Payment and contact with Statistics NZ for information or about taking part in a survey follow statistically significant declines in Statistically Significant Decreases Five services recorded statistically significant decreases in service quality over the year: Used an 0800 number for health information Received outpatient services from a public hospital (includes A & E) Visited sorted.org.nz for information to help manage your personal finances or retirement income Importing goods into New Zealand or customs duties Living in a Housing New Zealand home Health Services Two of these services, received outpatient services from a public hospital (includes A & E) and used an 0800 number for health information had statistically significant increases when looking back to when they were first measured in Their 2016 declines follow historic highs (and significant increases) in With respect to the decline in outpatient services, the Ministry of Health (MoH) comments that for some time now, District Health Boards have been focused on improving patient experience in a range of hospital settings. Local initiatives have varied, but have included a focus on reducing the time people wait, minimising cancellations, considering alternative methods of delivery that better suit the client (such as non-contact outpatient appointments, or telehealth service delivery), communication between clinicians and other staff with patients and their families, improvements to physical facilities and services, and improving the mechanisms for booking appointments and ensuring these best meet client needs. 18 Kiwis Count 2016

21 The Health Quality and Safety Commission (HQSC) currently monitors the shift in adult inpatient experience via surveys, collated and reported on a monthly basis. Latest results suggest there has not been a significant change through the year. With respect to the decline in using an 0800 number for health information, an integrated Telehealth service was launched on 1 November This brought together seven 0800 health advice phone lines: Healthline, Quitline, Alcohol and Drug line; Depression line, Gambling Helpline, Poisons Advice and an Immunisation service as well as other health communication channels. In the 13 months between start up on 1 November 2015 and 31 December 2016 there were over 1 million contacts with New Zealanders. Focus groups have been run to receive customer feedback during this time and, from June 2017, customer surveying will be in place across all 0800 telehealth lines as well. Information received on the customer experience will be used to make meaningful improvements over time. The New Zealand Health Strategy: Future direction outlines the high level direction for New Zealand's health system over the 10 years from 2016 to Supporting the Strategy is a Roadmap of Actions for the next five years to help make the Strategy happen. MoH and the wider health sector are partners in the Strategy, and are actively engaged in a range of local and national initiatives to make gains in the areas identified. Through these actions, MoH comments that it is anticipated that we can make meaningful improvements in the area of patient experience, both for outpatient hospital services, and wider across hospital, primary and community settings. Sorted.org.nz A rebrand and a rebuild for the sorted.org.nz website was launched in March Since then the bounce rate for the site has decreased, the number of registered sorted.org.nz users has increased and feedback to the Commission for Financial Capability (CFFC), which manages the site, has been positive. However, the CFFC reports that immediately post launch there was a two month period of ironing out final bugs in the new site which may have impacted user satisfaction. Importing Goods The importing goods into New Zealand or customs duties service has been dropped from the Kiwis Count questionnaire from 2017 due to low use of the service and that cognitive testing of the service undertaken at the end of 2016 raised questions about how respondents rate this service and, therefore, the usefulness of the measure. Over the same period, the Customs New Zealand s Goods Client Survey shows satisfaction for customers who have had retail goods delivered from overseas remains favourable and has not changed significantly between 2015 and Housing New Zealand home Although the 2016 result for living in a Housing New Zealand home was statistically significantly less than the 2015 result, it was based on a sample count of less than 50 (i.e. this question was answered by less than 50 people). Therefore this result should be used with caution. Kiwis Count

22 Sample counts for this service have always been low. To try to increase the sample count, the scope for this service has been widened from Housing New Zealand (HNZ) surveys 500 tenants each quarter about their satisfaction with their home and services provided by HNZ. A decline in satisfaction between 2015 and 2016 is not seen in the HNZ results. Respondents who answer this service in the Kiwis Count survey are likely to be both HNZ tenants and non-tenants as the Kiwis Count question asks the respondent to rate the service If they have used or had contact about Living in a Housing New Zealand home. 6 The new wording for the service is: In the last 12 months have you applied for, or lived in a publicly subsidised house (eg a house owned by Housing NZ or a community housing provider or a local council). 20 Kiwis Count 2016

23 Chapter 3: Customer Insight Channel Use and Drivers of Satisfaction Kiwis Count measures service satisfaction and trust in government services at the macro level. Agencies are also encouraged to measure satisfaction with their services at a detailed level to help them understand how they are doing in improving areas which matter most to New Zealanders, and where to focus resources for the greatest impact. Ten years ago, the State Services Commission undertook research to identify the key factors (or drivers) that have the greatest influence on New Zealanders satisfaction with, and trust in, public services. These results were published in the Drivers Survey 7 (2007). Since 2012 Kiwis Count has asked respondents to answer questions about aspects of their satisfaction (the driver questions) based on the channel used for the most recent service interaction. These responses show:- New Zealanders access public services most often via the general (face-to-face or through correspondence) channel (70%). This has not changed much since 2012 (69%). The phone is the second most used channel (14% in 2016). Use of the phone channel has declined from 18% in Transacting online follows as the next most used way to access public services. Transacting online has increased from 7% in 2012 to 10% in Accessing information online is the least used channel (6% in 2016) and use of this channel is relatively unchanged from 2012 (7%). Improving Satisfaction One way of improving satisfaction with public services is for agencies to focus on these key drivers (Drivers Survey, 2007): The service experience met your expectations Staff were competent Staff kept their promises, they did what they said they would do You were treated fairly You feel your individual circumstances were taken into account It s an example of good value for tax dollars spent Kiwis Count has measured the drivers of satisfaction since Not all drivers are equal: the service experience met your expectations is the most important driver and accounts for nearly one third of satisfaction with public services. Results for this driver across different channels are shown in figure 9. 7 The full report on the 2007 Drivers Survey can be found at and the summary report can be found at Kiwis Count

24 How well did each service channel meet expectations? The results for met expectations on the online channels are impressively high in % of respondents had their expectations met when looking for public service information online (an 8 percentage point increase over the year and a steady increase from 73% in 2012). 85% of respondents had their expectations met when transacting online with the public service (a steady increase from 79% in 2012). The general channel (face to face contact or through correspondence) has not shown much of an increase since 2012, but 2016 is still a healthy result at 81%. There is a marked decline over the year on the Phone Channel. Satisfaction on the Phone Channel has always been lowest for the met expectations driver. We noted in 2015, a level of increase in satisfaction on the Phone Channel being the best of any channel since Over 2016, however, there has been a statistically significant decline in users of the phone channel having their expectations met (74% down to 62%). In 2016, the digital channels appear to be outperforming the non-digital channels. This is the first year that looking for information online has outperformed the general channel on the met expectations driver. However, it is worth noting that interactions on the non-digital channels are likely to be more complex. Figure 9: Met expectations driver 22 Kiwis Count 2016

25 All Drivers There is a similar pattern to that described for the met expectations driver when looking at results for all drivers: improving performance on the online channels, fairly similar performance on the general channel and poor/declining rates of satisfaction on the phone channel. Kiwis Count

26 Appendix 1: Background and Technical Information At the start of each month, the research provider, Gravitas Limited, sends out 432 survey invitations 8. The annual information included in the report draws on the experiences of 2,137 New Zealanders who completed Kiwis Count between January 2016 and December Background In 2007, for the first time, the State Services Commission asked a sample of New Zealanders about their experiences and views of public services. Known as the Kiwis Count survey, this provided rich information on how New Zealand s public services were performing in the eyes of the people who use them. The survey ran for a second time in These first two surveys were point-in-time surveys. In 2012 the survey moved to a continuous collection methodology. This is the fifth report of annual results from the Kiwis Count survey since the survey became continuous in Based on the methodology of a Canadian government survey called Citizens First, Kiwis Count measures satisfaction in public services. Public services means all services provided by government and includes central and local government services, tertiary institutions, schools and hospitals. Kiwis Count is an integral part of the New Zealanders Experience Research Programme (NZE), an SSC research initiative begun in 2007 and designed to find out how New Zealanders experience public services and to develop tools through which services can improve. The other two parts of NZE, designed to work together and complement and enhance each other are work on the drivers of satisfaction and the Common Measurements Tool. New Zealanders Experience Research Programme 8 More survey invitations were sent out in the first half of 2014 to ensure a sufficient sample of parent / primary caregivers answered the survey for a one-off module for the Ministry of Education. 24 Kiwis Count 2016

27 The Drivers of Satisfaction The Drivers Survey 9, published in July 2007, identified the key factors (or drivers) that have the greatest influence on New Zealanders satisfaction with, and trust in, public services. The most effective way to improve satisfaction with public services is for agencies to focus on these key drivers. Kiwis Count has measured the drivers of satisfaction since it began in 2007 and, since 2012 Kiwis Count has measured the drivers by channel (general: face to face or by correspondence, telephone, looking for information online and transactions online). The Common Measurements Tool Kiwis Count measures services satisfaction and trust in government at the macro level. Agencies are also encouraged to measure satisfaction with their services at a detailed level to help them understand how they are doing in improving areas which really matter to New Zealanders, and where to focus resources so they can have the greatest impact. The Common Measurements Tool (CMT) is a databank of customer satisfaction questions which agencies can use as they develop their surveys. This saves them from developing their own question banks and allows them to benchmark their results against similar agencies. Approximately half of our agencies use CMT. Survey Approach The Kiwis Count team have published a survey methodology report on the SSC website ( New Survey Provider Gravitas began collecting the survey for SSC in July To provide comparability with the earlier results Gravitas collected the survey using the same survey materials and overall method as previously. There are some differences in the demographics of those who completed the survey since July Looking at the unweighted sample, younger respondents made up a larger proportion of completions, while older respondents (particularly those 65+) made up a lower proportion. However, as expected, once weighted to the population, the proportions of each demographic are very similar to previously. Analysis by Gravitas shows no evidence of this difference in the unweighted sample having an impact on the headline results in this report. In general, the difference has improved the robustness of the survey results as the unweighted sample is now more representative of the population for younger and older respondents. 9 The full report on the Drivers Survey can be found at and the summary report can be found at 10 From January 2012 to June 2016, the survey collection provider was Nielsen. Kiwis Count

28 Changes to Reporting Cycle These results are slightly different to those previously published. The difference is due to Kiwis Count changing its reporting period. The base data and calculations remains the same. Continuous surveying for Kiwis Count began in January 2012 and the first data point (June 2012) was used as the 2012 result and subsequent reporting used data from July-June years as annual results. However, at December 2016, with five years of continuous data, Kiwis Count has moved to a December year end annual reporting cycle. Annual data points for 2012, 2013, 2014 and 2015 (which were based on a June year) have been recalculated as annual results based on a calendar year. From June 2012 to June 2015, quarterly reports of rolling six month average results were publicly provided. When analysed, the State Services Commission found that these results were not widely used publicly, although six month results were still useful for agencies. Therefore, quarterly reporting ceased in June Questionnaire Content The Kiwis Count survey is modular. At the heart of the survey are questions about the public services that New Zealanders use most frequently. These core questions have been fixed since 2012, with new questions added only as required to reflect actual changes in services. The modular part of the questionnaire is designed to change as required to focus on service delivery priorities: In the 2012 calendar year the survey included a module of questions on channel use and preferences. This repeated a module of questions which was included in the 2009 survey. Starting in 2013 a module of questions about the ease of transacting with government in the digital environment replaced the previous module. The new module, developed with the team responsible for Result of the Better Public Services programme, will be one of a suite of measures used to report on the progress of Result 10. For the first half of 2014, a new module of questions was included. It was about parent/primary caregiver satisfaction with education services. In the second half of 2014, at the request of the Result 10 team, new questions were added: - To the Government and the digital environment module, and - To the main body of the survey, at A10, about experiencing public services. In 2015, new questions were added, at A13, about the privacy of personal information for the Office of the Chief Privacy Officer. In 2016 a comprehensive review of the questionnaire was undertaken to ensure its continued usefulness. This new questionnaire is being used from the beginning of Of the 42 services measured from 2012, 29 have stayed the same (with some having been reworded for clarity), two services have each been divided into two separate services (making four services 11 People have easy access to Public Services, which are designed around them, when they need them. 26 Kiwis Count 2016

29 in total), two services have been amalgamated into one service, seven services have been deleted and nine have been added. The changes do not affect the results in this report but will affect the 2017 annual results. The new questionnaire can be downloaded or viewed at Encouraging Online Participation Fifty eight percent of respondents chose to complete the survey online in the year to December Online completion rates have been consistent with this since This compares to online rates of 17% in 2009 and 8% in Response Rate The response rate between January 2016 and December 2016 was 45%. This is at the low end of response rates achieved since the continuous approach was adopted (response rates have ranged between 45% and 53%). This reflects greater emphasis on sampling younger people and Māori both groups tend to have lower response rates. Key drivers of response rates include sampling, questionnaire length and incentives provided to complete surveys. Analysis undertaken by the Kiwis Count team shows a correlation between response rates and questionnaire length. Sample Size Numbers of New Zealanders who Collection Year answered the survey A total of 22,255 New Zealanders have completed the survey since it began. Service Quality Scores The Kiwis Count survey asks New Zealanders to rate services or express opinions using a scale from 1 to 5. To enable comparisons between Kiwis Count and Citizens First to be made, we have adopted the Canadian approach of converting five point rating scales to service quality scores ranging from 0 to This was the number of respondents to the main 2007 survey where 6000 people were invited to complete the survey. Prior to the main survey, a pilot survey with 500 invitations was also undertaken. Kiwis Count

30 The overall Service Quality Score is calculated by rescaling the result from each respondent s five point scale (1,2,3,4,5) to a 101 point scale (0,25,50,75,100) then calculating an average of these scores from all the services used. The overall average uses all service experiences, so a respondent who has used ten services contributes ten observations to the overall score and a respondent who has used one service contributes one observation to the overall score. Example: the service quality question 28 Kiwis Count 2016