Federal Democratic Republic of Ethiopia Ministry of Capacity Building in Collaboration with PSCAP Donors

Transcription

1 Federal Democratic Republic of Ethiopia Ministry of Capacity Building in Collaboration with PSCAP Donors "Woreda and City Administrations Benchmarking Survey III Citizens Report Card (CRC) Survey of Service Delivery Satisfaction Status Final Addis Ababa July, 2010

2 ACKNOWLEDGEMENT The survey work was lead and coordinated by Berhanu Legesse (AFTPR, World Bank) and Ato Tesfaye Atire from Ministry of Capacity Building. The Demand side (Citizens Report Card) has been designed and analysis was produced by Gil Yaron while the data was collected by Selam Development Consultants firm with quality control from Mr. Sebastian Jilke. The survey was sponsored through PSCAP s multi-donor trust fund facility financed by DFID and CIDA and managed by the World Bank. All stages of the survey work was evaluated and guided by a steering committee comprises of representatives from Ministry of Capacity Building, Central Statistical Agency, the World Bank, DFID, and CIDA. Large thanks are due to the Regional Bureaus of Capacity Building and all PSCAP executing agencies as well as PSCAP Support Project team in the World Bank and in the participating donors for their inputs in the Production of this analysis. Without them, it would have been impossible to produce. ii

3 Table of Contents Executive Summary... vi 1 Introduction Methodology Sample design Survey instrument design and the review process Comparing WCBSII and WCBSIII CRC data Findings: Changes since 2008 (WCBSII) Issues to bear in mind when comparing 2008 and 2010 CRC data Water supply satisfaction levels Health use of Government facilities & satisfaction levels Agriculture use of extension services, impact & satisfaction Solid waste collection satisfaction with services Taxation payment & satisfaction with collection Payments outside those legally required for government services Access to information on government services Do you know what taxes and fees you are legally required to pay? Information on various Woreda or City Administration activities Are Woreda/City Administration Council meetings open to the public? Consultation with local government: officials seeking views of the public Consultation with local government: citizens raising concerns over service quality Confidence local government service quality concerns will be addressed Understanding who has access to local government services Who has obtained improved water services in the past two years? Findings for City Administrations Findings for rural Woredas Causes of satisfaction and dissatisfaction with water supply Overview City Administrations Primarily rural Woredas Perceptions of government health service quality levels and changes Who believes services from government health facilities have improved? City Administrations Primarily rural Woredas Health service satisfaction and poverty status Health service satisfaction and informal payments Solid Waste Disposal Knowledge of which taxes and fees must be paid Public information on Woreda/CA activities Holding service providers to account Perceptions of crime Perceptions of social shengo (courts) Perceptions of corruption Measures of corruption and perceived corruption Factors determining perceived corruption Residents priorities for service delivery improvements Annex 1 Additional statistical results Annex 2 List of Woredas/City Administrations surveyed iii

4 Index of Tables Table 1: Comparing poverty profiles WCBSII and III... 6 Table 2: Perceived change in water quality in the past two years... 7 Table 3 Satisfaction with dimensions of government health facility services 2008 (WCBSII)... 8 Table 4 Satisfaction with dimensions of gov t health facility services 2010 (WCBSIII)... 9 Table 5 Perceptions of changes in the quality of Government health services since 2008 (WCBSIII)... 9 Table 6 During the last year was there any instance when you had to make an informal payment or a bribe to get health services?... 9 Table 7 Taking everything into account are you satisfied with the quality of agricultural extension services? (WCBSII) Table 8 Taking everything into account are you satisfied with the quality of agricultural extension services? (WCBSIII) Table 9- Taking everything into consideration, how satisfied or dissatisfied are you with the quality of household rubbish (solid waste) available to you? Table 10 - Proportion paying any of the following taxes or fees? Table 11 - Satisfaction levels with tax office operation Table 12: Have you been informed about various activities of Woreda/CA Government? Table 13 - Are Woreda/City Administration Council meetings open to the public? Table 14 - Have any Woreda/City Administration officials actively sought the views of people in your Kebele concerning: Table 15 - If drinking water is not satisfactory what would you do about it? Table 16 - For those who would raise their concern with water service quality, how confident are you that these concerns would be addressed? Table 17 - For those who would raise their concern with government health facility service quality, how confident are you that these concerns would be addressed? Table 18: Ordered logistic regression of determinants of perceived changes in water service quality in City Administrations within Regional States Table 19: Ordered logistic regression of determinants of perceived changes in water service quality in primarily rural Woredas within Regional States Table 20: The odds of being satisfied with your water supply if you are in a City Administration are determined by the following (logistic regression) Table 21: The odds of being satisfied with your water supply if you are in a primarily rural Woreda are determined by the following (logistic regression) Table 22: How do you compare the quality of government health services that you currently get with those you got two years ago? Table 23: levels of service Table 24: Have Government Health Services improved, stayed the same or got worse in the past two years (City Administration sample)? Table 25: Have Government Health Services improved, stayed the same or got worse in the past two years (Woreda sample)? Table 26: If Woreda/City Admin leaders do not keep promises how much can an ordinary person do to hold them to account? Table 27: Likelihood of being punished if you complain about services or misuse of funds Table 28: Percentage who think it is likely or very likely people will be punished by Woreda officials if they complain about poor quality services or misuse of funds Table 29: Feared crime happening in your own home? Table 30: How often would you say that social shengo courts solve problems by gender? Table 31: How often would you say that social shengo courts solve problems by region? iv

5 Table 30: Some people say that many of the following people are involved in corruption but others say they are not. Of the following statements which one do you think is most accurate? Table 31: Logistic regression, dependent variable is the probability that City Administration officials are seen as mostly or all corrupt (for City Administrations) Table 32: Logistic regression, dependent variable is the probability that Woreda officials are seen as mostly or all corrupt (for Woreda) Table 33: What would be your highest priority for the Woreda/City Admin to address? Table 34: What would be your highest priority for the Woreda/City Admin to address? Index of Figures Figure 1: Satisfaction levels with water supply... 6 Figure 2: Woreda (rural) satisfaction levels with water supply... 7 Figure 3: City Administration (urban) satisfaction levels with water supply... 7 Figure 4 - Overall, how would you rate the information provided by the agent? Figure 5 - What difference to income has the service made? Figure 6 - If you face times of food shortage, has this support reduced the amount of food shortage you face? Figure 7 - Are Woreda/City Administration Council meetings open to the public? Figure 8: Changes in perceived water quality in City Administrations by Region Figure 9: Changes in perceived water quality in rural Woredas by Region Figure 10: Proportion using Government health facilities in the past year Figure 11: City Administration & Woreda health satisfaction indices by Region Figure 12: Health satisfaction index disaggregated by wealth category Figure 13: Proportion who strongly agree that you will get a better service from a government health facility if you make an informal payment (City Administration) Figure 14: Proportion who strongly agree that you will get a better service from a government health facility if you make an informal payment (Woreda) Figure 15: Taking everything into consideration, how satisfied or dissatisfied are you with the quality of household rubbish (solid waste) collection services available to you Figure 16: Proportion who know which taxes & fees must be legally paid Figure 17: Percentage who know their Woreda/CA budget Figure 18: Percentage who know their Woreda/CA Strategic Plans, Council agenda & decisions44 Figure 19: Proportion believing they can do nothing or very little if Woreda/City Admin leaders do not keep their promises Figure 20: Proportion who did NOT fear crime in their own home in the past year Figure 21: Percentage of people who think Woreda Officials are mostly or all corrupt Figure 22: Percentage who think the police in their Woreda are mostly or all corrupt v

6 Executive Summary Introduction The Woreda and City Government Benchmarking Survey (WCBS) was initiated through a high level Government of Ethiopia (GoE) and Bilateral and Multilateral Donors agreement to jointly undertake four rounds of benchmarking survey over the life time of the Public Sector Capacity Building Program (PSACP) as a core monitoring instrument. The first WCBS carried out in 2005 established a base line in a limited number of Woredas and Cities. The 2008 WBCS II extended the coverage and scope of WCBS I and added a demand side to the analysis in terms of collecting data on citizens perceptions regarding the performance and service delivery of Woredas and Cities. Specifically, the demand side component aims to collect and analyze the views, opinions and perceptions of citizens with regard to governance and service delivery at the local administrations level. It provides a snapshot of citizen s views at the time of the survey but cannot attribute levels or changes in these views to any particular program or intervention. It is important that policy makers, executives and civil servants take this information on board to review policy and strengthen service delivery. Citizen s perspectives are also required to validate and place supply-side findings (from local officials) into context. This report presents the findings of the primary quantitative instrument for collecting demand-side information in WCBSIII - a random sample survey of 10,567 citizens, known as the Citizen s Report Card survey (CRC). Fieldwork was undertaken in early 2010 covering Addis Ababa and the 9 Regional States. The objective of this study has been three fold: Water 1. To allow comparison of major indicators in the WCBSII and WCBSIII CRC surveys. The WCBSIII sample contains nearly 4000 observations from 23 Woredas that are compared with observations from the same Woredas (but not necessarily the same enumeration areas) from the full WCBSII sample. 2. To provide a detailed understanding of key demand-side factors determining local public service delivery and governance. 3. To enable comparison of supply and demand-side perspectives on critical local public service delivery and governance issues this comparison is undertaken in the supply-side report. In major urban areas (City Administrations), satisfaction levels have risen overall between 2008 & 2010 and the proportion saying they are very satisfied has risen sharply (24% to 35%). In primarily rural areas (Woredas), satisfaction levels have fallen overall and the proportion saying they are very dissatisfied has risen sharply (16% to 29%) These findings are consistent with the results of asking people in WCBSIII to say whether water services have improved in the past two years. Just over half of those in City Administrations say water service quality has improved compared to just over one in three vi

7 of those in rural Woredas. However, the situation varies significantly between Regions (particularly for rural areas): from the Woreda data set Gambella was the only Regional State in which a majority (55%) of respondents felt that water service quality had improved over the past two years. In contrast, 70% of residents in Benishangul-Gumuz believe quality has fallen. Ordered logistic regression for each Region to investigate who is likely to have benefited from improved water quality indicates that: The poor or very poor are much less likely to report improvements in water quality than the non-poor in most City Administrations (those in Harari and Amhara are the two exceptions). The same is true for primarily rural Woredas with the exception of those in Afar. For example, in Benishangul-Gumuz someone who describes themselves as very poor (33% of the sample in this area) is 4.4 times more likely to have experienced declining (rather than no difference or improving) water service quality than someone who categorizes themselves as non-poor. In cities, stand-pipe users (where water is delivered outside the compound) are much less likely to report improvements than those where water is piped inside the compound/home. In general, men and women have the same views on whether water service quality has improved except in City Administrations in Tigray and there is generally no differentiation by age either. Using logit analysis to determine why a person is satisfied or dissatisfied with their water supply we see that: For city residents, experiencing worsening water service quality increases the odds of being dissatisfied far more than improving service quality increases the odds of satisfaction (13:4). This suggests that local government needs to maintain a focus on preventing service deterioration as well as expanding improved services to new customers. As expected, the better educated and non-poor are more likely to be satisfied with the service they receive (in both rural and urban areas). Being informed by the City/Woreda Administration about services provided or even how your Woreda compares to another does not increase the odds of satisfaction with water supply. Consultation by City Administration/Woreda Officials does appear to make a positive difference (once we have controlled for improved supply). However, if you think that others in the Kebele rather than you personally are consulted you are also more likely to have a greater satisfaction with water supply which suggests that consultation may be focussed on areas where people have a reasonable water supply rather than areas where satisfaction is low. Confidence that the City/Woreda Administration will address concerns over water supply issues raised by citizens is closely associated with service satisfaction. Although there are alternative explanations for this it does suggest that local government can improve satisfaction with water supply by demonstrating to citizens that their concerns with service quality will be addressed. Across all primarily rural Woredas, the odds of being satisfied with water supply are twice as high for residents with a protected water supply and nearly four times as high for those vii

8 with stand-pipes outside their residence in comparison to citizens relying on unprotected water sources (all other variables held constant). Health Comparison of the 2008 and 2010 data reveals a very large increase in the proportion of those in City Administrations using Government Health facilities (50% to 73%) and a large increase for those in primarily rural Woredas (77% to 87%) 1. The proportion of respondents who had used government health facilities during the past year (of those who had used any health facilities) ranged from nearly 100% in Benishungul- Gumuz and Gambella to around 70% in Harari (the average nationally was 84%). There is much less difference in terms of government health facility access between City Administration (79%) and Woreda (86%) samples than we have seen for access to drinking water. The increase in access to government health facilities has been accompanied by a perception of increased service quality over the past two years. Nearly 60% of those in rural Woredas and nearly 70% in City Administrations say that the quality of Government health facilities has improved. Approximately two-thirds of all respondents are satisfied using a number of indicators of service quality. Access to prescribed medicines and medical supplies does, however, appear to be an area of widespread dissatisfaction. Focus group discussions in a number of locations confirm this and identify that many people feel that doctors too often direct patients to private dispensaries where they have an interest. A government health facility service index (-16 to +16) has been compiled from the extent of agreement with the following 8 statements: A. I was satisfied with the length of time I had to wait for service B. The facility is at a convenient distance from my Home C. The medical staff were readily available D. The facility had all the necessary medicines and supplies E. The medical staff were courteous and helpful to me F. The buildings are in good condition and well maintained G. I would get better service if I paid a small informal Fee H. I received good medical attention by qualified staff Average scores for Cities and primarily rural Woredas in each Region are shown in the Figures below. The average government facility health index satisfaction score for City Administrations is 4 but city residents in Somali, SNNP and, in particular, Amhara have lower satisfaction scores. In contrast, those in Oromiya, Harari, Addis Ababa and, in particular, Tigray have higher service satisfaction scores. The mean index values for primarily rural Woredas tell another story. For this group the average score is 1.2 i.e. satisfaction levels with government health services are more than 1 This comparison is for all those who had used a health facility in the previous year. The proportion using a health facility (of any kind) is somewhat higher in the WCBSIII data (74%) than in the previous round (65%). viii

9 three times higher in City Administrations than rural Woredas. Scores in Amhara match the national average and those in Oromiya are slightly higher. Health service satisfaction index scores in Somali are more than twice the rural average and are more than four times this average in Gambella and Tigray. In Benishungul-Gumuz the average index score is just below zero and this falls to -1.1 in SNNP implying moderate levels of dissatisfaction taking all dimensions of the index into account but it is twice as far below zero in Afar suggesting much greater dissatisfaction. Mean of City Admin health service satisfaction index Tigray Addis A Harari Oromia Somali SNNP Amhara REGION Mean of Woreda health service satisfaction index Tigray Gambell Somali Oromia Amhara Benisha SNNP Afar REGION In City Administrations both the relatively rich and the poor, the young and the old appear to be benefiting from improved government health facilities. The story is very different in rural Woredas. In most Regions the very poor are significantly less likely than the non-poor to identify an improvement in health services. Only in Gambella and Amhara are those who identify themselves as poor or very poor more likely to identify improvements than other groups. More research is needed to identify if this reflects an investment in particular types of service and if lessons can be drawn for other regions. ix

10 Agricultural extension A large majority of respondents from the matched set of Woredas who were involved in any form of agriculture reported that they had an agricultural extension officer in their Kebele (95% in WCBSII and 94% in WCBSIII). This reflects a near 100% coverage for sample sites in Amhara, Oromiya, SNNP and Tigray and 82% coverage for the WCBSII sample in the Afar region. Overall, there has been a significant decline in the proportion of respondents who find that the information provided by their extension agent is very useful (52% to 38%). In 2008 most respondents said that the information received was very useful but in 2010 most felt it was only quite useful. This reflects reduced impacts of extension services on agricultural income and food security. These results were seen across Woredas in Afar, Amhara, Oromiya and SNNP but not Tigray where the opposite was actually the case. This has to be seen in the context of below normal seasonal rains in most areas during the past two years. Our results imply that the perceived value of agricultural extension services is closely associated with levels of rainfall. Indeed, Focus Group Discussions (Butcher, 2010) suggest that a combination of increasing expense of inputs such as fertilisers and poor weather conditions that have meant that farmers have not been able to realise expected returns. It is important to note that despite the decline in the proportion believing this service is very useful, overall satisfaction levels with agricultural extension services remain high (90% of those receiving this service in 2008 and 84% in 2010). Solid waste collection Questions on this public service were only asked in City Administrations. Overall, there has been an increase in satisfaction with this service from 70% to 74% between WCBSII and WCBSIII. In general, most residents are now quite or very satisfied with this service area but there are some significant regional variations. The proportion very satisfied with collection services ranges from 43% in cities in Tigray to 3% in Harari. The only significant levels of dissatisfaction are found in cities in Oromiya and, to a lesser extent, in Harar. Taxation There has been a significant increase in the proportion of respondents paying most taxes and fees between 2008 and 2010 particularly in City Administrations. The high levels of dissatisfaction with tax offices almost certainly reflect a more general dissatisfaction with paying taxes. Hence it is probably most useful to focus on the aspects of operation that are particularly unpopular and have become more so since These are: Times of opening of offices where payment is made; and x

11 The location of offices where payment is made People are slightly less dissatisfied with the information on tax and fee rates available and the honesty of officials and levels of dissatisfaction fell between 2008 and Nationally, nearly 60% of respondents knew which taxes and fees must be legally paid but there is considerable regional variation with less than 20% of residents in Afar and Gambella answering in the affirmative. Overall, the average is also lower for urban residents reflecting the greater payment of income tax rather than user fees. Public information, consultation and local governance Local government appears to be improving the delivery of information on their budgets, strategic plans and decisions taken by the Council although it is still less than a quarter of citizens who have this knowledge. However, this progress has not been matched in publicising the agenda for Council meetings. An important indicator is drawn from the response to whether any Woreda officials have actively sought the views of people in your Kebele concerning the most important development needs and the quality of public services. Approximately half of those interviewed in 2010 said that Kebele members had been consulted, suggesting a decline of some 5% since In contrast, only 17% said they had been consulted personally by local officials in the WCBSIII. A comparison of WCBSII and III results suggests that citizens are increasingly reliant on their most local institutions (e.g. water user committees or public meetings) rather than Woreda/City Administration leaders to resolve problems with water and health service delivery. For example, where there are concerns with services from local government health facilities those saying they are quite likely or very likely to speak with or write to their Woreda/City Administration Councillor on these issues has fallen from 68% in 2008 to 35% in 2010 (see Annex 1). The greatest fall has been in rural Woredas. This may well be a consequence of what appears to be a striking loss of confidence among rural (Woreda) respondents that the concerns they raise regarding water service quality will be addressed (e.g. the proportion very confident has fallen from 32% to 8%). Confidence among residents in City Administrations has remained largely unchanged. A similar loss of confidence by citizens of rural Woredas that concerns will be addressed is reported for users of government health facilities. Although for health services, the loss of confidence is shared to some extent by citizens in City Administrations. In response to the question If Woreda/City Administration leaders fail to keep their promises how much do you think an ordinary person can do to improve the situation? we find that just over half (53%) feel powerless but nearly half believe they can do at least something (47%). There is, however, very significant variation across Regions (and to a smaller extent by wealth category). For example, while approximately 35% of residents in SNNP and Addis Ababa feel unable to hold local government leaders to account this figure rises to more than 75% in Tigray. Given the Government s objective of improving local public service delivery, attention should be given to improving the ability and confidence of xi

12 0 10% 20% 30% citizens particularly in Tigray, Somali, Afar and Amhara Regions to hold their local leaders to account on such issues. Perceptions of corruption Our experience with this survey suggests that people in Ethiopia are often uncomfortable talking directly about corruption but are willing to talk in indirect terms about it. Only 5% say yes in response to the question Have you or anyone in your household had to make any extra payments outside the legal requirement for any government services in the past year? This is very likely to be an underestimate of all such payments as 8% of (the large majority) of respondents whose family had received government health services reported making an informal payment just to get health services in the past year. When we ask about the extent of corruption in local public institutions we find that perceived corruption levels among civil servants at Kebele and Woreda levels is significantly lower than for other groups. Taking most or all are corrupt as an indicator of corruption we can define three ranges: 1. Low approximately 9% (for local civil servants) 2. Medium - 16% - 19% (for Kebele and Woreda officials) 3. High 22%-23% (for the local Police and Courts) Considering perceived corruption by Woreda officials, residents of City Administrations perceive this aspect of corruption to be higher (23%) than those in primarily rural Woredas (17%). Even allowing for this, the divergence in the range of perceptions by Region is striking with this aspect of perceived corruption being at least 300% greater in Amhara and Tigray than in Harari, Afar and Benishangul-Gumuz. Amhara Tigray City Gambella SNNP Oromia Somali Addis Ab Woreda Harari Afar Benishan REGION CITY or WOREDA xii

13 It has to be emphasised that the variation in perceived corruption across regions reflects variation in trust in these officials and may not actually imply financial corruption. So, for example, Tigray has one of the lowest levels of informal payments to secure health services but a relatively high proportion of citizens say their Woreda officials are corrupt. The variable determining whether a resident of a City Administration believes that their Woreda officials are corrupt was investigated further using logistic regression. Taking all factors into consideration, age, gender and wealth have no significant impact. Consultation by Woreda officials does nothing to change the perception of whether they are corrupt or not. However, having information on the services provided by your Woreda reduces the odds of perceived corruption by 50%. It seems likely that citizens feel better able to judge how Woreda resources are being used if they are informed about which services are available and that this reduces the perception that resources are being misused. Individuals who do not feel empowered to hold their City Administration officials to account for promises made are much more likely to believe these officials are corrupt. This may well be the reality as the opportunity for corruption is reduced by public accountability. Yet it also illustrates that encouraging a culture that enables citizens to hold their public service providers to account is also likely to raise the status of these officials in the eyes of the public. Taking all other variables into account, fear that reporting poor service quality or misuse of funds will result in punishment by Woreda officials dramatically increases the odds that these officials will be regarded as corrupt. Moving from a situation where punishment is thought not very likely to likely makes it nearly three times as likely that a person will see their City Administration officials as corrupt. Again, there is likely to be a real effect here as a culture of deterring complaints from the public creates space to hide corruption. It also increases the perception that funds are being misused even if they are not. Crime The WCBSIII records reported incidence of crime and fear of crime over the past year across a number of categories ranging from never to very frequent. Overall, the vast majority of respondents (around 9 out of 10) did not fear crime in their own home. Women report very slightly higher levels of fear of crime in their home than men. There is a greater differentiation by socio-economic group with the very poorest 10% feeling significantly less secure in the home than other groups. Once again, however, the biggest differences are between Regional States with more than 20% of respondents in Gambella and Amhara reporting fear of crime in the home in comparison to less than 5% in the Somali Region and Addis Ababa. It is worth noting that there is virtually no difference in fear of crime by age group or between Cities and primarily rural Woredas. xiii

14 Social Shengo Courts Nationally around half of respondents believe social courts solve problems quite or more than quite often. There is no clear pattern of variation by poverty category but men are slightly more positive in their attitude to social courts than women. Regional differences are more pronounced with very positive perceptions of success in Tigray, Harari and Benishungul-Gumuz, much more modest perceptions in Addis Ababa and Somali regions and almost no use of these courts in Afar. Single greatest priority for improvement in local public service delivery Residents were asked to give their single greatest priority for local public service delivery improvement and the results are that: Improving the water supply is the most frequently expressed priority but for individuals in Afar, B-G, Harari, Oromiya, SNNP, Somali and Tigray it is the overwhelming priority for improvement. In general, though this is the highest priority in rural but not urban areas. While improving government health facilities is only the number one priority in Amhara it gets the second most votes in most Regions and is the main priority in City Administrations. Road improvement is a high priority in most but not all regions. Improving support to the poorest local families is the number one priority in Addis Ababa but is also a high priority in Somali and Tigray Regions. xiv

15 1 Introduction The Woreda and City Government Benchmarking Survey (WCBS) was initiated through a high level Government of Ethiopia (GoE) and Bilateral and Multilateral Donors agreement to jointly undertake four rounds of benchmarking survey over the life time of the Public Sector Capacity Building Program (PSACP) as a core monitoring instrument. The benchmarking survey aims to provide a systematic, empirical and representative assessment of key aspects of the public management capacity of Wereda and Municipal Governments (financial, human resource and participative capacities) and performance for some selected services. The first WCBS carried out in 2005 established a base line in a limited number of Woredas and Cities. The WBCS II extended the coverage and scope of WCBS I and added a demand side to the analysis in terms of collecting data on citizens perceptions regarding the performance and service delivery of Woredas and Cities. More specifically, the demand side component aims to collect and analyze the views, opinions and perceptions of citizens with regard to governance and service delivery at the local administrations level. It provides a snapshot of citizen s views at the time of the survey but cannot attribute levels or changes in these views to any particular program or intervention. It is important that policy makers, executives and civil servants take this information on board to review policy and strengthen service delivery. Citizen s perspectives are also required to validate and place supply-side findings (from local officials) into context. The primary quantitative instrument for collecting demand-side information in WCBSIII is a random sample survey of 10,567 citizens, known as the Citizen s Report Card survey (CRC). Fieldwork was undertaken by Selam Development Consultants in early 2010 covering Addis Ababa and the 9 Regional States and data was handed over for analysis at the end of April This report presents the results of this analysis and has three objectives: 1. To allow comparison of major indicators in the WCBSII and WCBSIII CRC surveys; 2. To provide a detailed understanding of key demand-side factors determining local public service delivery and governance. A qualitative demand-side module was also designed to focus on issues from the CRC and, although a separate report has been produced on these issues, this report attempts to draw from it at critical junctures. 3. To enable comparison of supply and demand-side perspectives on critical local public service delivery and governance issues. The CRC was designed to cover a core sub-set of issues tackled in the supply-side survey 2. This comparison is undertaken in the supply-side survey report. The remainder of this document is as follows: the CRC methodology is set out in Section 2; a comparison of major indicators with WCBSII can be found in Section 3; and Section 4 presents a detailed analysis of key determinants of local service delivery and governance. 2 The supply-side survey is based on interviews with officials in each local administration. 1

16 2 Methodology 2.1 Sample design The design of the CRC sample was complicated by the requirement to meet three criteria: 1. To deliver random sampling of citizens in the target Regions that meets Ethiopia s Central Statistical Agency (CSA) and international standards 2. To be drawn from 378 Supply-Side survey Woredas & City Governments (across all Regions but from a total of approximately 770 Woredas & City Governments). 3. To include 23 Woredas/City Administrations sampled in the 2008 WCBSII We are very grateful for the advice of Zelealem Hailegiorgis of the CSA on how to meet these objectives and his assistance in producing the sample frame. The sampling method uses multi-stage random sampling as follows: 1. The first step was to stratify all 378 Supply-Side Woredas by remoteness from the Regional capital and food security Woredas were then selected using probability proportional to size (PPS), where size is the population from the 2007 Census. 3. Second-stage sampling units are standard Enumeration Areas (EAs) within each selected Woreda selected from the sampling frame provided by CSA again using probability proportional to size, where size is the number of households in each EA (20 households were selected). 4. Finally, households were selected at random from within the EA by fieldworkers (who had identified all local households). From a technical perspective, the calculation of sample size for a population-based survey needs to take into account the variability of the key measured characteristics and the degree of confidence required. Based on a maximum confidence interval of +/- 3 percentage points at the 95 per cent confidence level, CSA calculated the required number of households for the regionally representative sample to be 10,440 households. In practice, it was not possible to work in two Woredas in Somali Region and 10,657 households were sampled from 68 Woredas. The full list of Woredas sampled is given in Annex Survey instrument design and the review process Following an external review of the WCBSII, the WCBSIII CRC questionnaire was based on a significantly shortened version of the WCBSII questionnaire which was developed in 2

17 cooperation with the Ministry of Capacity Building Steering Committee, the World Bank and other stakeholders 3. The areas covered in this round of the WCBSIII are: City Administrations Essential socio-economic and geographic location questions Water services Government health services Taxation Land management Solid waste services Public information, consultation and governance Woredas Essential socio-economic and geographic location questions Water services Government health services Taxation Agricultural extension services Public information, consultation and governance Training sessions were held for fieldwork staff and a detailed survey manual was produced. The questionnaire was translated and back-translated and piloted in four locations (rural and urban). Although data entry and cleaning was the responsibility of Selam Development consultants, Sebastian Jilke contributed by undertaking a detailed review of the initial CRC data and provided many helpful recommendations. 2.3 Comparing WCBSII and WCBSIII CRC data The sub-sample of WCBSIII Woredas matched to WCBSII Woredas is shown in Annex 2. The WCBS III matched Woreda sample contains nearly 4000 observations from 23 Woredas (and these are compared with approximately 9800 observations in WCBSII). Although the Woredas are common to both survey rounds it was not possible to match the enumeration areas within Woredas. Moreover, differences in sample design between the two rounds led to a lower proportion of City Administration respondents in the WCBSII sample (67% Vs 72%). In order to get the best match possible between rounds, the WCBSII data is weighted to give the same proportions in Woredas and City Administrations as the WCBSIII comparison sample 4. 3 The questionnaires have been distributed separately to this report as PDF files. 4 Working with the WCBSII CRC data proved to be something of a challenge as no variable or value labels were entered and the data entry protocol combined no and don t know responses. It was not possible to consult the WCBSII questionnaires as these had been disposed of. 3

18 3 Findings: Changes since 2008 (WCBSII) 3.1 Issues to bear in mind when comparing 2008 and 2010 CRC data Caution is required when comparing the CRC survey results from WCBSII and III. The following should be taken into account: 1. As noted previously, we have data on 23 Woredas that are common to both survey rounds but it was not possible to match enumeration areas within Woredas. There is some overlap but it is only partial; 2. The 2008 CRC covered 27 Woredas/City Administrations although the local contractor was only able to provide data on 23 of these for the 2010 CRC. These 23 matched Woredas have been used as the basis of comparison rather than the 2008 CRC report; 3. The CSA sample frame is designed to allow regional level analysis using the full WCBSIII CRC data set. This is undertaken in Section 4 of this report. However, the matched 23 Woredas are only a sub-set of the 2010 CRC data (only 58% of all observations in these regions) and so it is not sensible (in terms of being statistically robust) to disaggregate the matched data by region. With these health warnings in mind we compare poverty profiles 5 based on selfassessment wealth category criteria from the weighted, matched samples. As 5 We attempted to compare the data sets using age and education variables but this was not possible. Following the external review of the 2008 CRC the 2010 questionnaire has been significantly shortened. As part of this exercise data on age and education is now only collected for the respondent (rather than all family members) and as it proved impossible to identify which variable referred to each family member in the 2008 data these variables cannot be compared across the two survey rounds. 4

19 Table 1 below shows these results are similar which gives us some confidence in comparing the results of the two survey rounds at a national level. It is interesting to note that the 2010 data shows a reduction in the proportion in the poor (34.8% to 33.2%) and very poor (12.3% to 9.6%) and an increase in the neither poor nor non-poor group. Although it is tempting to say this reflects the high rate of economic growth since 2008 it could simply reflect changes in the sample mix. 5

20 Table 1: Comparing poverty profiles WCBSII and III WCBSII Poverty category Valid Missing Total very poor poor neither poor nor non-poor non-poor Total Sy stem Valid Cumulativ e Frequency Percent Percent Percent WCBSIII (comparative Woredas only) Which of the following categories would you put yourself in: Valid Very poor Poor Neither poor nor non-poor Non-poor Total Valid Cumulativ e Frequency Percent Percent Percent Water supply satisfaction levels Figure 1 below compares levels of satisfaction for water supply 6 across the two survey rounds. There is some positive news here as the proportion who are very satisfied with water services has increased (18% to 20%) but overall, satisfaction has fallen. The proportion satisfied has declined (42% to 37%) and there appears to have been a move from dissatisfied (28% to 22%) to very dissatisfied (13% to 22%). Figure 1: Satisfaction levels with water supply 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Strongly satisfied Satisfied Dissatisfied Strongly dissatisfied WCBSII WCBSIII 6 The WCBSII figures reflect the weighted average using both drinking and other water source questions 6

21 As Figure 2 and Figure 3 below show, these findings combine two quite separate trends. 1. In major urban areas (City Administrations), satisfaction levels have risen overall and the proportion saying they are very satisfied has risen sharply (24% to 35%). 2. In primarily rural areas (Woredas), satisfaction levels have fallen overall and the proportion saying they are very dissatisfied has risen sharply (16% to 29%) Figure 2: Woreda (rural) satisfaction levels with water supply 40% 35% 30% 25% 20% 15% WCBSII WCBSIII 10% 5% 0% Strongly satisfied Quite satisfied Quite dissatisfied Strongly dissatisfied Figure 3: City Administration (urban) satisfaction levels with water supply 60% 50% 40% 30% 20% WCBSII WCBSIII 10% 0% Strongly satisfied Quite satisfied Quite dissatisfied Strongly dissatisfied These findings are consistent with the results of asking people in WCBSIII to say whether water quality has improved in the past two years (see Table 2) just over half of those in City Administrations say it has compared to just over one in three of those in rural Woredas. Table 2: Perceived change in water quality in the past two years 7

22 3.3 Health use of Government facilities & satisfaction levels Comparison of the 2008 and 2010 data reveals a very large increase in the proportion of those in City Administrations using Government Health facilities (50% to 73%) and a large increase for those in primarily rural Woredas (77% to 87%) 7. There are some differences in the way the satisfaction questions were asked that should be noted when comparing satisfaction levels over time. In 2008, respondents were first asked if they were satisfied or dissatisfied with various aspects of service and then asked whether their satisfaction or dissatisfaction was complete or partial. Unfortunately, data was entered so that it is impossible to distinguish whether respondents were completely satisfied or completely dissatisfied! Hence we only report the first part of the 2008 question. In 2010, respondents were asked the extent to which they agreed or disagreed with statements on these service aspects e.g. I was satisfied with the length of time I had to wait for service. Comparing the results of Table 3 and Table 4 it seems that the large increase in the proportion of citizens using government health facilities may have been accompanied by a small reduction in satisfaction levels. In one area the availability of medicines the reduction in satisfaction is much greater (53% to 36%) and focus group findings confirm this is an issue of concern. However, as Table 5 shows, general satisfaction levels with government facility health services are actually likely to be rising. When health service users were asked in WCBSIII (2010) how they compared the quality of government health services they received now with those received two years ago, a significant majority identified an improvement. Nearly 60% of those in rural Woredas and nearly 70% in City Administrations say that the quality of Government health facilities has improved over the past two years. We look into this in further detail in the next section of this Report but note here that while perceived improvements from 2010 data may capture improved availability of services as well as quality it seems more likely, to this author at least, that the 2010 data is a more accurate barometer of service quality. Table 3 Satisfaction with dimensions of government health facility services 2008 (WCBSII) Satisfied Yes No Overall treatment you received 74% 26% Waiting time taken to attend to you 66% 34% Overall behaviour of staff/doctors with you 69% 31% Helpfulness of staff 66% 44% Quality of treatment 63% 47% Availability of medicine 53% 47% 7 This comparison is for all those who had used a health facility in the previous year. The proportion using a health facility (of any kind) is somewhat higher in the WCBSIII data (74%) than in the previous round (65%). 8

23 Table 4 Satisfaction with dimensions of gov t health facility services 2010 (WCBSIII) Agree strongly Agree somewhat Disagree somewhat Disagree strongly I received good medical attention by qualified staff 12% 45% 28% 15% I was satisfied with the length of time I had to wait for service 17% 46% 25% 12% The medical staff were courteous and helpful to me 16% 49% 26% 9% The facility had all the necessary medicines and supplies 7% 29% 35% 29% Table 5 Perceptions of changes in the quality of Government health services since 2008 (WCBSIII) Finally, in this sub-section we consider the extent to which respondents reported paying tips or bribes to get better treatment. Table 6 indicates that this measure has remained broadly constant at 4%. Table 6 During the last year was there any instance when you had to make an informal payment or a bribe to get health services? WCBSII (2008) WCBSIII (2010) Yes Yes 4.3% 3.9% 9

24 3.4 Agriculture use of extension services, impact & satisfaction A large majority of respondents from the matched set of Woredas who were involved in any form of agriculture reported that they had an agricultural extension officer in their Kebele (95% in WCBSII and 94% in WCBSIII). This reflects a near 100% coverage for sample sites in Amhara, Oromiya, SNNP and Tigray and 82% coverage for the WCBSII sample in the Afar region. As Figure 4 shows, there has been a significant decline in the proportion of respondents who find that the information provided by their extension agent is very useful 8 (53% to 32%). This has been accompanied by an increase in the proportion reporting it is Quite useful (37% to 46%) and a small increase in the proportion saying this information was not useful (3% to 6%). Figure 4 - Overall, how would you rate the information provided by the agent? 60% 50% 40% 30% 20% WCBSII WCBSIII 10% 0% Not useful Quite useful Very useful Cannot say In 2008 most respondents said that the information received was very useful but in 2010 most felt it was only quite useful. This result was seen across Woredas in Afar, Amhara, Oromiya and SNNP but not Tigray where the opposite was actually the case. In order to try and understand what is driving an apparent decline in satisfaction with this service we consider two areas in which agricultural extension services are likely to have an impact: agricultural income and food security. As the Figures below show, we observe declines in perceived performance in both these areas with a greater fall for food security. This has to be seen in the context of below normal seasonal rains during the past two years. According to the FAO/WFP Crop and Food Security Assessment Mission (CFSAM) report of February 2010, the 2009 national production of cereals and pulses estimated at 16.8 million MT is about five percent lower than the production in Our results do seem to imply that the perceived value of agricultural extension services is closely related to rainfall. 8 Reported for all those who say they had an extension agent in their Kebele