Ministerial Leadership in Health Program Ministerial Forum for Health Ministers

Ministerial Leadership in Health Program Ministerial Forum for Health Ministers Harvard University June 7 11, 201 In association with the Children s Investment Fund Foundation (UK) and with the support of the Bill and Melinda Gates Foundation EVALUATION REPORT Analysis and Compilation by Helene Perold and Aislinn Delany for Helene Perold & Associates i

Table of Contents 1 Introduction... 1 2 Overview of evaluation findings... 1 Evaluation methodology... 2 Competency barometer assessment....1 Changes in group feedback on self-reported competency levels....2 Changes in individual feedback on self-reported competency levels... Country 1... Country 2... 6 Country... 7 Country... 8 Country... 9 Country 6... 10 Country 7... 10 Country 8... 11 Country 9... 12 Country 10... 1 What is likely to change?... 1.1 Doing things differently... 1.2 Dealing with challenges... 1. Most useful aspects of the Forum... 16 6 Conclusion... 16 Annexure 1 Health Ministers Forum Agenda... 18 Annexure 2 Day 2 feedback report... 22 Annexure Day feedback report... 2 Annexure Day feedback report... 2 i

1 Introduction The annual Ministerial Forum for Health Ministers is a core part of the Ministerial Leadership in Health Program at Harvard University. The main goals of the Forum are to increase ministerial and political leadership effectiveness in the effort to improve health outcomes in developing and emergent countries by increasing efficiency in the use of national budgets and delivery of public health services, and establishing better standards of care. The 201 Forum convened participants from ten countries: Barbados, Georgia, Ghana, Ivory Coast, Jamaica, Lesotho, Malawi, Mozambique, Namibia, and Uganda. The Forum is an intensive five day learning experience (see Attachment I) lead by senior Harvard faculty, a select number of experts from other organizations, and a resource group of former and long serving health ministers. The curriculum is designed to facilitate discussion of new approaches and impart practical skills for effective ministerial leadership and sustainable health sector strengthening with specific attention to health financing, more efficient utilization of health resources, improved standards of public health care and better outcomes. This report outlines the learning recorded by each of the participants over the course of the five days and documents how they rated changes in their knowledge and their competence to strengthen the health systems in their countries. The results are based on self-reported assessments on indicators related to the content of the program. It also reports on take-out interviews with a subset of randomly chosen participating ministers and includes the results of learning assessment tools administered each day of the four day Forum. In order to adhere to the assurance that all responses would be confidential, the countries represented in the report are numbered, not named. 2 Overview of evaluation findings At the start of the Forum, the participating Health Ministers reported mid-level confidence and comfort with a set of ten competencies. By the close of the program, the group had improved in knowledge and confidence across all ten of the competency indicators, with the average group score increasing from. (out of ) to.2. In the final assessment, all but one indicator received a score of or more, showing that levels of confidence were high by the end of the Forum. The group felt most equipped to: (a) define their leadership goals and priorities to leave a legacy following their term in office, and (b) increase collaboration with the Finance Minister in support of health sector strengthening. Several ministers mentioned being comfortable with this collaboration during the individual interviews. The areas that showed the largest gains in group knowledge and confidence were associated with health financing, namely: (a) Identifying fiscal space in the health budget through reallocation, savings and additional revenues, and (b) identifying opportunities for increased efficiencies in the use of the health budget in procurement, supply management, health worker productivity. The management of planning, implementation and monitoring to deliver health sector also showed a marked improvement. Although this was the item with the lowest score (.9 out of ) in the final assessment, it represents the third largest improvement over the group s baseline score (2.9 out of ) given at the start of the Forum. 1

These findings show that the Forum was successful in promoting a positive shift in the Health Ministers levels of confidence and knowledge on the core themes: transformational leadership, strengthening planning and policy implementation, the need for a focus on efficiency and obtaining value for money through improved procurement and supply chain management systems, and building a productive working relationship with the finance minister in order to strengthen health systems and service delivery. By the end of the Forum a number of the ministers still identified resourcing as their major challenge, but their comments and ratings indicate that they had acquired skills and tools that could enhance their leadership effectiveness in dealing with this constraint. Most said they planned to work closely with their finance minister to increase fiscal space for health and outlined in some detail the approach they would take on their return to their country. A number of ministers said that they would immediately focus on setting up delivery teams whom they could hold responsible for implementing measures to improve the delivery of health services, and would monitor progress more closely than they had done in the past. Some welcomed this approach as the only way of consistently working towards strengthening the health system whilst dealing with competing demands and managing crises. Evaluation methodology The evaluation process was designed to track the development of participants knowledge, skills and confidence, and to provide the Harvard team with daily feedback about the program content, presentations and discussion. Specific aims of the evaluation process were as follows: a. Provide comparative data that measures what the Ministers feel they have learnt in the course of the forum; b. Provide the Harvard team with daily feedback from the Ministers on the value of what they learnt on the day; c. Gain closer insight into the scores provided by the Ministers. Two types of self-assessment instruments were used to collect quantitative information about participants perspectives on the value of the different parts of the program and whether they have learnt anything new: A competency barometer was designed to produce baseline scores for self-reported impact of the workshop. At the beginning of the Forum ministers were asked to rate the extent to which they felt equipped to carry out 10 competencies required to strengthen the public health sector. Ministers were also asked to write a sentence explaining their scores, and this provided some insight into the ratings given. At the end of the forum the same instrument was administered to capture final scores needed to assess what the ministers had learnt over the three days. A daily assessment instrument was used to capture the extent to which the Ministers felt they had improved their knowledge and skill in relation to the issues covered. This produced information that helped presenters take the temperature of the forum on a daily basis. The indicators for this assessment were drawn from the content planned for each day. 2

Competency barometer assessment All ten ministers completed a baseline Competency Barometer assessment on the first day of the program, and a follow-up assessment at the close of the program. The Competency Barometer consisted of participants rating the extent to which they feel equipped to carry out the following ten competencies related to the supporting the strengthening of public health programs: 1. Define my leadership goals and priorities to leave a legacy following my term in office. 2. Develop a plan and implementation strategy to deliver health systems strengthening priorities.. Build political support and increase cross-sectoral collaboration for health system.. Organize an effective implementation management team in the ministry.. Manage planning, implementation and monitoring to deliver health sector strengthening priorities. 6. Increase collaboration with the Minister of Finance in support of health sector strengthening. 7. Identify fiscal space in the health budget through reallocation, savings and additional revenues. 8. Identify opportunities for increased efficiencies in the use of the health budget in procurement, supply management, health worker productivity. 9. Practice evidence-based decision-making. 10. Understand where to intervene to achieve sustainable health system strengthening. The participants used a scale of 1 (not at all equipped) to (very well equipped). The next sections compare the scores given at the beginning (baseline) and end of the Forum (follow-up assessment), with the aim of showing shifts that may have occurred over the course of the program. Section.1 considers changes in the scores provided by the group as a whole, while Section.2 presents the scores provided by individual ministers. Section captures the main insights provided by seven of the health ministers who were interviewed at the end of the Forum..1 Changes in group feedback on self-reported competency levels There was a positive shift in the average scores provided by the ministers, with the average overall score increasing from. at the baseline assessment to.2 (out of ) on the final assessment. Figure 1 compares the average group scores for the baseline (in blue) and final assessment (in orange).

Figure 1: Comparison of average group scores on the Competency Assessment Competency barometer - baseline and final comparison 1. Define my leadership goals and priorities to leave a legacy 10. Understand where to intervene to, 2. Develop plan & implementation achieve sustainable health system strategy to deliver health systems,1 strengthening,1,7,,2 9. Practice evidence-based decisionmaking. Build political support, increase crosssectoral collaboration,1,,2, 8. Identify opportunities for increased efficiencies in use of health budget,0 2,8 2,8,8 7. Identify fiscal space in the health,0,9. Manage planning, implementation and budget through reallocation, savings and monitoring to deliver health sector additional revenues, 6. Increase collaboration with Minister of Finance in support of health sector strengthening Baseline 2,9, Final,2. Organize an effective implementation management team in the ministry The results show that on each of the ten indicators the group showed an improvement over the course of the program. The following table shows the change between the baseline and final scores per indicator. 1 2 6 7 8 Competency indicator Define my leadership goals and priorities to leave a legacy following my term in office. Develop a plan and implementation strategy to deliver health systems. Build political support and increase cross-sectoral collaboration for health system. Organize an effective implementation management team in the ministry. Manage planning, implementation and monitoring to deliver health sector. Increase collaboration with the Minister of Finance in support of health sector strengthening. Identify fiscal space in the health budget through reallocation, savings and additional revenues. Identify opportunities for increased efficiencies in use of health budget in procurement, supply management, health worker productivity. Baseline average Final average Difference.7. 0.8.2.1 0.9.. 0.9..2 0.8 2.9.9 1.0.8. 0.7 2.8.0 1.2 2.8.0 1.2 9 Practice evidence-based decision-making..2.1 0.9 10 Understand where to intervene to achieve sustainable health system strengthening...1 0.7 These results indicate that by the end of the Forum, participants felt most confident about defining their leadership goals and priorities to leave a legacy following their term in office (.) and increasing

collaboration with the Minister of Finance in support of health sector strengthening (.). These two indicators also received the highest scores in the baseline assessment (rated.7 and.8 respectively), indicating that of all the competencies, these are two areas in which the ministers are particularly confident. As will be shown by the comments captured in section, a number of the health ministers are comfortable with engaging their finance ministers on different terms as suggested by the health financing content of the Forum. The biggest improvements in knowledge and confidence were reported for the following actions: Manage planning, implementation and monitoring to deliver health sector strengthening priorities (from 2.9 to.9) 7 Identify fiscal space in the health budget through reallocation, savings and additional revenues (from 2.8 to.0) 8 Identify opportunities for increased efficiencies in the use of the health budget in procurement, supply management, health worker productivity (from 2.8 to.0) All three of these indicators are related to aspects of health financing. The improvements in these scores show that participants made the largest gains in knowledge and confidence in this area. It is positive to note that all but one of the indicators received a score of or above on the final assessment, suggesting high levels of confidence on these competencies by the end of the Forum. The only indicator that scored less than on average was item : manage planning, implementation and monitoring to deliver health sector. However as noted above, this indicator showed notable improvement over the course of the program..2 Changes in individual feedback on self-reported competency levels The following section provides a review of the feedback given at an individual level. As in the previous section, the figures below present a comparison of baseline and final competency scores but at the level of individual participants. This section provides an indication of the competencies in which individual ministers grew in confidence, and areas of competency that require further attention and support. The individual results are interesting because they demonstrate how different parts of the program made an impact on the participants. Country 1 The minister from Country 1 showed an overall increase in confidence in the identified competency areas, with scores improving from the baseline to the final assessment on eight of the 10 indicators. The scores on the other two indicators remained the same.

Figure 2: Comparison of individual competency scores Country 1 10. Understand where to intervene to achieve sustainable health system strengthening 9. Practice evidence-based decisionmaking 8. Identify opportunities for increased efficiencies in use of health budget 7. Identify fiscal space in the health budget through reallocation, savings and additional revenues Country 1 1. Define my leadership goals and priorities to leave a legacy 2 1 0 6. Increase collaboration with Minister of Finance in support of health sector strengthening Baseline 2. Develop plan & implementation strategy to deliver health systems. Build political support, increase cross-sectoral collaboration. Organize an effective implementation management team in the ministry. Manage planning, implementation and monitoring to deliver health sector strengthening priorities Final At the final assessment, the scores ranges from (average) to (very high confidence). The highest scores of were given to: 2 Develop a plan and implementation strategy to deliver health systems strengthening priorities. Organize an effective implementation management team in the ministry. Manage planning, implementation and monitoring to deliver health sector strengthening priorities. 9 Practice evidence-based decision-making. These results indicate that the minister from Country 1 found greatest value in the content on planning, organising and managing evidence-based decision-making. Country 2 In the case of the Health Minister from Country 2, there was a fair amount of overlap between the scores given at the baseline and final assessments. Where only the final scores appear to be visible, it indicates that the scores at the baseline and final assessments remained the same. Four of the ten indicators showed an increase by the end of the Forum: Manage planning, implementation and monitoring to deliver health sector strengthening priorities (from to ) 8 Identify opportunities for increased efficiencies in the use of the health budget in procurement, supply management, health worker productivity (from to ) 9 Practice evidence-based decision-making (from to ) 10 Understand where to intervene to achieve sustainable health system strengthening (from to ) 6

Figure : Comparison of individual competency scores Country 2 10. Understand where to intervene to achieve sustainable health system strengthening 9. Practice evidence-based decisionmaking 8. Identify opportunities for increased efficiencies in use of health budget 7. Identify fiscal space in the health budget through reallocation, savings and additional revenues Country 2 1. Define my leadership goals and priorities to leave a legacy 2 1 0 6. Increase collaboration with Minister of Finance in support of health sector strengthening Baseline 2. Develop plan & implementation strategy to deliver health systems. Build political support, increase cross-sectoral collaboration. Organize an effective implementation management team in the ministry. Manage planning, implementation and monitoring to deliver health sector Final At the final assessment, all but one indicator was rated a very positive or. The only exception to this was organising an effective implementation management team in the ministry, which received a middling score of. Country The scores provided by this minister presented a varied pattern. Improvements in confidence were reported for three indicators, namely: 2 Develop a plan and implementation strategy to deliver health systems strengthening priorities (from to ) Organize an effective implementation management team in the ministry (from to ) 8 Identify opportunities for increased efficiencies in the use of the health budget in procurement, supply management, health worker productivity (from 2 to ) 7

Figure : Comparison of individual competency scores Country 10. Understand where to intervene to achieve sustainable health system strengthening 9. Practice evidence-based decisionmaking Country 1. Define my leadership goals and priorities to leave a legacy 2 1 0 2. Develop plan & implementation strategy to deliver health systems. Build political support, increase crosssectoral collaboration 8. Identify opportunities for increased efficiencies in use of health budget. Organize an effective implementation management team in the ministry 7. Identify fiscal space in the health budget through reallocation, savings and additional revenues 6. Increase collaboration with Minister of Finance in support of health sector strengthening. Manage planning, implementation and monitoring to deliver health sector Baseline Final The scores for six of the ten indicators remained the same, while the score for the final indicator understanding where to intervene to achieve sustainable health system strengthening dropped slightly from a rating of to. Country The feedback from the minister from the Country followed a slightly different pattern. In the baseline assessment, the minister allocated the highest possible score () to eight of the ten competencies, while the other two indicators relating to planning and implementation (item 2 and item ) received scores of. The scores provided in the final assessment were slightly lower, and perhaps more realistic. Five of the indicators remained the same (items 1 -, 6 and 9), while the other five were rated one point lower. Only one item received a middling score of in the final assessment, namely managing the planning, implementation and monitoring required to deliver health sector. 8

Figure : Comparison of individual competency scores Country 10. Understand where to intervene to achieve sustainable health system strengthening 9. Practice evidence-based decisionmaking 8. Identify opportunities for increased efficiencies in use of health budget 7. Identify fiscal space in the health budget through reallocation, savings and additional revenues Country 1. Define my leadership goals and priorities to leave a legacy 2 1 0 6. Increase collaboration with Minister of Finance in support of health sector strengthening Baseline 2. Develop plan & implementation strategy to deliver health systems. Build political support, increase cross-sectoral collaboration. Organize an effective implementation management team in the ministry. Manage planning, implementation and monitoring to deliver health sector Final Country The minister from Country reported increased levels of confidence on all of the competency areas. The one exception was item 6, which was rated at the highest score of on both the baseline and final assessment. Figure 6: Comparison of individual competency scores Country 10. Understand where to intervene to achieve sustainable health system strengthening 9. Practice evidence-based decisionmaking 8. Identify opportunities for increased efficiencies in use of health budget 7. Identify fiscal space in the health budget through reallocation, savings and additional revenues Country 1. Define my leadership goals and priorities to leave a legacy 2 1 0 6. Increase collaboration with Minister of Finance in support of health sector strengthening 2. Develop plan & implementation strategy to deliver health systems. Build political support, increase cross-sectoral collaboration. Organize an effective implementation management team in the ministry. Manage planning, implementation and monitoring to deliver health sector Baseline Final In the final assessment all of the indicators were rated or, reflecting high levels of confidence at the close of the program. 9

Country 6 The scores provided by the minister from Country 6 tended to be fairly low, although with one exception the scores improved or remained the same. The main area of concern for this minister was in practising evidence-based decision-making. This indicator received a score of in the baseline assessment, and this fell to 1 in the final assessment, suggesting the minister has no confidence in this area and requires further support. Figure 7: Comparison of individual competency scores Country 6 10. Understand where to intervene to achieve sustainable health system strengthening 9. Practice evidence-based decisionmaking 8. Identify opportunities for increased efficiencies in use of health budget 7. Identify fiscal space in the health budget through reallocation, savings and additional revenues Country 6 1. Define my leadership goals and priorities to leave a legacy 2 1 0 6. Increase collaboration with Minister of Finance in support of health sector strengthening 2. Develop plan & implementation strategy to deliver health systems. Build political support, increase cross-sectoral collaboration. Organize an effective implementation management team in the ministry. Manage planning, implementation and monitoring to deliver health sector Baseline Final The scores in the final assessment ranged from 1 to. The largest increase was seen on item 1: defining my leadership goals and priorities to leave a legacy. Three indicators received scores of 2 in both the baseline and final assessments, and would benefit from further attention: Organize an effective implementation management team in the ministry (2) Manage planning, implementation and monitoring to deliver health sector strengthening priorities (2) 10 Understand where to intervene to achieve sustainable health system strengthening (2). Country 7 There was a notable improvement in the scores given by the minister from Country 7 from the baseline assessment to the final assessment. On six of the competency indicators, the scores improved by at least 2 points on the scale: Organize an effective implementation management team in the ministry (from to ) Manage planning, implementation and monitoring to deliver health sector strengthening priorities (from to ) 6 Increase collaboration with the Minister of Finance in support of health sector strengthening (from 2 to ) 7 Identify fiscal space in the health budget through reallocation, savings and additional revenues (from 2 to ) 10

9 Practice evidence-based decision-making (from to ) 10 Understand where to intervene to achieve sustainable health system strengthening (from to ). The scores did not increase on only two items, and these remained at an already high score of. Figure 8: Comparison of individual competency scores Country 7 10. Understand where to intervene to achieve sustainable health system strengthening 9. Practice evidence-based decisionmaking 8. Identify opportunities for increased efficiencies in use of health budget 7. Identify fiscal space in the health budget through reallocation, savings and additional revenues Country 7 1. Define my leadership goals and priorities to leave a legacy 2 1 0 6. Increase collaboration with Minister of Finance in support of health sector strengthening 2. Develop plan & implementation strategy to deliver health systems. Build political support, increase cross-sectoral collaboration. Organize an effective implementation management team in the ministry. Manage planning, implementation and monitoring to deliver health sector Baseline Final All of the scores in the final assessment were high ( or ), confirming the significant increase in confidence by the end of the Forum. Country 8 The Mozambican minister also reported a general increase in confidence by the end of the Forum. Only one indicator remained the same (item 10 with an already high score of ). The other competencies all showed an improvement, with particularly large improvements being noted on the two items that received the lowest scores from this minister in the baseline assessment. Both of the items related to health financing competencies: 7 Identify fiscal space in the health budget through reallocation, savings and additional revenues (from 2 to ) 8 Identify opportunities for increased efficiencies in the use of the health budget in procurement, supply management, health worker productivity (from 2 to ). 11

Figure 9: Comparison of individual competency scores Country 8 10. Understand where to intervene to achieve sustainable health system strengthening 9. Practice evidence-based decisionmaking Country 8 1. Define my leadership goals and priorities to leave a legacy 2 1 0 2. Develop plan & implementation strategy to deliver health systems. Build political support, increase crosssectoral collaboration 8. Identify opportunities for increased efficiencies in use of health budget. Organize an effective implementation management team in the ministry 7. Identify fiscal space in the health budget through reallocation, savings and additional revenues 6. Increase collaboration with Minister of Finance in support of health sector strengthening Baseline. Manage planning, implementation and monitoring to deliver health sector Final Country 9 Similarly, the minister from Country 9 reported an overall increase in confidence over the course of the program, with all 10 of the competencies showing an improvement. The two areas that received scores of 2 in the baseline, and therefore were areas of concern at the start of the program, both improved markedly: 2 Develop a plan and implementation strategy to deliver health systems strengthening priorities (from 2 to ) 9 Practice evidence-based decision-making (from 2 to ). Figure 10: Comparison of individual competency scores Country 9 10. Understand where to intervene to achieve sustainable health system strengthening 9. Practice evidence-based decisionmaking 8. Identify opportunities for increased efficiencies in use of health budget Country 9 1. Define my leadership goals and priorities to leave a legacy 2 1 0 2. Develop plan & implementation strategy to deliver health systems. Build political support, increase cross-sectoral collaboration. Organize an effective implementation management team in the ministry 7. Identify fiscal space in the health. Manage planning, implementation budget through reallocation, savings and monitoring to deliver health and additional revenues sector 6. Increase collaboration with Minister of Finance in support of health sector strengthening Baseline Final 12

By the close of the program, the minister from Country 9 allocated high scores of or to all of indicators, underlining his increased comfort with carrying out these competencies. Country 10 Again, the minister from Country 10 reported a positive shift in confidence over the course of the Forum. Four indicators received low scores of 2 in the baseline assessment, but all four showed marked improvements in the final assessment: Build political support and increase cross-sectoral collaboration for health system (from 2 to ) Manage planning, implementation and monitoring to deliver health sector strengthening priorities (from 2 to ) 7 Identify fiscal space in the health budget through reallocation, savings and additional revenues (from 2 to ) 8 Identify opportunities for increased efficiencies in the use of the health budget in procurement, supply management, health worker productivity (from 2 to ). Figure 11: Comparison of individual competency scores 10. Understand where to intervene to achieve sustainable health system strengthening 9. Practice evidence-based decisionmaking 8. Identify opportunities for increased efficiencies in use of health budget 7. Identify fiscal space in the health budget through reallocation, savings and additional revenues Country 10 1. Define my leadership goals and priorities to leave a legacy 2 1 0 6. Increase collaboration with Minister of Finance in support of health sector strengthening 2. Develop plan & implementation strategy to deliver health systems. Build political support, increase cross-sectoral collaboration. Organize an effective implementation management team in the ministry. Manage planning, implementation and monitoring to deliver health sector Baseline Final Again, the increased levels of confidence are reflected in the fact that at the close of the Forum, the Country 10 minister allocated high scores of or to all of indicators in the final assessment. What is likely to change? Interviews were held with seven of the ten health ministers to tap their insights about the Forum and gather more detail about their assessment of its value to their effectiveness in their posts..1 Doing things differently When asked what they would do differently in their jobs on their return to their countries, the ministers mentioned working differently with their finance ministers; focusing on efficiency and budget effectiveness to get more value for the money at their disposal; planning according to goals 1

and priorities, managing the delivery process more effectively; and taking a public health approach to get better results from existing resources. It is interesting to note that most of the interviewees described in very similar terms how they would approach their finance minister in future, thus showing that the approach advocated in the Forum made a big impact on their thinking: I would use a very scientific approach to approach my Minister of Finance, but I have to get my ministry to do the homework. I m not going to be asking for money necessarily, but will make him a partner in my endeavor. There is the ability to start certain things from scratch, specially in regard to the supply chain and procurement processes. So it s a good time to bring them [finance officials] in because this is more finance oriented. (Country ) I m going to concentrate on more efficient ways of doing things. That means changing people and systems and more regular monitoring and evaluation and improve as we go along rather than waiting for long bureaucratic processes. I want to cut out this bureaucracy so we work more efficiently. At the moment we have bureaucratic ways of doing things, duplicated systems and offices it s inefficient at the moment. (Country 9) I will protect some time to make sure I come up with a plan, a strategy, and find a team of people to help me implement that plan which I did not have... When I have in place a delivery unit I won t have to scratch around for who will help me with this, that. We ll do more with the Minister of Finance to explain what we plan to do and what we expect from him. (Country 7) We don t have a detailed procurement plan. We also don t have efficient supply chain management. So if we can work on those there can be savings and efficiencies. (Country 6) With this week I can think of some measurements that I can look for we have our strategic plan, which includes most of what was explained here, but I can produce my own to control and monitor my officers. (Country 8) The ministers all commented that on returning home they would work differently with their finance ministers. Their comments suggest that they see this as a turning point in their ability to stretch their resources for better health service delivery: In the past it s always been a question of this is my budget allocation, it s not enough, what are we going to do about it? that sort of environment. It s always been finance and health on different sides of the table. Now we need a round table so we re all in it together. (Country ) I ll have to explain what efforts we are making in health for savings, for value for money for more people that should be part one. Then we start asking about the part we want him to play to help us. (Country 7) She has not cut our budget, but would like us to discuss inefficiencies, especially at the tertiary hospital, and also the fact that the monies are returning [to the fiscus]. So it s not going to be difficult. I propose meeting her asking her to release some of her officers to help me, and go the Ministry of Planning and establish this group, the delivery team, so they can push ours to deliver and have a joint plan to present to her. (Country 6) The finance minister always says health comes first, so I feel very confident that for the next years our relationship will become much better. What I have learnt here is how to negotiate. I think people know this is a priority, but you must convince them and negotiate. (Country 8) 1

Only one of the ministers indicated that while he works closely with the finance minister, it is up to Parliament, not the finance minister, to decide on the resource allocations made to health: The Minister of Finance is a good guy, but he is not the decision-maker to give more or less budget for health care. The state budget is approved by Parliament and this is an annual discussion when the Minister of Finance and the Minister of Health should present arguments why to go this way, and this being approved. During preparation of budget we have close and intensive collaboration with the Ministry of Finance and in both ministries there are deputies specifically delegated to budgetary issues who are in a working group. (Country 2).2 Dealing with challenges As might be expected, the ministers identified resourcing as being their major challenge, but showed clearly that the approaches and tools discussed at the Forum gave them the confidence to find more effective ways of dealing with these challenges: The biggest challenge of course is the resources. The health fund is the process I ll follow. Which part is for the Minister of Health and which part is for the Minister of Finance was a grey area for me, but now it s clear. Very helpful. (Country 7) We don t have enough doctors in the public sector, nurses and pharmacists. That s why before I came here I initiated a programme called outreach to the village, myself being part of that, and at least I energized the colleagues from the public and private sector coming together. We re going to improve the district hospital and build up a permanent team in the district. The session did help in terms of budgeting space, but more important is the enhancement of efficiency in the system rather than getting more money. It is the way to persuade the Minister of Health or the Prime Minister or even the President to say we need money, and this is what we do. So for me this is more useful. I m not from a finance background, I m more from a clinical background, but I m learning a lot. (Country 9) We must decentralize the budget to the district level the district must have their plans linked to the activities and the budget allocated. So at a three-monthly forum where we meet, the district management team should report and the data should be there to monitor which is happening, live on my desk, so it s easy to check. (Country 6) There are lots of challenges but one of them is the staff, the health workers some work without motivation, even though the government is trying to improve the salary... Another big challenge is medicines we re buying a lot, but I feel we re not buying the right medicines and we are spending more money than we should. So this I will change I will not ask for more money. I know if we improve our efficiency and try to be more open and more honest we can increase our efficiency and after that if I feel the money is not enough, I ll ask for more. If I control the procurement process myself I can ask for more. These are the two biggest challenges. The other is infrastructure some people walk 0 kms to reach a health centre. I have to work with the budget I have and then convince the Minister of Finance that this money is not enough. In this course the way the topics were introduced and put, I can produce my own tools and show the Minister of Finance how I tried to work with the budget he gave me. I ll show him tools and simple examples and I think we can work together. (Country 8) 1

Another challenge mentioned was staying focused on leadership priorities whilst constantly dealing with crises: The distractions are huge and they are constant. The ability to have a process that is working seamlessly that can cope with all those distractions is key. Otherwise you just get swamped by a crisis like a cholera outbreak, Ebola, questions about whether we should be having clinical trials on certain vaccines or not, etc, etc they come out of the woodwork strikes by students, doctors, nurses all these are major distractions and they come one after the other and they gather momentum. So you need to have a mechanism that is still doing the everyday business notwithstanding the distractions. So how you set up the ministry after this is actually for me crucial because you have to set up a team that can multi-task and at the same time not be in key positions that have to deal with the crisis that happens. So the art of delegation, the art of giving responsibility to people who didn t have it before, becomes a key part of the process. (Country ). Most useful aspects of the Forum When asked what they found most useful about the Forum, the health ministers who were interviewed identified both content and process aspects of the program. Top of mind responses with respect to content, mentioned the role of the health minister (Country 7), health financing components (in particular creating fiscal space through improved efficiency and cost savings) (Country ), and coordinating different players and building a coalition to support health systems strengthening (Country 6). With regard to process, ministers referred to the value of learning how their circumstances are similar to or different from those in other countries; the practical methods used in presenting and discussing new concepts; the use of systemic and analytic approaches to problem-solving; and being able to relate these discussions to circumstances back home: Two things: [you realize] one, that you re not alone and two, that you re unique. This came out for me quite clearly and it actually empower you and gives you a little more strength. (Country ) The interactions and direct case discussion is practical and educates [better than] theory. (Country 9) This was amazing because we were talking about the issues and topics that seemed well known for us, but I was able to look at the issues from a deeper, different point of view with systemic approaches, analytical approaches that gave me new possibilities to look at the issues I m dealing with on a daily basis and gave me the tools and expertise to analyze and look at the right direction. (Country 2) To see how things were explained and identifying what is happening back home. (Country 8) 6 Conclusion Across the group, the ten health ministers felt they improved their knowledge and confidence on all the indicators. Their comments demonstrate that the sessions on health financing were particularly valuable in that participants were introduced to new ways of thinking about how to get better value for money and stretch scarce resources to deliver improved health care. The focus on the delivery 16

approach gave them useful tools and practical ways of providing transformational leadership to strengthen their health care systems effectively. The group ratings show that by the end of the Forum the ministers felt their competence had improved from equipped to well equipped. In a number of cases individual feedback saw selfassessed competency ratings moving from poorly equipped and equipped to very well equipped. These ratings cannot predict how the new tools and approaches will be applied in practice, but do show that the participants have gained the confidence to take steps to changing how they do their jobs. By their own account the participants signalled that their improved knowledge and confidence will help them strengthen their leadership and management skills to improve their health systems. Two ministers mentioned how the Forum had given them time out to engage in structured thinking and that the program had provided them with practical ways of dealing with the challenges they faced in their countries, Participants variously described the Forum as being a very educational opportunity and as a great programme it must just go on. In their feedback during the week many of the ministers commented one the high calibre of the presenters and one person described this dimension as follows: It s a fantastic Forum. The calibre of people on it gives it credibility that is demonstrated in the knowledge that is imparted. (Country ) One minister indicated that he would like to maintain a close relationship with the Harvard School of Public Health and the MLIH Program in particular, while another mentioned how much he was looking forward to learning more at the annual convening of health ministers who had participated in the Forum on previous occasions. Another said that he was very interested in extending the engagement with the MLIH Program to his senior management team: This is very good and I d like to take it further by getting the team from finance, planning and health, as well as the directors of programmes, the senior management team, to work with Harvard and they should bring back a comprehensive plan. (Country 6) The evaluation thus found that the participants felt equipped with a number of practical skills that would strengthen their ability to provide transformative leadership for health systems strengthening. On this basis there seem to be improved prospects for them implementing policy more effectively and for adopting effective strategies to increase health financing and achieve value for money. o o o 0 o o o 17

Annexure 1 Health Ministers Forum Agenda Health Leaders Ministerial Forum Harvard Loeb House Cambridge, MA June 7-11, 201 Agenda Coach transportation to the Forum venue will leave promptly from the lobby of the hotel at times indicated below. Participants are expected to attend and fully participate in all sessions. Dress for the duration of the Forum is informal (no ties for men) with the exception of dinner on June 9 where business attire or the equivalent is requested. Sunday, June 7 2:0pm Ministers meet in hotel lobby for transfer to Harvard s Loeb House :00 :0pm Welcome Introductions and Overview Michael Sinclair, Executive Director, Ministerial Leadership in Health Program :0 :0pm Leadership and Legacy: How to Make Your Mark in Government Howard Koh, Harvard T.H. Chan School of Public Health and former Assistant Secretary for Health and Human Services, United States Ministers group work on vision statement and discussion :0 :0pm Framework for Health Systems Strengthening: Getting the Fundamentals Right Rifat Atun, Harvard T.H. Chan School of Public Health :0 6:00pm Ministers group work on health system diagnostic and revise vision statement 6:00 8:0pm Working Dinner Presentation and Discussion: Defining a Legacy by Getting Things Done Monday, June 8 7:1am Introduction Julio Frenk, former Minister of Health, Mexico and Dean, Harvard T.H. Chan School of Public Health Presentation and Discussion: Idris Jala, Minister in the Office of the Prime Minister & Director, Performance Management Development Unit (PEMANDU), Malaysia Michael Barber, former Director, Prime Minister s Delivery Unit, United Kingdom Ministers meet in hotel lobby for transfer to Loeb House 7: 9:0am Working Breakfast: Transformational Leadership I Marty Linsky, Harvard Kennedy School 9:0 9:am Refreshment Break Monday, June 8 cont. 18

9: 11:1am Transformational Leadership II Marty Linsky 11:1am 12:pm Organizing to Deliver on Policy Goals: The Malaysian Experience Michael Barber and Idris Jala 1:00 2:0pm Working Lunch Transformative Leadership in Action: The Case of HIV/AIDS Policy in South Africa; and Tobacco Control in the Philippines Moderator: Michael Sinclair Lead discussants: the Hon. Aaron Motsoaledi, Minister of Health, South Africa Enrique Ona, former Secretary of Health, the Philippines 2:0 :00pm Organizing to Deliver: cont. Michael Barber. :00 :1pm Refreshment Break :0 6:00pm Organizing for Implementation: Re-engineering the Ministry Michael Barber, Idris Jala Organizing the Ministry for Health Reform: The Mexican Experience Julio Frenk 6:00 8:0pm Working Dinner Mitigating Risks: Building Resilient and Responsive Health Systems Michael Barber Presenters: Michael van Rooyen, Harvard T. H. Chan School of Public Health Margaret Kruk, Harvard T.H. Chan School of Public Health Discussion leaders: Enrique Ona and Recep Akdag, former Minister of Health, Turkey Tuesday, June 9 7:1am Ministers meet in hotel lobby for transfer to Harvard s Loeb House 7: 9:0am Working Breakfast: Persuasion, Motivation and Negotiation in the Implementation of Policy Michael Barber Discussants: Leslie Ramsammy, former Minister of Health, Guyana, Hon. Kesete Birhane, Minister of Health, Ethiopia, Howard Koh 9:0 9:am Refreshment Break 9: 11:00am Managing and Monitoring Implementation Michael Barber, Idris Jala 11:00 11:1am Refreshment Break 11:1am 12:pm Managing and Monitoring Implementation Michael Barber, Idris Jala 12: 1:0pm Working Lunch: Ministers develop and present personal delivery plans Michael Barber Tuesday, June 9 cont. 19

2:00 :00pm Political Leadership in Pursuing Policy Reform Michael Reich, Harvard T.H. Chan School of Public Health, Case Study: Tobacco Control in South Africa Lead Discussant: Aaron Motsoaledi :00pm 6:0pm Return to Hotel Depart Hotel to Harvard Art Museum 7:00 10:00pm Reception Followed by Special Dinner Discussion Leadership in Health: From Vision to Reality Keynote Remarks: Donna Shalala, former Secretary for Health and Human Services, United States Julio Frenk, Dean, T.H. Chan School of Public Health and former Minister of Health, Mexico Leslie Ramsammy, former Minister of Health, Guyana Wednesday, June 10 7:1am Ministers meet in hotel lobby for transfer to Harvard s Loeb House 7: 9:0am Working Breakfast: Health Financing: Options and Outcomes Bill Hsiao, Harvard T.H. Chan School of Public Health 9:0 11:00am Sustainable Health Financing: Exploring Public-Private Opportunities Bill Hsiao The Case of Chile Lead Discussant: Jeanette Vega, Fondo Nacional de Salud, Chile 11:00 11:20am Refreshment Break 11:20am 1:00pm Achieving Greater Value for Money in Health: More Effective Health Budget Allocation and Increased Efficiency in Resource Utilization Bill Hsiao and Rifat Atun 1:00 1:pm Lunch 1: :1pm Tracking Progress: Intro to the Scorecard Robert Kaplan, Harvard Business School :1 :00pm Ministers group work on developing and present personal scorecard Robert Kaplan :00 :1pm Refreshment Break :1 6:0pm Presentation and Discussion Prioritizing Nutrition for Durable Health Returns: Leadership for Implementation Purnima Menon, International Food Policy Research Institute, India Discussants: Suresh Shetty, former Minister of Health, Maharashtra State, India Wafaie Fawzie, Harvard T.H. Chan School of Public Health 6:0pm Dinner 20

Thursday, June 11 7:1am Ministers meet in hotel lobby for transfer to Harvard s Loeb House 7: 9:1am Working Breakfast: Fiscal Space in Health Budgeting Ajay Tandon, World Bank 9:1 10:am The Mechanics of Achieving Greater Effectiveness and Efficiency in Health Care Delivery: Procurement, Supply Management, Productivity Rifat Atun, Michael Sinclair and Saad Rizvi, Delivery Associates, U.K. 10: 11:00am Refreshment Break 11:00am 12:0pm Ministers discuss Vision Statement and Implementation Plan Rifat Atun, Michael Barber & Michael Sinclair 12:0 2:00pm Health Leadership Awards Lunch Julio Frenk 21

Annexure 2 Day 2 feedback report Ministerial Leadership in Health Program Ministers of Health Feedback on Day 2 Monday 8 June 201 The Ministers of Health attending the June 201 Forum are asked to provide daily feedback on the extent to which they feel confident to carry out different actions related to the content of the training. On Day 2 the training focused on transformative leadership and organizing for delivery. Participants were asked to provide feedback on what they gained from the day in respect of the following seven actions: 1. Set priorities in line with my leadership vision and goals 2. Understand the role of the Minister in health systems management and development. Define and lead efforts to achieve sustainable health sector strengthening. Build a political coalition to support health sector. Establish effective management processes and monitoring systems for implementing health sector 6. Understand crisis management and risk mitigation strategies 7. Identify opportunities for improving standards of public health care through increased efficiencies in core service delivery functions Ministers rated their levels of knowledge and confidence using a -point scale, where 1 indicated very little knowledge and confidence and indicated fully knowledgeable and confident. The average scores given by the participants as a group are shown in the figure below. 22