Emil Tesliuc and Phillippe Leite November 23, 2009

Emil Tesliuc and Phillippe Leite November 23, 2009 ADePT SP (developed by HDNSP-SSN SSN team and Development Research Group -Poverty Team ) ADePT SP is a Stata routine built as a special module in ADePT. Through consistent set of methods and assumptions, ADePT SP produces up to 24 standardized tables and 3 graphs to examine how equitable, efficient and effective social protection programs are. ADePT SP generate indicators about benefits or beneficiaries across quintiles and deciles of a monetary welfare or a non-monetary welfare; and broken-down by other population groups. THE QUALITY OF OUTPUTS DEPENDS ON DATA QUALITY AND DATA MANIPULATION GARBAGE IN GARBAGE OUT 2 1

ADePT SP (developed by HDNSP-SSN SSN team and Development Research Group -Poverty Team )) Value Added of ADePT SP Current practice: estimate only a partial set of indicators; without sensitivity testing; ti different, non-transparent t algorithms by different researchers. ADePT SP: 1. A simplified framework for judging the impact of a SN program on poverty based on a set of intermediate outcome indicators dpg = f(coverage, Generosity, Targeting accuracy) 2. Can be used for ex-ante analysis of a new or re-designed d social protection program 3. Facilitates benchmarking, comparisons across programs and countries (generated using a standard procedure) 3 First step preparation of input data Main source of information: Household Surveys (HBS, IES, LSMS) Key Variables: Household identification Location of the household (Urban or Rural) The welfare level of the households, typically expressed as a per capita or per adult equivalent consumption or income; Poverty line (s) Expansion factor (individual or household) Sample frame design Social programs benefits or beneficiaries (either individual amount or household amount) 4 2

Individual Input File Amount Amount received Amount Household Adult Age of the Total received Participation in Household Individual Urban location =1; Household Head of the Poverty by the household received by the STRATA PSU expansion equivalent household household from old scholarship Identification Identification Rural location=2 Size household line from household from factor scale member income age programs Oportunidades Pro Campo pensions id_hh id_ind strata psu urban hhweight hhsize adul_eq head age hh_income pob_ing apos becas_ toport tprocam 20060150282 1 1 2 2 305 3 2 1 18 2459.34 938.61 0 180.49 20060150282 2 1 2 2 305 3 2 0 18 2459.34 938.61 0 180.49 20060150282 3 1 2 2 305 3 2 0 1 2459.34 938.61 0 180.49 20060150280 1 1 2 2 305 7 6 1 56 9094.69 938.61 0 334.24 20060150280 2 1 2 2 305 7 6 0 53 9094.69 938.61 0 334.24 20060150280 3 1 2 2 305 7 6 0 29 9094.69 938.61 0 334.24 20060150280 4 1 2 2 305 7 6 0 26 9094.69 938.61 0 334.24 20060150280 5 1 2 2 305 7 6 0 15 9094.69 938.61 0 334.24 20060150280 6 1 2 2 305 7 6 0 13 9094.69 938.61 0 334.24 20060150280 7 1 2 2 305 7 6 0 7 9094.69 938.61 1 334.24 20060150030 1 1 1 1 777 4 3 1 77 18183.37 938.61 1403.81 0 20060150030 2 1 1 1 777 4 3 0 51 18183.37 938.61 0 20060150030 3 1 1 1 777 4 3 0 43 18183.37 938.61 0 20060150030 4 1 1 1 777 4 3 0 9 18183.37 938.61 0 20060150040 1 1 1 1 777 1 1 1 92 4458.78 938.61 1604.35 0 20060150050 1 1 1 1 777 2 2 1 83 6397.05 938.61 1640.45 0 20060150050 2 1 1 1 777 2 2 0 39 6397.05 938.61 0 20060150060 1 1 1 1 859 5 2 1 41 12988.27 938.61 0 20060150060 2 1 1 1 859 5 2 0 32 12988.27 938.61 0 20060150060 3 1 1 1 859 5 2 0 11 12988.27 938.61 0 20060140410 1 1 7 1 638 10 6 1 56 10730.62 938.61 0 514.18 20060140410 2 1 7 1 638 10 6 0 58 10730.62 938.61 0 514.18 20060140410 3 1 7 1 638 10 6 0 86 10730.62 938.61 1411.48 0 514.18 20060140410 4 1 7 1 638 10 6 0 30 10730.62 938.61 0 514.18 20060140410 5 1 7 1 638 10 6 0 29 10730.62 938.61 0 514.18 20060140410 6 1 7 1 638 10 6 0 10 10730.62 938.61 0 514.18 20060140410 7 1 7 1 638 10 6 0 9 10730.62 938.61 0 514.18 20060140410 8 1 7 1 638 10 6 0 4 10730.62 938.61 0 514.18 5 Household Input File Household Identification Individual Identification STRATA PSU Urban location =1; Rural location=2 Household Household expansion Size factor Adult Age of the Head of the equivalent household household scale member Total household income Poverty line Amount received from old age pensions Participation in scholarship programs Amount received by the household from Oportunidades Amount received by the household from Pro Campo id_hh id_ind strata psu urban hhweight hhsize adul_eq head age hh_income pob_ing apos becas_ toport tprocam 20060150282 1 1 2 2 305 3 2 1 18 2459.34 938.61 0 180.49 20060150280 1 1 2 2 305 7 6 1 56 9094.69 938.61 1 334.24 20060150030 1 1 1 1 777 4 3 1 77 18183.37 938.61 1403.81 0 20060150040 1 1 1 1 777 1 1 1 92 4458.78 938.61 1604.35 0 20060150050 1 1 1 1 777 2 2 1 83 6397.05 938.61 1640.45 0 20060150060 1 1 1 1 859 5 2 1 41 12988.27 938.61 0 20060140410 1 1 7 1 638 10 6 1 56 10730.62 938.61 1411.48 0 514.18 6 3

ADePT SP snapshot (developed by HDNSP-SSN SSN team and Development Research Group -Poverty Team ) 8 4

ADePT SP snapshot (developed by HDNSP-SSN SSN team and Development Research Group -Poverty Team ) 9 ADePT SP snapshot (developed by HDNSP-SSN SSN team and Development Research Group -Poverty Team ) 10 5

ADePT SP snapshot (developed by HDNSP-SSN SSN team and Development Research Group -Poverty Team ) 11 ADePT SP snapshot (developed by HDNSP-SSN SSN team and Development Research Group -Poverty Team ) 12 6

ADePT SP snapshot (developed by HDNSP-SSN SSN team and Development Research Group -Poverty Team ) 13 What ADePT SP tables will tell you? Focusing on few tables Tbl Table 1: Sample and Population sizes Table 5: Coverage Table 7: Distribution of Benefits Table 9: Generosity Table 10: Undercoverage and Leakage Table 11: Impacts of program on Poverty and Inequality Table 19: Cost-Benefit ratios 14 7

Table 1 : Sample and Population Sizes Table 1 : Sample and Population Sizes Sample size* Population** Transfer Households Individuals Recipient Households Individuals Recipient amount Level*** declared? All observations 20,875 83,624 26,639,430 105,434,304 For households that receive the indicated transfer only All social protection 6,777 30,104 11,936 7,704,894 33,188,158 12,627,162 Yes HHold All social insurance 2,122 7,471 2,329 2,885,613 10,049,597 3,163,608 Yes HHold Old Age Pension 2,116 7,439 2,322 2,875,903 9,986,142 3,151,370 Yes Ind Occupational injury / Sickness benefits 7 36 7 12,238 73,567 12,238 Yes Ind All labor market programs 0 0 0 0 0 0 Yes HHold All social assistance 4,838 23,474 9,690 5,056,774 24,152,556 9,577,819 Yes HHold Fee waivers, education 647 3,121 739 714,702 3,479,775 817,540 Yes Ind Conditional Cash Transfers 3,908 19,462 7,998 3,913,758 19,168,138 7,806,388 Yes Ind Non contributory pension 1,074 5,097 1,091 1,053,189 4,784,885 1,068,693 Yes Ind All remittances 4,977 18,026 5,985 6,245,825 22,155,677 7,479,059 Yes HHold Remittances 4,977 18,026 5,985 6,245,825 22,155,677 7,479,059 Yes HHold *) The sample size columns show the number of households, individuals and recipients of SP programs in the survey. **) The population columns show the number of households, individuals and recipients of SP programs, expanded to the population using expansion factors. ***) The Level collumn specifies whether the information on program participation is collected at individual or household level Table 5 : Coverage - Direct and indirect beneficiaries Table 5 : Coverage Direct and indirect beneficiaries Quintiles of pc consumption Poverty Status Area of Residence Total Q1 Q2 Q3 Q4 Q5 P NP Urban Rural All social protection 31.5 54.6 34.8 25.2 20.8 22.0 55.6 25.9 23.1 59.5 All social insurance 9.5 2.2 6.1 8.7 13.1 17.6 2.1 11.2 11.0 4.7 Old Age Pension 9.5 2.2 6.0 8.5 13.0 17.5 2.1 11.2 10.9 4.7 Occupational injury / Sickness benefits 0.1 0.0 0.1 0.1 0.1 0.0 0.0 0.1 0.1 0.0 All labor market programs 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 All social assistance 22.9 53.2 30.1 17.8 8.5 4.8 54.4 15.7 12.8 56.9 Fee waivers, education 3.3 3.4 4.1 3.5 3.1 2.4 3.2 3.3 3.1 4.1 Conditional Cash Transfers 18.2 48.4 25.3 11.7 4.2 1.3 50.0 10.9 9.0 49.0 Non contributory pension 4.5 10.5 4.3 4.4 2.1 1.4 10.7 3.1 1.7 14.2 All remittances 21.0 21.7 21.6 22.3 21.3 18.2 21.6 20.9 18.7 28.8 Remittances 21.0 21.7 21.6 22.3 21.3 18.2 21.6 20.9 18.7 28.88 Notes: Program coverage is the portion of population in each group that receives the transfer. Specifically, coverage is: (Number of individuals in the group who live in a household where at least one member receives the transfer)/(number of individuals in the group). Program coverage is calculated setting as expansion factor the household expansion factor multiplied by the household size. 8

Table 7: Distribution of Benefits (Targeting Accuracy) Table 7: Distribution of Benefits (Targeting Accuracy) Quintiles of pc consumption Poverty Status Area of Residence Total Q1 Q2 Q3 Q4 Q5 P NP Urban Rural All social protection 100.0 7.5 7.4 8.5 16.6 60.0 7.1 92.9 82.1 17.9 All social insurance 100.0 1.1 3.7 7.1 18.2 69.8 0.9 99.1 91.6 8.4 Old Age Pension 100.0 1.1 3.7 7.1 18.2 69.8 0.9 99.1 91.6 8.4 Occupational injury / Sickness benefits 100.0 0.0 17.4 14.4 58.0 10.1 0.0 100.0 57.9 42.1 All labor market programs n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. All social assistance 100.0 33.0 21.9 13.8 10.1 21.2 31.4 68.6 44.2 55.8 Fee waivers, education 100.0 7.7 7.0 10.2 12.1 63.1 7.0 93.0 86.2 13.8 Conditional Cash Transfers 100.0 47.6 29.9 13.4 6.6 2.5 45.6 54.4 36.9 63.1 Non contributory pension 100.0 16.8 13.6 16.5 16.6 36.5 15.6 84.4 35.7 64.3 All remittances 100.0 6.5 11.2 16.9 21.2 44.2 5.9 94.1 70.8 29.2 Remittances 100.0 6.5 11.2 16.9 21.2 44.2 5.9 94.1 70.8 29.2 Notes: Benefits' incidence is the transfer amount received by the group as a percent of total transfers received by the population Specifically, benefits' incidence is: (Sum of all transfers received by all individuals in the group)/(sum of all transfers received by all individuals in the population). Aggregated transfer amounts are estimated using household size-weighted expansion factors. Table 7: Distribution of Benefits with standard errors Table 7: Distribution of Benefits (Targeting Accuracy) Quintiles of pc consumption Q1 Q2 Q3 Q4 Q5 Conditional Cash Transfer 47.6 29.9 13.4 6.6 2.5 standard error 0.8 0.7 0.6 0.5 0.5 Confidence interval Upper 45.9 28.5 12.3 5.5 1.4 Lower 49.2 31.4 14.6 7.6 3.6 9

Figure 1: Concentration curves Social assistance programs Figure 1: Concentration curves Social assistance programs 100 per capita consumption Cash or near cash transfer Conditional Cash Transfer 80 Line of equality Cumulative % of benefits 60 40 20 0 0 20 40 60 80 100 Cumulative % of population, ranked by consumption 10

Table 9 : Generosity - Direct and indirect beneficiaries Table 9 : Generosity Direct and indirect beneficiaries Quintiles of pc consumption Poverty Status Area of Residence Total Q1 Q2 Q3 Q4 Q5 P NP Urban Rural All social protection 18.8 15.5 11.9 12.6 18.7 22.7 15.7 19.1 20.6 13.5 All social insurance 26.4 33.8 27.0 24.3 26.0 26.6 34.5 26.4 26.4 26.7 Old Age Pension 26.5 33.8 27.4 24.5 25.9 26.7 34.5 26.4 26.5 26.7 Occupational injury / Sickness benefits 6.6 n.a. 7.0 3.7 15.1 2.1 n.a. 6.6 4.2 30.6 All labor market programs n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. All social assistance 8.3 14.1 8.3 5.9 5.6 7.2 14.5 6.9 6.9 9.8 Fee waivers, education 4.8 7.3 3.0 3.4 2.8 6.2 8.0 4.7 5.0 4.1 Conditional Cash Transfers 7.8 13.0 7.7 5.1 4.3 2.3 13.2 5.8 7.0 8.4 Non contributory pension 8.9 10.1 9.8 7.9 10.1 8.1 10.2 8.7 7.4 10.0 All remittances 20.7 26.9 24.0 23.3 19.6 19.1 27.2 20.4 18.7 28.2 Remittances 20.7 26.9 24.0 23.33 19.6 19.11 27.22 20.4 18.7 28.22 Notes: Generosity is the mean value of the share transfer amount received by all beneficiaries in a group as a share of total welfare aggregate of the beneficiaries in that group. Generosity is calculated setting as expansion factor the household expansion factor multiplied by the household size. Generosity expressed in LCU. Table 10 : Undercoverage and Leakage Table 10 : Undercoverage and Leakage Direct and indirect beneficiaries Coverage of the poor (1) Under- coverage (2) Leakage (# of Leakage beneficiaries) (benefits) (3) (4) Targeting differential (5) = (1) - (3) All social protection 55.6 44.4 67.0 92.9-11.3 All social insurance 2.1 97.9 95.9 99.1-93.8 Old Age Pension 2.1 97.9 95.9 99.1-93.8 Occupational injury / Sickness benefits 0.0 100.0 100.0 100.0-100.0 All labor market programs n.a. n.a. n.a. n.a. n.a. All social assistance 54.4 45.6 55.7 68.6-1.3 Fee waivers, education 3.2 96.8 81.7 93.0-78.5 Conditional Cash Transfers 50.0 50.0 48.6 54.4 1.5 Non contributory pension 10.7 89.3 56.1 84.4-45.5 All remittances 21.6 78.4 80.88 94.1-59.2 Remittances 21.6 78.4 80.8 94.1-59.2 Notes: Undercoverage is percent of poor individuals that do not receive transfer. Leakage is percent of individuals that receive transfer and are not poor. Sample of all htaouseholds. Undercoverage and leakage are calculated across this sample, setting as expansion factor the household expansion factor multiplied by the household size. The targeting differential is the difference between the coverage rate and the participation rate for nonpoor. 11

Table 11 : Impact of SP programs on poverty simulating the absence of the program Table 11 : Impact of programs on Poverty and Inequality measures - simulating the absence of the program Poverty Indicator Inequality FGT0 FGT1 FGT2 Gini GE(0) GE(1) GE(2) Indicator 0.187 0.061 0.029 0.504 0.445 0.516 1.163 Indicator without listed transfer All social protection 0.211 0.076 0.039 0.510 0.466 0.530 1.225 All social insurance 0.197 0.065 0.031 0.503 0.445 0.519 1.203 Old Age Pension 0.197 0.065 0.031 0.503 0.445 0.519 1.203 Occupational injury / Sickness benefits 0.187 0.061 0.029 0.504 0.445 0.516 1.163 All labor market programs n.a. n.a. n.a. n.a. n.a. n.a. n.a. All social assistance 0.200 0.072 0.037 0.511 0.465 0.527 1.183 Fee waivers, education 0.188 0.062 0.029 0.504 0.446 0.516 1.164 Conditional Cash Transfers 0.197 0.070 0.035 0.510 0.462 0.525 1.178 Non contributory pension 0189 0.189 0.063063 0.030030 0.505 0448 0.448 0518 0.518 1.166166 All remittances 0.219 0.080 0.042 0.516 0.493 0.541 1.235 Remittances 0.219 0.080 0.042 0.516 0.493 0.541 1.235 Notes: The simulated impact is the change in a poverty or inequality indicator due to transfer, assuming that household welfare with diminish by the full value of that transfer Table 19 : Cost-Benefit Ratios Table 19 : Cost-Benefit Ratios Simulated poverty gap At Actual Difference without poverty gap (dpg) transfer Total amount spent in the program (X) Cost- Benefit (dpg0/x) All social protection 7,546,296,736 6,100,966,849 1,445,329,887 14,686,891,534 0.098 All social insurance 6,470,215,583 6,100,966,849 369,248,734 11,738,290,941 0.031 Old Age Pension 6,470,215,583 6,100,966,849 369,248,734 11,726,179,515 0.031 Occupational injury / Sickness benefits 6,100,966,849 6,100,966,849 0 12,111,426 0.000 All labor market programs n.a. n.a. n.a. n.a. n.a. All social assistance 7,146,362,530 6,100,966,849 1,045,395,681 2,948,600,598 0.355 Fee waivers, education 6,138,825,322 6,100,966,849 37,858,473 459,205,888 0.082 Conditional lcash Transfers 6,944,402,673 402 673 6,100,966,849 843,435,825435 825 1,689,155,069 155 0.499 Non contributory pension 6,253,200,955 6,100,966,849 152,234,106 800,239,640 0.190 All remittances 7,951,441,508 6,100,966,849 1,850,474,659 12,299,478,320 0.150 Remittances 7,951,441,508 6,100,966,849 1,850,474,659 12,299,478,320 0.150 Notes: Cost-Benefit is the poverty gap reduction in $ for each unity ($1) spent in the social program. Amounts in LCU. 12

Figure 3: Cost-Benefit Ratios for the total (upper) poverty line Conditional Cash Transfers All social assistance Non contributory pension Remittances All remittances Fee waivers, education 0.1.2.3.4.5 $ reduction in poverty gap for each $1 spend in the program Next Steps Baseline: All poor OVC receives KHS500 a month Hypothesis: We assume we have a good welfare indicator for all households in Kenya Scenario 1: Only poor OVC according to a score index receives a transfer of KHS X a month Hypothesis: We don t know the welfare indicator but we have a simple score card that classifies households as poor or non poor using easy to verify indicators. Scenario 2: Only poor OVC according to a PMT receives a transfer of KHS X a month Hypothesis: We don t know the welfare indicator but we have a PMT score that classifies households as poor or non poor using easy to verify indicators. 26 13

27 Next steps on Manual and on the Agenda Add sections: How to perform sensitivity analysis? How to simulate new programs, or changes in the parameters of existing programs? Receive feedback The team plans to develop both on-line and on-site training materials on how to use ADePT SP lecture format interactive exercise face-to-face training distance learning Next steps on the Agenda 28 Production of a Social Protection World Atlas By the end of the fiscal year 09, the HDNSP-SSN SSN team has piloted the Social Protection World Atlas (SP Atlas) using the ADePT SP and ECA region is working on a regional study. 14

Next steps on the Agenda 29 100 90 80 70 60 50 40 30 20 10 0 Overall Social Assistance Coverage of Poorest Quintile (%) Next steps on the Agenda 30 Upgrades: Developing outcomes to examine dependency d on SA income(s) by able-bodies beneficiaries. Profile of the typical beneficiary of each individual SP program, compared with the "typical household" in the country Incorporating simulation feature to simulate a new program, or to simulate changes in few parameters of existing programs. Generating more graphs based on the current tabular output based on the sensitivity analysis (different welfare counterfactuals) 15