Demand and supply of microcredit in presence of selection

Transcription

1 Demand and supply of microcredit in presence of selection February 22, 2018 Please click HERE for the updated version. Abstract We study whether BRAC, a Microfinance Institution (MFI) based in Bangladesh, uses household performance in a livestock transfer program as a signal of creditworthiness to improve targeting in its subsequent microcredit program. Using sample selection and Random Forest (RF) models, we show that a livestock efficiency score, estimated using a value-added stochastic livestock production function, helps explain subsequent loan approval decisions by BRAC. Our results suggest that information on efficiency gleaned through monitoring of program beneficiaries was incorporated into decision making by BRAC loan officers. Loan officer based targeting enables BRAC to selectively approve loan application of households with a higher return to capital and subsequently increases productive utilization of microcredit. We show that borrowers observable characteristics do not proxy efficiency score sufficiently, therefore, the applicability of pure machine-based credit scoring model may not be the best alternative option to traditional loan officer-based approach in microcredit market. keywords: Microcredit, Livestock Transfer, Targeting, Machine Learning(ML) model JEL class: C53, D14, G21, O12, P46 Incomplete Version 1

2 1 Introduction Asymmetric information generates moral hazard and adverse selection in the financial market that can result in credit rationing and mispricing of risk (Crawford et al., 2015; Hulme and Mosley, 1996). Financial institutions use collateral requirement to screen out poor-quality borrowers to minimize risk related to information asymmetry (Stiglitz and Weiss, 1986). Microcredit lenders use group lending, regular repayment schedules, and non-refinancing threats instead of a collateral requirement (Cull and Morduch, 2017; Giné et al., 2010; Armendáriz de Aghion and Morduch, 2000; Zeller, 1998; Besley and Coate, 1995). When making individual loans, microcredit lenders tend to rely on the subjective judgment of loan officers as well as credit scoring algorithms to assess borrower credit worthiness (Agier and Szafarz, 2013). Credit scoring has the potential to reduce inefficiencies or inequities in credit access arising from the biases of loan officers. However, borrower credit worthiness and entrepreneurial potential may depend on characteristics that are not easily observable. Subjective judgment may still be useful for identifying successful borrowers if hard-to-observe determinants of credit worthiness are poorly correlated with the variables typically used in credit scoring algorithms. The screening problem in the credit market can be partly attributed to the substantial heterogeneity in returns to capital among entrepreneurs. For example, De Mel et al. (2008) show that return to capital ranges from 0% to 5.3% per month among microenterprises in Sri Lanka. In some instances, it is difficult to estimate an accurate return to capital for small-scale enterprises because of their low engagement in market transactions. Several studies show that ex-ante peer-referral can improve credit market outcomes significantly. Bryan et al. (2015) show that peer referral can add information on loan approval in South Africa. Maitra et al. (2017) show that borrowers referred to lenders by local trader agents have 27% higher cash crop production and earn 22% more farm income than self-selected recipients of group loans in India. Similarly, Rigol et al. (2017) show that entrepreneurs in 2

3 the top tercile of community ranking in terms of growth potential and business profitability gain a three-fold higher return than average return for all entrepreneurs. An alternative approach to the above ex-ante prediction mechanisms can be the use of ex-post information on the performance of potential borrowers from a prior program. For example, MFIs who administer multiple programs including microcredit in the same location, they can use households performance in a prior intervention as a proxy of credibility for microcredit. In this paper, we study whether BRAC uses beneficiary household s performance from a prior livestock transfer program, named as targeting the ultra-poor (TUP), as a signal of credibility for its subsequent microcredit program. Specifically, we examine whether efficiency score in livestock activity during the TUP experiment can explain microcredit approval or rejection decision by BRAC after controlling for current observable information, for example, asset holding, labor endowment, and income. TUP program otherwise known as antipoverty graduation model offers livestock, entrepreneurship training, and subsistence stipends to poor women in rural areas of Bangladesh. Using a randomized control trial, Bandiera et al. (2017) show that TUP program increases aggregate labor supply, earnings, and asset accumulation, and reduces poverty. Direct TUP benefits last for two years, after which beneficiaries are expected to graduate to self-sustaining entrepreneurial activities. To encourage entrepreneurism, TUP offers loans to qualified and interested beneficiaries. During the TUP intervention period, BRAC officers visit beneficiary households periodically to monitor transferred assets, provide training, and discuss potential investment plans. While BRAC collects observable information from beneficiaries that can be used in lending decisions, BRAC may also incorporate information gleaned from monitoring visits into subjective assessments of potential borrowers. If subjective assessments proxied by efficiency score, which is unobservable otherwise, remains as a significant determinant of loan approval after controlling observable information, it will demonstrate that MFIs pre-target potential borrowers and only credit scoring based loan approval strategy may not be effective in microcredit market. It will also show the importance of subjective judgment of loan officers in 3

4 loan approval. We use data from the livestock transfer experiment conducted by Bandiera et al. (2017). We analyze credit market outcomes of 3,677 beneficiary households from surveys conducted in 2009 and 2011, after the start of TUP. We find that 37% of beneficiary households take at least one loan from BRAC-microcredit program over four years and another 8% were rejected to take a loan from BRAC. 1 We begin by estimating a bivariate probit model, where one equation represents the decision to apply for credit and the other equation represents BRAC s decision to supply credit conditional on having applied. We correct for selection into the sample of applicants by combining the Heckit method with the bivariate probit model. To assess the role of the individual judgment of BRAC program officers in credit outcomes, we control for an efficiency score estimated from a stochastic value-added livestock production function in both equations of the credit model. To identify the bivariate probit model with sample selection, we control a binary indicator on risk-adjusted income in the credit participation model only. According to Samphantharak and Townsend (2017), enterprise activities are usually associated with risks; therefore, a risk-adjusted return on capital is a better proxy of demand for credit. We justify the exclusion restriction based on the assumption that BRAC as large MFI consider only aggregate level risk and do not have enough information on individual level risk-adjusted income. Conditional on the decision to apply for a loan, we find that a one-point increase in the livestock efficiency score reduces the probability of rejection by 31 points. Correlates of livestock performance those are relatively easy to measure through survey data collection, including total livestock assets and livestock income, are also important predictors of loan approval, as are total household assets, number of male household members, and having a male household head. We conclude that information collected through household surveys as well as impressions gleaned through BRAC s intensive monitoring activities play important roles in loan approval. One limitation in the sample selection model is that households with no demand for 1 Surveys have information on when a loan is taken but no information on when loan is rejected. Therefore, we had to assume that rejection occurred in any time during these four years. 4

5 microcredit are composed of eligible but no demand, ineligible and no demand, and ineligible and no demand because they anticipated rejection up front. As we don t have detailed information on each group, we focus on only the supply decision of microcredit for rest of the analysis. To model loan supply decision conditional on demand for microcredit, a supervised Machine Learning (ML) model can be a more reliable alternative than regular regression model. ML model is entirely a data-driven procedure to predict an outcome variable using a set of predictors which eliminates concern on functional form and variable selection in the model. ML models can also provide additional insights on other important policy questions, for example, if BRAC s ultimate goal is to lend microcredit to these beneficiary households, would BRAC be able to predict final credit outcomes upfront? We use RF model using information sets from different survey waves, baseline (2007), first follow-up (2009), and second follow-up (2011), to predict microcredit market outcomes, approved and rejected by BRAC. In addition to the observable predictors, we also include efficiency score with the follow-up rounds predictor sets. We find that overall accuracy of prediction ranges from 71% to 79%. When we use predictor sets from the follow-up rounds, the true prediction of rejected households increases from 6% (baseline predictor set) to as high as 46%. Our results also show that efficiency score in livestock activity, income, asset holding, and age of the household head are most important variables in explaining microcredit market outcomes. Two important takeaways for this analysis are, first, information from the follow-up rounds are important over baseline information set specially to identify rejected household and second, efficiency score or subjective assessment of loan officer work as a complementary to the observable loan approval criteria. Both points reiterate the importance of administering the livestock experiment that improves household capacity to apply and take loan later. It is expected that improved targeting strategy by BRAC will have implication in terms of having a better borrower pool followed by productive utilization of loan and lower default rate. We do a placebo test to examine whether returns to capital varies by loan approval status using information prior to loan application. If pre-loan return to capital 5

6 varies by loan approval status, it will confirm that BRAC was able to selectively approve the application of high-return households. Our results show that elasticity of livestock profit with respect to livestock capital is 0.34 percentage point lower for rejected households as compared to approved households. This difference falls to 0.20 percentage points when we control for the livestock efficiency score. These results reinforce our earlier findings that BRAC was able to identify good borrower to reduce miss-targeting and efficiency score explains part of the targeting success. In the absence of information on default rate, we test the role of observable information sets along with efficiency score in predicting loan utilization of credit approved households. We show that predictor sets from the follow-up rounds have better accuracy in predicting loan utilization in productive activities. Efficiency score as an additional predictor does not improve overall prediction, but it remains one of the important variables in prediction. It implies that after the initial selection, efficiency score may not have enough variation on the top of other predictors in explaining subsequent loan utilization. As an additional check, we use the sample selection model to estimate role of efficiency on productive utilization conditional on loan approval and find that a one-point increase in efficiency score increases the probability of productive utilization by 0.39 points. As we find that efficiency in the livestock production is an important determinant of loan approval by BRAC, it is intriguing to examine whether BRAC would have been able to predict efficiency score had there is no such monitoring opportunity. In other words, can observable characteristics predict efficiency score sufficiently and if yes, does predicted efficiency varies by household loan approval status? If observable factors are enough to predict efficiency, it will assert that machine-based credit scoring method will be sufficient for loan approval decision and there is no need for monitoring or individual judgment of loan officers. We apply RF method to predict efficiency score using observable predictor sets from different survey rounds as before. We find that observable characteristics perform well in predicting efficiency for households at the center of efficiency distribution. The mean absolute error (MAE) of prediction is lower for predictor sets from the follow-up rounds 6

7 compared to baseline predictor set. Predicted efficiency score also varies by household loan approval status in case of predictor sets from the follow-up rounds. However, it is evident that observable information set cannot proxy efficiency score sufficiently. Therefore, a mixmethod credit scoring model combining both observable factors and subjective judgment of loan officer is a better alternative option in the microcredit market. Our study complements earlier studies as well as adds new evidence on using the ex-post information to improve targeting in the microcredit market. Studies by Bryan et al. (2015), Maitra et al. (2017), and Rigol et al. (2017) show effectiveness of using ex-ante information on subsequent return to capital or repayment. We show that monitoring households performance in a prior program improves targeting in the microcredit market. This finding aligns with earlier literature on the importance of combining both individual judgment and credit scoring method in the microcredit sector. Vogelgesang (2003) and Van Gool et al. (2012) also suggest using a combined method of borrower selection in the microcredit market. Our study also shows possibilities of gaining economies of scope by the MFIs; large MFIs like BRAC can use one program to improve performance in another program. Previous studies on the economics of scope by MFIs show that MFIs cannot take advantage of using both savings and lending together and economics scope is mainly driven by savings in fixed cost (Hartarska and Parmeter, 2009). We add an additional aspect to the literature on livestock transfer program. Studies by Bandiera et al. (2017), Banerjee et al. (2015a), and Emran et al. (2014) concentrate mainly on the direct effect of livestock transfer. We show that demand for microcredit is around 44% among the beneficiary households in the livestock program, which is higher than typical demand for microcredit (Banerjee et al., 2015b). Additionally, the livestock program has the potential to work as a pre-screening stage for subsequent large programs such as microcredit. Finally, we also contribute to the growing literature on using machine learning (ML) method in predicting policy outcomes. Although machine learning methods are applied on various aspects in economics literature (Chalfin et al., 2016; Jean et al., 2016; Aulck et al., 2016), its application in targeting issues is relatively new. Rigol 7

8 et al. (2017) use causal forest model to predict entrepreneurs marginal return to capital in India and McKenzie and Sansone (2017) use Post-LASSO, Support Vector Machines, and Boosted Regression to predict successful entrepreneurs in Nigeria. Our results complement McKenzie and Sansone (2017) that prediction performance of ML method in targeting is not substantive. 2 Context 2.1 TUP program BRAC launched TUP program in 2002 targeting asset-poor women in rural Bangladesh, a group who are among the hardest to reach through conventional anti-poverty programs. BRAC initiated TUP program based on its experience from the income generation for vulnerable group development (IGVGD) program, a food transfer accompanied with skill training scheme, launched by BRAC and the World Food Program (WFP) jointly in Although IGVGD program was successful in increasing income of the participant households, it largely failed to generate a sustainable impact due to mistargeting and ineffective service packages (Ahmed et al., 2009; Hashemi, 2001). Based on lessons from IGVGD program, BRAC introduced TUP program in 2002 as an asset-based transfer scheme along with an overhauled targeting strategy. TUP program primarily transfers productive assets such as livestock or other assets for small-scale retail operation. As a complementary to physical assets, it also provides skill training for better utilization of transferred assets and bi-weekly subsistence allowance in the first 40 weeks to compensate for any shortfall in income. Additionally, beneficiary households receive health care and social awareness services from BRAC staffs during the intervention period. Livestock assets are transferred in six different combinations of either one or two types of livestock, for example, cow, goat, or chicken. Bandiera et al. (2017) mention that about 91% of beneficiary households select an asset bundle containing at least one cow. All offered asset bundles had a similar value at around $140. Participants are encouraged to 8

9 retain transferred assets for at least two years; however, they can exchange current assets for other income generating assets during this period. Skill training includes an initial classroom training, weekly visit of BRAC staffs for the first two years, and monthly or bi-monthly visits of livestock specialists for the first year (Bandiera et al., 2017). 2.2 Targeting strategy of TUP program BRAC follows a three-stage targeting strategy to identify eligible households for TUP program. In the first stage, BRAC identifies poorest districts and sub-districts in Bangladesh using the poverty map for Bangladesh by WFP. Later, BRAC identifies all poorest communities within each sub-district in consultation with local BRAC staffs. In the second stage, BRAC divides all selected communities into clusters of households and conduct a focus group discussion (FGD) in each cluster to rank households based on their wealth level. All households in bottom 2/3 categories are selected for the final stage of screening; this group is termed as community-defined ultra-poor. In the final stage, BRAC conducts a household level survey among all selected households to identify who meet at least three out of five inclusion conditions and none of the exclusion conditions. Inclusion criteria are as follows: household is dependent on female domestic work or begging, holds less than 10 decimals of land, has no adult active male member, school-aged children are engaged in paid work, or possesses no productive work. Exclusion criteria are as follows: household has no active adult women, is a microfinance participant, or is a beneficiary of government or non-government development project. Finally, households who meet both inclusion and exclusion criteria are selected to participate in the program. 2.3 From TUP to the Microcredit program The objective of TUP program is to build a sustainable livelihood for beneficiary households, which BRAC terms as Graduation model. According to BRAC a household graduates from poverty if it satisfies some economic and social indicators, for example, no self-reported food deficit, holds multiple income sources, uses a sanitary latrine, has access to clean drinking 9

10 water, has cash savings, sends children to school, and adopts family planning. The full graduation process takes a maximum of two years and according to BRAC, 95% of the beneficiary households satisfy those conditions at the end of the two-year-long intervention. Throughout the graduation period, beneficiary households become more capable to utilize financial assets effectively, and BRAC takes initiatives gradually to transfer households from TUP program to the microcredit program. After the initial asset transfer, BRAC designates all beneficiary households to the nearest microfinance branch and village organization (VO) 2. BRAC arranges an orientation meeting about the microcredit program for all beneficiary households before their graduation. All TUP and microfinance officers from the local area attend the orientation meeting. During the orientation meeting, BRAC provides detailed information about the microcredit program and its terms and conditions to all beneficiary households. At the end of the TUP intervention, BRAC transfers savings amount of the beneficiary households from the TUP branch to the pre-assigned microcredit branch. At this point, a beneficiary can withdraw all her savings if she does not want to continue savings. A TUP member can apply for microfinance loan from their respective branches after graduation; beneficiary households can also get loan before graduation if they have potential investment plans. 3 Data We use data from an RCT experiment conducted bybandiera et al. (2017) to evaluate the second phase of the TUP program. The second phase of the program named as Challenging the Frontiers of Poverty Reduction (CFPR) was initiated in 40 districts of Bangladesh in Following the targeting strategy of TUP program, the research team followed a multistage sampling procedure to identify sample of this study. First, the research team randomly selected 20 sub-districts from 13 poorest districts of Bangladesh. Within each sub-district, one BRAC branch office is randomly selected as treatment and one as control 2 VO is a platform to collect installment and regular discussion between BRAC and microcredit borrowers. 10

11 branch. All villages within 8 km radius of each branch are considered as the study area. Following all targeting stages as mentioned above, this study covers 13 districts, 20 subdistricts, 40 branches, and 1,409 communities. Total sample of this experiment is 6,305 who were surveyed in 2007(baseline), 2009(medium term), and 2011(longer term). 3,691 households are in the treatment group and 2,614 are in the control group. We use information of 3,691 households from the treatment group over the period of 2009 and Among the treatment households, 14 households had no livestock activity in any round, so we drop those observations. Therefore, this study is based on 3,677 households who were involved in livestock rearing in at least one round. 4 Summary statistics Table 1 shows summary statistics of the beneficiary households by survey round. A substantial proportion of the targeted women, who were respondent in the baseline survey, have no formal education; average years of education is only 0.60 years. The average age of the targeted women is 38 years in the baseline. A male member heads approximately 63% of the beneficiary households, which is lower than the national average (87%) implying that some these households do not have any adult male member at all (Joshi, 2004). Household heads also have less than one years of formation education or no education at all. TUP households are typically less populated; average household size is approximately 3.5, which is 4.50 nationally. Many TUP households have no working aged male member; the average working aged male member is less than one. Lack of working aged member in the targeted households aligns with the targeting strategy of the program as one of the inclusion criteria of the TUP program is that households have no working aged male member. As expected TUP households do not hold many productive assets. The average number of cow was only 0.08 and 0.15 for goat. Average land holding is only 1.43 decimals, which reflects that most of the TUP households even do not own any homestead land. 11

12 [Table 1 here] Our results show that only 37% of beneficiary households received at least one loan from BRAC from 2008 through Among the recipient households, 22% received loan during , 36% received loan during , and 42% received loan in both periods. Around 8% households applied for a loan but were rejected by BRAC. The remaining 55% of households did not apply for a loan for distinct reasons. Other than the microcredit loan from BRAC, TUP households are not much involved in financial markets irrespective of their loan status from BRAC. We present household loan amounts from various sources by TUP loan status in appendix table 1. It is evident that loans from formal banks are virtually zero for both groups over time. Relative or friends are main sources for loans to both groups, although there is no significant difference in loan amount. [Table 2 here] 5 Methodology We report in the previous section that 45% of the beneficiary households applied for at least one loan from BRAC. Conditional on demand for microcredit from BRAC, 82% received at least a loan and 18% were rejected from receiving a loan. Various socioeconomic, demographic, and other factors can affect demand and supply decision of microcredit in a different direction and magnitude. Therefore, a sample selection model will be more appropriate than a Tobit-type model to examine microcredit outcomes. Let s assume that whether a household i faced a positive supply of microcredit (S i ) is a function of vectorx and whether household i has a positive demand for microcredit (D i ) is a function of vector Z. We define both demand and supply equations as follows, S i = (X iβ + u 1i > 0) (1) 12

13 D i = (Z iβ + u 2i > 0) (2) where u 1 N(0, 1), u 2 N(0, 1), and corr(u 1, u 2 ) = ρ. We observe S i if and only if D i = 1. If D i = 0, we have no information about S i. We can term equation (1) as the outcome equation and equation (2) as the participation equation. Given the participation and outcome, we have three types of households in our study: D i = 0, D i = 1 & S i = 0, and D i = 1 & S i = 1. We can form the log-likelihood function as follows, ln L = n i=1 {D is i ln Φ 2 (Xβ, Zγ; ρ) + D i (1 S i ) ln[φ(xβ) Φ 2 (Xβ, Zγ; ρ)] + (1 D i ) ln Φ( Xβ)} (3) To test our main hypothesis on the role of monitoring by BRAC, we control household efficiency score estimated by using a stochastic value-added livestock production function in both participation and outcome equations. Although the livestock intervention lasted only first two years (2007 to 2009), we use information of both follow-up rounds (2009 and 2011) to estimate efficiency score. It is expected that when BRAC officers judge a household performance, they consider both current indicators as well as the future prospect. Utilizing both current and future information is especially information in livestock activity because it takes a relatively longer time to get a return from livestock activity. 3 As we have two rounds information, we took the benefit of panel data and estimate both time fixed and time-varying efficiency score for each beneficiary household. We describe estimation procedure in detail in the appendix. Figure 1 shows distribution of efficiency score and appendix table 2 shows estimated production function. Efficiency or performance of a household is usually unobservable to microcredit lender and arguably known to households 4. However, as BRAC officers monitor the beneficiary households during this livestock intervention, they can rank each household in terms of their efficiency score and use that information as a 3 According to BRAC TUP program stuffs, BRAC usually transfers cow or goat that needs to rear up for couple of years to have a calf or earn any income. So judging household performance based on initial two years may not reflect an accurate scenario. 4 With a small geographic area, household can compare output relative to other households given their input cost. 13

14 determinant of loan approval. [Figure 1 here] We need at least one additional exogenous variable in the participation equation to get unbiased estimates in the outcome equation. We control a binary indicator on whether household s risk-adjusted income is positive or not in the participation equation. Following Samphantharak and Townsend (2017), we argue that demand of microcredit for investment will depend on beneficiary households risk-adjusted income realization from the livestock activity. For most of the beneficiary households, livestock activity sponsored by the TUP intervention was the first or main entrepreneurial activity, therefore, their demand for any subsequent investment will depend on the risk-adjusted gain from the current livestock activity. Although it is possible that some households will apply for loan for consumption or other non-productive purposes, but each applicant needs to show a potential investment plan. Therefore, we expect risk-adjusted livestock income is an important determinant of demand for microcredit. Whether BRAC (Supply side or outcome equation) also cares about risk-adjusted income is arguable. BRAC as a large MFI is expected not to consider individual-level risk and consider average risk of all borrowers. While estimating the risk-adjusted income, we assume measurement error is random if any. We explain detailed estimation strategy of risk-adjusted income in the appendix. 6 Findings We use the Bivariate Probit model with sample selection to estimate determinants of demand and supply of microcredit. Results are shown in column 1-2 of table 3. Form the demand equation, we find that value of assets, number of male working aged member, and male headship have a significant and positive effect on microcredit demand from BRAC. Although income and risk-adjusted income indicators have a positive effect on loan demand, they are not statistically significant. We find that efficiency in the livestock production is not a 14

15 significant determinant of microcredit demand. [Table 2 here] Turning to the loan supply decision by BRAC, we find that efficiency score significantly reduces the probability of loan rejection. A one-point increase in efficiency reduces the probability of loan rejection by 31 points. This confirms our hypothesis that BRAC uses performance in livestock activity as a signal of creditworthiness. Therefore, monitoring the beneficiary households throughout the intervention period helped loan officer to gather additional information which would not have been possible if there is no livestock intervention. Among other observable characteristics, value of asset, household income, and education of household head reduce the probability of loan rejection. Surprisingly, we find that presence of additional male members in the households increases the probability of loan rejection. Column 3-4 of table 3 show result without sample selection correction for which we use Bivariate Probit model. We find a similar result as before. In appendix table 3, we show results considering time-varying inefficiency; once again we find a similar result here as well. One important finding on demand side is the role of male members on demand for microcredit. Although TUP program is designated towards the female, the presence of male member is a crucial factor for sustainability of this program like any other development programs in Bangladesh. Importance of male member is expected given the social and cultural context in Bangladesh. Earlier literature shows that female participation in economic activities is significantly lower than that of men; according to BBS (2010), the rate of labor force participation among the population aged 15 years or older is only 36% for females compared to 83% for males. Female workers in the agricultural sector work in activities such as vegetable gardening, livestock production, and aquaculture within or near their homesteads, and many of them tend to be unpaid laborers (Khan et al., 2009; Kabeer, 2012). Although women started livestock activity due to the transfer from BRAC, they might depend on the presence of male member when it s about taking a loan or investing in economic activities. In fact, Roy et al. (2015) find that livestock transfer program increased women s ownership 15

16 of productive assets, but new investment is largely owned by men. Overall, we notice that performance in livestock production and other observable characteristics are important to explain loan supply decision by BRAC. It presses the importance of combining both individual judgment and credit scoring methods together in the microfinance sector. Previous studies on the feasibility of using only credit scoring methods also conclude in a similar fashion. For example, Vogelgesang (2003) finds that credit scoring model predicts poorly in microfinance loan decision in Bolivia; he finds that model correct prediction rate is only 59% even with a threshold value of 15%. Van Gool et al. (2012) find that performance of automated credit scoring method as an alternative to human-intensive process is very weak due to significant risk associated with microcredit client in Bosnia-Herzegovinian. 7 Machine learning (ML) model to predict microcredit market outcome In the previous section, we use the sample selection model and show that household s performance is an important determinant of microcredit supply by BRAC. However, the accuracy of our findings depends on the validity of the model selection and its functional form, and exclusion restriction. Results from the previous section also do not tell us anything about the net contribution of efficiency score on loan approval. Ideally, we could include interaction terms of efficiency score with all observable variables in the previous model and sum-up all interaction coefficients to estimate net role efficiency score, but again there is a potential problem of variable selection and overfitness of the model. One alternative option is to use a supervised Machine Learning (ML) model which is entirely a data-driven procedure to predict an outcome variable using a set of predictor. The ML model can also provide additional insights on targeting potential borrowers. For example, on a narrow perspective, if BRAC s ultimate goal is to lend microcredit to beneficiary households, it will be cost effective if BRAC could predict final credit outcomes upfront, before the livestock transfer. Pre-identification of potential borrower could enable BRAC to transfer livestock only to households who are 16

17 likely to get loan and increase coverage of the program to other location. It raises an interesting question that can BRAC use the baseline information to predict final credit outcomes, e.g. who will be approved and who will be rejected? If the baseline information does not predict credit market outcomes sufficiently, it will indicate that BRAC needs information from the follow-up rounds, so as the livestock experiment. We use the Random Forest (RF) model to predict credit market outcomes. RF method has advantages over other prediction models in reducing over-fitness of the model Zhang and Ma (2012). It is a tree-based classifier that generates a set of decision trees based on a collection of random variables to predict an outcome variable. Let X is a vector observable characteristics, Y is a credit market outcome (accept or reject), and f(x) is a prediction function using a real-valued vector of input X = (X 1,...X p ) T on the response variable Y. f(x) minimizes the expected value of the loss function, E XY (L(Y, f(x))), where L(Y, f(x))equals to, 0 if Y = f(x) L(Y, f(x)) = I(Y f(x)) = 1 otherwise (4) The minimized expected loss function sets equal to the maximum value of prediction (arg max P (Y = y X = x)) based on a set of base learners,h 1 (x),..., h j (x). The j th base learner, alternatively j th tree, is defined as, where is a collection of random variables. Therefore, in a classification algorithm is the most frequently predicted classf(x) = arg max J j=1 I (Y = h j (x)). In the appendix, we describe RF method and all the steps in detail. We have observable information sets from three survey waves, baseline (2007), first follow-up (2009), and second follow-up (2011). Adding efficiency score with two follow-up waves, we get total five alternative predictor sets. In the observable predictor set, we consider asset holding, income, characteristics of the household head, and working-aged members; these indicators are typically used by BRAC microcredit program in loan approval decision. Our predicted variable is a binary indicator of loan approved or rejected. To judge the performance of alternative 17

18 predictor sets, we estimate overall accuracy and true prediction rate using out-of-sample prediction. The RF method can also estimate variables of importance in prediction based on a held-out or out-of-bag sample as described in the appendix. Results are presented in table 3. We find that overall accuracy of prediction ranges from 71% to 79%. Predictor set from the second follow-up round including efficiency score has highest overall prediction accuracy of 79%. True prediction of loan approved and rejected household ranges between 82% to 93% and 6% to 45%, respectively. It is important to note that the true prediction rate of rejected household rises significantly when we use predictor set from the follow-up rounds; true prediction of rejected category increases from 6% (baseline set) to as high as 46% for the predictor sets from the second follow-up rounds. We notice that true prediction of approved household falls when we use follow-up round predictor sets, but the margin of falling is smaller compared to the margin of raise in prediction accuracy for the rejected category. Inclusion of efficiency score in prediction has no clear role of improvement or deterioration in true prediction. [Table 3 here] We list variables in descending order in terms of their importance in predicting credit outcomes for all specifications. RF model uses permutation test by rearranges value of a predictor and estimates its effect on degradation in prediction accuracy in a held-out or out-of-bag samples. Variable of importance is calculated based on a null hypothesis that a predictor is not important if the above permutation test does not change prediction accuracy. We describe variable of importance estimation in the appendix in detail. We find that age of household head, income, and asset are most important variables to predict credit outcomes. Efficiency becomes an important predictor once we include it in the predictor set. It is important to note that RF method list variable of importance in terms of their prediction power, it does not tell us whether a variable is positively or negatively related to efficiency score. 18

19 8 Benefits of improved targeting In previous two sections, we show that BRAC is more likely to supply microcredit to creditworthy borrowers using households efficiency score as a proxy for performance during the livestock transfer program. the goal of BRAC is to lend to a better borrower pool who are more likely to utilize loan in productive activities, e.g. investment in income generating activities. It implies that loan approved households will be better performing group compared to rejected group. We test this using return to capital as our first indicator followed by whether approved households use loan in a productive activity or not. 8.1 Does return to capital differ by the selection? It is obvious that financial institutions would likely to lend to households who have a high return to capital or at least have a prospect of high-return on their investment. As an MFI, BRAC is no different and expected to lend microcredit to households with a high-return prospect. If we can show that return to capital is different between approved and rejected households prior to microcredit approval, it will affirm our previous findings that BRAC was able to sort out households and selectively supplied loan to high-return households. Consider the following regression model to estimate return to capital by approval decision, profit ij = α 0 + α 1 capital ij + α 2 rejected i + α 3 capital ij rejected i + λx + ε ij (5) where prof it shows household income net of input cost in activity j. We consider livestock and other self-employment activities separately (j = 1, 2). capital reflects the value of all physical capital at replacement cost excluding building or infrastructural items. Finally, rejection is a binary indicator of loan approval or rejection status of the household i. α 3 will show if return to capital varies by loan approval decision. One potential problem with the specification in the equation (5) is that household rejection status can be endogenous. From the demand side, we expect no selection bias as 19

20 both approved and rejected households had the intention to take microcredit from BRAC. On the supply side, we know that BRAC were more likely to lend to more efficient households that can affect profit as well. To overcome this problem, we can control efficiency score as a proxy of unobservable factors in the model. Another major problem with the equation (5) is the timing of loan approval by BRAC. Ideally, we would like to estimate the return to capital using information from a pre-microcredit period and test whether it differs by the subsequent loan approval status. However, almost all the targeted households in the TUP experiment had no productive asset or entrepreneurial activity before the intervention, so we cannot examine this hypothesis using the baseline information. Therefore, we needed to use information from the follow-up rounds, which has at least two additional problems. If households receive a loan, then any differences in returns on capital will be composed of individual heterogeneity as well as the effect of credit. Access to credit can have a multiplier effect on unobserved ability and affect profit simultaneously. It can also increase the productivity of household labor in self-employment activities De Mel et al. (2008). Therefore, we need to estimate returns on capital such that it is not affected by credit access. In our data among the loan approved households, 21% households received loan only during the period of , 36% received loan only during , and rest of the households received loan in both periods. We use information from the first follow-up survey (2009) for households who borrowed only during the period and households who were rejected to estimate the equation (5). Aligning to the impact evaluation literature, we can term this regression as a placebo test ; testing whether treatment assignment (loan approval) is random or whether treatment arms are balanced in the pre-intervention period. Results are presented in table 4. We show results by livestock and non-livestock activity as well as total self-employment activity consecutively. Non-livestock activity includes other self-employment activities such as non-farm business, nursery, fishery, and small retail shop. Column 1-6 show results using data with the base form and columns 6-12 shows result where profit and capital variables are transformed into the inverse-hyperbolic-sign format 20

21 (IHS) 5. When we use data in the base form, we do not find any precise differences in returns on capital by rejection status. The sign of the coefficients of interaction term are all negative for livestock and aggregate specifications but returns on capital differ by rejection status at 10% level of significance when we control efficiency in the regression for livestock activity. When we use IHS transformed data, we find that returns on capital vary significantly by rejection status for both livestock and aggregate models. We find that elasticity of return to capital is 0.34 percentage point less for rejected households compared to approved households in the livestock sector; the differences reduce to 0.20 percentage points when we control efficiency in the model. We also find that elasticity of return to capital is 0.51 percentage point less for rejected households compared to approved households in the aggregate self-employment sectors; similar to the livestock activity, the differences reduce to 0.40 percentage points when we control efficiency in the model. Appendix table 4 shows results after winsorizing values of the bottom and top 5% by value at 5 th and 95 th percentile value, respectively. [Table 4 here] From table 6, we also notice that aggregate return to capital is zero or negative in the livestock sector and positive in non-livestock activity. As sample in table 6 is only a subgroup of all beneficiaries, we estimate returns to capital from all the beneficiary households irrespective of their loan status. Results are presented in appendix table 5. We find that elasticity of returns on capital is 0.29 for livestock, 0.26 in non-livestock, and 0.21 in aggregate self-employment activity. 8.2 Implication of targeting on utilization of microcredit If borrowers use microcredit for consumption or other non-productive purposes, they are more likely not to repay regularly or even to be a defaulter. We use information on main expenditure sector of each loan from BRAC and find that borrowers use loan in productive 5 The inverse hyperbolic sine transformation is estimated as log(yi+(yi2+1)1/2). It produces a similar scenario as the standard logarithmic variable. 21

22 activity in 40% cases. We test whether efficiency score in livestock activity as an additional screening mechanism over observable characteristics can predict post-loan utilization. If efficiency score can predict utilization, it will reassert that individual monitoring is an important strategy in the microcredit market. We use the RF model to predict outcome variables as before. As before we vary predictor set by phases and efficiency score and use five alternative sets of predictor. Table 5 shows prediction results for productive utilization. We find that overall accuracy of prediction remains same at around 60%. However, predictor sets from the follow-up rounds have better accuracy in predicting which households will use microcredit in productive activities. Adding efficiency as an additional predictor does not improve prediction. From the list of variables of importance, we find that income is the most consistent predictor of productive use of loan. Efficiency becomes the most important predictor among the first follow-up predictor set and second best among the second follow-up predictor set. Findings from panel A have two important dimensions. First, predictor sets form the follow-up rounds out-perform the predictor set from the baseline survey implying the need of the livestock experiment for better targeting. Second, as BRAC already screened-out efficient households at the initial stage, efficiency score does not hold enough predictability power to predict utilization over other predictors. In appendix table 6, we use the sample selection model as before where the participation equation is based on whether a household is approved or not and the outcome equation shows whether approved loan is utilized in productive activities or not. We find that conditional on loan approval, a one-point increase in efficiency score increases probability of productive loan utilization by 0.39 units. [Table 5 here] 22

23 9 Is efficiency predictable by observable characteristics? We show that efficiency in the livestock production is an important determinant of loan approval by BRAC along with other observable characteristics. This raises an important question that would BRAC be able to predict efficiency score if there is no such monitoring opportunity? In other words, can observable characteristics predict efficiency score sufficiently? If yes, does it vary by household loan approval status? If observable factors are enough to predict efficiency, machine-based credit scoring method will be equivalent to the loan officer-based loan approval. Therefore, monitoring household performance need not be an important criterion in loan decision. We use the RF method to predict efficiency score as before except that our predicted variable is continuous in this case. For a continuous predicted variable,f(x) minimizes expected value of the loss function, E XY (L(Y, f(x))), where L(Y, f(x)) shows how close f(x) to Y. A common choice of L is absolute error loss,l(y, f(x)) = Y f(x), in a regression-based RF method. The minimized expected loss function sets f(x) equal to the conditional expectation function of Y ((f(x) = E(Y X = x)) based on a set of base learners, h 1 (x),..., h j (x). We plot the distribution of actual efficiency score and predicted score using different predictors in figure 2. It is clear that the actual distribution and predicted distributions are not equivalent especially at the tails of the distribution. Kolmogorov-Smirnov test of equality of distributions also confirms that predicted distributions are statistically different from the actual efficiency score distribution. It is also noticeable that predictor sets from the follow-up rounds do a better job in predicting the actual distribution. In table 4, we present mean absolute error (MAE) in predicting actual efficiency score for each set of predictors. We find that predictor set from the second follow-up round have lowest MAE compared to other sets of predictor. We also present MAE by each decile of the distribution of actual efficiency 23

24 score. There is a clear pattern that estimated MAE increase in both tails of the efficiency distribution. [Figure 2 here] [Table 6 here] We use predicted efficiency score by different predictor sets and plot them by household loan approval status. A predictor set will be more reliable to proxy efficiency score if it can distinguish efficiency score by loan approval status. We show predicted cumulative distribution function (CDF) of efficiency score by loan approval status in figure 3 and 4. Figure 3 shows that baseline predictor set cannot distinguish efficiency score by loan approval status; equality test of CDF by loan status also confirms that both CDFs are statistically indifferent. Figure 4 shows efficiency by loan status for predictor sets from two follow-up rounds. We find that both predictor sets can distinguish efficiency score by loan status of household. In both cases, we find that rejected households have lower efficiency rate. Finally, we list variable of importance in descending order in terms of their importance in predicting credit efficiency score in panel B in of table 4. We find that number of working-aged male member, age and gender of household head, income, and asset are the most important variables to predict efficiency score. [Figure 3 here] [Figure 4 here] 10 Conclusion We study whether BRAC uses performance of beneficiary households in a prior livestock transfer program as a signal of credibility for a subsequent microcredit program. We estimate efficiency score in the livestock production using a stochastic production function as a proxy for their performance and show that it significantly explains loan approval decision by BRAC. 24