Running head: PICKING PROFITABLE INVESTMENTS 1. Decision Making. Jan K. Woike

Transcription

1 Runnng head: PICKING PROFITABLE INVESTMENTS 1 Pckng Proftable Investments: The Success of Equal Weghtng n Smulated Venture Captalst Decson Makng Jan K. Woke Max Planck Insttute for Human Development, Center for Adaptve Ratonalty (ARC) Ulrch Hoffrage and Jeffrey S. Petty Unversty of Lausanne, Faculty of Busness and Economcs (HEC) Author Note Correspondence concernng ths artcle should be addressed to Jan K. Woke (woke@mpbberln.mpg.de, Max Planck Insttute for Human Development, Center for Adaptve Ratonalty (ARC), Lentzeallee 94, Berln, Germany). We thank Suzanne de Trevlle, Marc Gruber, Sebastan Hafenbrädl, Chrs M. Whte, the specal ssue edtors, and three anonymous revewers for helpful comments on prevous versons of ths manuscrpt, and the Swss Natonal Scence Foundaton for ther fnancal support (SNF /1 and _140503). Preprnt submtted to Journal of Busness Research December 4, 2014

2 PICKING PROFITABLE INVESTMENTS 2 Abstract Usng computer smulaton, we nvestgate the mpact of dfferent strateges on the fnancal performance of VCs. We compare smple heurstcs such as equal weghtng and fast and frugal trees wth more complex machne learnng and regresson models and analyze the mpact of three factors: VC learnng, the statstcal propertes of the nvestment envronment, and the amount of nformaton avalable n a busness plan. We demonstrate that the performance of decson strateges and the relatve qualty of decson outcomes change crtcally between envronments n whch dfferent statstcal relatonshps hold between nformaton contaned n busness plans and the lkelhood of fnancal success. The Equal Weghtng strategy s compettve wth more complex nvestment decson strateges and ts performance s robust across envronments. Learnng only from those plans that the smulated VC nvested n, drastcally reduces the VC s potental to learn from experence. Lastly, the results confrm that decson strateges dffer n respect to the mpact of added nformaton on the outcomes of decsons. Fnally, we dscuss real-world mplcatons for the practce of VCs and research on VC decson makng. Keywords: decson makng, VC, nvestment, smple heurstcs, smulaton

3 PICKING PROFITABLE INVESTMENTS 3 1. Introducton Each year venture captal (VC) funds nvest bllons of dollars n entrepreneurs and companes that seek to commercalze ther nnovatve products and servces. VC-backed companes receve much needed captal as well as prvleged access to fnancal and commercal networks, so beng selected by a VC as a portfolo nvestment s consdered by many to be one of the keys to success for nnovatve hgh growth ventures (Gorman & Sahlman, 1989; Meggnson & Wess, 1991). Yet, only one to three per cent of the hundreds of deals receved each year by any gven VC frm are ultmately selected as portfolo nvestments (Bruno & Tyebjee, 1985; Maer & Walker, 1987; Petty & Gruber, 2011). Gven the perceved benefts of recevng VC nvestment combned wth the apparent selectvty on the part of the VCs, t s lttle wonder that VC decson makng has prompted multple streams of research. The exstng research on VC decson makng has focused on two man ssues, the overarchng selecton process (Tyebjee & Bruno, 1984; Rquelme & Rckards, 1992; Fred & Hsrch, 1994) and the crtera used by VCs when evaluatng deals (MacMllan et al., 1985; Dxon, 1991; Hsrch & Jankowcz, 1990; Hall & Hofer, 1993; Muzyka et al., 1996; Franke et al., 2008). Multple studes have demonstrated that VCs generally rely upon a mult-stage screenng and evaluaton process that s desgned to lmt nformaton asymmetry whle reducng the probablty of mssng a potentally proftable opportunty, thus ultmately maxmzng ther profts (Tyebjee & Bruno, 1984; Hall & Hofer, 1993; Fred & Hsrch, 1994; Gompers, 1998). In terms of the actual nvestment decson, research has repeatedly shown that when forecastng the vablty of a deal, ts relatve attractveness s determned by the VC s assessment of the many assocated nformatonal cues or characterstcs (Tyebjee & Bruno, 1984; MacMllan et al., 1985; Rquelme & Rckards, 1992; Zacharaks & Meyer, 2000). However, although dfferent VCs

4 PICKING PROFITABLE INVESTMENTS 4 appear to engage n the same selecton process and focus on the same general categores of cues when evaluatng deals, all VCs do not necessarly employ the same forecastng and decson makng strateges when assessng the potental proftablty of deals and ultmately selectng portfolo nvestments (Gupta & Sapenza, 1992; Muzyka et al., 1996; Shepherd, 1999a; Shepherd & Zacharaks, 2002). Hoffman, one of the poneerng researchers n the feld, concluded that VCs are lkely to dffer substantally n ther nvestment propensty and practces to the same set of nvestment stuatons (1972, p.182). However, two research questons central to VC decson makng have receved only lmted attenton n the lterature: (1) how do dfferent decson makng strateges mpact the VC s overall performance, and (2) what are the factors that nfluence how well VCs can learn from outcome feedback. These ssues are of partcular mportance n the VC context gven the falure rates of new ventures and the varyng rates of returns of VC nvestment portfolos (Sahlman, 1990; Mason & Harrson, 2002; Dmov & De Clercq, 2006). The objectve of ths paper s to estmate the relatve effectveness of dfferent decson makng strateges whch may be employed by a VC whle at the same tme accountng for the mpact of learnng when screenng and selectng deals over the lfe of a fund. Ths goal s acheved by means of computer smulatons n whch the relaton between the varables descrbng a busness plan and the success probablty of ths plan s systematcally manpulated. 2. Related Lterature A recurrng theme n the VC decson makng lterature over the past forty years has been the attempt to dentfy the nformatonal cues that VCs deem to be relevant when revewng an nvestment proposal or a busness plan. Specfcally, researchers have endeavored to develop lsts

5 PICKING PROFITABLE INVESTMENTS 5 or categores of the key cues and then determne how these cues are used by the VC to evaluate and select, or more commonly reject, an nvestment proposal (Hoffman, 1972; MacMllan et al., 1985; Robnson, 1987; Muzyka et al., 1996; Zacharaks & Meyer, 1998; Kaplan & Strömberg, 2004). As a result, a multtude of cues have been recorded and there s general agreement between researchers that these cues can be grouped wthn the followng broad categores: (a) product characterstcs, (b) market characterstcs, (c) a company s fnancal poston and outlook, (d) the trats of the entrepreneur or management team, and (e) other cues such as the nterest of another VC n a busness plan under consderaton or the ablty of a VC to add value to a deal (Tyebjee & Bruno, 1984; Muzyka et al., 1996; Zacharaks & Meyer, 2000; Petty & Gruber, 2011). Regardless of how these cues may be lsted or arbtrarly grouped, t s mportant to note that there s no correspondng agreement amongst VCs as to whch of the cues s consdered to be the most mportant when evaluatng and selectng a deal (Rquelme & Rckards, 1992; Hall & Hofer, 1993; Gompers & Lerner, 1999; Shepherd, 1999b). In realty, the perceved relatve sgnfcance of any gven cue s often context specfc and may be nfluenced by the nvestment clmate, characterstcs of the VC frm, or the preferences of the ndvdual VC. Several expermental studes nvolvng VCs have establshed that the evaluaton of a deal s often nfluenced by dosyncratc factors such as an ndvdual s ntuton, prevous ndustry experence, educatonal background, or nvestng experence (Hsrch & Jankowcz, 1990; Rquelme & Rckards, 1992; Zacharaks & Meyer, 2000; Franke et al., 2008). However, despte beng consdered experts at selectng hgh potental deals and spendng on average one-thrd of ther tme screenng and evaluatng deals (Robnson, 1987), there s lttle evdence that VCs are aware of ther underlyng decson makng strateges. To the contrary, as a result of ther

6 PICKING PROFITABLE INVESTMENTS 6 performance n a varety of study settngs, the ablty of VCs to accurately dentfy successful companes has often been called nto queston by researchers (Khan, 1987; Zacharaks & Meyer, 1998; Shepherd et al., 2003; Zacharaks & Shepherd, 2005). Another area that has receved lttle attenton s the role of learnng and ts mpact on VC nvestment decson makng and frm performance. The few studes that have examned learnng wthn VC frms (Chan et al., 1990; Bergemann & Hege, 1998; Sorenson & Stuart, 2001) have focused on contractng and portfolo management actvtes, whch take place after the VC has made ther nvestment decson. Unlke exstng research, our study focuses on VC learnng durng the pre-nvestment phase of the process. Unlke the post-nvestment actvtes that are only performed wth the few deals that are selected as portfolo companes, VCs repeatedly perform the pre-nvestment screenng routne on each of the thousands of proposals they receve and conduct evaluatons on hundreds of deals over the lfe of any gven fund (Tyebjee & Bruno, 1984; Fred & Hsrch, 1994). Exstng vews of the VC s selecton and evaluaton of the cues that are used when makng nvestment decsons have been bult prmarly upon survey and expermental desgn-based studes. As a result, the majorty of the research n ths stream remans cross sectonal n desgn and only ndrectly lnked to the VC s combned objectve to both enhance process effcency and maxmze the frm s return on nvestment (ROI) (Gfford, 1997; De Clercq & Sapenza, 2005; Shepherd et al., 2005). Our smulatons provde a comparson of the effectveness of varous forecastng and decson makng strateges wth prescrptve mplcatons for VC decson makng. As Herbert Smon (e.g. 1955) repeatedly asserted, strateges and envronments have to be consdered jontly. Buldng on ths nsght, the noton of ecologcal ratonalty (Hogarth & Karelaa, 2007; Todd et al., 2012) mples that the effectveness of strateges can only be evaluated n a gven context. We

7 PICKING PROFITABLE INVESTMENTS 7 report how well strateges for nvestment decsons perform n envronments that vary along the followng dmensons: () selectvty of outcome nformaton (nformaton about the success or falure of a busness plan s avalable for all plans versus only for those plans a VC has nvested n), () the relatve mportance of cues, and () the number of cues avalable to the VC. Note that our nvestgaton on the nfluence of the selectvty of feedback s one of the frst that addresses ths mportant aspect of learnng n VC decson makng. 3. Setup One way to determne the performance of strateges would be to mplement what Brunswk (1955) called a representatve desgn (see also Dham et al., 2004). In the present case ths would requre a randomly drawn set of real busness plans, all descrbed on the same set of features and wth nformaton avalable about ther performance n case these plans had found an nvestor and had been realzed. There are two reasons why we dd not evaluate the decson strateges ths way. The frst s pragmatc: To the best of our knowledge, such a data set does not exst, nor s t lkely to be collected and made accessble to academc nqury n the near future. The second reason s of a methodologcal nature: Even f t exsted and were at our dsposal, smply observng how the strateges fare n a gven set of busness plans for whch we would not manpulate anythng, would make any causal clams and a systematc study of the dependency of strateges success on partcular nformaton structures mpossble. In an attempt to overcome the dffcultes assocated wth representatve desgn, Hammond (1966) dfferentated between the concepts of substantve stuatonal samplng and formal stuatonal samplng. The former mplements the orgnal dea of representatve desgn by focusng on the content of the task and usng real stmul that have been representatvely sampled

8 PICKING PROFITABLE INVESTMENTS 8 from the envronment. The latter, n contrast, permts the constructon and presentaton of stmul that are representatve n terms of the formal nformatonal propertes of the envronment (.e., number of cues, ther values, dstrbuton, nter-correlatons, and ecologcal valdtes). We followed Hammond s approach and created fcttous busness plans that our smulated VCs used as nput when makng ther success forecasts and nvestment decsons. Ths procedure gave us perfect control over the statstcal structure wthn our world. It s mportant to add that the statstcal propertes n ths artfcal world were nformed by the lterature and our knowledge of the real world, that s, the generaton of busness plans was based on a set of plausble assumptons that had an emprcal bass. Each of the plans that we generated s assocated wth a set of k bnary cues C {0000,, c1c 2c3ck,,1111}, and a dchotomous (future) outcome varable whch can take the values +1 (success or postve return on nvestment) and -1 (falure or negatve return on nvestment). Each cue s probablstcally related to ths outcome so that the vector of cue values (henceforth, the cue profle) s useful nformaton when t comes to predctng whether a busness plan wll result n success or not. Note that the cues n the present smulatons are devod of any semantcs, that s, they could nclude varables specfed n the busness plan (e.g., experence of the management team) or varables descrbng the broader context (e.g., market characterstcs) The Envronments The term envronment refers to the set of probabltes used when creatng a set of busness plans. Specfcally, an envronment s characterzed by the followng probabltes. Each of the possble cue profles s assocated wth a probablty p ( o C ) that t occurs, and wth a probablty p C ) that t wll be successful, so that knowledge of ths set of probabltes s s(

9 PICKING PROFITABLE INVESTMENTS 9 suffcent to determne the probablty of success ( ) for a randomly sampled busness plan from ths envronment Cue Weghtng Structures We were nterested n the queston whether the performance of deal selecton strateges depends on how cues dffer from each other wth respect to ther statstcal relatonshps wth the outcome varable. Therefore, we generated envronments n whch the cues are equally mportant when determnng outcomes, and other envronments n whch some cues are more mportant than others. Specfcally, the outcome s determned by comparng a weghted sum of the cues and a random error component wth a threshold value. In the Equal Weghts envronments, all cues are weghed equally. In the Arthmetc Weghts envronments, a constant value s added to cue weghts for each subsequent cue n a fxed sequence, whereas n the Geometrc Weghts envronments, cue weghts n the sequence are multpled by a constant factor. Parameter values and detals for generatng these envronments for dfferent numbers of cues are descrbed n Appendx A Overall Success Probablty In our smulatons, we created envronments wth a predetermned overall success probablty. Because the ndvdual success probabltes of the varous cue profles are known (see Appendx A), t was possble to manpulate the occurrence probabltes of the ndvdual cue profles such that the resultng overall success probablty matched the predetermned probablty. The detals of ths procedure are descrbed n Appendx B The Decson Strateges Even experts have been shown to have major dffcultes when engagng n unaded judgmental forecastng (Dawes, 1979; Tetlock, 2005; Burgman et al., 2011). In the present paper, we use smulated VCs who execute four well-defned strateges to make these selectons. Such a smulaton approach s

10 PICKING PROFITABLE INVESTMENTS 10 removes any nose that humans produce when usng strateges unrelably, and hence allows us to determne the upper bound of aded decson makng. Each of the four deal selecton strateges we consder n ths paper s responsve to experence and ntegrates new nformaton n systematc ways. However, the strateges dffer wth respect to how nformaton about cue profles s processed and how ther parameters are computed from a set of busness plans for whch the cue profles and outcome nformaton are gven. Equal Weghtng s an algorthm that smply bases the decsons on the number of cue values n a cue profle that pont to success. Logstc Regresson s a procedure to estmate weghts n a lnear combnaton of cues, whch, n turn, s used to make decsons. CART (Breman et al., 1984) s a tree-buldng algorthm that s flexble enough to produce the same decsons as any other possble strategy for bnary cues. The Fast and Frugal Tree heurstc (Martgnon et al., 2003, 2008) s a tree-buldng algorthm as well, but t s more restrcted n the sense that the set of trees t can possbly generate s reduced. Note that each of these strateges needs to estmate parameters, whch s acheved on the bass of a sample of cue profles and ther assocated outcome values, henceforth referred to as the learnng sample L. Equal Weghtng combnes the prncple of unt weghtng of cues, as nstantated by Dawes Rule (see Dawes, 1979; Czerlnsk et al., 1999; Haste & Kameda, 2005) and ndex methods (see Burgess, 1939; Armstrong & Graefe, 2011) and the prncple of makng dentcal predctons for sets of objects wth the same number of postve cue values, as, for example, mplemented n the Mappng Model (von Helversen & Reskamp, 2008). Fast and Frugal Trees are smple bnary decson trees that look up cue values sequentally and that are potentally, dependng on the cue value observed, able to make decsons wthout consultng further cues. They embody the prncple of one-reason decson makng and have been analyzed and tested n the context of the smple heurstcs program Todd et al. (2012). Logstc regresson and CART are procedures often used n machne learnng and classfcaton tasks across dscplnes.

11 PICKING PROFITABLE INVESTMENTS 11 Ordered wth respect to complexty when estmatng ther parameters, Equal Weghtng s the smplest strategy, followed by the Fast and Frugal Tree, wth Logstc Regresson and CART beng the most complex strateges of the set. The detals of ther mplementaton n the present study are specfed n Appendx C The Learnng Task: The Standard Condton and Its Varatons The Standard Condton In ths paper, we smulated a number of VCs over the perod of one fund. Durng the lfe of a fund, four VCs were confronted sequentally wth 10 2 randomly generated busness plans. The four VCs dffered wth respect to whch of the four forecastng and deal selecton strateges descrbed above they used to make ther 10 2 nvestment decsons. A gven condton, whch was defned by a partcular set of parameter combnatons (descrbed below), was realzed 100 tmes, correspondng to 100 dfferent sets of 10 2 randomly generated busness plans. Each of these dfferent sets was shown to new VCs, so that none of the smulated VCs could use knowledge from a prevous fund. The followng propertes defne what we refer to as the standard condton that we use as a reference pont when presentng the results of the smulaton: Each VC started wth an experence base of busness plans wth known cue profles and outcome nformaton, generated by the same process as the 10 2 plans for whch the VCs had to make ther nvestment decsons. When learnng ts parameter values from the experence base before startng a new fund, each deal selecton strategy used the same plans. In the standard condton, each busness plan was characterzed by eght cues and ther values, and all ths nformaton was gven, wthout any cue readng errors, to the VCs. The plans were randomly generated usng the arthmetc cue weghtng structure dsplayed n Table 2 and an overall success probablty =.2. s

12 PICKING PROFITABLE INVESTMENTS 12 For each of the 10 2 busness plans n a gven fund, the VC had to decde whether or not to nvest an arbtrarly chosen nvestment unt (the same for each nvestment). After a feedback lag of 6 2 plans (reflectng the fact that feedback on whether or not a busness succeeds s usually delayed, Shepherd & Zacharaks, 2002), the busness plan s cue profle and ts outcome was added to the VC s experence base regardless of the nvestment decson. Each tme a profle was added to ths experence base, all strateges estmated ther parameters agan as descrbed above and wthout any bas. An nvestment decson for any of these plans led to a return of 150% of the nvestment unt f the busness was successful, and to ts complete loss f t faled Varatons Takng the standard condton as a startng pont, we manpulated a number of parameters. In ths paper we focus on three parameters that we consdered to be most nterestng and for whch we report the effects below: Frst, the selectvty of the feedback relates to the queston of whether feedback on the outcome varable was provded for all plans or only for those n whch a VC actually nvested, resultng n a based sample (compare Denrell, 2003). Second, the cue weghtng structure (as descrbed n Table 2) determned how predctve the cues were relatve to each other (Zacharaks & Meyer, 2000). Thrd, the number of cues specfed how many peces of nformaton the strateges had at ther dsposal when makng decsons (Zacharaks & Meyer, 1998). In addton, we manpulated a number of other factors: Overall success probablty (Chrsman et al., 1989; Shepherd, 1999b), fnancal consequences of success and falure (Bruno & Tyebjee, 1986; Gompers & Lerner, 2001), sze of the expertse base (Shepherd et al., 2003; Franke et al., 2008), delay of outcome 1 Ths translates nto =.5 o, o = 1, o = 0, and o = 0 s f ns nf 2 3, and =, see Appendx C.

13 PICKING PROFITABLE INVESTMENTS 13 feedback (Enhorn & Hogarth, 1978; Harvey & Fscher, 2005), amount of error n determnng success (Khan, 1987; Zacharaks & Meyer, 1998). These manpulatons served as a senstvty analyss and ther effects were ether mnor or ntutve. More mportantly, these robustness checks revealed that our major results seem not to hnge on specfc parameter constellatons, therefore specfc results are not reported here. 4. Results 4.1. The Standard Condton The man measure we use to evaluate the behavor and the monetary performance of the strateges s the accumulated proft, that s, the total ROI after the frst k nvestments. Note that the number of nvestments should not be confused wth the number of encountered plans. Ths dfference s mportant as the decson strateges dffer wth respect to how many nvestments were made for a gven number of presented plans, or, conversely, how many plans were presented before makng k nvestments. If the fund closed before k nvestments were made, then the proft for k nvestments was defned as the proft that had accumulated at the tme the fund closed. Fgure 1 dsplays the proft of the strateges after k nvestments, averaged across the 100 smulated funds, for the parameter confguraton that we defned as the standard condton.

14 PICKING PROFITABLE INVESTMENTS 14 Fgure 1: Return after k nvestments n the standard condton The curves, except for the Fast and Frugal Tree, have smlar shapes and can be dvded nto four dstnct phases. In the frst phase, the learnng phase, proft gradually ncreased wth the number of nvestments, reflectng the fact that VCs refned ther parameters and contnued to beneft from a growng experence base. In the second phase, the nformed phase, the slope (.e., the average dfference wth respect to accumulated return) remans relatvely stable, reflectng the fact that the parameter estmates no longer changed substantally as new feedback was obtaned (whch amounts to sayng that VCs were relatvely well nformed about ther envronment). In the fourth phase, the slope fnally approaches zero, reflectng the fact that by the tme the fund was closed (.e., after 10 2 plans) each VC had made a certan number of

15 PICKING PROFITABLE INVESTMENTS 15 nvestments ( m ) and the VC s proft s defned to be constant for any k > m. In the ntermedate thrd phase, the slope smoothly decreases, reflectng the fact that the curve for a gven strategy s aggregated over 100 funds, each managed by a dfferent VC wth a dstnct m. Had we used twce as many plans, the learnng phase would have looked exactly the same, the nformed phase would have lasted more than twce as long (because there would be no necessty to start learnng from scratch), and the fnal proft would have been more than twce as hgh (because the nformed phase, durng whch proft s hgher compared to the learnng phase, would have lasted more than twce as long). The Equal Weghtng strategy had the shortest learnng phase, endng at approxmately 5 20 nvestments, and the hghest slope durng the nformed phase. CART started negatve whle learnng but was able to recover and to make postve profts. Stll, t was less proftable than Equal Weghtng and Logstc Regresson, and t never reached ther fnal level of profts. The Fast and Frugal Tree dd not make any proft at any tme. Table 1 provdes some addtonal nformaton about the behavor and performance of the strateges n the standard condton. The measures ntroduced n ths table correspond to the phases that we dentfed n Fgure 1. The frst data column reports the average return per nvestment computed over the frst 25 nvestments ( ROI 125). If a VC made less than 25 nvestments, ths VC s nvestments were stll ncluded n the analyss. As Fgure 1 shows, some strateges had a learnng phase longer than 25 nvestments, but t also seems to be the case that none of them had completed learnng before ths pont. Equal Weghtng, whch had the shortest learnng phase, also made the most proft durng ths phase. The second data column reports the return averaged across the 25 nvestments between the 51 st and 75 th ( ROI 5175). Agan, not every VC contrbuted 25 data ponts to the analyss. As

16 PICKING PROFITABLE INVESTMENTS 16 Fgure 1 shows, ths perod falls nto the nformed phase of Equal Weghtng and the Regresson strategy, whle t seems to be at the very begnnng of CART s nformed phase. And agan, Equal Weghtng performed the best. The next data column n Table 1 reports the average return across all nvestment decsons made by the VCs usng a gven strategy ( ROI all ). The next four data columns report how many nvestments the 100 smulated VCs who used a gven strategy made across the 210 plans. Frst, the varance and the range of ths number (across the VCs who used the same strategy) were qute large. Second, the dfference between the strateges wth respect to the mean number of nvestments that VCs made was also consderable. The entres for the mnmal and maxmal number of nvestments roughly mark when the slope durng the nformed phase starts to decrease and when t approaches zero, respectvely. Even though Equal Weghtng had a hgher slope durng ts nformed phase, VCs who used ths strategy nvested, on average, less than those usng Regresson. As a consequence, the curves cross and Equal Weghtng ended up wth a lower fnal return (see Table 1, last two columns). Table 1: Results for the Standard Condton. The performance of the strateges, as summarzed n Fgure 1 and Table 1, can be related to the cue weghtng structure. In the present envronment wth ts lnear and compensatory set of

17 PICKING PROFITABLE INVESTMENTS 17 weghts (arthmetc cue weghtng structure, see Table 2), nether of the two tree algorthms was compettve. The Fast and Frugal Tree dd not fnd the sngle cue that s vald enough for onereason decson makng, smply because ths cue dd not exst. CART, n contrast, has the potental to deal wth an arthmetc cue weghtng structure, but ths requres the constructon of a complex tree wth a hgh number of nodes, whch, n turn, requres an extended experence base. Note that at a gven stage durng the constructon of the tree, the busness plans are dstrbuted over ts end nodes and a small sample of plans prevents these end nodes from beng splt up further. Even though Equal Weghtng does not match the weghtng structure n the present envronment, ts ablty to make robust nferences Dawes (1979) was able to compensate for ths msmatch at least to such an extent that t was able to outperform the Logstc Regresson Selectvty of Feedback In the standard condton, the VCs could nclude all busness plans n ther experence base, no matter whether they had decded to nvest or not to nvest. Whle t s of vtal mportance to montor the success or falure of busness plans a VC nvested n, such nformaton about plans that a VC rejected s often not avalable be t because outcome nformaton dd not reach the VC after a dfferent nvestor funded the unmodfed proposal, or because no other nvestor has been found and the busness hence faled to materalze (Bruno & Tyebjee, 1986). A hgh confdence n one s ablty to make good nvestment decsons, eventually coupled wth the belef that outcome nformaton about rejected proposals s rrelevant, may lower the motvaton to actvely search for such (potentally costly) nformaton. The based samplng of busness plans creates a stuaton n whch the set of plans that VCs encounter cannot be consdered to be a representatve sample of the busness plans n the populaton. The psychologcal lterature s rch n examples demonstratng that performance s

18 PICKING PROFITABLE INVESTMENTS 18 severely mpared due to such a sample selecton bas (see Brunswk, 1955; Fedler, 2000; Dham et al., 2004). To see how ths bas affects nvestment results, we also created a condton n whch the smulated VCs were only able to nclude n ther learnng sample those busness plans they nvested n (note that ths reducton of feedback was only mplemented after the fund started, but t dd not affect the experence base that VCs were equpped wth before they started to make decsons). The results are dsplayed n Fgure 2. As the comparson between ths fgure and that of the standard condton shows, all strateges perform worse f the feedback was selectve be t because the sze of the learnng sample was smaller, or because the learnng sample was based, or both. Even though the order of the four strateges wth respect to ther proft was the same for both learnng condtons, there are some strkng dfferences. The strategy that was most affected by restrctng the learnng sample was CART: As n the standard condton, the slope starts out negatve, but when nformaton on rejected busness plans was lackng, ths strategy was not able to enter an nformed phase n whch the constructed trees could make substantal profts, and, as a consequence, also not able to compensate for the losses made durng ts learnng phase. Whle n the standard condton, Equal Weghtng has a slope of about.3 durng ts nformed phase (see also Table 1, ROI 5175), when feedback was selectve ths slope s reduced by roughly 20% to about.24. Note that slopes of.3 and.24 correspond to 86.6% and 82.7% successful outcomes among busness plans that receved fundng, respectvely, whch should be compared to the a pror success probablty of.2. For Logstc Regresson the slope was reduced from about.25 by roughly 28% to about.18, correspondng to 83.3% and 78.7% of successful outcomes among all nvestment decsons. Thus, not only dd Equal Weghtng have a hgher

19 PICKING PROFITABLE INVESTMENTS 19 slope than Logstc Regresson n both feedback condtons, t also suffered less when outcome nformaton on rejected plans no longer entered the learnng sample; that s, t was more robust when sample selecton bas was ntroduced. To put the absolute effect between unselected and selected feedback receved durng the nformed phase n context, note that a drop n profts of.06 (Equal Weghtng; from.3 to.24) and.07 (Logstc Regresson; from.25 to.18) s stll more than 10% of the proft for a successful deal, whch was.5 nvestment unts. Fgure 2: Return after k nvestments whle learnng from nvestments only The drop n performance due to selectvty of outcome feedback for three of the four strateges can also be seen when consderng ROI all. For Equal Weghtng, Logstc Regresson,

20 PICKING PROFITABLE INVESTMENTS 20 CART, and the Fast and Frugal Tree, ( ROI all ) was.21,.16,.00 and -.44, respectvely, when feedback was selectve. Comparng these values to the last column of Table 1 (where feedback was gven for all plans), the drop n performance due to ths manpulaton was roughly.06,.05,.11 for the frst three strateges. The Fast and Frugal Tree mproved by roughly.08, whle the ROI all stayed negatve The Cue Weghtng Structure For the standard condton, we used arthmetc cue weghts when generatng busness plans. As explaned n the Setup secton, we also generated busness plans for whch the cues were equally predctve (equal weghts), or for whch ther weghts adhered to a geometrc dstrbuton (see Table 2). We reran the standard condton except that we used busness plans generated wth the arthmetc and geometrc cue weghtng structure. The performance of the strateges n the two new envronments s shown n Fgure 3. It s evdent that the relatve success of the four strateges depends on the envronment n whch they are tested. In the Arthmetc Weghts Envronment, and especally n the Equal Weghts Envronment, Equal Weghtng was the best strategy (n terms of average return on nvestment durng the nformed phase). In the Geometrc Weghts Envronment, Equal Weghtng was the worst performng strategy, whle n contrast, the Fast and Frugal Tree, whch had faled n the Equal Weghts Envronment, now performed best, almost convergng aganst the theoretcally optmal slope of.5. CART, whch had shown the worst performance durng the learnng phase and the second worst performance durng the nformed phase n the other two envronments, showed vrtually the same (good) performance as Logstc Regresson n the Geometrc Weghts Envronment.

21 PICKING PROFITABLE INVESTMENTS 21 For the Equal Weghtng strategy, the average number of nvestments dd not vary much between the envronments (94.4, 92.9, and 82.7 for Equal Weghts, Arthmetc Weghts and Geometrc Weghts, respectvely). In contrast, both for the Logstc Regresson (127.8, 157.7, and 181.6, respectvely) and for CART (126.0, 147.1, and 182.0), ths number ncreased as the skewness of the dstrbuton of cue weghts ncreased (note that a hgher skew made t easer to dentfy what the predctve cues n the envronment were). The dfference was most pronounced for the Fast and Frugal Tree whch made, on average, 2.5 nvestments n the Equal Weghts Envronment, 2.0 n the Arthmetc Weghts Envronment, and n the Geometrc Weghts Envronment. The cue weghtng structure turns out to be a crucal factor n the study of the ecologcal ratonalty of strateges: a strategy tended to perform well f ts archtecture matched the propertes of the envronment (here, the cue weghtng structure). Equal Weghtng excelled n an envronment n whch the cues were equally predctve, but n the Geometrc Weghts Envronment n whch the dstrbuton of cue weghts s hghly skewed, ths strategy was punshed for usng a non-matchng weghtng structure. Treatng some cues as more mportant than others (as Fast and Frugal Trees do n the extreme) makes sense only f some cues are ndeed more mportant than others n the respectve envronment. Therefore, t showed the opposte pattern compared to Equal Weghtng. Martgnon et al. (2008, Result 2) have shown that each lnear threshold model wth non-compensatory weghts shows the same decson behavor as a correspondng Fast and Frugal Tree usng the same (or a subset of the) cues used n the lnear model. Therefore, the Fast and Frugal Tree can match the Geometrc Weghts Envronment as the latter can be descrbed as a lnear threshold classfer wth non-compensatory weghts.

22 PICKING PROFITABLE INVESTMENTS 22 Envronment Fgure 3: Return after k nvestments for the Equal Weghts and the Geometrc Weghts These two examples were extreme n the sense that the two strateges had an extreme archtectural bas (Equal Weghtng s based towards flat weghtng, and the Fast and Frugal Tree s based towards a steep herarchy) and the two envronments had extreme propertes (flat and hghly skewed cue weghtng structures). How dd the strateges perform n the Arthmetc Weghts Envronment whose cue weghtng structure was n between that of the other two envronments (see Table 2)? Consstent wth the ttle Dawes (1979) used for hs classc paper ( The Robust Beauty of Improper Lnear Models n Decson Makng ), Equal Weghtng was robust enough to fare well when the cue weghts followed an arthmetc structure and more than ths, t was even the best performng strategy. In contrast, ths cue weghtng structure was

23 PICKING PROFITABLE INVESTMENTS 23 not skewed enough to allow the Fast and Frugal Tree to realze ts potental. As Table 1 shows, VCs usng ths strategy could not construct a tree that led to a substantal number of nvestments and those few nvestments obtaned, on average, a negatve return. In contrast to Equal Weghtng and the Fast and Frugal Tree, the other two strateges were, at least n prncple, flexble enough to adjust to a wde range of envronmental propertes. One could therefore argue that n the Geometrc Envronment, CART should be able to match the performance of the smpler Fast and Frugal Tree, snce the structure of the tree constructed by the latter strategy s wthn CART s search space. However, ths s not necessarly true as the larger sze of CART s search space requres a longer learnng phase to fnd the structure of ths tree (and even wth an unlmted learnng phase ths s not guaranteed). Gven the parameters we used n our smulatons, even though CART s performance came close to that of the Fast and Frugal Tree, t dd not reach t (the slope n the nformed phase was about 7% lower). CART also has the potental to construct trees that match the performance of Equal Weghtng. As t turned out, however, t struggled n the Arthmetc Weghts Envronment and even more so n the Equal Weghts Envronment. Ths can be explaned by the fact that the trees necessary to capture the cue weghtng structures of these envronments need to be much larger than those for the Geometrc Weghts Envronment and even CART s prolonged learnng phase for those two envronments was not suffcent to construct a compettve complex tree. In a smlar ven, Logstc Regresson had the potental to perform well n all three envronments. It owes ts flexblty to the fact that t estmates cue weghts wthout any restrctons. Indeed, there s no envronment n whch VCs usng ths strategy are heavly outperformed. Nevertheless, there s not a sngle envronment that allows Logstc Regresson to wn the competton aganst all three other strateges. The lack of a stronger bas towards a

24 PICKING PROFITABLE INVESTMENTS 24 partcular cue weghtng structure and the mpact of samplng error durng the learnng phase prevents the strategy from fndng the cue weghts that are optmal for the envronment, and hence, from reachng the performance that could theoretcally be acheved The Number of Cues In the standard condton, a busness plan was descrbed by 8 cue values. We manpulated ths number and set t to be ether 6, 8 (standard), 10, or 12. The manpulaton of the number of cues dd not result n a change of the orderng of the strateges, except for the fact that CART s performance decreased (compared to that of the other strateges) as the number of cues ncreased. Fgure 4 shows how each of the four strateges was affected by ths manpulaton. The Equal Weghtng strategy performed almost the same n the four cue number condtons, except that t made more nvestments as the number of cues ncreased. For Logstc Regresson, n contrast, an nterestng pattern can be observed: durng the learnng phase, the VCs usng ths strategy made better decsons when the number of cues was smaller, whereas ths relatonshp was reversed when consderng the nformed phase. In addton to the resultng crossng of the return curves, t can be observed that the learnng phase was longer and the number of nvestments was hgher when more cues were avalable. For CART, the curves do not cross and n each phase, ths strategy both made fewer nvestments and performed better when the number of cues was smaller. Fnally, the nablty of the Fast and Frugal Tree to cope wth an Arthmetc Weghts Envronment (that we descrbed above for the standard condton) was robust aganst ths manpulaton, except that performance was worse for smaller numbers of cues.

25 PICKING PROFITABLE INVESTMENTS 25 Fgure 4: Return after k nvestments for dfferent numbers of cues These results agan demonstrate how mportant t s to consder strateges and envronments jontly (Todd et al., 2012). The performance of a strategy can only be determned n a gven envronment and strateges performance can be affected dfferentally by changes of envronmental features. Whereas the number of cues (almost) dd not affect Equal Weghtng and the Fast and Frugal Tree, CART and Logstc Regresson performed dfferently n the varous cue number condtons. For CART, the effect of the number of cues s readly explaned: the strategy obvously found t easer to construct a well performng tree when the number of cues (and, hence, the unverse of possble trees) was smaller. For Logstc Regresson, t s easer to estmate

26 PICKING PROFITABLE INVESTMENTS 26 the parameters when the number of cues s smaller (hence the shorter learnng phase for a smaller numbers of cues), but once the strategy s n the nformed phase and estmates are relatvely stable, the strategy benefts f more nformaton can be processed. 5. Dscusson and Concluson One of our central smulaton results was that the Equal Weghtng strategy was compettve wth more complex nvestment decson strateges. Ths fndng s consstent wth a general pattern found across a wde range of tasks and domans: Under specfc condtons, smple strateges can outperform complex strateges, presumably not despte of ther smplcty but because of t (Martgnon & Hoffrage, 2002). Specfcally, ther success when decdng upon new busness plans can largely be attrbuted to the robustness of ther parameter estmates (see also Dawes, 1979). Complex strateges have an advantage when t comes to fttng known data. Ths was not the stuaton, however, n the present set of smulatons, nether s t a relevant condton for VCs who wll have to make nvestment decsons for new busness plans outsde the learnng sample, n partcular, f ths learnng sample s small. In such a stuaton, smple strateges that are adapted to the envronmental structure wll preval. For nstance, when Armstrong & Graefe (2011) compared forecastng methods n the context of the last 29 U.S. presdental electons, they found that an equal weghtng model based on 59 bographcal varables of presdental canddates (whch they called ndex method ) outperformed n crossvaldaton Gallup polls, predcton markets, and three respectable econometrc models. Moreover, they found that a model based on ths bo-ndex was compettve wth seven econometrc models n forecastng votng shares (see also Graefe, ths ssue).

27 PICKING PROFITABLE INVESTMENTS 27 In a smlar ven, Ggerenzer & Brghton (2009) showed for small learnng samples that a lexcographc decson strategy outperformed several more complex strateges n a wde range of condtons (see also Brghton & Ggerenzer, ths ssue), von Helversen & Reskamp (2008) demonstrated that a smple strategy whch used equal weghtng of predctors outperformed multple regresson n an estmaton task, and DeMguel et al. (2009) demonstrated that the smple 1/N heurstc that allocated wealth equally across N assets could not be consstently outperformed by any of the optmzng allocaton model that they used as benchmarks. Goldsten & Ggerenzer (2009) offer a more extensve evaluaton of smple forecastng strateges. They demonstrate the successful performance of smple predcton rules n tasks as dverse as portfolo selecton, the predcton of future customer purchase actvty and future hgh-value customers, the forecastng of tenns and soccer results, and the forensc practce of geographc crme proflng. Our results are n lne wth these fndngs: The performance of the smple Equal Weghtng strategy was robust across envronments n the sense that t ether outperformed the other strateges (n Arthmetc and Equal Weghts Envronments) or dd not lag much behnd the wnner (n a Geometrc Weghts Envronment). VCs who do not know the statstcal structure of the envronment are thus well-advsed to use ths smple and robust strategy as a startng pont and may, after a substantal number of nvestments followed by outcome nformaton, consult and carefully evaluate the experence base to check whether the mllons or bllons they have at ther dsposal should better be nvested by adaptng a dfferent plan for makng nvestment decsons. Our fndngs are also consstent wth the noton of ecologcal ratonalty (Todd & Ggerenzer, 2007), and demonstrate how mportant t s to consder strateges and envronments

28 PICKING PROFITABLE INVESTMENTS 28 jontly: () The performance of a strategy can only be determned wthn the context of a gven envronment and () strateges performance can be dfferentally affected by changes of envronmental features. Buldng on these observatons, VCs are well advsed to learn more about ther envronment. Dfferent VCs are lkely to operate n envronments wth dfferent statstcal structures, n other words, there wll be varance across tme, regons, markets and fund szes. Whle our smulatons used parameter ranges that were nformed by the real world, VCs need to know whch parameter constellaton s relevant for them n partcular. Consequently, VCs may gan a compettve advantage from systematcally studyng ther decson makng context, for nstance, by usng the nformaton stored n ther archves, or, even better, by creatng protocols that allow for convenent forms of data analyss. In ths learnng process VCs should, however, consder that learnng from outcomes of ther own nvestments alone may lead to based parameter estmates for the decson strateges, as our smulatons have shown smply because the amount of decson makng experence, and learnng as a result of the same, by any VC s too lmted to enable them to estmate cue weghts wth enough precson to apply the model. The consderaton of ther own nvestments should thus be complemented by securng other nformaton sources to obtan a more complete representaton of the statstcal propertes of ther envronment. Whle the vast majorty of research on VC decson makng has focused more on the VCs assessment of specfc cues, our focus was on the performance of forecastng and deal selecton strateges that dffer wth respect to how they process cues. Smlarly, Zacharaks & Meyer (2000) used two bootstrappng models to predct the success of ventures and compared the performance of these models to that of actual VCs (see also Armstrong, 2001). Ther frst model used cue weghts that were ftted to data from an earler study and ther second model

29 PICKING PROFITABLE INVESTMENTS 29 weghted the cues equally. The authors found that the equally weghted bootstrap model was clearly better at selectng successful ventures than both the alternatve model and the actual VCs. They conclude that bootstrap models are useful decson ads for VCs a concluson that we fully endorse (see also Zacharaks & Meyer, 1998; Shepherd & Zacharaks, 2002). Our Equal Weghtng strategy outperformed more complex strateges n two of the three types of envronments we analyzed. Zacharaks and Meyer s bootstrap model wth weghts that have been ftted to earler data corresponds to the Logstc Regresson strategy n our smulaton wth regresson weghts that have been ftted to the VC s experence base. Thus, n both studes the Equal Weghtng strategy s more robust and hence more useful than the regresson model when t comes to forecastng the success of prevously unseen busness plans. Smlarly, Ǻstebro & Elhedhl (2006) found that a smple conjunctve model wth separate counts for good and bad cues also turned out to be extremely successful when modelng experts forecasts of early-stage ventures performance. They concluded that smple decson heurstcs can perform well n a natural and very dffcult decson-makng context (p. 395). In the real world, VCs may provde value-added actvtes to ther portfolo companes that wll enhance ther performance (Sapenza et al., 1994; Cummng et al., 2005). Also, durng tmes of ncreased VC nvestment actvty or n the case of an exceptonally promsng proposal, VCs are often forced to compete for the opportunty to nvest n selected deals (Gompers & Lerner, 2001; Lockett & Wrght, 2001). Whle we have not explctly consdered these factors so far, our smulatons do allow for ther ncluson. In the frst nstance, when evaluatng a deal a VC wll be assessng the need for, as well as ther ablty to provde, any necessary hands-on nvolvement or other value-added actvtes. As such, the perceved potental for the VC to provde support for a deal beyond the fnancal nvestment can be one of the cues that are

30 PICKING PROFITABLE INVESTMENTS 30 evaluated by a VC. Secondly, the actons of another VC nterested n a potental deal, ether observed drectly by the focal VC or reported by the entrepreneur or a thrd party, can be one of the cues that are used n our smulatons. The decson strateges we have studed are descrbed n a way that wll assst future researchers further the scope of emprcal testng n the VC settng. The lterature to date has provded many valuable nsghts on the content and process aspects of the core VC actvtes but has predomnantly provded descrptve accounts that offer lmted nput to the VC communty. Our work bulds upon ths body of research and evaluates the VC nvestment decson task through a more prescrptve approach. Moreover, n our smulatons the term VC may represent an ndvdual VC or a team of VCs wthn a frm so we hope that our fndngs wll provde the bass for more successful nvestment decson strateges, at both the ndvdual and frm levels. Fnally, we beleve that ths paper can serve as a bass for more longtudnal research that focuses on the ntersecton of the weghtng of nformatonal cues, the forecastng and decson makng process, and VC performance and learnng. Extendng beyond the VC context, our smulatons are equally relevant to angel nvestment networks, the managers of corporate venture funds and prvate equty funds. Although each of these nvestors has ther own unque preferences and nvestment objectves, they all seek proftable deals and draw upon multple nformatonal cues when evaluatng potental nvestment opportuntes. Addtonally, we beleve that our results wll nform others who engage n evaluaton and selecton routnes such as human resource professonals or any manager responsble for selectng job canddates. Gven the robustness of the pattern obtaned n our own smulaton and n the related research dscussed above, we predct that smple strateges for forecastng and for decson makng wll preval n these domans as well.

31 PICKING PROFITABLE INVESTMENTS 31 Appendx A: The cue weghtng structures 5.1. Eght cues Whether a busness plan wth a gven cue profle has a postve or negatve outcome value ( f C ) = 1 vs. f ( C ) = 1) s determned by comparng the value of ( u C ) = w c, (1) ( j, j j=1 k where w are cue weghts as specfed below and ǫ s a random varable (whose values are drawn from a unform dstrbuton over [ e; e], where the parameter constant e represents the amount of uncertanty), wth a threshold : 1, u( C ) f ( C ) = (2) 1, u( C ) <. The probablty for each cue profle to be assocated wth a postve outcome value s therefore determned as follows: k 1, w jc, j e > j=1 k w jc, j k k =1 p ( C j ) =.5, w c, e w c, e 2e (3) s j j j j j=1 j=1 k 0, w jc, j e < j=1