Minority subcontracting goals in government procurement auctions

Transcription

1 Minority subcontracting goals in government procurement auctions PRELIMINARY AND INCOMPLETE: DO NOT CITE Dakshina G. De Silva y, Timothy Dunne?, Georgia Kosmopoulou?, Carlos Lamarche? March 22, 2009 Abstract Programs that encourage the participation of disadvantaged business enterprises (DBE) have been a part of government procurement auctions for over three decades. This paper examines the impact of DBE programs on highway procurement auctions, focusing, in particular, on how DBE requirements a ect bidder participation and bidding behavior in federally funded projects. We rst apply matching techniques and use propensity scores to study di erences in the winning bids across di erent DBE participation goals. Our study shows little di erence in the winning bids that have DBE goals compared to once that do not. Additionally, within a symmetric independent private value framework, we use the equilibrium bidding function to obtain the cost distribution of rms undertaking projects either with or without subcontracting goals. We then use nonparametric estimation methods to uncover and compare the cost of rms bidding on a class of asphalt projects related to surface treatment in Texas. The analysis suggests that there are no signi cant di erences in the cost structure uncovered from the bids. JEL Classi cation: H4, H57, D44. Keywords: Government procurement, auctions. 1 Introduction Policies that create preferences for disadvantage business enterprises (DBE) have been an important feature of many government procurement programs. Both the US federal government and individual state governments have instituted such policies with the general goal of providing DBE contractors greater access to government procurement contracts. A number of studies have attempted to quantify the e ect of these policies on minority business creation and growth. The results of this literature are mixed and documented in Holzer and Neumark (2000). Our objective in this paper is not to examine the impact of a rmative action policies on DBE rms success, but rather to examine how government Texas Tech University y ; University of Oklahoma?. 1

2 procurement costs are impacted by such policies. In this e ort, we focus on contracts auctioned o. We recognize the challenge of comparing the cost of contracts related to heterogeneous projects and focus on matching closely projects with very similar characteristics, comparing winning bids and structurally uncovering cost di erences. A signi cant amount of government purchasing activity is undertaken in procurement auctions and these auctions often incorporate DBE set-aside or preference policies. For example, one large procurement program with DBE requirements is in road construction projects nanced by the US Federal government but administered by individual states. States are expected to meet speci c DBE contracting goals on their federally funded projects, usually determined as a percentage of the dollar value of contracts or subcontracts awarded to DBEs. These contracts are awarded through a competitive bidding process; however, there has been little analysis of the quantitative e ects of these DBE requirements on the outcome of auctions or on the behavior of bidders. In this paper, we examine how DBE policies in uence the participation of bidders, the bids submitted, and the winning bids in highway procurement auctions. Our study uses data from the Texas Department of Transportation (TxDOT) auctions on all highway construction projects awarded over the period 1999 to Like other states, TxDOT holds regularly scheduled bid lettings for highway construction projects in a rst-price, sealed-bid auction framework. The DBE program states a percentage goal of the dollar amount of the awarded contract to be performed by disadvantaged business enterprises. In Texas, these percentages range from 0 to 12 percent and are only included on federally funded projects. The state is responsible for determining which projects to assign DBE goals and the level of the goal for each project. The project goal is achieved typically through the subcontracting out of speci c tasks by the general contractor to a DBE subcontractor. In terms of magnitudes, TxDOT procured federally-funded highway construction services that averaged $2.4 billion dollars a year over the period 2000 through 2007 and had annual DBE goals that averaged $280 million on these projects. The most closely related study is by Marion (2007b). This study looks at federal and state funded projects in California that have bid preference and subcontracting participation goals in place. He examines the elimination of the DBE bid preference program for state-funded projects that occurred 2

3 in March of Marion nds that winning bids declined after the elimination of the program and attributes the decline to changes in the make versus buy decision and to the productivity of the subcontractors involved. Our study shows little di erence in the winning bids on federally funded projects that have DBE goals and those that do not. Moreover, we nd no signi cant di erence in the cost structure uncovered from the bids. In our analysis, we recognize the fact that contracts vary greatly in their characteristics, and DBE requirements. Their location, level of complexity, duration and size can have a great impact on the cost they impose to the state. With that in mind, we try to carefully match projects with similar characteristics. We do that in two ways: In the one case, we apply matching techniques and use propensity scores to study di erences in the winning bids across di erent DBE participation goals within the set of federally funded projects. Alternatively, we isolate a class of asphalt projects related to surface treatment and structurally estimate di erences in the cost of projects with and without DBE subcontracting requirements. Both analyses, along with a set of reduced form models, yield similar results. The paper proceeds as follows. The next section provides a brief review of the contracting literature that focuses on a rmative action policies. Section 3 presents a simple bidding model. Section 4 describes the data and section 5 presents a reduced from analysis of participation and bidding. Section 6 provides matching estimation results. Section 7 provides a structural analysis of the bid and cost distributions and section 8 concludes. 2 A rmative Action in Contracting. Literature Review In 1967, the Small Business administration program initiated an e ort to increase minority representation in small businesses through increased access to government procurement contracts. Minorities at that point were 15-18% of the population receiving less than 1% of federal contracts (Lanoue, 1992). The proportion of federal contracts awarded to minorities and women increased dramatically in the following years. By 1993 a rmative action in contracting and procurement resulted in representation of minorities proportional to their size in the population (Stephanopoulos and Edley, 1995). Despite 3

4 some attempts to evaluate the costs and bene ts of those programs very little work has been published on the e ect of a rmative action on e ciency and performance within procurement. The opponents of those programs argue that preferential treatment in awarding contracts may help promote weak companies that in turn may raise the cost of government contracting. The Supreme Court decision on the case of the city of Richmond versus J.A. Croson in 1989 mandated a stricter standard by which state and local (but not federal) programs were to be assessed from that point on. Its decision led to the suspension of programs in a large number of jurisdictions. A number of di erent programs are in e ect at the federal level some restrict the bidding for a set of contracts exclusively to minorities, others establish certain percentage bidding preference for qualifying rms, and others specifying minority subcontracting goals. We focus on programs setting subcontracting goals. Such programs can potentially create ine ciencies by introducing distortions at the level of participation or bidding. Denes (1997) compares bid prices relative to engineering cost estimates between solicitations restricted to small businesses and unrestricted solicitations. He nds that costs are no higher for restricted solicitation. Other studies that have been done focus on whether companies that bene t from a rmative action in procurement continue to succeed after the programs are no longer in e ect (Holzer and Neumark, 2000). Amendments to the program leading to elimination after a speci ed period have been shown to be e ective. Krasnokutskaya and Seim (2007) study bid preference programs. They analyze the bidding and participation decisions in highway procurement auction to evaluate the California small business program that is implemented in the form of a discount. A project is awarded to a small rm if it submits a bid that is within 5 percent of the overall lowest level. They nd that the preferential treatment of small businesses creates losses in e ciency (since the small rms have higher costs on average) but no change overall in the cost of procurement. The favored bidders participate more frequently at these auctions and are awarded more contracts. Their results are also used to simulate the outcome of the auction without preferential treatment and evaluate the program. In a related study of the California state procurement auctions, Marion (2007a) found that the distortion in participation patterns 4

5 is responsible for a 3.8 % increase in the cost of the winning rm. Despite this evidence, the e ect of such programs on the state s cost in general is ambiguous. By invoking bid preferences the state gives an advantage to minority bidders and compels the non-minority bidders to bid more aggressively and win contracts at a lower bid. At the same time, since the competitive pressure becomes lower for minority bidders they bid less aggressively than otherwise and when the item is awarded to them they impose additional cost to the state (McAfee and McMillan, 1989 and Maskin and Riley, 2000). With those counteracting factors in mind, procurement preferences can lead to ine ciencies but in some occasions they can help increase bidding competition and lower the expected price paid by the government in procurement or contracting depending on the composition of the bidder pool. Ayres and Crampton (1996), for example, provide such evidence in a case study of the FCC auctions of the regional narrowband spectrum. They show that bid preferences can be e ective if the percentage discount is optimally selected to re ect relative competition within an auction. The potential for e ciency distortions can be quite di erent for programs setting minority subcontracting goals. These programs are widely used in federal procurement contracts and may not introduce distortions in the participation decision since an e cient rm is not prevented from winning and subcontracting to another e cient minority rm. They may however, impose a cost to the government procurement agency if there is a lack of su cient competition in minority subcontracting or if subcontracting compels an e cient contractor to outsource a task. The exibility of the institutional rules for setting DBE goals across projects can have a large e ect on the success of these programs. There is very little research that has been done on the e ectiveness of subcontracting goals and our approach di ers signi cantly from others. 3 Model There are n risk neutral bidders who compete for a government contract in a rst price sealed bid auction where the low bidder is awarded the contract. There are two types of projects, indexed by j, those that have no subcontracting goals and those that do (i.e., j = 0; 1). The cost of contract j to a bidder i, is private and denoted by c ij. The density of the private cost c ij is f j and is strictly positive 5

6 on the support [c Lj ; c Hj ]. In a procurement auction, a bidder who is awarded contract j at a bid of b ij receives a net pro t of b ij c ij. Each bidder is maximizing expected pro t given by: E[ ij (b 1j ; b 2j ; :::; b nj ; c ij )] = (b ij c ij ) (1 F j ('(b ij ))) n 1 : In the symmetric independent private value case, the equilibrium bid function is where b ij = (c ij ) and '(b ij ) = c ij : (c ij jf j ; n) = c ij + 0 (c ij )[1 F j (c ij )] (n 1)f j (c ij ) Notice that the cost of the contract consists of the sum of the cost of various tasks comprising the project some or potentially all of which may be undertaken by the primary contractor. In projects having subcontractor participation goals, a number of tasks representing a minimum percentage of the estimated cost, have to be undertaken by minority or women owned subcontractors. In our analysis, we will focus on the cost to the state of mandating minority subcontracting. We ask if there is a di erence in bidding distribution between projects that have subcontracting goals in place and those that do not and whether the combined cost of the project is di erent across j 0 s. It is obvious that if the minority subcontractors are less e cient they will impose a cost to the state agency. 4 Data Description and Background on Texas Policy As with other states, the Texas Department of Transportation (TxDOT) holds regularly scheduled highway procurement auctions that incorporate goals for the awarding of subcontracts to DBEs. For selected federally-funded projects, TxDOT assigns a proportion of the dollar amount of a contract that must be carried out by DBEs. The overall annual goal is about 12% of federally-funded projects; however, Texas achieves one-half to two-thirds of that goal through race neutral awards, as shown in Table 1. The remainder is achieved through auctions with speci c DBE goals. TxDOT only considers projects that are estimated to cost at least $400,000 for assignment of DBE goals. Figure 1 presents the distribution of the DBE goals for federal projects over $400,000. The goals run from zero to 12 percent with 63% of projects having DBE goals above zero. State-funded projects do not have DBE 6

7 goals. Like in other states, the Department of Transportation in Texas chooses which projects to assign DBE status and the level of the DBE goal. The state makes its decisions by considering a number of factors including the geographic location of project and the availability of pre-quali ed DBE subcontractors that can do speci c tasks in speci c locations. The idea is to choose projects where there is an adequate supply of DBEs that can do the speci c tasks listed for a project in a given area. TxDOT has a separate o ce, distinct from the o ces that design and let the projects, that manages these assignments. The objective of our empirical analysis is to see whether bidding di ers systematically in auctions with and without DBE goals. The TxDOT bid data that we have access to contain information on all road construction projects o ered for bid letting in Texas for the period from September, 1999 through June, These projects include road construction and paving projects, bridge construction and maintenance projects, concrete, erosion control, grading and drainage as well as tra c signal projects. Projects are auctioned o on a monthly basis using a rst-price, sealed-bid auction format. For each project, we have information on the location of the project, a detailed description of the tasks involved, the estimated length of the project (in calendar days), the engineering estimate of the project s total cost, whether the project is federally or state funded, and DBE participation requirements. From the bid letting, we know the identity of the plan holders - the rms that purchase the plans for a project, which plan holders submit bids, the dollar value of each bid submitted, the winner and the winning bid. Looking rst at data on bids, Table 2 shows average bid statistics for projects by DBE intensity. Comparing projects with no DBE goal to projects with DBE goals, we see that relative bids the bid submitted by a rm divided by TxDOT s engineering cost estimate for a project and relative winning bids are generally lower for projects with no DBE goals. This is true in our federal sample and for the more speci c group of paving contracts. In the analysis that follows, we will perform more detailed analysis on paving contracts. However, as DBE goals increase from 4% to 10%+, there appears no strong pattern in the average relative bids and relative winning bids are essentially at across the DBE groups from 4% to 10%+. 7

8 Table 3 provides a summary of key features of the data, disaggregating the data into several di erent sub-samples. The rst column presents the data for all Texas auctions, the second column for statefunded auctions and the remaining four columns for federally-funded auctions. As we discussed above, auctions with DBE goals are federally-funded auctions, and in Texas, only federally-funded projects above $400,000 are eligible for DBE status. So, in the econometric analysis that follows, we will restrict the samples to federally-funded projects that exceed $400K. The table also breaks out paving projects into a separate category as we examine this speci c group of auctions in more detail below. Overall, there are 6562 projects in our data set, with state-funded projects and federal-funded projects of less than $400K accounting for 48 percent of all projects. These projects do not have DBE goals. There are 3419 federally-funded projects that exceed the $400,000 level and these are the projects eligible for DBE s assignment. Of the eligible projects, 2154 (63%) have positive DBE goals with the average goal being 6.1%. Compared to federal projects greater than 400K that are not assigned DBE goals, these projects are much larger, generate a somewhat larger number of bids, have higher relative average bids, and higher relative winning bids. These DBE projects also have a signi cantly higher number of bid components, indicating the projects are more complex. It is not surprising that projects with a larger number of tasks are more likely to have positive DBE goals. This is because DBE status is assigned based upon the availability of DBEs to undertake speci c work. Projects with a larger number of tasks are more likely to contain elements that DBEs are quali ed to undertake in a speci c geographic area. The paving sample, while representing on average smaller jobs, has similar patterns as the federal sample. The paving sample represents a set of projects that are more uniform in nature but still include a substantial number of DBE auctions. Relative bids and winning relative bids are higher for paving auctions with positive DBE goals. In addition, project complexity is greater in auctions with DBE goals. 8

9 5 Reduced Form Estimation In order to better understand the patterns of bidding in DBE auctions, we present a set of reducedform regressions that show how participation and bidding varies with the DBE status of an auction. Throughout this analysis we only use data on the federally-funded projects that have engineering cost estimates that exceed $400,000 (i.e. DBE eligible projects). 5.1 Variable De nitions Our main dependent variables are the number of plan holders and number of bidders in the participation models and the log of the bid and log of the winning bid in the bid models. We also have estimated all models using the relative bid in place of the log bids and the results are very similar. Throughout the analysis, we employ two alternative measures of DBE status of an auction. The rst is a simple dummy variable that indicates whether an auction has a DBE goal or not. The second includes a set of DBE dummies that allow the e ect of DBE status to di er by DBE intensity. In this case, we use the four classi cations presented in Figure 1 and create indicator variables based on these groupings. The regressions include controls for project, bidder and rival characteristics when appropriate. Project characteristics include the DBE status (discussed above), the date of the auction, the location of the project, the type of project, the engineering cost estimate of the project, the number of individual tasks or bid items that make up a project, and the number of calendar days. The location of the project is identi ed down to the county level but in the regression models we utilize a broader geographic area known as a construction division. TxDOT divides the state of Texas into 25 construction divisions. Projects are classi ed into ve broad types paving (primarily asphalt), earth-work, structure- and bridge-work, tra c and signals, and subgrade projects. Miscellaneous and maintenance projects that are often quite small are excluded from the analysis. The engineering cost estimate is the state s estimate of costs to complete the project. In Texas, the engineering cost estimate is public information available to potential bidders before the bid letting. The number of bid items re ects the number of distinct tasks or items that make up a project. In our setting, we view the number of bid items as re ecting the overall complexity of a project. More involved projects, holding the size of the project 9

10 constant, should involve a larger number of bid items. The number of calendar days measures the length of time in days that a project is expected to take. The bid letting data include a list of all plan holders, a list of all bidders and their submitted bids, and the winner and the winning bid. From this information, we identify the speci c plan holders and bidders in an auction which allows us to construct variables based on prior bidding histories (discussed below) and to incorporate rm e ects in the bid models. A plan holder is a pre-quali ed rm that purchases a plan for a speci c project. The list of plan holders is public information available to all bidders prior to the bid letting. In addition to variables that characterize the project and auction bidding, we also construct a set of variables related to bidder characteristics. Similar to previous work (Porter and Zona (1993) and Bajari and Ye (2003)), we use measures of capacity utilization, the distance of the rm to the project, and whether the rm has ongoing projects nearby the proposed project to proxy for di erences in rm costs. The capacity utilization rate is the current project backlog of a rm divided by the maximum backlog of that rm during the sample period. For rms that have never won a contract, the utilization rate is set to zero. Data from the year 1998 are used to construct a set of initial starting value for the capacity utilization variable. The backlog variable is constructed as follows. For each awarded project, TxDOT pays the monthly work completed amount. A contract backlog is constructed in each month by summing across the remaining value of all existing contracts in Texas for a rm. As projects are completed, the backlog of a rm goes to zero unless new contracts are won. Another control for di erences in rm costs is the distance between a project and the location of the rm. It is constructed as the logarithm of the distance between the county the project is located in, as measured by the longitude and latitude of the county seat, and the distance to the rm s location. We also construct a variable to identify a rm that is bidding on a project in a location where it has an ongoing project in the same county. The idea here is that staging costs will be lower if a rm already has ongoing work in a nearby location. Finally, we have a small number of rms that participate in the auctions that are DBE rms. The work that these rms perform on auctions they win would count against the DBE goal for the auction. Hence, these rms may need to rely less (or not at all) on DBE subcontractors in 10

11 ful lling the DBE goals of the project. In our bid models, we control for di erences in the DBE status of the bidder when the rm is bidding on DBE projects. We also construct a set of variables related to the rival s a bidder face. These variables include measures of the minimum backlog and the minimum distance from the set of rivals a rm faces in the auction. The set of rivals is determined by the plan holder list. We also utilize a measure of the rivals past success in auction. This variable is constructed as the average of the ratio of past wins to the past number of plans held across rivals in an auction. This variable incorporates two aspects of past rival bidding behavior. It incorporates both the probability of a rival bidding given they are a plan holder and the probability the rival wins an auction given that they bid. These probabilities are updated monthly using the complete set of bidding data in Texas. The probabilities are initialized using data from Participation Rates Our rst analysis looks at how plan holder and bidder participation varies by DBE status of the auction. We estimate a set of count data models using the number of plan holders and the number of bidders as our dependent variables. As control variables, we use auction characteristics including the log of the engineering cost estimate, the log of complexity (number of bid items), the log of the number of calendar days, a set of indicator variables for construction division, and time dummies. We also include a set of variables to control for the distribution of tasks in each project. These are the dollar share of tasks in each of the project categories. In the paving contracts, we incorporate a more detailed set of tasks 11 distinct groups and include these shares in the participation and bid models. To control for DBE status, we estimate two alternative speci cations one with a single indicator variable for whether or not the project has a DBE goal and one with the three indicators that based the level of the goals 1%-5%, 6%-7%, and > 7%. We estimate the models using a Poisson regression for the sample of federally-funded auctions over $400K and for a sample that focuses speci cally on paving projects. Table 4 presents the marginal e ects, along with robust standard errors. The results presented in Table 4a show that for the sample of federally-funded projects above 11

12 $400K auctions with DBE goals actually have greater participation in terms of the number of plan holders and number of bidders. This is true across both DBE speci cations. In fact, in auctions with higher DBE goals, participation in terms of the number of plan holders and the number of bidders is higher. Looking at the other project characteristics, there are a greater number of plan holders and bidders in auctions where the project size is larger, the calendar days longer, and the number bid items less. Thus, conditional on other factors, it appears project complexity reduces participation. In the paving runs (Table 4b), the DBE coe cients are positive but generally not statistically signi cant. Participation, in terms of plan holders and bidders, is higher for larger projects. More complex projects have a lower number of plan holders but the relationship is not statistically signi cant in the number of bidder models. Overall, the evidence suggests that participation in DBE auctions is generally higher compared to non-dbe auctions in the federal sample, with less of a statistical di erence in the paving sample. 5.3 Bidding This section presents a set of reduced-form regressions that describe bidding in the TxDOT auctions, focusing on the di erences in bidding that occurs in DBE and non-dbe auctions. Two sets of analysis are performed a regression model examining all bids and a regression model examining winning bids. The rst analysis reports the results of regressions that use all submitted bids. We estimate a xed-e ects bid regression with controls for auction, bidder, and rival characteristics. We include a set of bidder control variables capacity utilization, distance to project, ongoing project in the location, and DBE status of the bidder, along with the rival variables rivals minimum distance and backlog. A very small number of our bidders are DBE contractors. An interaction variable between the DBE status of the bidder and the DBE status of the auction is included in several of the speci cations. This interaction is included to see if DBE rms bid di erently than non-dbe rms in DBE auctions. All models include rm, district, and time e ects. The results of this analysis are shown in Table 5. We do not show all the parameter values from the estimation to conserve space, but these are available from the authors. 12

13 With respect to the DBE parameters, they are positive but not generally statistically signi cant across the various speci cations. Thus, the di erence in average bids between DBE and non-dbe auctions that we saw in Table 2 is no longer apparent once we control for bidder, rival and auction characteristics. The same pattern is true in models where the dependent variable is measured as the relative bid (Table A1). Not surprisingly, bids rise with project size and decline as the number of plan holders increase. Longer projects and projects that have more tasks (complexity) have higher bids. In addition, higher capacity utilization and greater distance to a project are associated with higher bids. A bidder with ongoing projects in the same location bids somewhat more aggressively. Average rival past winning percentage has a negative sign but is not statistically signi cant. The next set of models we turn to examine how the winning bid in an auction varies with DBE status of the auction. In these models, we use the log of the winning bid as the dependent variable along with winning bidder, rival, project and auction characteristics, as control variables. The models are estimated for three alternative DBE speci cations in the case of federal sample that includes all project types. For the paving sample, we do not include the DBE interaction terms because we have relatively few DBE rms bidding in these auctions. The results are presented in Table 5. With respect to the DBE e ects, we see that the coe cients on the DBE variables are positive but generally not statistically signi cant. The one exception is for DBE projects with greater than 7% DBE goal, where we see a positive and statistically signi cant di erence. Average winning bids are 2.6% higher in these auctions. The other variables in the models generally conform to expectations. Auctions with a greater number of plan holders have lower winning bids, the interpretation being that the number of plan holders is a good proxy for the potential competition that exists for a project. Alternatively, projects that are more complex, as measured by the number of distinct bid items, have higher winning bids. Winning bidders that have ongoing projects nearby or face tougher rivals bid more aggressively. The results are consistent in regression models that substitute the relative bid variable for the log of the bids (Table A4). Overall, the bid models suggest a relatively small e ect of DBE status on bidding in these auctions. This makes sense as the DBE share of project totals average about 6% in the sample. So even if tasks done by DBE s cost more, the 13

14 relatively low share will restrict their impact on overall project costs. In addition, we nd no evidence (in fact, we nd the opposite) that DBE status decreases participation in these auctions. However, one concern that we have is that we may not be making as tight a comparison as possible between DBE and non-dbe projects. In the next section, we explore this by using matching techniques in order to compare more similar projects. 6 Matching Estimation We use matching techniques to evaluate the whether the winning bids are di erent among DBE and non-dbe goal projects. Rosenbaum and Rubin (1983) proposed propensity score matching as a method to evaluate a treatment e ect. 1 They de ned propensity score as the conditional probability of receiving a treatment given observable (pre-treatment) characteristics. When considering our issue, the basic idea of the matching method is to compare the outcomes of DBE and non-dbe goal projects that have similar distributions conditioning on observable auction characteristics. Let D = 1 when projects are assigned a DBE goal and D = 0 when there are no DBE goals assigned. The variables W 0 and W 1 are winning bids for non-dbe and DBE projects, respectively, and we are interested in the di erence in W 0 W 1. Following Becker and Ichino (2002), the Average E ect of Treatment on the Treated (ATT) can be written as follows: = EfW 1i W 0i jd i = 1g = EfEfW 1i W 0i jd i = 1; p(x i )gg = EfEfW 1i jd i = 1; p(x i )g EfW 0i jd i = 0; p(x i )gjd i = 1g where X is de ned as observable auction characteristics. To derive the above, three assumptions need to be satis ed: 2 balancing of observable variables, unconfoundedness, and the common-support condition. When the balancing property is met, observations with the same propensity score have the same distribution of observable auction characteristics independent of DBE status. Unconfoundedness 1 Imbens (2000) and Lechner (2001) extend the method of propensity score matching to multiple mutually exclusive programs. Frolich (2002) discusses di erent impact evaluation methods, including those based on the conditional independent assumption in a similar context. Also see Lechner (2002a and 2002b.) 2 For a formal proof, see Rosenbaum and Rubin (1983) and Imbens (2000). 14

15 assumes that, conditioning on observed auction characteristics, DBE assignment is independent of the winning bid for non-dbe cases. Finally, the common-support condition assumes that, for each DBE auction or treated unit, there are non-dbe or control units with similar observable auction characteristics. When these assumptions are met, the observed outcome of non-dbe projects can be used to estimate the counterfactual outcome of DBE projects in the case of no DBE goals. Our situation is not the typical matching case. In the bidding environment, TxDOT assigns DBE goals to a project the treatment based on the availability of quali ed DBE subcontractors to perform project-speci c tasks. The assignment is not random and certain types of projects are more likely to be selected than other types of projects. We control for this non-random assignment by matching DBE and non-dbe projects using a well de ned set of auction characteristics. We use engineering cost estimate, number of potential bidders, days to complete the project, number of tasks, market condition variables, location, and month and year dummies. In addition, we control for the distribution of tasks by including cost shares of various bid items. We also ensure that the balancing property is satis ed while estimating the propensity score. Next, we follow Becker and Ichino (2002) when estimating propensity score and calculating matching estimators for ATT. We use a probit model to estimate the propensity score. The results of the probit models are reported in Table 7. The results indicate that high-value and more complex projects have a higher probability of being assigned DBE goals. We do not observe a signi cant correlation between DBE assignment patterns and days to complete the project. When considering project shares (not reported due to considerations of space), auctions with high miscellaneous and maintenance shares have a higher probability of being assigned as DBE projects. This is somewhat expected as many DBE rms are quali ed for tasks such as transportation of equipment, moving, landscaping, and maintenance. IIn Table 8, we report the e ect of DBE assignment on relative winning bids. We use relative winning bids as a way to control for di erences in project size. Here, we use three matching techniques: (1) nearest neighbor matching, (2) radius matching, and (3) Kernel density matching. 3 In radius 3 Detail discussion about these matching techniques is given by Becker and Ichino (2002.) 15

16 matching, we specify the radius to be.001. We observe no statistical di erence in winning bids between DBE and non-dbe projects. This agrees with the ndings in Table 6 with regard to the DBE indicator variable. Next, we examine if there is a di erence depending upon the magnitude of the DBE goal. Here, we re-estimate the propensity score for DBE goals less than six percent, from six through seven percent, and above seven percent with non-dbe goal projects separately. These results are reported in Table 9. In radius matching, we set the radius to be.005. Again, we see that there is no di erence between DBE and non-dbe winning bids. 7 Structural Estimation This section uses nonparametric estimation methods to uncover the cost of rms bidding in procurement auctions. Within the symmetric independent private value framework presented above, we use the equilibrium bidding function to obtain the cost distribution of rms undertaking projects either with subcontracting goals or without subcontracting goals. The nonparametric identi cation and estimation follows closely Guerre et al. (2000) and Li et al. (2000). 4 In what follows, we investigate whether the policy for disadvantage business enterprises led to e ciency losses in the period Identi cation and nonparametric estimation Let G 0 (b) be the distribution function of bids in projects without subcontracting goals and G 1 (b) the distribution function of bids in projects with subcontracting goals. Let g 0 (b) and g 1 (b) be the associated densities. Considering the standard monotonicity condition imposed on the equilibrium bid function (c), we write f(c) = g(b) 0 (c) and F (c) = F ( 1 (b)) = G(b): If we substitute these expressions into the equilibrium bidding function, we nd that the latent cost of undertaking a project without subcontracting goals can be written as, c 0 = b 0 1 n G 0 (b 0 ) ; (1) g 0 (b 0 ) 4 The implementation of the approach is related to Li, Perrigne and Vuong (2002), Flambard and Perrigne (2006) and Marion (2007). For alternative estimation methods see, for example, Haile et al. (2003) and Krasnokutskaya (2004). 16

17 where n 0 is the number of rms bidding in projects without subcontracting goals. Similarly, the latent cost associated with a project that has subcontracting goals is, c 1 = b 1 1 n G 1 (b 1 ) ; (2) g 1 (b 1 ) where n 1 is the number of rms bidding in projects with subcontracting goals. Below, we consider n 0 = n 1 = n: The right hand side of these equations can be estimated considering nonparametric methods using the observed vector of bids b = (b 0 0 ; b0 1) 0 and the number of bidders n. The projects considered in this study are not identical. As such, it is likely then that project characteristics shift the distribution of bids G j (b). If observed heterogeneity is present, standard nonparametric methods may produce biased estimates. Therefore, we incorporate auction speci c characteristics replacing the unconditional distribution functions G j (b) and g j (b) in equations (1)-(2) by conditional distributions of a form G j (bjx) and g j (bjx). The vector x 2 R p includes variables capturing observed projects heterogeneity. These conditional functions can be estimated by considering the empirical version of standard de nitions, ^g j (b j jx j ) = ^g j (b j ; x j )= ^f j (x j ) and ^G j (b j jx j ) = ^G j (b j ; x j )= ^f j (x j ), and the following estimators: ^g j (b j ; x j ) = ^G j (b j ; x j ) = ^f j (x j ) = 1 nl j h 2 jg 1 nl j h jg L j X l=1 i=1 L j X l=1 i=1 L j nx b K g 1 X x K f L j h jf l=1 nx x K G h jf xjl bjil ; x x jl ; h jg h jg h jg : xjl 1 fb jil bg ; where 1 fg is an indicator function, K g (); K G (); and K f () are continuously di erentiable kernel functions de ned over a compact support, and h g ; h G ; and h f are the associated bandwidths. Several kernels satisfy these conditions, including the triweight kernel, K(u) = u2 3 1 fjuj 1g : We use this triweight kernel to estimate the density f j (x j ) and the distribution function G j (b j ; x j ). Moreover, we consider the product of two triweight kernels for estimating the density g j (b j ; x j ). 17

18 An important aspect of the estimation method is related to the choice of the bandwidth because the shape of the empirical distributions can be dramatically a ected by the selection of h. Both the rates in Guerre et al. (2000) and the factors associated to the choice of the triweight kernel (see, e.g, Hardle 1991) suggest employing bandwidths of the form h jg = c^(b j )(nl j ) 1=5, h jg = c^(b j )(nl j ) 1=6, and h jf = c^(x j )(nl j ) 1=5 ; with (b) de ned as the standard deviation of b and c = 2:978 1:06. Following Guerre et al. (2000) and Li et al. (2000), the estimation of the conditional version of equations (1)-(2) is completed in two steps. In step 1, we estimate the pseudo cost c j separately for each equation, and in the second step, we use the pseudo values ^c j and the project characteristics x j to estimate the conditional cost distribution of rms bidding in auctions either with or without subcontracting goals. 7.2 Asphalt project data and empirical considerations The identi cation strategy relies on having similar projects. We use a sample of asphalt projects. It has been argued that in asphalt projects one has to rely more on the individual rm s state of equipment and internal e ciency to determine the cost. From related literature (see Hong and Shum (2002), De Silva et al. (2008)) and our discussions with state highway and civil engineers, we believe that asphalt projects appear best described by the independent private value framework. The sample of asphalt projects includes auctions that have the largest component of the engineer s cost estimate on asphalt work. 5 Although asphalt projects are less heterogeneous than the full sample of projects, they may include some work on di erent components such us bridge, subgrade, etc. We made two attempts to obtain an even more homogeneous sample. First, we select maintenance contracts related exclusively to surface treatment. We present descriptive statistics of this sample in the rst four columns of Table 10 (we call this sample Projects (a)). We consider the sample for all levels of participation in the rst two columns and the subsample with 3 and 4 bidders that has the highest frequency in the following two 5 The original sample of procurement auctions includes auctions on earthwork, subgrade and base course, surfacing and structures. The diversity of the projects ranges for instance, from bridge work to roadway excavation. Projects typically require work on these very di erent tasks, but it is possible to categorize the auctions based on the main component of the engineer s cost estimate. 18

19 columns. Second, we restrict attention to asphalt maintenance projects with an estimated cost between 1 million and 20 millions, asphalt component higher than 50 percent of the engineer s cost estimate, and bridge and earthwork components less than 5 percent. Bridge and earthwork components introduce uncertainty in the cost that is more common to all bidders. We also restricted the sample to projects with no subgrade and base course tasks. Those tasks not only introduce common uncertainty in costs but appear typically in the construction of new roads. New road construction has very distinct features from road maintenance. The descriptive statistics for this sample, called Projects (b) in what follows, are presented in the last four columns of Table 10. Figure 2 presents standard Gaussian kernel estimates for projects with and without subcontracting goals (DBE and Non DBE) considering projects with 3 and 4 bidders. While the upper panels show the densities for the observed bids, the lower panels present the densities for the logarithm of observed bids, considering Table s 10 samples (a) and (b). In the upper panels, we see that the distribution of bids is skewed to the right due to few outliers in the upper tail. This is an empirical issue that is commonly reported in the literature (see, e.g., Marion 2007, Li et al. 2000). A possible way out of this is to consider the data-driven scheme introduced by Guerre et al. (2000). They propose to use a bandwidth h to de ne lower and upper bounds, [b min + s(h); b max s(h) ]; for trimming bids given a known function s(). The small marks at the top of the gures indicate [b min + h; b max h], where h was selected by the standard Silverman rule of thumb. The gures at the top suggest that if we apply the truncation device on the observed bids, we will trim more heavily the lower tail than the upper tail loosing many observations. Note however that if we consider the logarithm of bids instead, we avoid trimming out many observations overcoming the di culties associated to using standard kernel density estimation at the boundaries of the support [0;max(b)]. Given the potential bene ts of using the logarithm of bids rather than bids, we consider the logarithmic transformation for the variable of interest c j (see, e.g., Li et al. 2000, Marion 2007). We de ne the pseudo cost ^c as follows: exp(aj )(1 m ^c j = j (a j ; z j )) if max fh jg ; h jg g a jil a j max max fh jg ; h jg g +1 otherwise (3) 19

20 where the variables a j = log(b j ); z j = log(x j ), and m j (a j ; z j ) = 1 n 1 1 ^Gj (a j jz j ) : ^g j (a j jz j ) The upper bound of the support includes a variable a j max de ned as maxfa j1 ; :::a jnlj g: In the rst stage, we now use equation (3) to obtain ^c 0 and ^c 1 ; and in the second stage, we use these pseudo costs and the engineer s cost estimate to estimate the conditional distributions ^g 0 (^c 0 jx 0 ) and ^g 1 (^c 1 jx 1 ): Figure 3 presents the conditional densities evaluated at the median of the engineer s cost estimate considering the analysis samples described in Table 10. The continuous lines show kernel density estimates for the cost of rms bidding in projects without subcontracting goals (Non DBE), and the dashed line present estimates for the cost of rms bidding in projects with subcontracting goals (DBE). Because the bid s distributions are not comparable in cases of di erent number of bidders, we estimate the vector of pseudo cost ^c j separately for 3 and 4 bidders. Then we pool the values for di erent number of bidders to estimate the conditional density of cost ^g j (^c j jx j ) (see Li et al. 2002). The upper panel in Figure 3 shows that the cost distributions of rms bidding in surface treatment auctions are similar, suggesting that there are no signi cant di erences between the cost of rms undertaking projects with subcontracting goals and the cost of rms undertaking projects without subcontracting goals. The lower panel shows results obtained from the sample called Projects (b). We nd that, once again, there are no noticeable di erences in the cost distributions. In Table 11 we brie y evaluate the robustness of our previous results. First, we consider the previous logarithmic transformation based on a di erent conditioning variable. In the rst and second stages, we replace the logarithm of the engineer s cost estimate by the complexity of the project. The rst two rows present statistics based on the estimated costs distribution. The last two rows show descriptive statistics for the cost distributions obtained from using a di erent two-stage approach. Haile et al. (2003) propose an alternative method to account for observed heterogeneity that avoids the estimation of the conditional densities and distribution functions. 6 The method uses residuals obtained from a mean regression model (x), where () is a known 6 The advantage of using this method relative to the approach presented before is that allows controlling for more than one auction-speci c characteristic without increasing the sample size. 20

21 function and x is a vector of observed auction characteristics. In this study, we assume that () is a linear function and x includes the logarithm of engineer s cost estimate, a quadratic term on the logarithm of engineer s estimate, and complexity of the project. 7 The second stage simply uses the homogenized bids to estimate the cost s distribution of rms bidding in auctions either with or without subcontracting goals. Table 11 shows that the empirical distributions seem to be similar irrespective of both the estimation method and sample. 8 Conclusion This paper examines how DBE policies in uence the participation of bidders, the bids, and the winning bids in federally funded highway procurement auctions. Our approach is to carefully match projects with similar characteristics. We then analyze the data in two ways: We rst apply matching techniques and use propensity scores to study di erences in the winning bids across di erent DBE participation goals. Then, we isolate a class of asphalt projects related to surface treatment and structurally estimate di erences in the cost of projects with and without DBE subcontracting requirements. Both analyses yield similar results. We nd no statistically signi cant di erence in the winning bids based on the matching techniques, suggesting that these programs may not be costly on the budget. Moreover, we nd no signi cant di erence in the cost structure uncovered from the bids. 9 References Becker and Ichino (2002) Estimation of Average Treatment E ects based on Propensity Scores. The Stata Journal, 2002, 2, 4, Chay Kenneth and Robert w. Fairlie, (1998) "Minority Business Set-Asides and Black Self- Employment" University of California Santa Cruz, working paper. 7 We evaluated the sensitivity of our results to the choice of the mean function letting () to be a smooth function. We estimated the function considering standard local polynomial regression and generalized additive methods. In our application, the evidence suggests that the conclusions are not sensitive to the choice of the conditional mean function. Therefore, we assume that () is a linear function that is estimated by standard least squares methods. Moreover, in order to provide a more direct comparison between the estimated costs obtained from the two alternative estimation methods, we transformed the homogenized bids considering the rst order statistic of the residual vector. 21