A Technical Description of the STARS Efficiency Rating System Calculation

Size: px

Start display at page:

Download "A Technical Description of the STARS Efficiency Rating System Calculation"

Griffin Boone
5 years ago
Views:

1 A Techical Descriptio of the STARS Efficiecy Ratig System Calculatio The followig is a techical descriptio of the efficiecy ratig calculatio process used by the Office of Superitedet of Public Istructio s (OSPI) Studet Trasportatio Allocatio Reportig System (STARS). It is ot desiged for the geeral populatio, but for those mathematically iclied idividuals wishig more detailed techical iformatio regardig the calculatio process used to develop the efficiecy ratigs ad cohort district weightigs. The STARS efficiecy system was developed as a itegral part of the fudig model developed by Maagemet Partership Services (MPS) uder cotract to the Office of Fiacial Maagemet (OFM) ad this documet was primarily based o MPS s report to the Washigto State Office of Fiacial Maagemet Developmet of Studet Trasportatio Fudig Methodology Optios for Washigto State, November 21, The core of the efficiecy ratig process is a Data Evelopmet Aalysis (DEA) calculatio. DEA is a established ad widely accepted statistical process. To build a properly costructed DEA model, oe first idetifies the iputs (the resources used), the outputs (the services provided), ad the site characteristics (the factors that ifluece costs, but are beyod the direct cotrol of the school district). The DEA model idetifies, for each school district, a empirically based ad mathematically soud miimum expediture level that allows the district to provide trasportatio services, while recogizig each district s local site characteristics. I the STARS process, there are two iputs: operatig expeditures ad buses; ad two outputs: the umber of basic ad special program studets trasported. At this poit, OSPI has idetified the followig six school district features that are used as site characteristics: lad area, average distace from stops to school, the umber of destiatios, studets per mile of roadway (used as a measure of studet desity) ad roadway miles per lad area (used as a measure of roadway desity). I the March 2013 ad March 2014 ratigs, the umber of kidergarte routes was icluded. However, the collectio of that data was discotiued i the school year. Geeral Descriptio The key cocept of the DEA model is the idetificatio for each district of a target district, which is a weighted average of all the school districts i Washigto State. DEA uses the optimizatio power of liear programmig to idetify, for each school district, how much weight to place o every other district to produce a target district that simultaeously reduces expeditures ad buses by the largest possible percetage, while maitaiig the umber of studets trasported. The target district does so while operatig uder the same or worse site characteristics.

2 As a simplified example, suppose that the liear program for District A idetifies its target district to be 60% of District B, 30% of District C, ad 10% of District D ad that lad area is the oly site characteristic. The the expeditures, buses, regular educatio riders, special educatio riders, ad lad area of District A s target district equal 60% of the value at District B, plus 30% of the value at District C, plus 10% of the value at District D. The table below shows hypothetical data for Districts A through D ad for District A s target. District Weight Expeditures Buses Basic Riders Special Riders Lad Area A --- $900, B 60% $1,000, C 30% $100, D 10% $2,000, Target $830, The expeditures for District A s target would be (0.6)*($1,000,000) + (0.3)*($100,000) + (0.1)*($2,000,000) = $600,000 + $30,000 + $200,000 = $830,000. The same calculatio would be performed for buses, basic program riders, special program riders, ad lad area. The importat observatio is that District A s target performs better tha District A. The target speds $70,000 less, uses 5 fewer buses, ad trasports the same umber of studets, while it has more lad area, a factor kow to icrease cost. Yet, it is reasoable to assume that the performace of the target district is achievable sice it is a weighted average of actual districts (if lad area was the determiig characteristic of school district trasportatio performace). Note that if District A were operatig efficietly, the it would have placed 100% of its weight o itself ad its target would be idetical to District A. Details of the Calculatio Process Let be the umber of school districts. The DEA literature would refer to school districts as a service delivery uit or decisio-makig uit (DMU). For this discussio, SD will be used to idicate the school district as the uit uder cosideratio. Let X ij be amout of iput i cosumed by SD j, for i = 1, 2,, I ad j = 1, 2,,. I STARS, where the iputs are prior year expeditures ad buses, X 1j would be the prior year expeditures for SD j ad X 2j would be the umber of buses. Let Y rj be the amout of output r produced by SD j, for r = 1, 2,, R ad j = 1, 2,,. The outputs represet the productio levels of the DMUs. I STARS, the outputs are the basic program studet cout (Y 1j ) ad the special program studet cout (Y 2j ). I DEA theory, while the choices of iputs ad outputs must capture the essece of each DMU s productive operatios, it is ofte difficult to describe completely all the

3 iputs that a DMU cosumes ad all the outputs that it produces. As a result, every effort must be take to defie the iputs ad outputs as fully as possible. Fortuately, i the evaluatio of school trasportatio, the iputs ad outputs are fairly straightforward. Let S kj be the value of site characteristic k at SD j, for k = 1, 2,, K ad j = 1, 2,,. A site characteristic describes a feature of a school district that iflueces its ability, favorably or ufavorably, to trasport studets per dollar of expediture. For example, the umber of roadway miles per square mile is a favorable site characteristic because a larger value idicates that a school district is likely to sped less moey ad use fewer buses to deliver give umbers of basic ad special program studets to school. Two questios eed to be aswered with respect to each site characteristic. First, does it belog i the model? There may be reasos to suspect that it has a ifluece o the ability of a school district to operate efficietly, but it may ot be obvious that the effect is real. Secod, if the effect is real, is the site characteristic favorable or ufavorable? I some cases, there is little questio about the favorable or ufavorable ature of the site characteristic, but i other cases it may ot be clear. The STARS process addresses both questios by costructig a multiple regressio model usig operatig expeditures as the depedet variable ad usig all of the outputs ad all of the site characteristics as potetial idepedet variables. STARS uses operatig expeses as the depedet variable, because the formula focuses o reducig the total cost of operatios. Some data elemets have bee coverted usig the atural logarithm to esure that all the stadard regressio model assumptios are satisfied. The site characteristics that remai i the regressio model are used i the efficiecy ratig system ad the sigs of the coefficiets reveal the ature of the site characteristics. If its coefficiet is positive, the the site characteristic is ufavorable (higher values of the site characteristic are associated with higher operatig costs) ad if its coefficiet is egative, the the site characteristic is favorable (higher values of the site characteristic are associated with lower operatig costs). A quality measure represets the degree of excellece or complexity associated with the overall performace of the SD. The STARS model curretly does ot icorporate ay measures of quality or safety, due to lack of appropriate statewide data. However, it is possible that future versios of the model will iclude quality measures, such as average time spet o the bus, or appropriate safety measures such as out-of-service school bus ispectios. I that case, Q mj would be the value of quality measure m at SD j, for m = 1, 2,, M ad j = 1, 2,,. A iput orietatio is used because the objective is to fid the lowest possible cost for deliverig the SD s outputs. A variable returs to scale model is used i may DEA applicatios where the DMUs display a wide variatio i sizes. A variable returs to

4 scale model helps to reduce the bias that may be exhibited by a costat returs to scale model. A variable returs to scale model is appropriate, sice the school districts i Washigto display sigificat variatios i size. The problem is the to solve a iput-orieted DEA model with variable returs to scale, which requires the solutio of oe liear program for each SD. The liear program for SD d, d = 1, 2,,, is: Mi E d (1) subject to j=1 λ j X ij E d X id for i = 1, 2,, I (2) j=1 λ j Y rj Y rd for r = 1, 2,, R (3) j=1 λ j S kj or or = S kd for k = 1, 2,, K (4) j=1 λ j Q mj or Q md for m = 1, 2,, M (5) j=1 λ j = 1 (6) λ j 0 for j = 1, 2,, (7) E d 0 (8) Note that settig λ d = 1, λ j = 0 for j d, ad E d = 1 is a feasible, but ot ecessarily optimal, solutio to the liear program for SD d. This implies that E d *, the optimal value of E d, must be less tha or equal to 1. The optimal value, E d *, is the overall efficiecy of SD j. The left-had-sides of Equatios (2)-(5) are weighted averages, because of Equatio (6), of the iputs, outputs, site characteristics, ad quality measures, respectively, of the SDs. At optimality, that is with the λ j replaced by λ j *, the left-hadsides of Equatios (2)-(5) would be called the target iputs, target outputs, target site characteristics, ad the target quality measures, respectively, for SD d. Equatio (2) implies that each target iput will be less tha or equal to the actual level of that iput at SD d. Similarly, Equatio (3) implies that each target output will be greater tha or equal to the actual level of that output at SD d. The ature of each site characteristic iequality i Equatio (4) depeds o the maer i which the site characteristic iflueces efficiecy. For a favorable site characteristic (larger values imply higher efficiecy, o average), the less-tha-or-equal to sig is used, while for a ufavorable site characteristic (larger values imply lower efficiecy,

5 o average), the greater-tha-or-equal to sig is used. I the future, a site characteristic could be icluded usig a 0-1 idicator variable to reflect membership i a category, such as operatig i a rural area. If this category were used, whe aalyzig a school district operatig i a rural area, oly other districts operatig i rural areas would be allowed to appear with positive weight i the target SD. I such cases, the equal sig is used. Thus, Equatio (4) implies that the value of each target site characteristic will be the same as or worse tha the actual value of that site characteristic at SD d. Similarly, the ature of ay quality measure iequality i Equatio (5) depeds o the scale of the quality measure. If a quality measure is such that larger values represet higher quality, the the greater-tha-or-equal-to sig is chose, while if the quality measure is such that smaller values represet higher quality, the choice would be the less-tha-or-equal-to sig. Thus, Equatio (5) implies that the value of each target quality measure will be the same as or better tha the actual value of that quality measure at SD d. As a result, the optimal solutio to the liear program for SD d idetifies a hypothetical target SD d* that, relative to SD d; cosumes the same or less of every iput (uses less moey or buses), produces the same or more of every output (trasports as may or more studets), operates uder the same or worse site characteristics, ad achieves quality measures that are the same or better. Moreover, the objective fuctio expressed i Equatio (1) esures that the target SD d* cosumes iput levels (expeditures ad buses) that are reduced as much as possible i across-the-board percetage terms. A essetial premise is that a SD could i fact operate exactly as does SD d*. I the theory of productio, this is the assumptio, made uiversally by ecoomists, that the productio possibility set is covex. I this cotext, the productio possibility set is the set of all vectors {X i, Y r S k,q m } of iputs, outputs, site characteristics, ad quality measures such that it is possible for a SD to use iput levels X i to produce output levels Y r uder site characteristics S k while achievig quality measures Q m. The covexity assumptio assures that SD d* is feasible ad that it is reasoable to expect that SD d could modify its performace to match the performace of d*. Applyig the DEA Process i Washigto State I the academic year, the 288 school districts providig trasportatio services i Washigto State icurred $405,623,489 i operatig expeditures ad used 7,455 school buses to trasport 355,051 basic program studets ad 25,182 special program studets to ad from school. These couts are pro-rated from the Fall, Witer ad Sprig reports from the school year. Thus, the DEA model has = 288 DMUs, I = 2 iputs (each district s operatig expeditures ad buses), ad R = 2 outputs (each

6 district s basic program studet cout ad special program studet cout). There are K = 5 site characteristics idetified usig a regressio model that used the atural logarithm of operatig expeditures as the depedet variable. The followig table shows the five site characteristics that were statistically sigificat i this process ad whether higher values of each site characteristic are associated with more or less favorable operatig coditios. Also icluded as site characteristics are four biary variables, each with its ow costrait, that idicate the percetile groupig of the school district with respect to total studets trasported. The equal sig is used as the costraits for the quartile site characteristics, which implies that a school district ca oly place positive weight, λ j *, o other school districts i its ow quartile. Site Characteristic Lad Area Average Studet Distace to School Number of Locatios Served Road Miles per Square Mile of Lad Area Studets per Roadway Mile Favorable/Ufavorable Ufavorable Ufavorable Ufavorable Favorable Favorable To perform these calculatios, OSPI uses a macro developed by MPS writte i Visual Basic for Applicatios (VBA) to solve the liear programs sequetially ad save the results i a spreadsheet. The VBA uses the Solver add-i (Frotlie Systems, Ic., Iclie Village, NV) i Microsoft Excel to solve the liear programs. While both Solver ad VBA are available i all versios of Excel, the stadard versio of Solver is limited to 200 variables ad 200 costraits, which limits the size of the problems to o more tha 199 school districts ad o more tha 199 iputs, outputs, site characteristics, ad quality measures, combied. As a result, OSPI uses Premium Solver, available for purchase from Frotlie Systems, Ic. The VBA program produces several worksheets ad charts that summarize the results of the DEA. Oe worksheet shows the overall ad factor efficiecies for each district alog with actual ad target values for each iput. Aother worksheet cotais iformatio o the efficiet referece set of cohort districts ad associated weights for each district. This worksheet is imported ito STARS. The software also provides statewide frequecy tables ad charts. Coclusio While the creatio ad implemetatio of a efficiecy ratig system is challegig, the rewards ca be substatial. Besides the measurable beefit of lowered costs, a efficiecy ratig process may alter the maer i which school districts make importat strategic ad tactical decisios. A efficiecy ratig system should lead to more

7 efficietly maaged school district trasportatio operatios. By cotrast, the STARS fudig approach is based o expected costs ad will ted to lead to average, rather tha best, performace. Adapted by Alla J Joes from Usig DEA to Improve the Efficiecy of Pupil Trasportatio, Maagig Service Productivity, Iteratioal Series i Operatios Research & Maagemet Sciece Volume 215, 2014, pp Thomas R. Sexto, Ph.D. Professor ad Associate Dea College of Busiess Stoy Brook Uiversity Stoy Brook, NY Alla J. Joes Director of Studet Trasportatio Office of the Superitedet of Public Istructio Old Capitol Buildig 600 Washigto St. S.E. Olympia, WA Ady Forsyth Vice Presidet Maagemet Partership Services, Ic Traville Gateway Drive #363 Rockville, MD Herbert F. Lewis, Ph.D. Lecturer College of Busiess Stoy Brook Uiversity Stoy Brook, NY

Overlapping Generations

Overlapping Generations Eco. 53a all 996 C. Sims. troductio Overlappig Geeratios We wat to study how asset markets allow idividuals, motivated by the eed to provide icome for their retiremet years, to fiace capital accumulatio