A Common Weighted Performance Evaluation Process by Using Data Envelopment Analysis Models

Size: px

Start display at page:

Download "A Common Weighted Performance Evaluation Process by Using Data Envelopment Analysis Models"

Abner Nelson
5 years ago
Views:

1 A Commo Weighted Performace Evaluatio Process by Usig Data Evelopmet Aalysis Models Chig-Hsiag Lai & Meg-Yig Wei Departmet of Iformatio Maagemet Chug Sha Medical Uiversity aichug aiwa Abstract he fiace literature searches for a li betwee productio ad performace cotrollig for variables such as sales firm size employee umber etc. that ifluece productio pacages. he performace idices are desiged accordig to the resultig effects ad determie whether the performace model is appropriate or ot ad/or whether the performace of system is good or ot. I this paper a process based o data evelopmet aalysis (DEA) is developed to evaluate ad ra the relative importace of ey performace idices (KPIs). he relative importace of each KPI is evaluated by performace loss measure ad each KPI is weighted accordig to the measure. he the relative performace of each uit is the ratio of weighted output to weighted iput based o the commo weights. Keywords data evelopmet aalysis (DEA) raig ey performace idex (KPI) commo weights efficiecy. I. INRODUCION Decisio-maig problems ivolve both quatitative ad o-quatitative factors. he o-quatitative factors are ot usually well defied or are subectively determied by the decisio-maer. Such factors caot be icluded i the mathematical models while the quatitative factors are modeled as multiple obective liear programmig (MOLP). he coefficiets i MOLP may obtaiable well defied or ot sesitive to the fial solutio. A example of MOLP may be proects of govermet ivestmet i which the miimizatio obective fuctios (iputs) may be mapower machies costructio costs operatio costs other cotrollable costs ad ucotrollable costs while the maximizatio obective fuctios (outputs) may be reveues rate of populatio growth ad growth of ecoomic improvemet. I real world problems there exist peer groups of decisio-maig uits (DMUs) such as corporatios hospitals ad cities etc. which use multiple resources to geerate multiple products. or example hospitals may use labor iputs of physicias urses ad techicias ad capital iputs of medical equipmets ad beds to produce service of ipatiet ad outpatiet care ad researches. May studies focus o evaluatig the allover performace of DMUs based o their resources cosumed ad products geerated. It is expected that multiple iputs ad outputs could be trasformed ito oe performace measure such as the ratio of aggregated outputs to aggregated iputs. his provides the performace of DMUs to be evaluated ad raed by the correspodig measure. he sigificace of various performace idices (iputs ad outputs) affect such id of performace measure of DMUs so the weights for various idices are eeded to determie i order to obtai the measure. he performace idices are desiged accordig to the resultig effects ad determie whether the performace model is appropriate or ot. Buyuoza [] proposes a measure amely the e-maretplace success idex (e-msi) to quatify the performace of e- Maretplace. As the performace idices are collected ad selected by some maagerial iformatio ad experiece. he sigificace of performace idices are set to determie whether the performace evaluatio system is good or ot. Whe the ey performace idices (KPIs) are idetified by the expert opiios or other maagerial iformatio the relative performace of each DMU is higed o the weights of the KPIs. here are some ways to determie the prioritizatio (weights) of KPIs. he aalytic hierarchy process (AHP) developed by Saaty [2] is desiged to solve complex multi-criteria decisio problems. AHP requires the decisio maer to provide udgmets about the relative importace of each criterio ad the specify a preferece for each DMU usig each criterio. he Delphi method was coceived as a group techique whose aim was to obtai the most reliable cosesus of opiio of a group of experts by meas of a series of itesive questioaires with cotrolled opiio feedbac [3]. It is a method of structurig commuicatio i a group of people who ca provide valuable cotributios i order to resolve a complex multi-criteria problem [4]. So we could adopt expert opiios through Delphi method to specific the weights. Moreover we could also use statistical method such as pricipal compoet aalysis (PCA) to determie the weights [5]. he above methods all provide the relative weight correspod to each KPI. Data Evelopmet Aalysis (DEA) is a robust ad valuable methodology for the frotier estimatio [6]. Based o mathematical programmig techiques it is particularly suited to estimatig multiple iput ad output productio correspodece. I the last two decades DEA has become a popular method for aalyzig the efficiecy of various orgaizatio uits [7] which differ both i the quatities of iputs they cosume ad i the outputs they produce ad does ot require ay subective or ecoomic /07/$ IEEE 827

2 Proceedigs of the 2007 IEEE IEEM parameters (weights prices etc.). May studies have bee cocered with the efficiecy of productio. It is clear that DEA is ow playig a wider role i operatioal research/maagemet sciece area. I particular DEA approaches have assumed importat status withi the toolits of ivestigators cocered with multiple criteria decisio-maig [8 9]. I this research we propose a process based o the optimizatio techique DEA to evaluate the relative importace of KPIs ad determie relative weight of each KPI. he the relative performace of all DMUs based o commo weights is follows. he ext sectio reviews papers related to select ad ra KPIs. Sectio three itroduces the aalysis for usig DEA to prioritize ad weight the give KPIs. Sectio four uses a data set comprised 20 corporatios of persoal computer (PC) idustry i aiwa to illustrate this aalysis. Coclusio ad discussio are preseted i sectio five. II. LIERAURE REVIEW Shashua & Goldschmidt [0] develop a formal aalytical model to costruct a idex for evaluatig the allover performace of a firm relatively i the same idustry. he idex also serves as a idicator of ecoomic progress. Philbi & Reegar [] idicate that a egieerig performace idex method ca be used to provide performace measuremet ad feedbac for a etire egieerig departmet idividual egieers idividual performace categories idividual proects ad idividual elemet tas uits. he world becomes more complex as we eter the iformatio age. We fid that almost every importat realworld problem ivolves more tha oe obective ad decisio maers fid it imperative to evaluate solutio alteratives accordig to multiple criteria [2 3]. We ow eed to exted the sigle performace idex problems to the multiple idices problems. Glado et al. [4] preset that aggregate idices reflect dimesios of hospital fiacial performace ad simplify the iformatio i fiacial ratios for the decisio-maig aid. heir study reviews the developmet ad use of hospital fiacial performace measures ad lays the groudwor for research ito derivig a multi-dimesioal measure. he KPIs are critical assessmet. Chag & Yeh [5] use AHP DEA ad PCA to build a evaluative model for selectig ad raig KPIs i Liquid Crystal Displayer (LCD) idustry i aiwa. Chi et al. [5] idetify the criteria ad factors for maagig supplier quality (MSQ) through literature review ad a mail survey of maufacturers i Hog Kog. Usig the AHP idustry experts were ivited to determie the relative weights of these criteria ad factors. As the KPIs are idetified by expert opiios the prioritizatio or the relative weights of KPIs is a importat issue i trasformig multiple idices ito a aggregate performace measure. DEA is also a popular techique that eables multiple iputs ad outputs are trasformed ito oe performace measure. If the performace reduces due to remove a specific KPI we could compare the quatity of performace differeces amog all KPIs ad ra these values to determie the prioritizatio of the KPIs. he methodology of usig DEA to prioritize PKIs is developed i the ext sectio. III. MEHODOLOGY DEA was itroduced ito the operatios research by Chares et al. [6] which exteded the wor of arrell [6]. DEA is a methodology based o mathematical programmig ad used for assessig the relative efficiecy of observed DMUs with the same multiple iput ad output. his study employs the oparametric approach to evaluate the sigificace differeces i KPIs based o the productio frotier literature. DEA is adopted to proect KPIs to performace frotier with miimal assumptio o performace correspodece. Suppose a set of DMUs umbered = is uder evaluatio. or DMU it cosumes x i i I={ m} resource to produce y r r O={ s} products. he arrell [6] efficiecy measure for a specific DMU say DMU based o all KPIs i I ad O is obtaied by usig the followig model: s.t. = mi θ = = λ x λ y θ x y λ 0 =. i r r i i = m; r = s; I case of = DMU is o the performace frotier ad is called efficiecy while < DMU is iside the performace frotier ad is called iefficiecy. I geeral suppose M ad S are subsets of idex sets I ad O respectively. I the correct versio we compute: ( M S) = mi θ s.t. = = λ x λ y θ x y λ 0 =. i r r i i M; r S; or each DMU which differs from model () i that our efficiecy measure is costructed from KPIs i M ad S oly. A iterestig result emerges from further ispectio of the solutios to of a optimal solutio to (M S) ad () (2). Because (M S) is also a feasible 828

3 Proceedigs of the 2007 IEEE IEEM solutio to i the aalysis of corporatio. Hece (M S) for all idex subsets M I ad S O. We ow itroduce a formal defiitio of a performace loss. Defiitio. or corporatio PL (M S) = (M S) (3) is a measure of performace loss due to remove KPIs from I ad O to M ad S. PL is a measure of performace loss because is reveals whether the best practices of corporatio are differeces betwee allover KPIs ad a deletio of some KPIs. It is obviously the measure of performace loss has the followig properties: ) PL 0. Sice (M S) 0 the result follows straightforward. 2) or corporatio if (M S) = PL = 0. A. Weightig Iput Idices or a specific iput idex we tae M =I \{} ad compute (M O). he the measure PL (M O) could be regarded as the performace loss while iput is abset i the idex set. I other words PL (M O) measure could be regarded as the efficiecy cotributio to DMU while iput is added ito idex set M. If the performace loss for deletig iput is greater tha the measure for deletig iput we coclude that the ifluece of iput should be larger tha the ifluece of iput o the performace of DMU. hat is iput is more importace tha iput with respect to DMU if PL (M O) > PL (M O). However we are iterested i the relative weights (importace) of KPIs for evaluatig all DMUs or possibly for evaluatig a subgroup of DMUs i the same idustry. he rule to determie the relative weights of KPIs is developed as the followig: Rule. he importace of iput is prior to the importace of iput with respect to all DMUs if PL ( M O) > PL( M O). (4) Where PL( M O) = Mea { PL ( M O) = }. Based o Rule the relative weight v of iput is assiged to equal to the measure PL ( M O). Hece we have: v = PL( M O) i = m. (5) B. Weightig Output Idices or a specific output idex we tae S =O \{} ad compute (I S ). he the measure PL (I S ) could be regarded as the performace loss while output is removed from the idex set. I other words PL (I S ) measure could be regarded as the efficiecy cotributio to DMU while iput is added ito idex set S. If the performace loss for deletig output is greater tha the measure for deletig output we coclude that the ifluece of output should be larger tha the ifluece of output o the performace evaluatio of DMU. hat is output is more importace tha output if PL (I S ) > PL (I S ). Rule 2. he importace of output is prior to the importace of output with respect to all DMUs if PL I S ) > PL( I S ) (6) ( Where PL( I S ) = Mea { PL ( I S ) = }. Based o Rule 2 the relative weight µ of output is assiged to equal the measure PL I S ). Hece we have: ( µ = PL( I S ) r = s. (7) C. Performace Evaluatio i Commo Weights I order to discrimiate the exact ifluece of each performace idices we eed to re-scale the data of idices that eable the weighted values of idices could reflect the idex importace ad affect the performace i a suitable quatity. he iputs ad outputs of DMUs are re-scaled as follows: xˆ i = xi xi i = m = ; = (8) yˆ r = yr yr r = s =. = his re-scalig provides the total weighted ifluece of idex is greater tha idex if weight of idex is greater tha idex. herefore the relative performace of each DMU is give by: s θ ˆ ˆ = µ r yr vi xi =. (9) r= m i= IV. PERORMANCE O AIWAN S PC INDUSRY A data set comprises with 20 corporatios (DMUs) of the PC peripherals ad compoets idustry (D-D20) idustry i 2005 [7] i aiwa is listed i able I. Suppose 4 iput KPIs ad 2 output KPIs have bee idetified accordig to the expert opiios. he iput output set is as follows: Iput: Assets (x ) Shareholders equity (x 2 ) Capital (x 3 ) ad Employee umber (x 4 ). Output: Sales (y ) ad Profit (y 2 ). We are iterested i prioritizig the importace ad settig the relative weights of the give KPIs for 829

4 Proceedigs of the 2007 IEEE IEEM evaluatig the relative performace of all DMUs. Our obectives iclude: (i) prioritizig KPIs with respect to a specific DMU (ii) prioritizig KPIs with respect to the 20 DMUs i PC idustry (iii) evaluate the relative weights of each iput/output idices ad (iv) evaluate the relative performace of all DMUs i commo weights of KPIs. ABLE I DAA O 20 PC PERIPHERALS CORPORAIONS DMU Iput Output x x 2 x 3 x 4 y y 2 D D D D D D D D D D D D D D D D D D D D Uit: hudred millio New aiwa Dollars (NDs). All of the 20 corporatios are evaluated by model () ad (2). he computatioal results are preseted i able II. he origial efficiecy scores i the secod colum are evaluated by usig model (). he other values are the measure of performace loss PL of each KPI to DMU which is the differece value betwee through deletig KPIs from I or O. ad (M S) We first cosider D4 ad D3 as the target DMUs to illustrate result of obective (i). he prioritizatio of KPI is obtaied by raig the PL values. hus we have the importace ras as: x > x 3 > x 2 = x 4 ad y 2 > y for D4 while x > x 4 > x 3 > x 2 ad y > y 2 for D3. I case of prioritizig KPIs for the PC idustry the prioritizatio of KPI is obtaied by raig the mea PL values. Obective (ii) results that: x > x 4 > x 3 > x 2 ad y > y 2 with respect to all DMUs. or obective (iii) the relative weights of idices are: v =0.253 v 2 = v 3 =0.006 v 4 = µ =0.857 ad µ 2 = ABLE II HE ORIGINAL SCORES AND PERORMANCE LOSS O EACH INDEX DMU Score PL of Iputs PL of Outputs ( ) x x 2 x 3 x 4 y y 2 D D D D D D D D D D D D D D D D D D D D Mea ABLE III HE RANK AND RELAIVE PERORMANCE DMU Score CCR Weighted Performace New Differece ( ) ra a iput output θ ra b of ra c D D D D D D D D D D D D D D D D D D D D a he CCR ra is based o origial CCR scores. b he ew ra is based o our relative performace θ. c he differece betwee CCR ra ad ew ra where + idicates improvemet ad idicates deterioratio respectively. 830

5 Proceedigs of the 2007 IEEE IEEM urther the result of obective (iv) could be obtaied from Eq. (9) ad is preseted i able III. All DMUs are raed accordig to the relative performace. here are slightly differece ( 3~+3) betwee CCR ra ad our ew performace ra for most of DMUs. But some DMUs have a strogly chaged i ra. he most sigificat are that: D9 move from ra to ra 9 ad D5 move from ra to ra 6. able III has also show that both D6 ad D7 are deteriorated 4 ras. By usig our evaluatio process of commo weights it results that the 7 ad 2 DMUs are improved deteriorated ad uchaged their ras respectively. V. CONCLUSION AND DISCUSSION he DEA methodology possesses may valuable applicatios. Istead of pre-assigig weight to each KPI idividually free the weight lets us evaluate the DMUs i each oeself best practice coditio. Our obective is to weight all KPIs based o their performace loss measures the re-evaluate DMUs based o the give weights. As the ra of KPIs is determied corporatio or idustry ca follow the iformatio to improve their performace effectively. or example assets decremet is prior to employee umber decremet i order to improve the performace of D4 while profit augmeted is prior to sales icremet i performace improvemet of D6. However the PC idustry should first decrease their assets ad icrease sales to improve the idustry performace effectively ad efficietly. he commo weighted evaluatio process is more realized the real-world problems. It could rule out the ureasoable efficiet DMUs from the efficiecy set ad provide a full raig evaluatio for the decisio problems. So this process will ehace the fie quality of fial decisio. he geeral multi-idices models are used to measure the efficiecy of some difficult public service policies or o-profit issues e.g. road costructio uclear power plat sited ad locatio of a airport etc.. Our process may provide a better aspect to the relative efficiecy. REERENCES [] G. Buyuoza A success idex to evaluate e- Maretplaces Productio Plaig & Cotrol vol. 5 o. 7 pp [2]. L. Saaty he Aalytic Hierarchy Process McGraw-Hill New Yor 988. [3] N. Daley ad O. Helmer A experimetal applicatio of the Delphi method to the use of experts Maagemet Sciece vol. 9 o. 3 pp [4] H. Listoe ad M. uroff he Delphi Method: echiques ad Applicatios Addiso-Wesley New Yor 975. [5] C. L. Chag ad C. H. Yeh he ey performace idices model of product data maagemet system for LCD idustry Iteratioal Joural of Electroic Busiess Maagemet vol. 3 pp [6] A. Chares W. W. Cooper ad E. Rhodes Measurig the efficiecy of decisio maig uits Europea Joural of Operatioal Research vol. 2 pp [7] M. Norma ad B. Stoer Data Evelopmet Aalysis: the Assessmet of Performace Joh Wiley & Sos New Yor 99. [8]. Joro P. Korhoe ad J. Walleius Structural compariso of data evelopmet aalysis ad multiobective liear programmig Maagemet Sciece vol. 44 o. 7 pp [9] J. Saris A comparative aalysis of DEA as a discrete alterative multiple criteria decisio tool Europea Joural of Operatioal Research vol. 23 pp [0] L. Shashua ad Y. Goldschmidt A idex for evaluatig fiacial performace he Joural of iace vol. 29 o. 3 pp [] L. Philbi ad B. Reegar ederal Express Corp. develops performace measuremet idex for idirect sector Idustrial Egieerig vol. 2 o. 8 pp [2] R. L. Keeey ad H. Raiffa Decisio with multiple obectives: prefereces ad value tradeoffs Joh Wiley & Sos New Yor 976. [3] R. E. Steuer Multiple criteria optimizatio: theory computatio ad applicatio Joh Wiley & Sos New Yor 986. [4] G. L. Glado M. Coute K. Holloma ad J. Kowalczy A aalytical review of hospital fiacial performace measures Hospital & Health Services Admiistratio vol. 32 o. 4 pp [5] K. S. Chi I. K. Yeug ad K.. Pu Developmet of a assessmet system for supplier quality maagemet he Iteratioal Joural of Quality & Reliability Maagemet vol. 23 o. 7 pp [6] M. J. arrell he measuremet of productive efficiecy Joural of the Royal Statistical Society Series A vol. 20 pp [7] CommoWealth Magazie Co. CommoWealth Magazie vol CommoWealth Magazie Co. aipei aiwa

A New Approach to Obtain an Optimal Solution for the Assignment Problem

A New Approach to Obtain an Optimal Solution for the Assignment Problem Iteratioal Joural of Sciece ad Research (IJSR) ISSN (Olie): 231-7064 Idex Copericus Value (2013): 6.14 Impact Factor (2015): 6.31 A New Approach to Obtai a Optimal Solutio for the Assigmet Problem A. Seethalakshmy