Integrating Customer Value Considerations into Predictive Modeling

Transcription

1 Iegrag Cusomer Value Cosderaos o Predcve Modelg Saharo Rosse, Ea Neuma Amdocs Ld {saharor, ea}@amdocs.com Absrac The success of predco models for busess purposes should o be measured by her accuracy oly. Ther evaluao should also ake o accou he hgher mporace of precse predco for valuable cusomers. We llusrae hs dea hrough he example of chur modelg elecommucaos, where s obvously much more mpora o defy poeal chur amog valuable cusomers. We dscuss, boh heorecally ad emprcally, he opmal use of cusomer value daa he model rag, model evaluao ad scorg sages. Our ma cocluso s ha a o-rval approach of usg decayed valueweghs for rag s usually preferable o he wo obvous approaches of eher usg o-decayed cusomer values as weghs or gorg hem.. Iroduco Successful aalyss ad modelg of daa ca corbue grealy o he success of busesses. A key grede fulfllg hs promse s o egrae a udersadg of he rue goals, processes ad crera for success of he busess o he daa aalyss process. I prevous papers we have ackled he problems of correc cosderao of busess goals model evaluao [] ad of calculag ad ulzg cusomer lfeme value []. I hs paper we prese aoher key ssue successful modelg for busess purposes: cosderao ad cluso of cusomer-value formao all phases of he daa aalyss process: sgh dscovery, predcve modelg, model evaluao ad scorg. As a cocree example, cosder he problem of chur aalyss wreless elephoy. The predco ask s clearly a bary oe, amely o predc wheher or o a cusomer wll dscoec ad swch o a compeor. However he loss curred by correc predco depeds o he dvdual cusomer value: wrogly predcg he behavor of a low value cusomer should o worry he elecommucao compay a all, whle makg a msake o a premum cusomer ca lead o grave cosequeces. Calculag a cusomer s curre value s usually a sragh forward calculao based o he cusomer s curre or rece formao: usage, prce pla, paymes, colleco effors, call ceer coacs, ec. A example for he cusomer value ca be The facal value of a cusomer o he orgazao. Ths value ca be calculaed from receved paymes mus he cos of supplyg producs ad servces o he cusomer. Le us assume, herefore, ha we kow how o calculae cusomer value from avalable daa. I hs case, alhough he cusomer value s a kow quay, sll plays a mpora role evaluag he performace of predco models for ukow quaes such as a chur dcaor. The key sascal queso s wha s he correc use of hese cusomer values he modelg sage, o creae models ha are mos useful for he weghed loss. We ackle hs problem heorecally seco 3 ad expermeally seco 6. We cosder he wo ave exreme approaches:. Igore cusomer values he modelg sage.e. rea he problem as a sadard modelg problem. Opmze cusomer-value weghed loss o he rag daa We show ha hese are boh geerally sub-opmal ad ha a ermedae approach of usg decayed cusomer-value weghs for rag s usually preferable o boh. Addoal sages he modelg ad scorg process where cusomer values should be cosdered are: - Preseg kowledge dscovery resuls (e.g. paers) o a user. We advocae preseg boh he cusomer-value weghed ad o-weghed resuls o compleme each oher. A example ca be see seco 5. - Model evaluao. Model evaluao o usee ( es ) daa should clearly ake o accou he specfc way whch fuure loss s defed. Thus should use he cusomer values o wegh he loss he same way as he busess would. See seco 4. - Scorg for predco. Ths s he deployme sage of he model. The way whch he scorg process should use he cusomer values depeds o he way whch he resulg scores are o be used. For example, f he scores are gog o be used for choosg a segme o ru a reeo campag o, he he dvdual cusomer chur scores should be mulpled by cusomer values ad he sored o fd he opmal campag populao. I seco 6 we gve a dealed example of he use of cusomer value he varous daa aalyss sages wh he Amdocs CRM Aalycs module. I llusraes he

2 mporace ad usefuless of correcly usg cusomer value. Our formulao of he learg problem ca be erpreed as a cos-sesve learg ask where he coss dffer by sace raher ha by class. [4] meos as oe of he uder-researched ypes of cos-sesve learg. Several auhors have preseed ad dscussed smlar problems hs coex. For example, [8] arbue coss o fraud cases elecommucao, whle [] cosders doao amous as cusomer values a doao solcao drec markeg campag. Our dscusso adds o prevous work wo ma pos: Raher ha cocerag o he specfc sages, we rack he use of cusomer value hroughou he kowledge dscovery process a chur aalyss sysem. I he modelg sage, we prese our ovel approach of usg decayed cusomer values as weghs model rag. Ths approach s jusfed boh heorecally ad emprcally.. The role of cusomer value Whe buldg predco models, we usually have md a loss fuco whch descrbes he measure of accuracy we are gog o apply o our predco model. I he chur aalyss example hs loss fuco may cosder he cos of losg a cusomer because he model dd o classfy hm/her as a churer (.e. a false egave ); ad he cos of makg a eedless reeo effor o a loyal cusomer because he model classfed her/hm as a churer (.e. a false posve ). Deoe he former cos by c ad he laer cos by c. A ave modelg effor would arge fdg a model, whch mmzes he loss whe appled o fuure, usee daa,.e. mmze P( false egave) c + P(false posve) c over he populao dsrbuo. A more sophscaed vew would cosder cusomer value as a releva quay deermg he loss. I parcular, a reasoable modelg goal would be o mmze expeced cusomer-value weghed loss,.e. look for a model, whch mmzes: E{ V [ I(false egave) c + I(false posve) c]} where V s a radom varable descrbg he cusomer s value, ad I s he dcaor fuco. More geerally, we could cosder havg a oweghed loss fuco L, whch depeds o he observed resposes ad he predced oes, where our real goal would be o mmze he cusomer-value weghed expeced loss,.e. we wa o fd a model f(x) ha mmzes E [V L(Y, f(x))] () Suaos where such a value-weghed loss would be aural for he problem are acually que prevale varous areas, such as cred card fraud [], loa approval daa [9], survey aalyss [3] ad more. The modelg asks volved could be classfcao, regresso or eve o-predcve parameer esmao. 3. The use of cusomer values modelg We ow cocerae o vesgag he correc use of cusomer values (or more geerally observao mporace weghs) whe buldg predco models, from a heorecal perspecve. The geeral framework s: We have daa (x, y )= ad we also have observao value formao (v )=. These daa come from a jo dsrbuo o (X,Y,V). Our goal s o buld a model, whch mmzes some expeced loss L, weghed by observao value,.e. we wa our model f o mmze E XYV V L (Y, f(x)). The ma queso we wa o aswer, s how should we use he rag observao values v order o buld good models. More specfcally, we cosder mmzg a weghed loss fuco o our rag daa of he followg form: p v L( y, f ( x )) = () Takg p= amous o weghg he emprcal loss by he cusomer values, whle akg p=0 meas we are gorg he values compleely buldg our model. We could ceraly cosder oher famles of rasformaos for he cusomer values, oher ha he power rasformao used here, bu for clary ad brevy we lm hs dscusso o hs famly oly. As caddaes for f we wll cosder he famly of lear models our predcors f(x) = β x. Our resuls wll have broader mplcaos ha for wha s usually called lear models oly, sce may o-lear modelg echques ca be descrbed as lear models some alerave predcor space: kerel suppor vecor maches [4], boosg [3] ad logsc regresso are a few examples. We wll sar by aalyzg lear regresso,.e. he loss s squared error loss. For ha case we ca derve rgorous resuls abou he effecs of weghg o he predco error of our model. The resuls ad sghs we ga from hs aalyss wll serve us as uo for udersadg ad erpreg he expermeal resuls we ge for less mahemacally-fredly suaos. 3.. Theorecal aalyss of lear regresso Cosder he smple case of lear regresso, usg squared error loss: L( y, fˆ( x)) = ( y fˆ( x)) The famly of models we are cosderg s lear models:

3 f ˆ ( x) = βˆ x Our goal s o mmze he expeced value-weghed fuure squared error loss: E XYV [ V ( Y β X ) As (), we esmae β by mmzg weghed squared error loss o our rag daa, wh he weghs beg decayed versos of he acual observao weghs: ˆ ( β p) = arg m v β ˆ ( p ) = p ( y ] β x ), p [0,] Fdg β s sraghforward. If we deoe by Z he daa marx whose rows are he x observaos ad deoe by W(p) a dagoal * marx, wh he dagoal p v elemes beg, he s easy o oba ha ˆ ( p) ( ( ) ) β = Z W p Z Z W ( p)y Noe ha alhough hs esmaor s he same as he sadard weghed leas squares esmaor (e.g. [5]), he uderlyg sascal heory s dffere, as our case creased wegh does o correspod o decreased varace. Ths affecs our resuls heorems ad below, whch do o hold for weghed leas squares. Le us ow add a couple of addoal assumpos for he purpose of our sascal aalyss:. Our rue model s of he form Y = f(x) + ε, where f(x)=e(y x) s he bes oracle predco a hs x, ad ε has mea 0 ad varace σ.. We are eresed oly he predcos a he gve rag se of x ad v values. I oher words, he oly radomess we are cosderg s he dsrbuo of he respose. Ths s a exeso of he fxed-x assumpo usually made o faclae easer aalyss of lear models. Ths exeso s somewha problemac as he sochascy of he cusomer value plays a mpora role our dscusso. However he sghs we ga from our resuls wll rema vald eve whe we allow radom cusomer values (see dscusso below). The key o our aalyss wll be he decomposo of he expeced squared error loss o hree compoes: rreducble error, squared bas ad varace. For a dervao of hs decomposo, see for example [4]. I our case, we apply hs decomposo o he weghed squared error loss he fxed x ad v case o ge ˆ ( (deoe by f ( x ) he predco β ˆ p ) x ): ve( Y fˆ( x )) = v σ + = = v ( f ( x ) Efˆ( x )) + ve( fˆ( x ) Efˆ( x )) (3) = = Where all he expecaos are over he dsrbuo of boh he sample y s ad he fuure y s. The frs erm s he rreducble error, whch a deal predco would cur because of he here varably he respose. The secod erm s he value-weghed squared bas, somemes referred o as approxmao error. The hrd erm s he varace of he predco, or esmao error, sce: E ( fˆ( x) Efˆ( x)) = Var( fˆ( x)) The ma dea behd hs decomposo s ha he wo reducble addve compoes usually rade-off as a fuco of model complexy. Makg he model more complex (for example, addg dmesos o he x predcor vecor) decreases he bas by gvg our model more flexbly o represe he real fuco f, bu creases he varace sce we are esmag a more complex model. I our curre coex urs ou ha he decay parameer p also serves o rade-off bas ad varace. Seg p=,.e. usg he o-decayed cusomer values rag, mmzes he value-weghed bas. Seg p=0 ad gorg he cusomer values rag, mmzes he varace. These wo coceps are capured he followg heorems, whose proofs ca be foud [9]. Theorem : The varace of he f s mmzed whe p=0: ˆ (0),, ( ) ( ˆ ( p) p x Var β x Var β x) Ths heorem shows ha he varace erm (3) s uformly mmzed by gorg he cusomer values compleely. I oher words, gorg he values makes mos effce use of he rag sample o esmae he coeffce vecor β. Theorem : The average value-weghed squared bas s mmzed whe p=: = () ( p, v ( f ( x ) ( E ˆ β ) x ) v ( f ( x ) ( E ˆ β = p) ) x ) Ths heorem shows ha he bas erm (3) s mmzed by usg he o-decayed cusomer values. I oher words, usg he values makes mos represeave use of he rag sample o esmae he coeffce vecor β. Combg heorems ad gves us some uo abou he bas-varace radeoff volved cusomervalue weghed aalyss. We see ha he varace s uformly mmzed whe we ake p=0, whle he

4 average value-weghed bas s mmzed whe we ake p=. The opmal power p* wll hus be deermed by he balace bewee hese wo effecs. I geeral, f he reducble error s domaed by varace ( parcular f he model s ubased), we ca expec o ge p* 0, whle f he bas effec s much bgger ( parcular f we have a large rag sample ad o may parameers) he we ca expec o ge p*. I s mpora o oe, hough, ha he bas resuls were based o he oo ha he values rema he same o he ``fuure'' copy of he daa. Ths assumpo srogly bases our resuls owards favorg value weghg. For example, mage ha realy he cusomer values of usee cusomers are all..d from some V-dsrbuo, depede of he values of (X,Y). I ha case, heorem s compleely rreleva (sce we cao fer ay useful formao from he rag cusomer value) ad we ca easly show ha we ca do o beer ha he soluo wh p=0 erms of value weghed predco error. However, f he usee cusomer values are correlaed wh he rag se values (as would geerally be he case real-lfe daa), heorem sll has mer as a dcaor ha use of he rag se values s lkely o decrease fuure value-weghed bas. 3.. Ierpreao ad dscusso If our loss s o squared error loss, parcular f he predco ask a had s a classfcao ask, he he erms bas ad varace do have a clear mahemacal defo. However he cocepual deas of approxmao error ad esmao error sll descrbe he sources of error our model. Approxmao error ells us how good of a model our mehod could buld f had fe daa, whle esmao error dcaes how far from hs bes model are we ca expec o be our fe-sample case. Ad sads o reaso ha he rade-off we observed he squared error loss case would sll be effec: Usg cusomer values as observao weghs he modelg sage mproves approxmao because gves us a more relable descrpo of wha our fuure loss looks lke. Igorg cusomer values he modelg sage mproves esmao because gves us more effecve observaos o esmae he model wh. Ths uo s cofrmed emprcally [9], as well as expermes we made wh logsc regresso preseed seco The use of cusomer values model evaluao ad deployme Our formulao of he predco problem s rue loss () as expeced value-weghed loss mples ha evaluao of our predco models - wheher usg cross valdao, a es-se, or real-lfe performace - should use value-weghed average loss as s performace measure: v L( y, f ( x )) = Noe ha f(x) s sll a model for y, ad he way whch we buld he model (dscussed he prevous seco) affecs oly he way we esmae f(x) bu o he basc fac ha models he respose y. Thus, he observao mporace weghs sll eed o be accoued for explcly he evaluao sage. I may cases he suao s o as sragh forward as he formulao we have descrbed so far. I parcular, he rue loss may o have he form (), ad may o eve be clearly defed whe we buld he model. Ths akes us back o he ssue of correc busess oreed evaluao of predco models, wh he added complcao ha we also eed o ake cusomer value o accou. The uve way o overcome hs complcao s o use he same evaluao measure oe would choose for he problem f all cusomer values were equal, ad he modfy o ake cusomer values o accou ad perform value-weghed evaluao. For example, cosder usg evaluao measures relaed o he lf or he ROC curve (see [] for defo of he mehods ad dscusso of her equvalece ad [7] for a dscusso of he desrable properes of he ROC). These measures requre calculag scores for all es-se observaos, sorg hem ad calculag raos of coverage of respoders ad o-respoders a a predeermed cuoff po. If we wa o calculae hese measures wh regard o a cusomer-value weghed objecve, we should re-calbrae he scores by mulplyg hem by dvdual cusomer values. I he example of chur aalyss, he value-lf a percele x would he correspod o: perceage of chur cusomer value he op x% of sored ls / perceage of oal cusomer value he op x% of he sored ls. We dscuss hs model evaluao approach more deal seco 5.3, ad gve a llusraed example seco 6. I he model deployme (or scorg) sage, whe he model we have bul s acually used for supporg ad gudg busess or markeg decsos, we should smlarly cosder cusomer value ay scorg process. I parcular, cusomer propesy scores should be mulpled by cusomer value o gve expeced value loss scores. Such value-weghed scores should also gude seleco of campag populaos. 5. Value weghed aalyss Amdocs CRM Aalycs module The Busess Isgh Professoal Servces u of he CRM dvso a Amdocs alors aalycal soluos o

5 he busess problems of Amdocs cusomers he elecommucao dusry. Amog he soluos are Chur ad reeo aalyss, Fraud deeco [5], [0], Lfeme Value modelg [], Bad Deb aalyss, Produc aalyss ad more. For he purpose of chur maageme ad aalyss, we use he CRM Aalycs module, whch allows he users o perform all sages of he chur maageme process wh oe sysem. Cusomer value cosderaos come o play may dffere sages of s workflow: - Rule Dscovery - Segmeao ad Aalyss sesso - Modelg - Model Evaluao - Scorg We wll ow descrbe he ma sysem compoes ad he way whch cusomer value feaures each oe of hem. The Kowledge Dscovery process sars wh daa colleco, cleag, pre-processg ad rasformg o ge a fla fle whch s he pu o he aalyss process. I our coex we eed o make sure ha cusomer value varables are cluded he pu daa, parcular ha a cusomer value calculaed feld has bee added o he fla fle. Aalycs offers a flexble value calculao erface, based o he formao he cusomer daa-mar. 5.. Rule dscovery, segmeao ad aalyss The frs aalyc sep Aalycs s rule dscovery. The algorhms used are decso ree ad rule duco. Is oupu s a colleco of rules (a.k.a. paers) descrbg cusomer segmes wh srog edecy owards chur or loyaly. These rules are preseed o he aalys, who ca vew ad erpre hem, modfy hem ad add ew rules ha represe busess kowledge o capured by he auomaed dscovery. Fgure gves a example of a rule as vewed he applcao. I coas he codos defg he segme hs case: a leas call ceer coacs ad specfc hadses (whch are old ad usophscaed). I also dsplays varous graphc ad umerc llusraos of s sascs. For example, he coverage feld ells us wha populao sze hs segme covers - covers 5 cusomers he sample, whch are 8.4% of he oal cusomer sample. The expeced h deoes he esmaed chur probably, whch s 7.%. All of hese sascs gore cusomer value,.e. are based o coug cusomers regardless of her value. Fgure. Rule vew by umber Fgure. Rule vew by value However o gve he aalys a relable pcure of he moeary mpac of hs segme s chur behavor seems eresg o gve a vew of he segme s sascs calculaed by value raher ha by umber. Fgure shows he same segme as fgure bu usg he by value vew sead of he by umber vew. Cosder he coverage feld fgure. I ells us ha he combed cusomer value of all cusomers hs segme s 5,499.53, whch s acually 5.7% of he oal cusomer value our cusomer sample. We see ha hs segme covers 8.4% of he cusomer populao bu oly 5.7% of he oal cusomer value. Also of eres s he dfferece he accuracy feld bewee he wo paels. I ells us ha 7.% of he cusomers hs segme are churers, bu ha hese represe 6.5% of he oal cusomer value for all cusomers he segme. To summarze our coclusos abou hs segme of cusomers ha have usophscaed hadses ad have coaced he call ceer a leas wce he pas moh from hs dual vew: - Ther average cusomer value s 50% less ha ha of he geeral populao - Wh hs segme, he o-chur cusomers ed o have hgher cusomer value ha he churers (cdeally, ha dfferece s abou 50% as well a easy calculao based o he umbers gve above). Ths example llusraes he mer combg a sadard cusomer-based vew ad cusomer-value based vew, whch ogeher allow us o udersad he chur behavor of our cusomers ad how relaes o reveue moveme. The fal oupu of he aalyss sage s a se of useful ad accurae segmes ha are used as pus o he acual model buldg sage. 5.. Model buldg The ma modelg ool Aalycs s logsc regresso, whch akes as pu boh he orgal varables ad bary feaures represeg he rules geeraed he aalyss sage. As we dscussed seco 3, he ma sascal focus of hs paper s how o correcly rasform he cusomer values o observao weghs he model rag sage. To udersad how logsc regresso ca be modfed o ake observao weghs, cosder he

6 geerc formulao (). Sce logsc regresso seeks o maxmze he bomal log lkelhood of he daa, we ca formulae s as a loss mmzao problem usg mus bomal log-lkelhood as our loss. The weghed crero aalogy o () for logsc regresso would hus be o mmze: w [ log( ˆ ) ( )log( ˆ y p + y p )] = (4) p, ˆ log ( ˆ ( p) w ) where = v p = β x, ad β s he vecor of coeffces whch we am o esmae. No surprsgly, hs formulao s equvale o havg w decal copes of observao our rag daa, whch also gves us a dea how we could esmae β usg mehods ha are esseally decal o hose used for o-weghed daa (see [6] for a dscusso of hese mehods). I should be emphaszed ha he oupu of logsc regresso wh hs weghed crero s sll a model, whch assgs a chur probably o each cusomer. All we have chaged s he way whch hese probables are esmaed. The logsc regresso compoe Aalycs uses he weghg rule w = v ½ for buldg predco models. Ths rule of humb s a resul of exesve expermes o varous daa ses ad represes a bas-varace compromse, whch eds, geeral, o perform reasoably. A alerave approach could have bee o make he power p a user seleced ug parameer, whch s problemac due o he dffculy erpreg hs parameer. Aoher alerave would be o add p as aoher parameer o he model opmzao process, whch s dffcul compuaoally ad preses addoal echcal dffcules Model evaluao ad scorg The ypcal use of a chur predco model s o for classfcao bu raher for wo que dffere asks: - Seleco of populaos for pro-acve reeo campags. Ths eals selecg a small par of he oal populao (ypcally a few perces) o make a cocered reeo effor o. The seleco crero ca be by populao segme or may jus ask for a ls of cusomers who represe he mos value a rsk o whom he reeo effor s mos warraed (we wll cocerae o he secod opo). - Maeace of a dvdual chur propesy score for all cusomers, bu parcular he mpora ad valuable cusomers. These scores may or may o correspod o acual probables ( may cases hey are jus he form of qualave levels of rsk), alhough geg good probably esmaes s ceraly advaageous ay case. For he frs ask a lf a x% measure would be approprae f he desred cuoff po s kow advace, oherwse a global lf measure lke area uder lf curve may be warraed. For he secod ask, seems lke a lkelhood measure may acually be he mos approprae oe, alhough msclassfcao rae ad he oal area uder he lf curve may be a reasoable surrogaes. Model evaluao Aalycs uses he lf measure, dsplayg umercally he model s lf a a large umber of cuoff pos, dsplayg he resulg lf curve ad he area uder he curve. Cusomer value cosderaos are egraed o he evaluao as descrbed seco 4. The es se scores are calculaed as a produc of propesy o chur gve by he model ad he (kow) cusomer values: Score = v * p (5) They are he sored descedg order ad he valuelf a x% s calculaed as: vi{cusomer s a churer}/ v I{Cusomer s a churer} I x = v / v I x = Where I x s he se of es-se observaos whose scores are he op x% of scores The model evaluao compoe Aalycs allows boh sadard lf evaluao of he model ad value-lf evaluao. As we observed he aalyss sage, he dffere vews ca gve dscly dffere resuls hs sage oo. Fgure 3 shows a par of lf curves, for he same model o he same es daa. Oe s a regular lf curve ad he oher s a value-lf curve. The wo curves are very dffere ad represe he esseal dfferece he wo evaluao mehods. See also he case sudy seco 6. Fgure 3. Regular lf (o boom) ad value lf (o op) A smlar wo-vews approach s ake by he applcao he scorg compoe, whch s used for predco oce he model s deployed. Cusomer chur propeses are esmaed usg he model ad value

7 propesy scores are calculaed from hem as (5) (recall ha he cusomer value s a fuco of he predcors ad s herefore kow eve for real predco asks). The busess user ca ask for he propesy scores, he value-propesy scores or boh, ad ca ulze eher oe selecg populaos for reeo campags or oher campags. 6. Case sudy We ow descrbe a case sudy of chur aalyss performed wh cusomer value cosderaos o real daa, usg he Amdocs CRM Aalycs. The daa source s a elecommucao servce provder ad he daa cosss of aroud 400 predcors. The daa was spl o a rag sample se coag 500 chur observaos ad 500 loyal observaos ad o a es sample se coag 750 chur observaos ad 750 loyal observaos. The cusomer value formula was defed Aalycs, usg several felds avalable he daa. The frs sage was o perform kowledge dscovery by rug he rule dscovery mechasm descrbed seco 5.. Fgure ( ha seco) dsplays oe of he rules auomacally dscovered by he applcao. The oal umber of rules dscovered was. The ex sage was o cosruc logsc regresso models usg a combao of bary varables represeg hese rules ad 50 of he orgal varables as predcors. Followg our approach of usg decayed cusomer values as observao weghs for modelg, we proceeded o buld several weghed logsc regresso models for hs daa.. The value rasformaos we used were:. No-decayed weghs: usg he rag cusomer values v as weghs for he logsc regresso. Square roo decay: usg v 0.5 as weghs for he logsc regresso rag. Ths s he defaul approach of Aalycs, as dscussed seco Srog decay: usg he quadruple roos v 0.5 as weghs for logsc regresso rag. 4. Igorg cusomer values compleely he modelg sage (.e. each observao has wegh ). The same rag daa se, as descrbed above, was used for buldg all models. The models were evaluaed o he leave-ou es sample from he same populao. The evaluao measure used was he lf, ad we calculaed s value a a rage of cuoff pos. We calculaed boh he value-lf as descrbed seco 5.3 ad he sadard, o-value-weghed lf. Table ad fgure 4 show he resuls for he value-lf evaluao ad able ad fgure 5 show he resuls for he sadard lf evaluao. 80% 60% 40% 0% 0% % 3% 5% 7% 9% % No-Decayed Igore Value 3% 5% Fgure 4. Value-lf resuls 7% 9% % 3% 5% 7% 9% Square-Roo Decay We observe ha for he value-lf calculaos, he model usg o-decayed values s bes for very low cuoff pos, bu for he vas majory of cuoff pos cosdered, he wo decayed models (models,3 above) seem o do sgfcaly beer ha he wo exreme models. I he wo ables we ca also see 95% cofdece ervals for he varous value-lf values a he varous cuoff pos, calculaed usg he hyper-geomerc approxmao descrbed []. We see ha he decayed models seem o do sgfcaly beer ha he wo exreme oes may cuoff pos. For he sadard lf calculaos our resuls seco 3 would lead us o expec he o-weghed model (model 4) o be he bes modelg approach. The acual dffereces we observe able ad fgure 5 are less srkg ha hose for he value weghed evaluao. We observe ha he srogly decayed model ad he o-weghed model (models 3,4) geerally perform bes, alhough he dffereces for much of he rage are o very bg. The o-decayed model, whch s supposed o be leas approprae for hs evaluao, does gve sgfcaly worse lf ha models 3,4 for he hgher cuoff pos cosdered, as we ca observe able. Table. Value-lf resuls Percele No-Decayed Square-Roo Decay Srog Decay Igore Value % 4.9% (.6,7.3).4% ( 9.3,3.5).5% (0.3,4.8) 3.9% (.5, 5.3) 4% 9.% (6.7,.8) 4.0% (.4,6.7) 6.6% (3.9,9.3) 8.7% ( 6.8,0.6) 0% 36.9% (34.,39.7) 45.% (4.5,47.7) 47.5% (44.9,50.) 38.9% (36.,4.6) 0% 6.4% (59.3,63.6) 65.8% (63.8,67.7) 67.3% (65.4,69.) 6.7% (60.6,64.8)

8 30% 74.% (7.6,75.8) 79.8% (78.5,8.) 77.7% (76.3,79.) 74.% (7.5,75.7) Table. Sadard lf resuls Percele No-Decayed Square-Roo Decay Srog Decay Igore Value %.% ( 9.0,3.) 9.8% ( 7.8,.8) 8.7% ( 6.8,0.6) 8.8% ( 6.9,0.7) 4% 6.4% (4.0,8.8) 5.5% (3.,7.9) 7.3% (4.8,9.8) 5.% (.7,7.5) 0% 36.6% (33.9,39.3) 39.% (36.4,4.8) 39.4% (36.7,4.) 36.7% (34.0,39.4) 0% 54.0% (5.6,56.4) 55.5% (53.,57.9) 60.% (57.9,6.3) 59.6% (57.4,6.8) 30% 69.6% (67.8,7.4) 75.% (73.7,76.7) 74.0% (7.4,75.6) 73.6% (7.0,75.) 80% 60% 40% 0% 0% % 3% 5% 7% 9% % 3% 5% 7% 9% % 3% 5% 7% $50k dfferece mohly he profably of reeo campags, cosderg boh he effec of mssg valuable churers ad wasg reeo effors o ovaluable oes or o cusomers who do o ed o chur. We feel ha hs case sudy cofrms he ma pos of our exposo:. Decay of rag cusomer values s beefcal for value-weghed predco. Value-weghed preseao ad evaluao of resuls s mpora for buldg beer predco models for busess purposes No-Decayed Square-Roo Decay Igore Value Fgure 5. Sadard lf resuls We ca emphasze he dfferece bewee he valuelf ad sadard lf resuls by observg hese comparso pos:. The model bul usg o-weghed daa (model 4) s o beer ha eher of he wo decayed models for praccally every cuoff po cosdered he value-lf evaluao (Table ). For he sadard lf evaluao model 4 s a he very leas compeve, somemes beer ha boh model ad model 3.. Comparg model ad model 4 s performace for he wo evaluaos show us ha o he value-lf evaluao hey behave que smlarly ad ed o do worse ha he decayed models. However o he sadard lf evaluao, model s clearly much less approprae ha model 4. Ths s wha we would expec heorecally, ad hs s wha we observe pracce. Lookg a our resuls, seems a bgger es se may have bee useful beer dffereag he varous models. Whle he dffereces her performace may be a few perces oly, cosder ha hese models are o be deployed o cusomer daabases coag mllos of cusomers, amog hem es of housads of churers each moh. Smple ROI calculaos show ha a dfferece of 3% value-lf ca easly correspod o 7. Summary I hs paper we have ackled he praccal use of cusomer values hroughou he daa aalyss process, parcular for chur aalyss elecommucaos. We have llusraed ha he modelg sage s beefcal o choose a rasformao of he rag daa cusomer values as weghs for learg, ad dscussed he correc use of cusomer values evaluao, scorg ad sgh dscovery. We have show how hese coceps are appled pracce he Amdocs CRM Aalycs ad llusraed her performace o real-lfe chur daa. There are may addoal eresg ad releva quesos ha come up whe cosderg daa wh observao mporace weghs, such as: - Wha are good modelg approaches (or algorhms) for buldg value-weghed predco models? - How ca we adjus sadard modelg ools o ake value o cosderao (hs problem has bee wdely addressed he mache learg leraure) - Wha ca we do whe cusomer values are o cera bu we have approxmae values? Wha ca we do whe hey are kow oly he fuure? 8. Refereces [] Cha, P. K., ad Solfo, S. J. (998) Toward Scalable Learg wh No-uform Class ad Cos Dsrbuos:

9 A Case Sudy Cred Card Fraud Deeco, Proc. KDD-98, pp [] Elka, C. (000) Cos-Sesve Learg ad Decso- Makg whe Coss Are Ukow. Workshop o Cos- Sesve Learg of he Ieraoal Coferece o Mache Learg (ICML'000), Saford Uversy, Calfora, Jue 000. [3] Kor, E.L., Graubard, B.I. (995) Examples of Dfferg Weghed ad Uweghed Esmaes from a Sample Survey. The Amerca Sasca, 49:9-95. [4] Hase, T., Tbshra, R., Fredma J. (00). The Elemes of Sascal Learg. Sprger. [5] Murad, U., Pkas, G. (999). Usupervsed Proflg for Idefyg Supermposed Fraud. PKDD-99: 5-6 [6] McCullagh, P., Nelder, J.A. (989). Geeralzed Lear Models. Chapma & Hall, secod edo, 989 [7] Provos, F.; Fawce, T. (997). Aalyss ad Vsualzao of Classfer Performace: Comparso uder Imprecse Class ad Cos Dsrbuo. I Proceedgs of KDD-97, pp Melo Park, CA: AAAI Press. [8] Provos, F., Fawce, T. (998). Adapve fraud deeco. Daa Mg ad Kowledge Dscovery, (3). [9] Rosse, S. (003). Buldg predco models for daa wh observao mporace weghs. I preparao. Draf avalable from: wwwsa.saford.edu/~saharo/papers/vwpaper.ps [0] Rosse, S., Murad, U., Neuma, E., Ida, I, Pkas, G.(999). Dscovery of Fraud Rules for Telecommucaos - Challeges ad Soluos. KDD- 99: [] Rosse, S., Neuma, E., Eck, U., Vak, N., Ida, I.(00). Evaluao of predco models for markeg campags. KDD-00: [] Rosse, S., Neuma, E., Eck, U., Vak, N., Ida, I.(00).Cusomer lfeme value modelg ad s use for cusomer reeo plag. KDD-00. [3] Rosse, S., Zhu, J., Hase, T. (00). Boosg as a regularzed pah o a maxmum marg classfer. Techcal repor, Dep. of Sascs, Saford Uv. [4] Turey, P.D. (000). Types of cos ducve cocep learg. Workshop o Cos-Sesve Learg a he Seveeeh Ieraoal Coferece o Mache Learg (WCSL a ICML-000), Saford Uversy, Calfora. [5] Wesberg, S. (985). Appled Lear Regresso, Joh Wley ad Sos, Ic.