Level estimation, classification and probability distribution architectures for trading the EUR/USD exchange rate

Transcription

1 Absrac Level esmaon, classfcaon and probably dsrbuon archecures for radng he EUR/USD exchange rae by Andreas Lndemann * Chrsan L. Duns * Paulo Lsboa ** ( * Lverpool Busness School, CIBEF and ** School of Compung and Mahemacal Scences, Lverpool John Moores Unversy) June 2003 In [Duns and Wllams, 2002, 2003] he auhors have shown he superory of a Mullayer percepron newor (MLP), ouperformng s benchmar models such as a movng average convergence dvergence echncal model (MACD), an auoregressve movng average model (ARMA) and a logsc regresson model (LOGIT) on a Euro/Dollar (EUR/USD) me seres. The movaon for hs paper s o nvesgae he use of dfferen neural newor archecures. Ths s done by benchmarng hree dfferen neural newor desgns represenng a level esmaor, a classfcaon model and a probably dsrbuon predcor. More specfcally, we presen he Mul-layer percepron newor, he Sofmax cross enropy model and he Gaussan mxure model and benchmar her respecve performance on he Euro/Dollar (EUR/USD) me seres as repored n [Duns and Wllams, 2002, 2003]. As urns ou, he Mul-layer percepron does bes when used whou confrmaon flers and leverage, whle he Sofmax cross enropy model and he Gaussan mxure model ouperforms he Mul-layer percepron when usng more sophscaed radng sraeges and leverage. Ths mgh be due o he ably of boh models usng probably dsrbuons o denfy successfully rades wh a hgh Sharpe rao. Keywords Confrmaon flers, Gaussan mxure models, leverage, mul-layer percepron newors, probably dsrbuon, Sofmax cross enropy newors, radng sraegy. Chrsan Duns s Groban Professor of Banng and Fnance a Lverpool Busness School and Drecor of CIBEF (E-mal: cduns@oalse.co.u). The opnons expressed heren are no hose of Groban. Paulo Lsboa s Professor n Indusral Mahemacs a Lverpool John Moores Unversy and Head of he Unversy s Graduae School (E-mal: p.j.lsboa@lvjm.ac.u). Andreas Lndemann s an Assocae Researcher wh CIBEF (E-mal: busalnd@lvjm.ac.u) and currenly worng on hs PhD hess a Lverpool Busness School. CIBEF Cenre for Inernaonal Banng, Economcs and Fnance, JMU, John Foser Buldng, 98 Moun Pleasan, Lverpool L3 5UZ.

2 . INTRODUCTION Neural newors are an emergen echnology wh an ncreasng number of real-world applcaons [Lsboa and Velldo, 2000]. A recen developmen n he area of fnancal forecasng deals wh he predcon of probably dsrbuons n conras o he more commonly used level and/or classfcaon predcon. The movaon for hs paper s o nvesgae he use of such a probably dsrbuon predcor. Ths s done by benchmarng hree dfferen neural newor archecures represenng a level esmaor, a classfcaon model and a probably dsrbuon predcor o forecas a arge value. More specfcally, we presen he Mul-layer percepron newor, he Sofmax cross enropy model and he Gaussan mxure model. We benchmar her respecve performance on he Euro/Dollar (EUR/USD) me seres as repored n [Duns and Wllams, 2002, 2003]. The resuls of our hree newors are compared o more radonal approaches, see also [Duns and Wllams, 2002, 2003] where he Mul-layer percepron s repored o ouperform models such as a movng average convergence dvergence echncal model (MACD), an auoregressve movng average model (ARMA) and a logsc regresson model (LOGIT). As urns ou, he Mul-layer percepron does bes when used whou confrmaon flers and leverage, whle he Sofmax cross enropy model and he Gaussan mxure model ouperforms he Mul-layer percepron when usng more sophscaed radng sraeges and leverage. Ths mgh be due o he ably of boh models usng probably dsrbuons o denfy successfully rades wh a hgh Sharpe rao. The res of he paper s organsed as follows. In secon 2, we presen he leraure relevan o he Gaussan mxure model. Secon 3 descrbes he daase used for hs research, acually he same as n [Duns and Wllams, 2002, 2003]. An overvew of he dfferen neural newor models s gven n secon 4. Secon 5 shows he emprcal resuls of all he models and nvesgaes he possbles of mprovng her performance wh he applcaon of more sophscaed radng sraeges. Secon 6 provdes some concludng remars. 2

3 2. LITERATURE REVIEW The movaon for applyng densy dsrbuon newors s ha a forecas of he whole probably dsrbuon offers more nformaon han a level and/or a classfcaon predcon. Many dfferen model archecures for densy esmaon have herefore been proposed n he leraure. A sraghforward approach nroduced by [Wegend and Nx, 994] uses one neural newor o generae a regresson funcon (level predcon) and anoher o predc he varance of he nose dsrbuon around he forecased value of he frs newor (local error bars). A shorcomng of hs approach les n he assumpon ha he nose s Gaussan. As a resul, he model s no able o capure sewed or mulmodal dsrbuons. A mehod o overcome hese shorcomngs s called sof hsogram and was nroduced by [Wegend and Srvasava, 995]. The dea s o redefne he forecasng problem as a classfcaon as. The newor has oupu nodes. Each node represens a ceran par of he range of he arge value. The learnng as s o map he npu vecor x o ha oupu node whose value range conans he arge value y. The densy dsrbuon for he arge value y may hen be found by lnearly nerpolang he oupu values of adjacen nodes. Ths approach s sensve o he choce of value range for each node and can lead o problems f one res o ge a well defned dsrbuon snce he necessary range sze mgh be smaller han he number of ranng samples can suppor, because of he large number of parameers n he model. The Sofmax cross enropy model whch we descrbe laer on s based on hs approach. More robus approaches use a funcon (such as a Gaussan funcon) as a buldng bloc of he densy dsrbuon and only a few adjusable parameers defnng how he buldng blocs are se ogeher o model he exac shape of he densy dsrbuon. Examples are Condonal Densy Esmaon Newors (CDEN) [Neuneer e al., 994], Mxure Densy Newors (MDN) [Bshop, 994], Dsored Probably Mxure Newors (DPMN) [Neuneer e al., 994], Mxure of Expers (ME) [Jordan and Jacobs, 994] and he Gaussan mxure Newor (GM) [Husmeer, 999]. These newors dffer manly n wo ways: frs n he choce of he funcon represenng he buldng bloc of he densy dsrbuon and second n he dependency of he parameers (whch defne he shape of he densy dsrbuon) from he acual npu daa x. An example when usng a Gaussan funcon as a buldng bloc could be he wdh of he Gaussan funcons. The wdh could eher be fxed and herefore be he same for he whole se of npu daa x or could be adjused each me o he acual npu daa. The followng research s based on he Gaussan mxure model (whch s presened n more deal n secon 6). Ths archecure uses he Gaussan funcon as a buldng bloc. Only he cenres of he Gaussan funcons are dependen on he acual npu daa vecor x of he model whle all he oher parameers (for example he wdh of he Gaussan funcons) are fxed once hey have been adaped durng he ranng process of he newor. We have chosen hs specal archecure because of s reduced model complexy whch s advanageous for compuaonal reasons and model robusness. 3

4 3. THE EUR/USD EXCHANGE RATE AND RELATED FINANCIAL DATA Our benchmar es s o rade he EUR/USD exchange rae based on daly forecass of s London closng prces. All me seres are daly closng daa obaned from a hsorcal daabase provded by Daasream and used n [Duns and Wllams, 2002, 2003]. Name of perod Tradng days Begnnng End Toal daase Ocober July 200 Tranng daase Ocober May 2000 Ou-of-sample daase [Valdaon se] May July 200 Table : The EUR/USD daase Fg. : EUR/USD London daly closng prces (oal daase) [Duns and Wllams, 2002, 2003] carred ou varable selecon and denfed he explanaory varables lsed n able 2. Number Varable Mnemoncs Lag US $ TO UK (WMR) EXCHANGE RATE USDOLLR 2 2 JAPANESE YEN TO US $ (WMR) EXCHANGE RATE JAPAYE$ 3 JAPANESE YEN TO US $ (WMR) EXCHANGE RATE JAPAYE$ 0 4 BRENT CRUDE Curren Monh, fob U$/BBL OILBREN 5 GOLD BULLION $/TROY OUNCE GOLDBLN 9 6 FRANCE BENCHMARK BOND 0 YR (DS) RED. YIELD FRBRYLD 2 7 ITALY BENCHMARK BOND 0 YR (DS) RED. YIELD ITBRYLD 6 8 JAPAN BENCHMARK BOND RYLD.0 YR (DS) RED. JPBRYLD 9 9 NIKKEI 225 STOCK AVERAGE PRICE INDEX JAPDOWA 0 NIKKEI 225 STOCK AVERAGE PRICE INDEX JAPDOWA 5 Table 2: Explanaory varables and Daasream mnemoncs EUR/USD s quoed as he number of USD per Euro: for example, a value of.2657 s USD.2657 per Euro. The EUR/USD exchange rae only exss from 4 January 999: was reropolaed from 7 Ocober 994 o 3 December 998 and a synhec EUR/USD seres was creaed for ha perod usng he fxed EUR/DEM converson rae agreed n 998, combned wh he USD/DEM daly mare rae. 4

5 The observed EUR/USD me seres s non-normal (Jarque-Bera sascs confrmed hs a he 99% confdence nerval) conanng slgh sewness and low uross. I s also nonsaonary and [Duns and Wllams, 2002, 2003] decded o ransform he EUR/USD as well as all he explanaory seres no saonary seres of raes of reurn 2. Gven he prce level P, P 2,, P, he rae of reurn a me s formed by: P R = [] P The summary sascs of he EUR/USD reurns seres reveal a slgh sewness and hgh uross. The Jarque-Bera sasc confrms agan ha he EUR/USD seres s non-normal a he 99% confdence nerval. Fg. 2: EUR/USD reurns summary sascs (oal daase) A furher ransformaon ncludes he creaon of neres raes yeld curve seres, generaed by: yc = 0 year benchmar bond yelds - 3 monh neres raes [2] We dvded our daase as follows: Name of perod Tradng days Begnnng End Toal daa se Ocober July 200 Tranng daa se 69 7 Ocober Aprl 999 Tes daa se Aprl May 2000 Ou-of-sample daa se [Valdaon se] May July 200 Table 3: The neural newors daases 2 Confrmaon of s saonary propery s obaned a he % sgnfcance level by boh he Augmened Dcey Fuller (ADF) and Phllps-Perron (PP) es sascs. 5

6 4. THE NEURAL NETWORKS FORECASTING MODELS Neural newors exs n several forms n he leraure. The mos popular archecure s he Mul-layer percepron. A sandard neural newor has a leas hree layers. The frs layer s called he npu layer (he number of s nodes corresponds o he number of explanaory varables). The las layer s called he oupu layer (he number of s nodes corresponds o he number of response varables). An nermedary layer of nodes, he hdden layer, separaes he npu from he oupu layer. Is number of nodes defnes he amoun of complexy he model s capable of fng. In addon, he npu and hdden layer conan an exra node, called he bas node. Ths node has a fxed value of one and has he same funcon as he nercep n radonal regresson models. Normally, each node of one layer has connecons o all he oher nodes of he nex layer. The newor processes nformaon as follows: he npu nodes conan he value of he explanaory varables. Snce each node connecon represens a wegh facor, he nformaon reaches a sngle hdden layer node as he weghed sum of s npus. Each node of he hdden layer passes he nformaon hrough a nonlnear acvaon funcon and passes on o he oupu layer f he calculaed value s above a hreshold. The ranng of he newor (whch s he adjusmen of s weghs n he way ha he newor maps he npu value of he ranng daa o he correspondng oupu value) sars wh randomly chosen weghs and proceeds by applyng a learnng algorhm called bacpropagaon of errors 3 [Shapro, 2000]. The learnng algorhm smply res o fnd hose weghs whch opmse an error funcon (normally he sum of all squared dfferences beween arge and acual values). Snce newors wh suffcen hdden nodes are able o learn he ranng daa (as well as her oulers and her nose) by hear, s crucal o sop he ranng procedure a he rgh me o preven overfng (hs s called early soppng ). Ths can be acheved by dvdng he daase no 3 subses respecvely called he ranng and es ses used for smulang he daa currenly avalable o f and une he model and he valdaon se used for smulang fuure values. The newor parameers are hen esmaed by fng he ranng daa usng he above menoned erave procedure (bacpropagaon of errors). The eraon lengh s opmsed by maxmsng he forecasng accuracy for he es daase. Fnally, he predcve value of he model s evaluaed applyng o he valdaon daase (ou-of-sample daase). 4. THE MULTI-LAYER PERCEPTRON MODEL 4.. The MLP newor archecure The newor archecure of a sandard Mul-layer percepron (henceforh MLP) loos as presened n fgure 3: 3 Bacpropagaon newors are he mos common mullayer newors and are he mos commonly used ype n fnancal me seres forecasng [Kaasra and Boyd, 996]. 6

7 [ ] x [ j] h y~ w j u j MLP where: [n] Fg. 3: A sngle oupu, fully conneced MLP model x ( n =,2, L, + ) are he model npus (ncludng he npu bas node) a me [m] h ( m =,2,..., j + ) are he hdden nodes oupus (ncludng he hdden bas node) y~ s he MLP model oupu u and are he newor weghs j w j s he ransfer sgmod funcon: S( x) s a lnear funcon: ( x) = = + e x x, [3] F [4] The error funcon o be mnmsed s: T ( j wj ) = ( y ~ y ( u j, wj ) E u, ), wh y beng he arge value [5] T = Emprcal resuls of he MLP model The resuls for he MLP acheved by [Duns and Wllams, 2002, 2003] are summarzed n able 4. The benchmar model naïve sraegy follows he rule ha he forecas for omorrow s oday s value. The performance measures are calculaed as shown n appendx A.. NAIVE MLP Sharpe Rao (excludng coss) Annualsed Volaly (excludng coss).6%.6% Annualsed Reurn (excludng coss) 2.3% 29.7% Maxmum Drawdown (excludng coss) -9.% -9.% Taen Posons (annualsed 4 ) 09 8 Table 4: Tradng performance of he benchmar models 4 The number of aen posons can dffer from he number of radng days due o he possbly o hold a poson for longer han day. 7

8 4.2 THE SOFTMAX CROSS ENTROPY MODEL The Sofmax cross enropy newor (henceforh SCE) s a neural newor wh a cross enropy cos funcon and a Sofmax acvaon funcon a he oupu nodes. The man dea of hs model s o approxmae he probably densy funcon for he arge value hrough an hsogram represenng he probably of he arge value beng whn a range of predefned sze. The oupu value of a SCE model s herefore a vecor wh as many elemens as here are oupu nodes (each node represenng one bar of he hsogram). The vecor elemens sum up o uny and represen he densy funcon for he arge value whle each vecor elemen sands for he probably ha he arge value les n he value range he vecor elemen represens. In order o apply he cross enropy cos funcon, he arge values of he ranng daa se have o be preprocessed so ha we ge a arge vecor (raher han a sngle arge value as wh he MLP), where he arge vecor has as many elemens as he SCE model has oupu nodes. The arge vecor consss of zeros and a sngle one. The value one ndcaes whch oupu node of he newor covers he value range where he orgnal arge value les n. Snce he newor forecass should be used as a densy funcon, one has o ae care ha he oupu vecor sums up o uny. Ths s done by supermposng he Sofmax funcon o he acual newor oupus. The Sofmax funcon eeps he nernal relaonshp beween he oupu values bu ransforms hem n a way ha her values add up o uny (see equaon [8] below). Durng he ranng phase (ha s when he newor weghs are adjused), he SCE model learns o map he npu vecor of he ranng daa se o he arge vecor of he same daa se. Snce each arge vecor consss of a sngle one represenng a nonoverlappng range of possble oupu values (whle he res are zeros), he SCE model res n fac o solve a classfcaon as. The newor mgh face a suaon where he same npu vecor s relaed o wo dfferen oupu values (a dfferen mes) so ha he newor has no oher chance han o map he npu vecor o more han one oupu node. In dong so, he newor generaes a densy funcon for he arge value, whle he negraed Sofmax funcon ensures ha he probables add up o uny. Fgure 4 shows a possble SCE newor oupu where he sum of he bars adds up o uny whle he x-axs shows he move sze (n percenage) of he forecased change of he EUR/USD me seres. 8

9 Fg.4: Example of he densy funcon as oupu of he SCE model 4.2. The SCE newor archecure The dfference n archecure wh a MLP les n he mulple oupu nodes. Whle he MLP has ypcally only one oupu node delverng a level esmaon, he SCE newor uses several oupu nodes o represen an approxmaon of he densy funcon (whle beng raned on a classfcaon as). [ ] x [ j] h [ q] y~ [ q] z~ u j wgj SCE where: [n] Fg. 5: A sngle oupu, fully conneced SCE model x ( n =,2, L, + ) are he model npus (ncludng he npu bas node) a me [m] h ( m =,2,..., j + ) are he hdden nodes oupus (ncludng he hdden bas node) [ g ] y~ ( g =,2,..., q) s he SCE model oupu before applyng he Sofmax funcon [ g ] z~ ( g =,2,..., q) s he newor value a he oupu node g u and are he newor weghs j w gj 9

10 s he ransfer sgmod funcon: S( x) = + e x, [6] s a lnear funcon: ( x) = s he Sofmax funcon ( g) F [7] x ( ~ y g ) ( ~ y ) exp A = ~ z g = [8] exp wh ~ y g beng he oupu of he lnear funcon g g The error funcon o be mnmsed s: T q yg, gj = yg log ~, wh y = = ( ) g beng he arge value [9] g zg u j, w j ( w ) E u j The algorhm o ran he SCE newors s shown n appendx A Emprcal resuls of he SCE model We apply he SCE model o he same EUR/USD reurn me seres whch we have used wh he MLP newor. Snce neural newors sar wh random nalsaon of her weghs, each newor (even wh he same archecure) s unque and produces slghly dfferen resuls. In order o ge sable and relable resuls from he SCE archecure, we have spl our nal nvesmen capal equally amongs 30 dencal (excep he nal weghs) models. The resul s herefore he average resul of a commee of 30 members. As before wh he MLP model, he problem of overfng s agan an ssue we have o deal wh. The smples way o do hs s o use early-soppng. Tha s (as before wh he MLP) an opmsaon of he newor parameers on a es daa se raher han on he ranng daa se self. In addon o hs mehod, we appled wegh decay o he SCE models. The mehod of wegh decay s realzed as a penaly erm added o he cos funcon. The penaly erm aes care of he wegh sze n a way ha prefers smaller weghs over bgger weghs. The raonaly behnd wegh decay s he assumpon ha bgger weghs go hand n hand wh overfng snce hey are used o f very specal paerns of he SCE model. We have appled many dfferen combnaons (wegh decay facors, range of he nal random weghs, number of nodes for he hdden and oupu layers, number of eraon seps). The one we have fnally chosen maxmsed he prof (obaned on he es daase) and s shown n able 5. 0

11 Parameer se for SCE newor Learnng algorhm Graden descen Learnng rae 0.3 Momenum 0.5 Regularsaon Wegh decay (0.0) Ieraon seps 000 Inalsaon of weghs N(0,) Sze range covered by each bn 5 0.3% Inpu nodes 0 Hdden nodes (layer) 5 Oupu node 6 Table 5: Parameer se of he SCE model We use he densy funcon of each of our 30 SCE models o calculae he probably for an upmove. Ths s smply done by addng up he las 3 of 6 values of he oupu vecor (snce hose 3 values cover he whole range of possble posve values for an upmove). Tang a long poson f he probably for an upmove exceeds 50% (and a shor poson vce versa), we are able o calculae he average radng resuls of he 30 SCE models as presened n able 6 below. NAIVE MLP SCE Sharpe Rao (excludng coss) Annualsed Volaly (excludng coss).6%.6%.6% Annualsed Reurn (excludng coss) 2.3% 29.7% 26.3% Maxmum Drawdown (excludng coss) -9.% -9.% -7.8% Posons Taen (annualsed) Table 6: Tradng performance resuls 4.3 THE GAUSSIAN MIXTURE MODEL The nex model s he Gaussan mxure newor (henceforh GM model) whch was frs nroduced by [Husmeer, 999]. The GM model represens he probably densy of he daa by a lnear combnaon of a fxed number of normal dsrbuons (where he dsrbuon wdh s adaped o he whole se of ranng daa whle he locaons of he dsrbuon cenres depend on he acual npu daa x and he dependen varable y ). Ths s done n a hdden layer where each node represens a normal dsrbuon. The acual newor oupu s no he densy funcon self bu he predcon of a sngle value 6 whch s he lelhood of he acual GM model parameers generang he observed value of he dependen varable y condoned on he npu daa x. 5 Acually, he ouer bns have o cover a bgger range. The value ranges for he 6 bns are: [- %;- 0.6%],]-0.6%;-0.3%], ]0.3%;0%];]0%;0.3%], ]0.3%;0.6%], ]0.6%; %]. 6 Neverheless, he whole densy dsrbuon could be consruced by varyng he value of y over he neresng range of he searched densy funcon.

12 To opmse he cos funcon (ha s o maxmse he sum of lelhood values), he weghs u j and w j, deermnng he locaon of he normal dsrbuon cenres (µ ), have o be adaped so ha he dsance beween y and µ s mnmal. Dong so, he cenres of he dsrbuon are close o y and herefore he lelhood and wh he value of he cos funcon are hgh. See fgure 6 o llusrae ha worng prncple The GM newor archecure The GM archecure dffers n hree man ways from our benchmar feedforward newor (MLP). Frs, can be shown [Husmeer, 999] ha n order o be a unversal approxmaor a leas a second hdden layer s necessary. Second, boh he ndependen and dependen varable (x,y) are used as npu daa, snce he am s no o predc y bu s densy dsrbuon P(y x) respecvely he correspondng lelhood value. Thrd, he newor uses Gaussan dsrbuons n he second hdden layer. u j w j [ ] x a P( y x ) y β P(ylx) µ 0 y 0 y GM model Fg. 6: GM newor archecure The followng funcons are appled whn he GM model: [n] x ( n =,2, L, + ) are he model npus (ncludng he npu bas node) a me y s he argumen of he densy funcon condonal on he values of he npus (noe ha he weghs of y are fxed o 7 ) [see fgure 6] 7 If we would no fx he wegh o he newor could decrease he cos funcon no only by adjusng he cenres of he Gaussan mxure funcons bu also by changng he orgnal arge value y. 2

13 u j and w j are he newor weghs β defne he nverse wdhs of he Gaussan dsrbuons a are he mxng coeffcens, wh = a s he number of appled Gaussan mxure dsrbuons j s he number of appled newor weghs j s he number of appled newor weghs u j The error funcon o be mnmsed s: T ( j wj,, a ) = ln( P( E u Gaussan dsrbuon: ( y µ ) wh µ ( x ): = wj S u j x, σ j Sgmod funcon: S( x) x w ( y µ ) 2 β β G = exp β,[0] 2π 2 =, β > 0, a 0, β = + e Lnear funcon: P( y x) = a G [ y µ ( x) ] β )) y, y x, u j, wj, β, a T = a =, [] β [2], wh beng he arge value [3] I s possble o updae he parameers of he GM model by graden descen, as was done wh he MLP newor. However hs algorhm, due o he archecural complexy of he GM ne, s very me consumng. The updae equaons for a faser learnng algorhm, namely he Expecaon-Maxmsaon Algorhm (ncludng he applcaon of he Bayesan Evdence Scheme for Regularsaon 8 ) are more complex and are herefore presened n appendx A Emprcal resuls of he GM model In order o apply he GM model o he EUR/USD reurn me seres, we have o opmse s parameers as well as he soppng pon 9 on he es daa se, whch s he same daase already used wh he MLP newor. Snce he ranng process of neural newors sars wh he random nalsaon of her weghs, each newor wll come up wh a smlar bu dfferen resul. To mnmze he varance of he newor forecass we have spl our nal nvesmen capal equally 8 There are wo man sraeges o preven overfng: early-soppng (whch we have descrbed a he begnnng of secon 4 on he MLP model) and regularsaon. The way regularsaon prevens overfng s by nroducng a penaly erm o he learnng algorhm n order o eep he wegh values small. The dea s ha a newor s only able o overf when s able o use exreme values for some of s weghs. 9 Even wh regularsaon, he addonal mplemenaon of early soppng mproved he resuls. The weghs were fxed a he bes resulon he es daa se durng ranng. 3

14 amongs 30 dencal (excep he nal weghs) GM models. The resul s herefore he average resul of a commee of 30 members. We have appled many dfferen parameer combnaons (number of nodes for boh hdden layers, number of eraon seps and he range of possble values for he wegh nalsaon a he begnnng of he ranng process) o he es daase. The one we have fnally chosen maxmsed he prof (obaned on he es daase) and s shown n able 7. Parameer se for GM newor Learnng algorhm EM Regularsaon Evdence scheme Ieraon seps 35 Inalsaon of weghs [-0.;0.] Inpu nodes 0 Hdden nodes (layer ) 5 Hdden nodes (layer 2) 5 Oupu node Table 7: Parameer se of he GM model The emprcal predcons for he densy funcons of he exchange rae, predced oneday-ahead, are shown below n fgure 7. On he rgh hand-sde are he probably dsrbuons of he 290 days of he valdaon daase (ou-of-sample daa) whle he lef hand-sde shows he predced densy funcon of a parcular pon n me. As can be seen from hese graphs, seems ha here s no mulmodaly n he EUR/USD exchange rae mare, a leas whn he nose level presen n he daa. densy values daa ses daa values Fg. 7: Probably dsrbuons of he valdaon se 4

15 Usng he densy funcons o calculae he probably for a posve exchange rae change as well as for a negave change and ang a radng poson where he probably s bgges (namely >50% snce boh add up o 00%) leads o he average radng resuls of he 30 GM models as presened n able 9 below. The decson o ae a commee of 30 dencal models raher han usng a sngle GM model s movaed by he wsh o mnmse he chance o pc an ouler model. Ths s parcularly mporan when radng on he als of he densy funcon. NAIVE MLP SCE GM Sharpe Rao (excludng coss) Annualsed Volaly (excludng coss).6%.6%.6%.6% Annualsed Reurn (excludng coss) 2.3% 29.7% 26.3% 24.2% Maxmum Drawdown (excludng coss) -9.% -9.% -7.8% -2.4% Posons Taen (annualsed) Table 8: Tradng performance resuls As can be seen, he performance of he GM commee where each model ndependenly predcs an upmove or downmove does no mprove he resul of he sngle benchmar MLP newor. However he GM model does provde more nformaon han s acually used wh hs smple radng sraegy as we have access o he complee dsrbuon of he predced move n he exchange rae. Ths should be helpful when applyng more sophscaed sraeges whch are nvesgaed n deal n he nex secon. 5. TRADING COSTS, FILTERS AND LEVERAGE Up o now, we have presened he radng resuls of all our models whou consderng ransacon coss. Snce some of our models rade que ofen, ang ransacon coss no accoun mgh change he whole pcure. We herefore nroduce ransacon coss as well as a flered radng sraegy for each model. The am s o devse a radng sraegy flerng only hose rades whch have a hgh probably of beng successful. Ths should help o reduce he negave effec of ransacon coss as rades wh an expeced gan lower han he ransacon coss should be omed. 5. Transacon Coss The ransacon coss for a radable amoun, say USD 5-0 mllon, are abou 3 pps ( EUR/USD) per rade (one way) beween mare maers. Bu snce he EUR/USD me seres s a seres of bd raes, we have o pay he coss only one and no wo mes per aen poson [Duns and Wllams, 2003]. Wh an average exchange rae of EUR/USD of for he ou-of-sample perod, a cos of 3 pps s equvalen o an average cos of 0.033% per poson = 0.033%

16 5.2 Confrmaon Fler Sraeges 5.2. Confrmaon Flers We wll now nroduce radng sraeges devsed o fler ou hose rades wh expeced reurns below he 0.033% ransacon cos. Due o he archecure of our models, he radng sraegy for he MLP newor consss of one sngle parameer whle he sraegy appled o he SCE and GM model uses wo parameers. Ths s because of he addonal avalable nformaon whch he SCE and GM models offer n erms of probably dsrbuons. Up o now, he radng sraeges appled o he models use a zero hreshold: hey sugges o go long when he forecas s above zero and o go shor when he forecas s below zero. In he followng, we examne how he models behave f we nroduce a hreshold d around zero (see fgure 8) and wha happens f we vary ha hreshold. The fler rule for he MLP model s presened n fgure 8 below. shor long < (-d) d d > (+d) Fg. 8: Flered radng sraegy for he MLP model Snce he forecas of he SCE and GM models provde more nformaon han our MLP models, we are able o nroduce a second parameer for he radng sraegy, whch s he probably level. As a resul, all hose radng sgnals are flered ou whch are (a) no ndcang a prce move (n eher drecon) bgger han he hreshold d (whch has o be a mulple of he bn sze n he SCE case) and n addon (b) no ndcang a probably hgher han x% for he forecased prce move (whch s he sum of he hsogram bars for he SCE model and he space under he densy funcon curve for he GM model). If boh condons are fulflled a he same me for an up- as well as for a downmove, he sraegy pcs he radng sgnal wh he hgher probably. shor relevan probables long < (-d) d d > (+d) Fg. 9 Flered radng sraegy for he SCE model 6

17 relevan probably shor long < (-d) d d > (+d) Fg. 0: Flered radng sraegy for he GM model Emprcal Resuls of he MLP, SCE and GM models In he followng, we presen he ou-of-sample radng resuls for he MLP, he SCE and he GM model. We proceed as follows. Frs, we fx for each model he hreshold for he radng sraegy. Ths s accomplshed by loong solely a he es daa se. The procedure of choosng he parameers s raher subjecve alhough we red o fnd hose values ha promsed a favourable mx beween prof and rs. Second, we show he radng resuls of several possble hreshold values, whch enables us o dscuss our parameer choce. The fnal parameers chosen for each model are presened n able 9. A vsualsaon of our choce can be found n appendx A.2.. Model Threshold (d) MLP = 0.00 SCE = 0.25 (move sze > 0.3% ) GM = 0.00 (probably > 0.0%) Table 9: Chosen parameers for each radng sraegy For he MLP, we leave he hreshold a zero (d=0.0) snce he prof on he es daase s bgges a hs value. The value of d=0.05 loos promsng from a Sharpe rao pon of vew bu loong a he prof and Sharpe rao values for he followng parameer values, hs pea n he Sharpe rao seems o be an ouler snce he whole rend s declnng. We sc herefore o d=0.0 o mnmze our rs. For reasons of beer comparably wh he oher benchmar models, we have fxed one of he wo parameers (namely he move sze ) of he radng sraegy for he SCE model. The value we have chosen for hs parameer s 0.3% (ha s, we are only loong a he probables for prce changes bgger han 0.3%). Ths decson has been made by loong a he whole resul marx (on he es daa se) spanned by he wo 7

18 parameers. The seleced value of he frs radng parameer provdes he bgges prof opporunes (condonal on he choce of he second parameer). The second parameer ( probably ) s se o 0.25 (~25%) because of he ncreasng Sharpe rao. Also for reasons of beer comparably, we have fxed one of he wo parameers (namely he probably ) of he radng sraegy for he GM model. We decded o fx he probably hreshold a a value of 0% snce he radng resuls are no mprovng wh an ncreasng parameer 2. The same s rue for he second parameer of he GM radng sraegy (see appendx A.2.), so ha he chosen parameer se for he GM s d=0.0 and probably = 0.0. Ths shows ha he confrmaon flers does no add value here (as shown n fgure ) and herefore he resuls gven n able 0 are wh no flers (respecvely wh a fler parameer se o 0). The ou-of-sample resuls for each possble parameer d are shown below: Fg. : Ou-of-sample resuls for poenal parameer values The ou-of-sample resuls for he chosen parameers are: NAIVE MLP SCE GM Sharpe Rao (excludng coss) Annualsed Volaly (excludng coss).6%.6% 8.5%.6% Annualsed Reurn (excludng coss) 2.3% 29.7% 22.7% 24.2% Maxmum Drawdown (excludng coss) -9.% -9.% -5.7% -2.4% Posons Taen (annualsed) Transacon coss 3.6% 3.9% 3.9% 5.3% Annualsed Reurn (ncludng coss) 7.7% 25.8% 8.8% 8.9% Table 0: Ou-of-sample resuls for he chosen parameers The whole resul marx for he es daa se s no shown n hs paper whle a graph s provded n appendx A.2. based on he row daa of ha marx defned by he fxed value of he frs radng parameer. 2 As before done wh he SCE model, our decson s based on he whole resul marx whch s no presened n hs paper. The resuls for he es daa se based on he seleced value of he frs parameer s shown n appendx A.2.. 8

19 We can see ha he smple MLP model sll ouperforms boh he oher models (based on annualsed reurn). Up o now, he nroducon of a flered radng sraegy has no mproved our resuls! 5.3 Leverage o explo hgh Sharpe Raos As we have seen, he applcaon of a flered radng sraegy does no mprove he resuls ha much, snce 2 of our 3 models sll sc o a hreshold of zero. The queson s wheher we can gan hgher rs-adjused profs by usng leverage. To selec he approprae parameers, we base our choce on he leveraged radng resuls of he es daa se (a vsualsaon can be found n appendx A.2.). The appled leverage facors are calculaed n a way such ha each model has a common volaly of 0% 3 on he es daa se. Snce we now have addonal nformaon (whch s he leveraged radng resuls based on he es daase), we can rehn our former choce of hresholds. The parameers whch we have fnally chosen are presened n able. The prof of he MLP model peas a d=0.05. The SCE model uses he same parameers as before whle he GM model seems o perform well wh d=0.35 (and a fxed probably of 0.25). Model Threshold (d) MLP = 0.05 SCE = 0.25 (move sze > 0.3% ) GM = 0.35 (probably = 0.25) Table : Parameers for he radng sraeges The ransacon coss are calculaed by ang 0.033% per poson no accoun, whle he coss of leverage (neres paymens for he addonal capal) s calculaed wh 4% p.a. (ha s 0.06% per radng day 4 ). For he combnaon of our 30 SCE and GM models, we approxmae he number of radng days by he average number of radng days of each of he 30 models. The resulng leverage coss for he model combnaon are herefore slghly oo hgh, snce some posons of he models would cancel hemselves ou, reducng he amoun of leverage capal needed. We show wo prof seres for he ou-of-sample se. The frs one s whou leverage. The second seres uses he leverage facor, whch, as menoned above, has been obaned by ransformng he models volaly on he es daa se o 0%. We can only use hs leverage facor, snce he facor ha guaranees a volaly of 0% for he ouof-sample perod s only avalable ex pos and herefore no nown a he me of radng. 3 Snce mos of he models (usng a hreshold of zero) have a volaly of abou 0%, we have chosen hs level as our bass. The leverage facors reaned are gven n able 2 below. 4 The neres coss are calculaed by consderng a 4% neres rae p.a. dvded by 252 radng days. In realy, leverage coss also apply durng non-radng days so ha we should calculae he neres coss usng 360 days per year. Bu for he sae of smplcy, we use he approxmaon of 252 radng days o spread he leverage coss of non-radng days equally over he radng days. Ths approxmaon prevens us from eepng rac of how many non-radng days we hold a poson. 9

20 The resuls for several parameer values for he ou-of-sample daa se are shown below n fgure 2. Those resuls are obaned for he SCE and GM model by fxng one of her wo parameers (as descrbed before). The ou-of-sample resuls when he second parameer s also allowed o vary could be seen n appendx A.2.2. Fg. 2: Resuls for possble parameer values on he ou-of-sample daa se The GM model shows an ncreasng prof funcon, whch peas a a value of abou 45% (whch s he hghes of all models). The only problem wh he GM model s he poenal for serous losses. By loong a he rgh handsde char for he GM model we fnd ou ha he losses go hand n hand wh a number of posons aen of only abou 6 (and below) durng he whole year. 20

21 The resuls for our chosen se of parameers are presened below: NAIVE MLP SCE GM Sharpe Rao (excludng coss) Annualsed Volaly (excludng coss).9% 3.4% 2.5% 2.2% Annualsed Reurn (excludng coss) 2.8% 30.8% 33.2% 46.4% Maxmum Drawdown (excludng coss) -9.3% -0.3% -8.5% -.3% Leverage Facor Posons Taen (annualsed) Transacon and leverage coss 3.7% 6.% 7.% 2.5% Annualsed Reurn (ncludng coss) 8.% 24.72% 26.% 33.94% Table 2: Tradng performance fnal resuls 7 We can see from hese resuls ha he SCE and GM model are able o ae advanage of he possbly offered by leverage n combnaon wh a radng sraegy. Boh models ge her hghes Sharpe rao when applyng he radng sraegy n combnaon wh leverage whle he MLP model s no able o mprove upon s prevous resuls. Ineresngly, from he hree dfferen neural newor archecures, he GM and SCE models are he ones mos smlar o each oher snce boh are dealng wh probably dsrbuons. 6. CONCLUDING REMARKS In hs paper, we have appled he Mul-layer percepron newor, he Sofmax cross enropy neural newor and he Gaussan mxure model o a one-day-ahead forecasng and radng as of he EUR/USD me seres. We have developed hese dfferen predcon models over he perod Ocober 994-May 2000 and esed her ou-of-sample radng effcency over he followng perod from May 2000 hrough July 200. Overall, we have seen ha radng sraeges ha should have flered ou poenally unsuccessful rades by usng a confrmaon hreshold have no wored ou. Even 5 The calculaon s done whou ransacon and leverage coss due o a beer comparably o oher publshed numbers (whch are generally calculaed n hs way). 6 The SCE and GM commees have acually aen more rades han repored n he able above (e.g. he GM model has acually aen 34 posons). The reason why we repor a smaller number of rades s ha SCE and GM commees are able o nves less han 00% of her oal capal per poson (hs s due o he fac ha he poson sze s deermned by he average number of commee members generang a radng sgnal). Snce our ransacon coss of 0.033% per poson are based on he assumpon of 00% of nvesed oal capal, we have o recalculae he 34 posons of parally nvesed oal capal no he equvalen number of posons wh 00% of nvesed capal (whch are he above shown 68 posons). 7 No aen no accoun are he followng effecs: a) The neres ha could be earned durng mes where he capal s no raded [non-radng days] and could herefore be nvesed; b) The SCE and GM commees are no forced o use 00% of her capal when radng (leavng ou a leverage facor <), snce he amoun s deermned by he averaged forecas of he 30 models. If he commees nves herefore only a few per cen of he capal avalable bu apply he leverage facor (>), he addonal capal has no o be borrowed (snce here s sll own money avalable) and herefore leverage coss would no be ncurred. Those savngs are no aen no accoun here. 2

22 ang ransacon coss no accoun, he bes resuls would have been acheved wh a hreshold of zero. Neverheless, he combnaon of a hreshold flered radng sraegy wh leverage has shown a dfferen pcure. Whle for he MLP model he addonal opon of leveragng he resuls has no posvely affeced s radng performance for a gven level of rs, he SCE and GM models were able o ae advanage of he combnaon of a confrmaon fler and leverage. The SCE and GM models seem o be able o use he hreshold sraegy o selec rades wh an expeced hgh Sharpe rao. Ths alone leads o reduced profs snce he number of posons aen also declnes. However, by leveragng he remanng posons, he expeced hgh Sharpe Rao s ransformed no acual hgher absolue and rs-adjused profs. Neverheless, a few fnal words of cauon seem approprae. The ranng, es and valdaon ses are all par of a major downrend (see fgure ) so ha a model raned on he srucure of he frs par of hs me seres mos probably exhbs a que promsng ably o forecas reurns on he ou-of-sample perod snce boh have he same underlyng srucure. If he ranng daa se and he valdaon daa se exhb dfferen characerscs, mgh well be possble ha he MLP ouperforms he SCE and/or he GM model snce s smpler and herefore more robus. We have appled our models o a EURO STOXX 50 me seres wh dfferen underlyng characerscs n he daa ses. The resul seems o suppor our suggeson ha n unsable suaons smpler neural newors could ouperform he more complex ones. 22

23 APPENDIX A. Performance measures The performance measures are calculaed as follows 8 : Performance Measure Annualsed Reurn wh R A R Descrpon N = 252 * R [4] N = beng he daly reurn Cumulave Reurn N A = 252 * * ( ) Volaly R R Annualsed Sharpe Rao Maxmum Drawdown R C = N = R N = [5] 2 σ [6] Maxmum negave value of ( ) A R SR = [7] σ A R over he perod MD = Mn R =, L, ; =, L, N j= j [8] Table 3: Tradng smulaon performance measures A.2 Emprcal resuls A.2. Resuls for he es daa se The graphs below are he resuls of he applcaon of he flered radng sraegy o he es daa se for dfferen values of d. The profs are annualsed and nclude ransacon coss. The arrows pon o he chosen values for he parameer d as shown n able 9. 8 For more deals see [Duns and Wllams, 2002, 2003]. 23

24 Fg. 3: Resuls for possble parameer values on he es daa se The graphs below show he resuls of he combnaon of he flered radng sraegy wh he leverage facor on he es daa se. The leverage facors are calculaed n such a way ha hey ensure a volaly of 0% for each model. 24

25 Fg. 4: Resuls for possble parameer values on he es daa se A.2.2 Resuls for he ou-of-sample daase The able below shows he radng resuls (based on he ou-of-sample daase) of he GM and SCE models for each possble combnaon of he wo radng sraegy parameers. The prof (n %) s leveraged. I ncludes ransacon and leverage coss and s annualsed. GM model Prof d prob Fg. 5: Ou-of-sample resuls for poenal parameer values (GM model) SCE model Prof d Move > 0% > 0.3% > 0.6% Fg. 6: Ou-of-sample resuls for poenal parameer values (SCE model) A.3 Tranng algorhm for he SCE model The SCE model s updaed usng he graden descen mehod as wh he MLP. The resuls concernng he dervaves of he sandard MLP could easly be exended o gan he updae equaons for he SCE model usng he followng relaon [Dunne and Campbell, 997]: 25

26 E() v ~ y = v j v j [9] wh: () v E beng he cross enropy error funcon ~ y beng he newor oupu vecor before applyng he sofmax funcon (whch s n fac he oupu of a MLP) z~ beng he fnal newor oupu vecor of he SCE newor y beng he arge vecor (conssng of zeros and a sngle one) ndcang he oupu node ha maches wh he arge value defned as ( z ~ - y ) v j beng a wegh of he SCE newor ~ y beng he already nown dervaves of he MLP v j A.4 Tranng algorhm for he GM model A.4. Random Vecor Funconal Ln (RVFL) The followng learnng algorhm s based on he Expecaon-Maxmsaon algorhm (EM) [Dempser e al., 977]. A drec applcaon of he EM should speed up he updang process of he weghs a, ß, and u. However, here s sll one problem lef: he remanng weghs u are enerng he cos funcon afer passng rough he sgmod funcon and herefore her dervave s an expresson of e. Ths mples ha he dervave of he cos funcon canno be se o zero for he weghs u and he EM algorhm canno be appled here eher. The weghs u would have o be updaed by he graden descen approach. Snce hs bolenec would slow down he ranng me, anoher adjusmen o he newor archecure s appled. The chosen echnque s called Random Vecor Funconal Ln (RVFL), ha s, he weghs beween he npu and he frs hdden layer (S-layer) are chosen randomly and ep fxed durng he followng ranng. Ths concep offers he possbly o solve our bolenec problem smply by eepng he roublesome weghs u as consans, so ha hey have no o be updaed. Alhough seems o be a reducon of complexy, he remanng newor s sll beng an unversal approxmaor as shown by [Igeln and Pao, 995]. In order o free up he nonlnear resources, he archecure s also slghly changed by addng addonal drec weghs from he npu nodes o he second hdden layer (Glayer). Ths allows he newor o exrac he lnear pars of he ranng daa. Snce hese weghs feed no he second hdden layer, hey belong o he wegh group w and wll herefore be updaed as such. 26

27 A.4.2 Learnng algorhm Wha follows s he presenaon of he updae algorhm ha s responsble for he adjusmen of he newor weghs, so ha maps he ranng npu daa wh he ranng oupu daa 9. The ranng algorhm s based on MacKay s approach [MacKay, 992] where he defnes no only he regularsaon α as a hyper-parameer 20 bu also he weghs a and β snce has been shown [Husmeer, 997] ha hs smplfes he calculaons and leads o beer resuls.. Weghs w / 2 Updae weghs w, usng ernel wdhs ( β ) prevous sep, accordng o: σ and hyper-parameer α from he = GΠ G α + β I w = GΠ y [20] wh G beng he ransposed marx of G y = ( y,..., ) yn, Π as dagonal NxN marx wh ( Π ) = π ( ) ' δ ', wh δ ' as elemens of he un marx agβ ( y µ ( x, q' )) π () = q = w = w,... a G y µ x, q' β wh ( w ) ( ( )) Gaussan dsrbuon ( ) = β x exp 2π 2 G = g( x ),..., g( x )) (WxN marx) wh ( ) ( N 2 β βx G wh σ = x β g conanng all he acves of all he nodes n he npu as well as n he frs hdden layer (S-layer) for he acual npu daa x Snce n pracce H = GΠ G have o handle boh cases. could be sngular or ll-condoned 2 [Husmeer, 999], we By usng he mehod of sngular value decomposon 22, one ges: 9 For furher deals see [Husmeer, 999]. 20 Alhough weghs are updaed n order o map npu and oupu daa, hey have o fulfl some resrcons. Those resrcons are made o preven he weghs from overfng. In our case we assume ha he weghs are normally dsrbued. The furher hey are away from zero he bgger a penaly erm ges. The mporance of he penaly erm n he learnng procedure s expressed by he regularsaon parameer α. 2 Tha s, when he marx s suscepble o round-off errors due o he varaon of s egenvalues over several orders of magnudes. 22 There exss a funcon n Malab, SVD, ha calculaes. 27

28 H UD D =, [2] wh U as a marx composed of he egenvecors and D as a marx of egenvaluesε. If hose egenvalues ε are below εmax 0-6, s bes o se hem o 0. ~ [ D] = [ ε f ε > θ ] [ 0 f ε θ ] 6, wh θ ~ ε as a possble suggeson. [22] max 0 2. Number of well defned parameers Calculae he so called number of well defned parameers γ by usng he egenvalues ε of he Hessan H for each group of weghs w ha feeds no a node of he second hdden layer (G-layer). To do so, he mehod of sngular value decomposon s requred. H N 2 ( β ) π () π () = 2 [ ]( y µ ( x ; w )) g( x ) g ( x ) = β GΠ G [23] Wh he egenvalues ε of of H one ges: W υ υ υ = ε + υ H defned as { ε } W wh W as he number of egenvalues υ= ε γ = [24] α 3. Oupu weghs a a = N N = () π [25] 4. Dsrbuon wdh ß β N π = = N () ( y µ ( x ; w )) π () γ = 2 [26] 5. Hyper-parameers α w w α = γ [27] Those seps are carred on n an erave manner unl he cos funcon s mnmsed. Snce hs algorhm conans regularsaon, namely he Bayesan Evdence Scheme, s no necessary o selec a soppng pon n he way descrbed n secon 4 for he MLP model Neverheless we found que helpful o combne regularsaon wh early soppng as we have menoned n secon (where we are runnng he GM model wh 35 ranng eraon seps bu we fx he weghs durng he 35 seps a he pon where he resul on he es daase s bes). 28

29 REFERENCES Bshop C. M. (994), Mxure Densy Newors. Inernal repor NCRG/4288, Deparmen of Compuer Scence and Appled Mahemacs, Ason Unversy, Brmngham, UK. Dempser, A. P., Lard, N. M. and Rubn, D. B. (977), Maxmum lelhood from ncomplee daa ve he EM algorhm, Journal of he Royal Sascal Socey,B39,, -38. Duns, C. and Wllams, M. (2002), Modellng and Tradng he EUR/USD Exchange Rae: Do Neural Newor Models Perform Beer?, Dervaves Use, Tradng and Regulaon, 8, 3, Duns, C. and Wllams, M. (2003), Applcaons of Advanced Regresson Analyss for Tradng and Invesmen, n C. Duns, J. Laws and P. Naïm [eds.], Appled Quanave Mehods for Tradng and Invesmen, John Wley, Chcheser (forhcomng). Dunne, R. A. and Campbell, N. A. (997), On he parng of he Sofmax acvaon and cross-enropy penaly funcons and he dervaon of he Sofmax acvaon funcon, Proc. 8 h Aus. Conf. on he Neural Newors, Melbourne, Husmeer, D. (997), Modellng Condonal Probably Denses wh Neural Newors. PhD hess, Deparmen of Mahemacs, Kng s College London, hp:// Husmeer, D. (999), Neural Newors for Condonal Probably Esmaon Forecasng Beyond Pon Predcons (Perspecves n neural compung), Sprnger, London. Igeln, B. and Pao, Y. H. (995), Sochasc choce of bass funcons n adapve funconal approxmaon and he funconal-ln ne, IEEE Transacons on Neural Newors, 6, Jordan, M. I. and Jacobs, R. A. (994), Herarchcal mxures of expers and he EM algorhm, Neural Compuaon 6, Kaasra, I. and Boyd, M. (996), Desgnng a Neural Newor for Forecasng Fnancal and Economc Tme Seres, Neurocompung, 0, MacKay, D. J. C. (992), Bayesan nerpolaon, Neural Compuaon, 4, Lsboa, P. J. G. and Velldo, A. (2000), Busness Applcaons of Neural Newors, v-xx, n P. J. G. Lsboa, B. Edsbury and A. Velldo [eds.] Busness Applcaons of Neural Newors: The Sae-of-he-Ar of Real-World Applcaons, World Scenfc, Sngapore. Neuneer, R., Herger, F., Fnnoff, W., and Ormone, D. (994), Esmaon of condonal denses: A comparson of neural newor approaches, n M. Marnaro and P. Morasso [eds.] Sprnger, Shapro, A. F. (2000), A Hchher s Gude o he Technques of Adapve Nonlnear Models, Insurance, Mahemacs and Economcs, 26,

30 Wegend, A. and Nx, A. N. (994), Predcons wh confdence nervals (local error bars), Proceedngs of he Inernaonal Conference on Neural Informaon Processng, Wegend, A. S. and Srvasava, A. N. (995), Predcng condonal probably dsrbuons: A connecons approach, Inernaonal Journal of Neural Sysems, 6, 2,