A Neural Network Approach to Time Series Forecasting

Transcription

1 A Neural Nework Approach o Tme Seres Forecasng Iffa A. Gheyas, Lesle S. Smh Absrac We propose a smple approach for forecasng unvarae me seres. The proposed algorhm s an ensemble learnng echnque ha combnes he advce from several Generalzed Regresson Neural Neworks. We compare our algorhm wh he mos used algorhms on real and synhec daases. The proposed algorhm appears as more powerful han exsng ones. Index Terms Tme seres forecasng, Box-Jenkns mehodology, Mullayer Perceprons, Generalzed Regresson Neural Neworks. I. INTRODUCTION A unvarae me seres s a sequence of observaons of he same random varable a dfferen mes, normally a unform nervals. The goal of unvarae me seres daa mnng s o predc fuure values of a gven varable by lookng a s behavour n he pas. Share prces, profs, mpors, expors, neres raes, populary rangs of polcans, amoun of polluans n he envronmen and number of SARS cases over me are some of examples of me seres. Lagged varables, auocorrelaon and nonsaonary are he major characerscs ha dsngush me seres daa from spaal daa. The dffcules posed by hese specal feaures make forecasng me seres noorously dffcul. In me seres forecasng, he magnude of he forecasng error ncreases over me, snce he uncerany ncreases wh he horzon of he forecas. When forecasng me seres, nerval esmaes are more nformave han smple pon esmaes. Whou a doub, he ARIMA (Auoregressve Inegraed Movng Average) modellng mehodology (popularzed by Box and Jenkns (1976)) and he GRACH (Generalzed Auoregressve Condonal Heeroskedascy) modellng mehodology (proposed by Bollerslev (1986)) are he mos popular mehodologes for forecasng me seres and fuure volaly, respecvely [1, 2]. Neural Neworks (NNs) are now he bgges challengers o convenonal me seres forecasng mehods [3-20]. A varey of NNs are avalable. However, mullayer perceprons (MLP) wh backproapagaon learnng are he mos employed NNs n me seres sudes. We presen a novel approach, usng a Generalzed Regresson Neural Neworks (GRNN) ensemble o he forecasng of me seres and fuure volaly. The remander of hs paper s organzed as follows: he new algorhm s descrbed n secon 2 (wh an overvew of GRNN n secon 2.1 and deals of he proposed algorhm n secon 2.2), Manuscrp receved February 15, I. A. Gheyas and L.S. Smh are boh wh he Deparmen of Compung Scence and Mahemacs, Unversy of Srlng, Srlng FK9 4LA, Scoland, UK (correspondng auhor s IAG, phone: +44 (0) ; fax: +44 (0) ; e-mal: ag@cs.sr.ac.uk). research mehodology n secon 3, resuls and dscussons n secon 4, followed by summary and conclusons n secon 5. II. DEVELOPING A NEW ALGORITHM We presen an mproved algorhm, based on GRNN, for he me seres forecasng. GRNN s a neural nework proposed by Donald F. Spech n 1991 [3]. Ths algorhm has a number of advanages over compeng algorhms. GRNN s non-paramerc. I makes no assumpons concernng he form of he underlyng dsrbuons. A major problem wh he ARIMA and GARCH mehodology and he MLP algorhm s ha hey are global approxmaors, assumng ha one relaonshp fs for all locaons n an area. Unlke hese algorhms, he GRNN s a local approxmaor. In hese algorhms local models are urned no heerogeneous forecasng models adequae o local approxmaon. GRNN s smpler han oher exsng algorhms. I has only one parameer (smoohng facorσ, where0 < σ 1) ha needs o be specfed, bu our research suggess ha he performance s no very sensve o he parameerσ. However, we face a dlemma when applyng he GRNN o he me seres forecasng ask. If we provde only he mos recen pas value, he GRNN generaes he smalles forecasng error bu does no accuraely forecas he correc drecon of change. On he oher hand, f we provde mulple pas observaons, he GRNN can forecas he drecon of change correcly, bu he forecasng error appears o proporonally ncrease wh an ncreasng number of npu values. In order o overcome hs problem, we propose a dervave of he GRNN, whch we call GRNN ensemble. Usng he MLP, he ARIMA & he GARCH mehodologes, ral-and-error mehods are appled n order o secure he bes fng model. The advanage of he proposed algorhm s ha s very smple o mplemen, neher he ral-and-error process nor pror knowledge abou he parameers s requred. A. Generalzed Regresson Neural Neworks In GRNN (Spech, 1991) each observaon n he ranng se forms s own cluser [3]. When a new npu paern x s presened o he GRNN for he predcon of he oupu value, each ranng paern y assgns a membershp value h o x based on he Eucldean dsance d = dx,y ( ) as n equaon 1. 1 h = 2πσ exp d 2 (1) 2 2σ 2 Where σ = smoohng funcon parameer (we specfy a defaul value:σ = 0.5).

2 Fnally, GRNN calculaes he oupu value z of he paern x as n equaon (2). ( h oupu of y ) z = (2) h If he oupu varable s bnary, hen GRNN calculaes he probably of even of neres. If he oupu varable s connuous, hen esmaes he value of he varable. B. Proposed Algorhm We propose a novel GRNN-based ensemble algorhm for me seres forecasng. Two GRNN ensembles, A and B, are bul. A GRNN ensemble s a collecon of GRNNs ha work ogeher. To make a forecas, every member of he ensemble forecass he oupu value. The overall oupu of he ensemble s he weghed average of he oupus produced by he member GRNNs. The GRNN ensemble A forecass he expeced fuure value, and he GRNN ensemble B forecass he expeced fuure volaly of he me seres. The raned GRNN ensemble A and he raned GRNN ensemble B are used o make successve one sep ahead forecass. Ths s done by rollng he sample forward one me sep, usng he forecas as an npu, and makng anoher one-sep-ahead forecas and so on. Pseudo code of he proposed algorhm // Consruc he GRNN ensemble A for forecasng condonal mean Saonarze he seres: Transform a non-saonary me seres no a saonary one by usng a logarhmc or square roo ransformaon and dfferencng. Normalze he values of saonary me seres n he range of 0 o 1. Selecon of npu varables: Measure he perods of waveforms (ha represens he me lags beween wo succeedng peaks or roughs) found n he whole observaon perod. Calculae he weghed average ( N A ) of all of he perods, where recen waveform perod carry more wegh han hose n he pas. Selec he N A mos recen pas values for he curren value of he seres. Creae he GRNN ensemble A wh N A separae GRNNs. Each GRNN s conneced wh a sngle npu node. The npu varable of each nework s dfferen. Tran each member GRNN on he pas values of he saonary me seres daa. Esmae weghs of each member GRNN: Presen ranng paerns o each GRNN separaely for forecasng purposes and calculae weghs for each GRNN: W j A = 1 where, A W j N =1 ( Z Z j ) N Z m, j 1,,T A m N (3). = wegh of he j-h member GRNN of he ensemble A, Z =Acual oupu of he -h paern, Z j =oupu of he -h paern predced by he j-h member GRNN, Z m = maxmum acual oupu n he ranng se, and N = number of observaons. Fnal oupu of he GRNN-ensemble A: The fnal oupu YA of GRNN ensemble A s he weghed average of he oupus of all s member GRNNs. Back-ransform he forecased condonal means no he orgnal doman. // Consruc he GRNN ensemble B for forecasng condonal varance Creae he ranng daase for he ensemble B: Presen ranng paerns o he GRNN ensemble A for predcon purposes and fnd he resdual seres a by subracng he predced value Z? from he acual value Z : a = Z Z 4. ˆ ( ) Normalze he squared resduals o le from 0 o 1. Idenfy predcors for he ensemble B: Coun he number of waves n he squared resdual seres and her assocaed perods. Calculae he weghed average of perods ( N B ). Selec he N B mos recen pas squared resduals for he ensemble B. Consruc he GRNN ensemble B wh N B separae GRNNs. Each GRNN consss of a sngle npu node. The npu of each member GRNN s a dfferen lagged squared resdual. Esmae wegh of each GRNN: Presen ranng paerns of he square resdual seres o each GRNN of he ensemble B for forecasng purposes and esmae weghs for each member GRNN as n equaon (5): W B j = 1 ( a a j ) a k m () 5 where W B j = wegh of he member GRNN of he ensemble B, a = acual square resdual, a j = square resdual predced by he j-h GRNN of he ensemble B, a m = maxmum acual resdual n he ranng se, and k = number of daa pons. The fnal oupu (predced square resdual) of he ensemble B s he weghed average of predcons of he member GRNNs. Calculae condonal varance a me lag (where =1, 2, 3.): Condonal varance a me lag = predced squared resdual me lag () 6 Compue 95% confdence nervals (CI) assocaed wh he condonal mean (predced by GRNN Ensemble A) as n equaon (7), assumng ha he me seres varable has a normal dsrbuon: 95% CI assocaed wh he expeced value a lag = condonal mean ± 1.96 ( condonal varance) ( 7)

3 III. RESEARCH METHODOLOGY An observed me seres can be decomposed no four componens: () Mean value of he daa: We arbrarly choose he mean value, () Long erm rend: We used lnear, exponenal, logarhmc, or polynomal funcons o esmae he rend a me pon, () Cyclcal change (Seasonaly/Perodcy): We esmae he perodcy s a me lag usng a snusodal model, and (v) Nose. The prncple of generang synhec me seres daases s o frs esmae he values of hese four componens usng mahemacal models and hen combne he values of componens no one based on he addve, mulplcave, or addve-mulplcave decomposon model. Fgures 1-2 show samples of synhec daa. We also red wh he Mahalanobs dsance provdng correlaon weghed dsance bu he performance dd no mprove. Our nal suspec s he hgh degree of redundancy among predcors. Mulcollneary among he predcors may lead o sysemac overesmaon of Eucldean (and Mahalanobs) lengh. Ths ssue deserves furher nvesgaon. However, our emprcal resuls demonsrae ha our new algorhm (GRNN-Ensemble) does no suffer from hs shorcomng anymore. V. SUMMARY AND CONCLUSION We propose a smpler and more effcen algorhm (GRNN ensemble) for forecasng unvarae me seres. We compare GRNN ensemble wh exsng algorhms (ARIMA & GARCH, MLP, GRNN wh a sngle predcor and GRNN wh mulple predcors) on fory daases. The one-sep process s eraed o oban predcons en-seps-ahead. The resuls obaned from he expermens show ha he GRNN ensemble s superor o exsng algorhms. IV. RESULTS AND DISCUSSIONS We compare our proposed algorhm (GRNN-Ensemble) wh exsng algorhms (ARIMA-GARCH mehodology, MLP, GS (GRNN wh a sngle predcor) and GM (GRNN wh mulple predcors)) on hry synhec daases and en real-world daases. The real-world daases (obaned from hp://sask.mahemac.unwuerzburg.de/meseres) are he Arlne Daa, he Bankrupcy Daa, he Elecrcy Daa, he Hong Kong Daa, he Nle Daa, he Share Daa, he Sar Daa, he Car Daa, he Sunspo Daa, and he Temperaures Daa. The algorhms are appled o make 10-sep-ahead ou-of-sample forecass. These algorhms were ranked n erms of her accuracy n he nerval esmaon, and nerval lengh. We assgn rank 1 o he bes algorhm, he rank 2 o he nex bes algorhm and so on. Table 1 summarzes he resuls. Obvously, he hgher he accuracy he beer. The lower he average nerval lengh, he beer he performance of he algorhms and he lower he sandard devaon, he more conssen and relable he algorhm. Appendx Tables A1-A4 gve an overvew of he sascal es resuls. Key Fndngs: GRNN-ensemble s sascally sgnfcanly superor o he oher four algorhms boh a very shor horzons (one sep-ahead) and a longer horzons (fve and en sep-ahead). The GRNN wh mulple predcors perform sgnfcanly worse compared wh he oher algorhms a all hree forecas horzons. I s dffcul o say for sure wha causes hs algorhm o generae a bad performance. One possble cause would be ha he GRNN does no assgn weghs o he npu varables. Lagged npu varables are hghly correlaed and hey mgh make each oher redundan o a grea exen. These algorhms mach paerns based on he Eucldean dsance beween npu paerns and sored reference paern. The dsance ncreases many fold above he acual dsance as he number of npu varables ncreases. REFERENCES [1] G.E.P. Box, and G.M. Jenkns, Tme seres analyss: forecasng and conrol, SAN Francsco: Holden-Day, [2] T. Bollerslev. (1986). Generalzed auoregressve condonal heerskedascy. Journal of Economercs. 31, [3] D.F.A. Spech. (1991). A general regresson neural nework. IEEE Transacons on Neural Neworks. 2, [4] R. Alev, B. Fazlollah, and B. Gurmov. (2008). Lngusc me seres forecasng usng fuzzy recurren neural nework. Sof Compung. 12(2), [5] R. Alev, B. Fazlollah, R. Alev, and B. Gurmov, (2006). Fuzzy me seres predcon mehod based on fuzzy recurren neural nework. Lecure Noes n Compuer Scence. 4233, [6] J. Hwang, and E. Lle. (1996). Real me recurren neural neworks for me seres predcon and confdence esmaon. IEEE Inernaonal Conference on Neural Neworks. 4, [7] M. Huken, and P. Sragge. (2003). Recurren neural neworks for me seres classfcaon. Neurocompung. 50, [8] V. Lopez, R. Huera, and J.R. Dorronsoro, (1993). Recurren and feedforward polynomal modellng of coupled me seres. Neural Compuaon. 5(5), [9] P. Coulbaly, and F. Ancl. (2000). Neural nework-based long erm hydropower forecasng sysem. Compuer-Aded Cvl and Infrasrucure Engneerng. 15, [10] R. Drossu, and Z. Obradovc, (1996). Rapd desgn of neural neworks for me seres predcon. IEEE Compuaonal Scence & Engneerng. 3(2), [11] G.P. Zhang, and M. Q, (2005). Neural nework forecasng for seasonal and rend me seres. European Journal of Operaonal Research. 160, [12] A.J. Conway. (1998). Tme seres, neural neworks and he fuure of he Sun. New Asronomy Revews. 42, [13] X. Yan, Z. Wang, S. Yu, and Y. L., Tme seres forecasng wh RBF neural nework. In: Proceedngs of he Fourh Inernaonal Conference on Machne Learnng and Cybernecs Guangzhou., Augus [14] E.S. Cheng, S. Chen, and B. Mulgrew. (1996). Graden radal bass funcon neworks for nonlnear and nonsaonary me seres predcon. IEEE Transacons on Neural Neworks. 7, [15] G. Ln, and L. Chen. (2005). Tme seres forecasng by combnng he radal bass funcon nework and he self-organzng map. Hydrologcal Processes. 19, [16] H.K. Cgzoglu. (2005). Applcaon of generalzed regresson neural neworks o nermen flow forecasng and esmaon. Journal of Hydrologc Engneerng. 10(4), [17] T. Yldrm, and H.K. Cgzoglu, Comparson of generalzed regresson neural nework and MLP performances on hydrologc daa forecasng. In: Proceedngs of he 9 h Inernaonal Conference on Neural Informaon Processng, vol.5, 2002.

4 [18] M.N. Islam, L.K.K. Phoon, and C.Y. Law, Forecasng of rver flow daa wh a general regresson neural nework. In: Proceedngs of Inernaonal symposum on negraed waer resource managemen, 2001(272), [19] H.K. Cgzoglu, and M. Alp, Generalzed regresson neural nework n modellng rver sedmen yeld. Advances n Engneerng Sofware, 37(2), [20] A. Chen, and M.T. Leung, Regresson neural nework for error correcon n foregn exchange forecasng and radng. Compuers & Operaons Research, 31, [21] S. Sngel, N.J. JR. Casellan, Nonparamerc sascs: for he behavoral scences, New York: McGraw-Hll, Second Edon. GRNN ensemble (GE) GRNN wh a sngle predcor (GS) GRNN wh mulple predcors (GM) ARIMA & GARCH (A&G) MLP Table 1: Summary Resuls One-sep-ahead forecas Fve-Sep-ahead forecas Ten-sep-ahead forecas Accuracy All Synhec Real All Synhec Real All Synhec Real daa (%) daa daa daa daa daa daa daa daa Rank Mean STD Max Mn CI wdh Rank Mean STD Max Mn CI wdh Rank Mean STD Max Mn CI wdh Rank Mean STD Max Mn CI wdh Rank Mean STD Max Mn CI wdh

5 Appendx Table A1: Fredman wo-way analyss of varance by ranks Hypohess Tes Sasc Tes Resul Ho: There s no dfference n rank oals of he one-sep-ahead-forecasng Ha: A dfference exss n rank oals of he 5 algorhms n one-sep-ahead forecasng. N=40 Ch-square= df=4 Asymp. Sg.=0.000 Ho: There s no dfference n rank oals of he fve-sep-ahead-forecasng Ha: A dfference exss n rank oals of he 5 algorhms n fve-sep-ahead forecasng. Ho: There s no dfference n rank oals of he en-sep-ahead-forecasng Ha: A dfference exss n rank oals of he 5 algorhms n en-sep-ahead forecasng. N=40 Ch-square= df=4 Asymp. Sg.=0.000 N= 30 Ch-square= df = 4 Asymp. Sg = he performance of one-sep-ahead-forecas across algorhms wh p<0.005 he performance of fve-sep-ahead forecasng across algorhms wh p<0.005 he performance of en-sep-ahead forecasng across algorhms wh p<0.005 Table A2: Resuls of Mulple Comparson Tess (n one-sep-ahead forecasng) GRNN- Ensemble GS GM ARIMA & GARCH MLP GRNN-Ensemble - Yes Yes Yes Yes GS Yes - Yes No No GM Yes Yes - Yes Yes ARIMA & GARCH Yes No Yes - No MLP Yes No Yes Yes * Yes ndcaes a sgnfcan dfference n rank oals beween wo algorhms a he 5% level of sgnfcance, whle No ndcaes no Table A3: Resuls of Mulple Comparson Tess (n fve-sep-ahead forecasng) GRNN-Ensemble GS GM ARIMA & GARCH MLP GRNN- Ensemble - Yes Yes Yes Yes GS Yes - Yes Yes No GM Yes Yes - Yes Yes ARIMA & GARCH Yes Yes Yes - No MLP Yes No Yes No * Yes ndcaes a sgnfcan dfference n rank oals beween wo algorhms a he 5% level of sgnfcance, whle No ndcaes no Table A4: Resuls of Mulple Comparson Tess (n en-sep-ahead forecasng) GRNN ensemble GS GM ARIMA & GARCH MLP GRNN-Ensemble - Yes Yes Yes Yes GS Yes - No No No GM Yes No - Yes Yes ARIMA & GARCH Yes No Yes - No MLP Yes No Yes No Yes ndcaes a sgnfcan dfference n rank oals beween wo algorhms a he 5% level of sgnfcance, whle No ndcaes no