Missing Data Prediction and Forecasting for Water Quantity Data

Transcription

1 2011 Inernaional Conference on Modeling, Simulaion and Conrol ICSIT vol.10 (2011) (2011) IACSIT ress, Singapore Missing Daa redicion and Forecasing for Waer Quaniy Daa rakhar Gupa 1 and R.Srinivasan Deparmen of Chemical Engineering & Technology, Insiue of Technology, BHU, Varanasi, India 2 Nalco Technology Cener, une, India Absrac. In indusrial applicaions, especially in waer reamen plans, i is necessary o obain flow daa (quaniy and qualiy) for a sysem over a broad range of ime. In mos cases i is no possible o obain he parameer of ineres in a closed form. Generaion of he daa over he ime period wihin he known range is no possible or may be exremely ime-consuming. Numerous mehods are available for inerpolaion or exrapolaion o deermine he unknown daa or missing daa wihin or ouside a range of known daa poins. In his paper wo developed mehodologies were analyzed, one o predic he missing values wihin a daa range and anoher o forecas he seasonal daa ouside he daa range for a raw waer quaniy daa. A proven mehod like Two-Direcional Exponenial Smoohing (TES) is applied for predicing he missing values for a raw sream flow daa and a seasonal daa was forecased using Exponenially Weighed Moving Average (EWMA) by using he known daa values of previous wo seasons. Boh he mehods prediced he daa wihin and ouside he period range of waer quaniy daa wih good resuls. Keywords: Missing Daa, Two-Direcional Exponenial Smoohing (TES), Exponenially Weighed Moving Average (EWMA), Forecasing. 1. Inroducion redicing he missing values wihin a ime period and forecasing daa for fuure periods is of common ineres for many indusrial applicaions. Replacing missing daa wih ime series wihin he range of known daa is crucial for more accurae design proposals and performance evaluaion. Especially, in waer reamen plans i is required o replace he missing values of he waer qualiy and quaniy daa o gain knowledge in he sysem and also o manage he waer resources effecively. The waer quaniy and qualiy daa are defined as ime series variables which are recorded a successive ime inervals. To use exising operaional daa as an inpu o a process simulaion model, he missing daa should be replaced. To provide a more accurae design proposal and sysem performance, a reasonable and reliable predicion of missing daa is highly needed o deermine he correc variabiliy of waer reamen plan daa. Mos common mehods available for predicing he missing values in ime series are replacing all missing values for a given variable wih mean, median or oher locaion saisics [1], SAS program using a ime-series model o predic missing values [2,3], Average Neares Observaion (ANO)[4], and Two-Direcional Exponenial Smoohing (TES) [5]. Also, forecasing and predicing he fuure daa can be very helpful in preparing in advance for any unexpeced values. The forecas of seasonal high or low oupu is crucial in aking care of he insalled capaciy sysems and herefore rigger alarms as needed according o he values. opular mehods like Auo Regressive Inegraed Moving Average (ARIMA) [6], Exponenially Weighed Moving Average Mehod (EWMA) [7], Thomas-Fiering mehod [8] are also used for forecasing daa from a known daa se. Comprehensive mehods for forecasing based on he exponenially weighed moving average are available [7] for series wih rend, non-seasonal, and seasonal series. + Corresponding auhor. Tel.: ; Fax: address: sramanahan@nalco.com 98

2 In his paper a proven mehod like Two-Direcional Exponenial Smoohing (TES) is applied for predicing he missing values wih ime series in a daa se. A sample daa was used o es he applicabiliy of he mehod by inenionally deleing some of he known values. Also, he raw sample daa se was exended o forecas he values using Exponenially Weighed Moving Average (EWMA). 2. Mehodology & Resuls The mehodology for predicing he unknown values wihin and ouside a known range by applying esablished mehods like Two-Direcional Exponenial Smoohing (TES) and Exponenially Weighed Moving Average (EWMA) respecively are discussed in he following secions. For applicaion of hese mehods, sample raw daa of sream flow over a period of 5 years was used [9]. Here, only he sream flow rae daa was considered for predicion and forecasing, assuming ha he waer flow daa was used by an indusrial plan nearby for waer managemen. A es daa se was creaed by removing some daa poins for predicing he sream flow wihin he known range and he sample es daa se for wo known seasons was used for predicing ouside he range. 2.1 Missing Daa redicion For predicing he missing daa of sream flow, Two-Direcional exponenial smoohing (TES) [5] was applied. Tes daa of sream flow rae by removing some daa poins over a period of five year is shown in Figure 1. Figure 1. Raw daa of sream flow rae [9] wih missing values for a 5 year period TES mehod was developed o replace missing daa. TES mehod depends on a suiable Exponenial Smoohing (ES) mehod and was developed by using Hol s linear rend algorihm mehod. The TES mehod esimaes missing daa poins based on he auocorrelaions of he ime series o accoun for he fac ha he missing values occur a non-random imes. The TES mehod is designed o represen boh forward and backward auocorrelaions in he ime series, can decrease he difference above caused by differen direcions. The firs sep in TES mehod is o generae he full daa se of daa using Average neares observaion (ANO) mehod [4]. The ANO mehod will replace he missing values wih he average of he neares previous and he following observaion i.e., he values are esimaed by a weighed average of he neares observaions wih higher weigh given o he closer observaion. Once he daa se is generaed using he ANO mehod, he missing values are prediced using a suiable Exponenial Smoohing mehod, Hol s linear rend mehod, in he forward and reverse direcion. An ES mehod could generae differen values depending on he direcion of he ime series. The Hol s Linear Trend algorihm can be represened as a = δ Y + (1 δ)(a -1 + b -1 ) (1) b = γ(a a -1 ) + (1 γ)b -1 (2) 99

3 where Y is he acual value a ime, a and a -1 are inerceps (smoohed levels) a ime and -1 respecively, b and b -1 are he slopes (smoohed rends) a ime and -1 respecively, δ and γ are smoohing consans ha are beween 0 and 1 [1,10]. The smoohing consans, δ is used o smooh he new acual and rend-adjused previously smoohed level and γ is used o smooh or average he rend, which eliminaes some of he random error refleced in he unsmoohed rend. The smoohing consans deermine he weigh given o mos recen pas observaions and herefore conrol he rae of smoohing or averaging. Values near 1 give weighage o more recen daa and near 0 disribue he weighs o consider daa from he more disan pas daa. The averaged forward and backward ES esimaes were used for predicing he missing daa poins. The TES mehod is a combinaion ime series mehod and represened for missing values as TES = (ES forward, + ES backward, )/ 2 (3) Figure 2 shows he flow diagram of TES mehod. This TES mehod is applied o he es daa (Figure 1). Figure 3 shows he raw sream flow rae daa (wih missing values) and he prediced missing sream flow values (by TES). For esimaing he ES forward and ES backward, he consans δ and γ were chosen as 0.7 and 0.9 by rial and error mehod. The replaced sream flow values deermined by he TES mehod are relaively close o he original values. Sream flow daa wih missing values Forward ANO daa wih puaive missing values replacemen Backward ANO daa wih puaive missing values replacemen Forward ES daa: ES forward, Backward ES daa: ES backward, TES = (ES forward, + ES backward, ) / 2 for missing daa Figure 2. Flow diagram of TES mehod Figure 3. Comparison beween he prediced values by TES mehod and raw daa wih missing values 2.2 Daa Forecasing The forecasing raio seasonal mehod [7] developed for predicing he sales rae was used for predicing he waer quaniy sream flow daa. The forecasing raio seasonal mehod can be used o predic he daa for nex season when he previous seasons share he comparable behaviour. I follows he exponenially weighed moving average mehod of smoohing he random flucuaions which is exremely easy o compue 100

4 wih minimum hisorical daa required. The prediced or he forecased value can be esimaed from he following exrapolaion equaion: ES + T = S ' + T N where S is he smoohed seasonally adjused rae in period, is he periodical adjusmen raio for he h period, N is he number of he periods in one seasonal and T is he number of forecas periods. The smoohed seasonally adjused rae in he period, is represened as S ' = AS + (1 A) S (5) where S is he value in period and A is a consan ha ranges beween 0 o 1, which deermines how fas he exponenial weighs decline over he pas consecuive periods. The curren seasonal adjusmen raio is obained by combining he curren raio of daa and he daa wih he seasonal adjusmen rae from a season ago. S ' B = + (1 B) S N where he consan B, deermines how fas he exponenial weighs decline over he pas season i.e., one period drawn from each season. By solving Equaion (5) and (6), he explici analyic expression for he new seasonal raio [7] can be given as A(1 B) A S ' = [ ] N S + [ ] S (7) AB AB B B(1 A) S = [ ] N + [ ] (8) AB AB S The above mehodology is used o predic he seasonal daa for he hird season by using he known values of previous wo seasons. The es daa published for sream flow rae was considered for seasonal variaion of 32 weeks for each season (high and low peaks) ou of 257 weeks [9]. The main objecive is o predic he hird season of 32 weeks daa considering he 64 weeks daa. Here he sample daa was carefully chosen o be in variaion wih seasonal changes from he available daa of 257 weeks. Figure 4 shows he raw es daa (wo seasons of 64 weeks) and he forecased daa for hird season (32 weeks). Here he consans A and B were chosen as 0.1 and 1 by rial and error mehod. The forecased daa for hese consan values seems o be more similar o he immediae previous season i.e., he second season. However, if he consan values are change o 0.2 and 0.1, as shown in Figure 5, he forecased daa is owards he firs season. The significance of he consans in he equaion deermines he daa correlaions o he immediae or previous season s daa. (4) (6) Fig.4. Comparison of forecased daa for one season wih he raw es daa [9] of wo seasons 101

5 . 3. Conclusions Fig 5.Comparison of forecased daa for one season wih he raw es daa [9] of wo seasons In his paper, a Two-Direcional exponenial smoohing (TES) mehod was applied for predicing he missing values wih ime series and exponenially weighed moving average (EWMA) was applied for forecasing he waer sream flow daa wih seasonal variaion. Boh he mehods prediced he daa wihin and ouside he period range wih good resuls. 4. Acknowledgemens This work was carried ou a he Nalco Technology Cener, une, as a par of an indusry-universiy collaboraive research inernship projec. The auhors would like o acknowledge Dr. Hari Reddy, Direcor, Nalco Technology Cener, une, India and Dr. A.K. Verma, Head, Deparmen of Chemical Engineering & Technology, Insiue of Technology, BHU, Varanasi, India for providing heir suppor o carry ou his projec. 5. References [1] DeLurgio, S. A.. Forecasing rinciples and Applicaions, McGraw-Hill, New York (1998) [2] H. Junninen, H. Niska, K. Tuppurainen, J. Ruuskanen, M. Kolehmainen. Mehods of impuaion of missing values in air qualiy daa ses. Amos. Environ. (2004), 38, [3] T. Scheider. Analysis of incomplee climae daa:esimaion of mean values and covariance marices and impuaion of missing values. J. Clim.14(5) (2001), [4] J. Huo. Applicaion of saisical mehods and process models for he design and analysis of acivaed sludge wasewaer reamen plans. hd disseraion (2005), The Univ. of Tennessee, Knoxville, Tenn. [5] J. Huo, C. D. Cox, W. L. Seaver, R.B. Robinson, Y. Jiang. Applicaion of Two-Direcional Time Series Models o Replace Missing Daa. J. of Environmenal Engineering. ASCE (April 2010) [6] G.E.. Box, G.M. Jenkins, Time Series Analysis Forecasing and Conrol. Holden-Day, San Francisco, 1976 [7] C. C. Hol, Forecasing Seasonals and rends by exponenially weighed moving averages, Inernaional J. of Forecasing 20 (2004) 5-10 [8] R.T. Clarke. Mahemaical Models in Hydrology, FAO of Unied Naions, Rome, 1984 [9] A. Kurunc, K. Yurekli, O. Cevik. erformance of wo sochasic approaches for forecasing waer qualiy and sream flow from Yeşilırmak River, Turkey. Environmenal Modeling & Sofware 20 (2005) [10] SAS Insiue Inc. SAS onlinedoc, version 8, SAS Insiue, Cary, N.C. (1999) 102