REGRESSION, THEIL S AND MLP FORECASTING MODELS OF STOCK INDEX

Transcription

1 International Journal of Computer Engineering and Technology (IJCET), ISSN (Print) ISSN (Online) Volume 1 Number 1, May - June (2010), pp IAEME, International Journal of Computer Engineering and Technology (IJCET), ISSN (Print), REGRESSION, THEIL S AND MLP FORECASTING MODELS OF STOCK INDEX K. V. Sujatha Research Scholar Sathyabama University, Chennai sujathacenthil@gmail.com S. Meenakshi Sundaram Department of Mathematics Sathyabama University, Chennai sundarambhu@rediffmail.com IJCET I A E M E ABSTRACT Financial Forecasting or specifically Stock Market prediction is one of the hottest fields of research lately due to its commercial applications owing to the high stakes and the kinds of attractive benefits that it has to offer. Financial time-series is one of the noisiest and non-stationary signals present and hence very difficult to forecast. In this paper we have made an attempt to forecast the daily prices of stock index using a Regression, Theil s and MLP models and the predictive ability of these models are compared using standard error measures. Keywords: Forecasting, Regression, Principal Component, Perceptron, MAPE. 1. INTRODUCTION Trading in stock market indices has gained unprecedented popularity in major financial markets around the world. However, the prediction of stock price index is a very difficult problem because of the complexity of the stock market data, and is affected by many factors including political events, general economic conditions, and investors expectations. Modeling the behavior of a market index is a challenging task for several reasons. There are two major approaches (fundamental and technical) for analyzing stock price prediction [1]. 82

2 Due to the lack of profound knowledge about interior running rules in nonlinear systems like stock system, we have no idea about the variables which are more influential and important and which are not. Input variables are selected only depending on opening and objective historical data in a stock market. To avoid missing important data influencing prediction from the historical data, Principal Component Analysis (PCA), is usually used. A functional principal component technique for the Statistical analysis of a set of financial time series highlights some relevant statistical features of such related datasets [3]. This method is to replace original variables with new ones, which are less in number and not mutually correlative, and contain most of the information of original variables [6]. Xiaoping Yang [4] used PCA to find the principal components that are taken as inputs for predicting stock prices using neural network. Variables high, low, open, volume and adjusted closing were considered for prediction of closing prices using Hybrid Kohonen Self Organizing Map [5]. Liu et al [7] used the back propagation neural networks using moving average, deviation from moving average, turnover moving average, and relative index for prediction. In Versace et al s work[8], values used are open, high, low, close and volume of a specific stock while Baba [9] used change of index, PBR, changes of the turnover by foreign traders, changes of current rates, and turnover in local stock market. MLP outperformed RBF in predicting weekly closing prices using the variables open, high, low and volume [10]. In the recent years, Artificial Neural Networks (ANNs) have been applied to many areas of statistics. One of these areas is time series forecasting [11-19]. The variables considered in this article for predicting the daily closing prices are the historic prices, daily opening, low and high prices of BSE Sensex from 1 st January 2009 till 31 st March Principal component analysis resulted in a single set of variable. The closing prices are predicted by fitting a parametric model Simple Linear Regression and also by classical Non parametric model Theil s Incomplete Method. Multilayer Perceptron is another non parametric model that is used to forecast the daily closing prices taking the principal component as the predictor variable. The forecast error values are measured which is the difference between the actual value and the forecast value for the corresponding period all three models. Error values MAPE, SMAPE and 83

3 MAE are related with how close the forecasted values are to the target ones. Lower the error values, better is the forecaster. 2 MODEL DESCRIPTION 2.1 PRINCIPAL COMPONENT ANALYSIS Principal component analysis is appropriate when there are number of observed variables and wishes to develop a smaller number of artificial variables (called principal components) that will account for most of the variance in the observed variables. The principal components may then be used as predictor or criterion variables in subsequent analyses. Principal component analysis is a variable reduction procedure. It is useful when there is redundancy in the data obtained on the number of variables. Here redundancy means that some of the variables are correlated with one another, possibly because they are measuring the same construct. Because of this redundancy it is possible to reduce the observed variables into smaller number of principal components that will account for most of the variance in the observed variables. Technically, a principal component can be defined as a linear combination of optimally weighted observed variables. Below is the general form for the formula to compute the first component extracted in a principal component analysis: C 1 = b 11 (X 1 )+ b 12 (X 2 )+.. b 1p (X p ) Where, C 1 = the first component extracted b 1p = the regression coefficient for the observed variable p, X p = the value of the observed variable. 2.2 SIMPLE LINEAR REGRESSION Simple linear regression fits a straight line through the set of n points in such a way that makes the sum of squared residuals of the model as small as possible. Regression has the following assumptions The dependent variable is linearly related to the independent variable. Residuals follow normal distribution. Residuals have uniform variance. 84

4 Regression parameters for a straight line model y = a + bx are calculated by the least squares method (minimization of the sum of squares of deviations from a straight line). This differentiates to the following formulae for the slope (b) and the y intercept (a) of the line 2.3 THEIL S INCOMPLETE METHOD A simple, non-parametric approach to fit a straight line to a set of (x,y)-points is the Theil's incomplete method which assumes that points (x 1, y 1 ), (x 2, y 2 )... (x N, y N ) are described by the equation y = a + bx The calculation of a and b takes place as follows: All N data points are ranked in ascending order of x-values. The data are separated into two equal size (m) groups, the low (L) and the high (H) group. If N is odd the middle data point is not included to either group The slope b i is calculated for all points of each group, i.e. b i = (y H,I y L,i )/ (x H,I x L,i ) for i=1,2,,m The median of the m slope values b 1, b 2,..,b m is calculated and it is taken as the best estimate of the slope (b) of the line, i.e. b = median(b 1, b 2,.. b m ). For each data point (x i,y i ) the value of intercept a i is calculated using the previously calculated slope b, i.e. a i =y i - bx i for i=1,2, N The median of the N intercept values a 1, a 2,... a N is calculated and it is taken as the best estimate of the intercept (a) of the line, i.e. a = median (a 1, a 2,... a N ). 85

5 2.4 MULTILAYER PERCEPTRON. A multilayer perceptron is a feed forward network model that maps sets of input data onto a set of appropriate output. It is a modification of the standard linear perceptron in that it uses three or more layers of neurons (nodes) with nonlinear activation functions, and is more powerful than the perceptron in that it can distinguish data that is not linearly separable. The MLP divides the data set in to three parts Training, Testing and Holdout. Training - This segment of data is used only to train the network. Testing - This segment of data is a part of the training data to prevent over training Hold out - This set of data used to assess the final neural network. Hold out data set gives an honest estimate of the predictive ability of the model. Multilayer Layer Perceptron has rescaling option which is done to improve the network training. There are three rescaling options: standardization, normalization, and adjusted normalization. All rescaling is performed based on the training data, even if a testing or holdout sample is defined. The activation function of the hidden layer can be hyperbolic tangent or sigmoid. The units in the output layer can use any one of the following activation function - Identity, Sigmoid, Softmax or Hyperbolic Tangent. 2.5 ERROR MEASURES Error Functions that are used are sum of square error and relative error. Sum of square error is defined as the sum of the squared deviation between observed and the model predicted value. Relative Error is the ratio of an absolute error to the true, specified, or theoretically correct value of the quantity that is in error MeanAverag eerror, MAE = 1 n n t= 1 A t P t MeanAveragePercentError, MAPE = 1 n n t= 1 A t P A t t Where 1 A Pt SymmentricMeanAveragePercentError, SMAPE =, n P At is the actual value and Pt is the predicted value. 86 n t t= 1 At + t

6 3 FINDINGS AND RESULTS Principal Component Analysis of the variable daily high, low and opening prices of BSE Sensex data resulted in the single principal component which is further used in predicting the closing prices by the methods discussed above. The factor determining the number of principal component, the eigen value and the factor loading of the principal components are given in Table 1. Table 1 Principal Component Analysis Eigenvalues Difference Proportion 81.97% 17.98% 0.03% 0.02% Cumulative 81.97% 99.95% 99.98% % Criteria: Kaiser Weights Factors F1 PCA PCA1 V V V V V V V V Exp. Var

7 Initial Descriptive analysis of the daily closing prices and the predictor variable (principal component variable) is given in Table 2. The assumptions of simple linear regression are checked and then with this set of observation the line of regression is fitted. Table 2 Descriptive Statistics Variable Mean Standard Deviation Skewness Kurtosis Daily Closing Principal Component Table 3 Tests of Normality Kolmogorov-Smirnov Shapiro-Wilk Statistic df Sig. Statistic df Significance Closing PCA Durbin Watson value is 2.11 clearly states the absence of autocorrelation. Normality tests Kolmogorov-Smirnov and Shapiro-Wilk were performed and the outcome were displayed in Table 3. From the Table 3 it is clear that both the tests imply that the condition of normality is not met. Using method of Least Squares, the Simple Linear Regression Model for the data is given by Y = X, where X is the principal component variable and Y represents the daily closing price of BSE. By the classical Nonparametric model Theil s method, the model is given by Y = X, where X is the principal component and Y represents the daily closing price. 88

8 For modeling the data with Multilayer Perceptron, the Principal component variable is taken as covariate and the daily closing prices of BSE is considered to be the target variable. Smoothing (standardized, normalized and adjusted normalized) of both the dependent variable and covariates are done successively. All possible combination, changing the activation function of the hidden layer (hyperbolic tangent and sigmoid) and that of the output layer (Identity, hyperbolic tangent and sigmoid) the sum of square error and relative error values are measured with different scaling options. The different combinations of the activation function of the output and the hidden layer with the three rescaling options of the input and target variables resulted in 30 models. The architecture for which the sum of square and relative error was minimum is the one in which the smoothing of both the dependent and covariates are normal with hyperbolic tangent as the activation function of the hidden layer and Identity for the output layer. Table 5 gives the MAE, MAPE, SMAPE and R square values for the above models discussed above. Figure 1 shows how the models predict the closing prices for the last 50 data point. Table 6 MAE, MAPE and SMAPE values Model MAE MAPE SMAPE R 2 Value Linear Regression Theil s Incomplete Method Multilayer Perceptron

9 Figure 1 shows how the models predict the closing prices for the last 50 data point. 4 CONCLUSION The best model for forecasting the daily closing prices was found to be linear regression. The model yielded the least error, only on average measured by the MAPE, on average measured by SMAPE and as the MAE value. The R square value is which indicates that the model is appropriate in predicting the daily closing prices when the daily opening, high and low prices are considered for predicting. This model out performed the nonparametric Theil s method and MLP models. It will be interesting to conduct further studies to compare the results with addition variables. 5. REFERENCES 1. Kai Keng Ang and Chai Quek, (2006), Stock Trading Using RSPOP: A Novel Rough Set-Based Neuro-Fuzzy Approach, IEEE Transactions of Neural Networks, 17(5): Brabazon. T., (2000) A connectivist approach to index modelling in financing markets, In Proceedings, Coil / EvoNet Summer School. University of Limerick. 3. Salvatore Ingrassia and G. Damiana Costanzo. (2005), Functional principal component analysis of financial time series, Vichi M., Monari P., Mignani S., Montanari A. (Eds.) New Developments in Classification and Data Analysis, Pages , Springer-Verlag, Berlin. 90

10 4. Xiaoping Yang (2005), The Prediction of Stock Prices Based on PCA and BP Neural Networks Chinese Business Review, ISSN , USA Volume 4, No.5 (Serial No.23), Page Mark O. Afolabi, Olatoyosi Olude (2007), Predicting Stock Prices Using a Hybrid Kohonen Self Organizing Map (SOM), Proceedings of the 40 th Hawaii International Conference on System Sciences, IEEE. 6. Huixin Ke, Jinghua Huang, Hao Shen (2007), Statistic Analysis in Investigation and Research, Beijing: Beijing Broadcast University Press, Qiong Liu, Xin Lu, Fuji Ren and Shingo Kuroiwa.( 2004), Automatic Estimation of Stock Market Forecasting and Generating the Corresponding Natural language Expression, IEEE Proceedings of the International Conference on Information Technology: Coding and Computing. 8. Versace M., Bhatt R., Hinds O. and Shiffer M. (2004), Predicting the exchange traded fund DIA with a combination of genetic algorithms and neural networks. Expert Systems with applications, Elsevier. 9. Baba N., Naoyuki I. and Hiroyuki A. (2000), Utilization of Neural Networks & GAs for Constructing Reliable Decision Support Systems to Deal Stocks. Proceedings of IEEE-INNS-ENNS International Joint Conference on Neural Networks. 10. Sujatha K. V. and S. Meenakshi Sundaram, (2010), A MLP, RBF Neural Network Model for Prediction in BSE SENSEX Data Set, Proceedings of National Conference on Applied Mathematics. 11. Katijani, Y., W.K. Hipel and A.I. McLeod, (2005), Forecasting Nonlinear Time Series with Feedforward Neural Networks: A Case Study of Canadian Lynx Data. Journal of Forecasting, 24: Yao, J., Y. Li and C.L. Tan, (2000), Option Price Forecasting Using Neural Networks. Omega, 28: Chakraborty, K., Merotra K., Mohan C.K. and Ranka S, (1992), Forecasting the Behavior of Multivariate Time-Series Using Neural Network, Neural Networks, 5: