Predicting Direction of Movement of Stock Price and Stock Market Index

Transcription

1 Chapter 3 Predicting Direction of Movement of Stock Price and Stock Market Index This study addresses problem of predicting direction of movement of stock price and stock market index for Indian stock markets. The study compares four prediction models, Artificial Neural Network (ANN), Support Vector Machine (SVM), Random Forest (RF) and Naive Bayes (NB) with two approaches for input to these models. The first approach for input data involves computation of ten technical parameters using stock trading data (open, high, low & close prices) while the second approach focuses on representing these technical parameters as trend deterministic data. Accuracy of each of the prediction models for each of the two input approaches is evaluated. Evaluation is carried out on 10 years of historical data from 2003 to 2012 of two stocks namely Reliance Industries and Infosys Ltd. and two stock market indices CNX Nifty and S&P Bombay Stock Exchange (BSE) Sensex. Experimental results suggest that for the first approach of input data where ten technical parameters are represented as continuous values, Random Forest outperforms other three prediction models on overall performance. Experimental results also show that the performance of all the prediction models improve when these technical parameters are represented as trend deterministic data. 11

2 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET Introduction Predicting stock and stock price index is difficult due to uncertainties involved. There are two types of analysis which investors perform before investing in a stock. First is the fundamental analysis. In this, investors look at intrinsic value of stocks, performance of the industry and economy, political climate etc. to decide whether to invest or not. On the other hand, technical analysis is the evaluation of stocks by means of studying statistics generated by market activity, such as past prices and volumes. Technical analysts do not attempt to measure a security s intrinsic value but instead use stock charts to identify patterns and trends that may suggest how a stock will behave in the future. Efficient market hypothesis states that prices of stocks are informationally efficient; which means that it is possible to predict stock prices based on the trading data (Malkiel and Fama). This is quite logical as many uncertain factors like political scenario of country, public image of the company, etc. will start reflecting in the stock prices. So, if the information obtained from stock prices is pre-processed efficiently and appropriate algorithms are applied, trend of stock or stock price index may be predicted. Since years, many techniques have been developed to predict stock trends. Initially, classical regression methods were used to predict stock trends. Since stock data can be categorized as non-stationary time series data, non-linear machine learning techniques have also been used. ANN (Mehrotra, Mohan, and Ranka) and SVM (Vapnik) are two machine learning algorithms which are most widely used for predicting stock and stock price index movement. Each algorithm has its own way to learn patterns. ANN emulates functioning of a human brain to learn by creating network of neurons while SVM uses the spirit of Structural Risk Minimization (SRM) principle. 3.2 Related Work (Hassan, Nath, and Kirley) proposed and implemented a fusion model by combining the Hidden Markov Model (HMM), ANN and Genetic Algorithms (GA) to forecast financial market behaviour. Using ANN, the daily stock prices are transformed to independent sets of values that become input to HMM. (Wang and Leu) developed a prediction system useful in forecasting mid-term price trend in Taiwan stock mar-

3 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET13 ket. Their system was based on a recurrent neural network trained by using features extracted from Autoregressive Integrated Moving Average (ARIMA) analysis. Empirical results showed that the network trained using 4-year weekly data was capable of predicting up to 6 weeks market trend with acceptable accuracy. Hybridized soft computing techniques for automated stock market forecasting and trend analysis was introduced in (Abraham, Nath, and Mahanti). They used Nasdaq-100 index of Nasdaq Stock Market with Neural Network for one day ahead stock forecasting and a neuro-fuzzy system for analysing the trend of the predicted stock values. The forecasting and trend prediction results using the proposed hybrid system were promising. (Chen, Leung, and Daouk) investigated the probabilistic neural network (PNN) to forecast the direction of index after it was trained by historical data. Empirical results showed that the PNN-based investment strategies obtained higher returns than other investment strategies. Other investment strategies that were examined include the buy-and-hold strategy as well as the investment strategies guided by forecasts estimated with the random walk model and the parametric GMM models. A very well-known SVM algorithm developed by (Vapnik) searches for a hyper plane in higher dimension to separate classes. SVM is a very specific type of learning algorithm characterized by the capacity control of the decision function, the use of the kernel functions and the scarcity of the solution. (Huang, Nakamori, and Wang) investigated the predictability of SVM in forecasting the weekly movement direction of NIKKEI 225 index. They compared SVM with Linear Discriminant Analysis, Quadratic Discriminant Analysis and Elman Backpropagation Neural Networks. The experiment results showed that SVM outperformed the other classification methods. SVM was used in (Kim) to predict the direction of daily stock price change in the Korea Composite Stock Price Index (KOSPI). Twelve technical indicators were selected to make up the initial attributes. This study compared SVM with Back-propagation Neural Network (BPN) and Case-Based Reasoning (CBR). It was evident from the experimental results that SVM outperformed BPN and CBR. Random Forest creates n classification trees using sample with replacement and predicts class based on what majority of trees predict. The trained ensemble, therefore, represents a single hypothesis. This hypothesis, however, is not necessarily contained within the hypothesis space of the models from which it is built. Thus,

4 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET14 ensembles can be shown to have more flexibility in the functions they can represent. This flexibility can, in theory, enable them to over-fit the training data more than a single model would, but in practice, some ensemble techniques (especially bagging) tend to reduce problems related to over-fitting of the training data. (Tsai et al.) investigated the prediction performance of the classifier based on ensemble method to analyse stock returns. The hybrid methods of majority voting and bagging were considered. Moreover, performance using two types of classifier ensembles were compared with those using single baseline classifiers (i.e. Neural Networks, Decision Trees, and Logistic Regression). The results indicated that multiple classifiers outperform single classifiers in terms of prediction accuracy and returns on investment. (Sun and Li) proposed new financial distress prediction (FDP) method based on SVM ensemble. The algorithm for selecting SVM ensemble s base classifiers from candidate ones was designed by considering both individual performance and diversity analysis. Experimental results indicated that SVM ensemble was significantly superior to individual SVM classifier. (Ou and Wang) used total ten data mining techniques to predict price movement of Hang Seng index of Hong Kong stock market. The approaches included Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), K- Nearest Neighbor (KNN) Classification, Naive Bayes based on Kernel Estimation, Logit Model, Tree based Classification, Neural network, Bayesian Classification with Gaussian Process, SVM and Least Squares - Support Vector Machine (LS-SVM). Experimental results showed that the SVM and LS-SVM generated superior predictive performance among the other models. It is evident from the above discussions that each of the algorithms in its own way can tackle this problem. It is also to be noticed that each of the algorithm has its own limitations. The final prediction outcome not only depends on the prediction algorithm used, but is also influenced by the representation of the input. Identifying important features and using only them as the input rather than all the features may improve the prediction accuracy of the prediction models. A two-stage architecture was developed in (Hsu et al.). They integrated Self-Organizing Map (SOM) and Support Vector Regression (SVR) for stock price prediction. They examined seven major stock market indices. Specifically, the self-organizing map was first used to decompose the whole input space into regions where data points with similar statistical

5 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET15 distributions were grouped together, so as to contain and capture the non-stationary property of financial series. After decomposing heterogeneous data points into several homogenous regions, SVR was applied to forecast financial indices. The results suggested that the two stage architecture provided a promising alternative for stock price prediction. Genetic Programming (GP) and its variants have been extensively applied for modelling of the stock markets. To improve the generalization ability of the model, GP have been hybridized with its own variants (Gene Expression Programming (GEP), Multi Expression Programming (MEP)) or with the other methods such as Neural Networks and boosting. The generalization ability of the GP model can also be improved by an appropriate choice of model selection criterion. (Garg, Sriram, and Tai) worked to analyse the effect of three model selection criteria across two data transformations on the performance of GP while modelling the stock indexed in the New York Stock Exchange (NYSE). Final Prediction Error (FPE) criteria showed a better fit for the GP model on both data transformations as compared to other model selection criteria. (Nair et al.) predicted the next day s closing value of five international stock indices using an adaptive ANN based system. The system adapted itself to the changing market dynamics with the help of Genetic Algorithm which tuned the parameters of the Neural Network at the end of each trading session. The study in (Ahmed) investigated the nature of the causal relationships between stock prices and the key macro-economic variables representing real and financial sector of the Indian economy for the period March, 1995 to March, 2007 using quarterly data. The study revealed that the movement of stock prices was not solely dependent on behaviour of key macro-economic variables. (Mantri, Gahan, and Nayak) estimated the volatilities of Indian stock markets using Generalized Autoregressive Conditional Heteroskedasticity (GARCH), Exponential GARCH (EGARCH), Glosten- Jagannathan-Runkle GARCH (GJR-GARCH), Integrated GARCH (IGARCH) & ANN models. This study used fourteen years of data of BSE Sensex & NSE Nifty to estimate the volatilities. It was concluded that there was no difference in the volatilities of Sensex & Nifty estimated under the GARCH, EGARCH, GJR GARCH, IGARCH & ANN models. (Mishra, Sehgal, and Bhanumurthy) tested for the presence of nonlinear de-

6 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET16 pendence and deterministic chaos in the rate of returns series for six Indian stock market indices. The result of analysis suggested that the returns series did not follow a random walk process. Rather it appeared that the daily increments in stock returns were serially correlated and the estimated Hurst exponents were indicative of marginal persistence in equity returns. (Liu and Wang) investigated and forecast the price fluctuation by an improved Legendre Neural Network by assuming that the investors decided their investing positions by analysing the historical data on the stock market. They also introduced a random time strength function in the forecasting model. The Morphological Rank Linear Forecasting (MRLF) method was proposed by (Araújo and Ferreira). An experimental analysis was conducted and the results were compared to Multilayer Perceptron (MLP) networks and Time-delay Added Evolutionary Forecasting (TAEF) method. This study focuses on comparing prediction performance of ANN, SVM, Random Forest and naive Bayes algorithms for the task of predicting stock and stock price index movement. Ten technical parameters are used as the inputs to these models. A Trend Deterministic Data Preparation Layer which converts continuous-valued inputs to discrete ones is proposed. Each input parameters in its discrete form indicates a possible up or down trend determined based on its inherent property. The focus is also to compare the performance of these prediction models, when the inputs are represented in the form of real values and trend deterministic data. All the experiments are carried out using 10 years of historical data of two stocks - Reliance Industries and Infosys Ltd. and two indices S&P BSE Sensex and CNX Nifty. Both stocks and indices are highly voluminous and vehemently traded in and so they reflect Indian Economy as a whole. 3.3 Research Data Ten years of data of total two stock market indices (CNX Nifty and S&P BSE Sensex) and two stocks (Reliance Industries & Infosys Ltd.) from Jan 2003 to Dec 2012 is used in this study. All the data is obtained from and websites. These data form our entire data set. Percentage wise increase and decrease cases (days) of each year in the entire data set are shown in Table 3.1.

7 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET17 Table 3.1: Number of increase and decrease cases (days) percentage in each year in the entire data set of S&P BSE Sensex Year Increase Increase (%) Decrease Decrease (%) Total Total This study uses 20% of the entire data as the parameter selection data. This data is used to determine design parameters of predictor models. Parameter selection data set is constructed by taking equal proportion of data from every year of the ten years. The proportion of percentage wise increase and decrease cases in each year is also maintained. This sampling method enables parameter setting data set to be better representative of the entire data set. The parameter selection data is further divided into training and hold-out set. Each of the set consists of 10% of the entire data. Table 3.2 depicts the number of increase and decrease cases (days) for parameter selection data set. These statistics are for S&P BSE Sensex. Similar data analysis is done for CNX Nifty, for Reliance Industries and Infosys Ltd. Optimum parameters for predictor models are obtained by means of experiments on parameter selection data. After that, for comparing ANN, SVM, Random Forest and Naive Bayes, comparison data set is devised. This data set comprises of entire

8 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET18 ten years of data. It is also divided in training (50% of the entire data) and hold-out (50% of the entire data) set. Details of this data set of S&P BSE Sensex is shown in Table 3.3. These experimental settings are same as in (Kara, Acar Boyacioglu, and Baykan). Table 3.2: Number of increase and decrease cases (days) in each year in the parameter setting data set of S&P BSE Sensex Year Training (Days) Holdout (Days) Increase Decrease Total Increase Decrease Total Total There are some technical indicators through which one can predict the future movement of stocks. Here in this study, total ten technical indicators as employed in (Kara, Acar Boyacioglu, and Baykan) are used. These indicators are shown in Table 3.4. Two approaches for the representation of the input data are employed in this study. The first approach uses continuous-valued representation, i.e., the actual time series, while the second one uses trend deterministic representation (which is discrete in nature) for the inputs. Both the representations are discussed here.

9 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET19 Table 3.3: Number of increase and decrease cases (days) in each year in the comparison data set of S&P BSE Sensex Year Training (Days) Holdout (Days) Increase Decrease Total Increase Decrease Total Total Continuous Representation - The Actual Time Series Ten technical indicators calculated based on the formula as shown in Table 3.4 are given as inputs to predictor models. It is evident that each of the technical indicators calculated based on the above mentioned formula is continuous-valued. The values of all technical indicators are normalized in the range between [-1, +1], so that larger value of one indicator do not overwhelm the smaller valued indicator. Performance of all the models under study is evaluated for this representation of inputs Discrete Representation - Trend Deterministic Data A new layer of decision is employed, which converts continuous-valued technical parameters to discrete value, representing the trend. This layer, proposed in this study, is thereby named as the Trend Deterministic Data Preparation Layer.

10 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET20 Table 3.4: Selected technical indicators & their formula (Kara, Acar Boyacioglu, and Baykan) Name of Indicators Simple n(10 here)-day Moving Average (SMA) Formula C t+c t 1 + +C t 9 n n(10 here)-day Exponential Moving Average (EMA) EMA(k) t = EMA(k) t 1 + α (C t EMA(k) t 1 ) Momentum (MOM) C t C t 9 Stochastic K% (STCK%) C t LL 100 HH LL Stochastic D% (STCD%) n 1 i=0 K t i 10 % 100 Relative Strength Index (RSI) ( n 1 i=0 UP t i/n)/( n 1 i=0 DW t i/n) Moving Average Convergence Divergence (MACD) MACD(n) t n+1 (DIF F t MACD(n) t 1 ) Larry William s R% (LWR) HH C t 100 HH LL Accumulation/Distribution (A/D) Oscillator (ADO) [(H t O t)+(c t L t)] [2 (H t L t)] 100 Commodity Channel Index (CCI) M t SM t 0.015D t

11 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET21 Here, C t is the closing price, O t is the opening price, L t is the low price and H t the high price of t th day, DIF F t = EMA(12) t EMA(26) t, EMA is Exponential Moving Average, EMA(k) t = EMA(k) t 1 + α (C t EMA(k) t 1 ), α is a smoothing factor which is equal to 2, k is the time period of k-day Exponential Moving Average, k+1 EMA(k) is the k-day moving average, LL and HH imply the lowest low and the highest high during the look back period (10 days here), respectively. M t = Ht+Lt+Ct SM t = ( n i=1 M t i+1) n, D t = n i=1 M t i+1 SM t ) DW t is the downward price change at time t. n 3,, UP t means upward price change while Each technical indicator has its own inherent property through which traders generally predict the stock s up or down movement. The job of this new layer is to convert this continuous values to +1 or -1 by considering this property during the discretization process. This way, the input data to each of the predictor models is converted to +1 and -1, where +1 indicates up movement and -1 shows down movement. Simple Moving Average (SMA) and Exponential Moving Average (EMA) help smooth price action and filter out the noise. SMA and EMA of 10-days will hug prices quite closely and turn shortly after prices turn. So when SMA and EMA at time t is greater than at time t 1, it suggests up movement for stock i.e. +1 and vice-a-versa for down movement i.e. -1. Stochastic K% (STCK%), Stochastic D% (STCD%) and Larry Williams R% are stochastic oscillators. These oscillators are clear trend indicators for any stock. When stochastic oscillators are increasing, the stock prices are likely to go up and vice-a-versa (Kim). MACD, RSI, CCI and A/D oscillators also follow the stock trend. Using these indicator values, the trend deterministic input set is prepared and given to the predictor models. Performance of all the models under study is evaluated, for this representation of inputs also. 3.4 Prediction Models ANN Model Inspired by functioning of biological neural networks, ANN are a dense network of inter-connected neurons which get activated based on inputs. A three layer feed-

12 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET22 forward neural network is employed in this study (Mehrotra, Mohan, and Ranka Han, Kamber, and Pei). Inputs for the network are ten technical indicators which are represented by ten neurons in the input layer. Output layer has a single neuron with log-sigmoid as the transfer function. This results in a continuous value output between 0 and 1. A threshold of 0.5 is used to determine the up or down movement prediction. For the output value greater than or equal to 0.5, prediction is considered to be the up movement, else the down movement. Each of the hidden layer s neurons employed tan-sigmoid as the transfer function. The architecture of the three-layered feed-forward ANN is illustrated in Figure 3.1. Figure 3.1: Architecture of ANN model (Kara, Acar Boyacioglu, and Baykan) Gradient descent with momentum is used to adjust the weights, in which, at each epochs, weights are adjusted, so that a global minimum can be reached. Comprehensive parameter setting experiments to determined parameters for each stocks and indices are performed in this study. The ANN model parameters are number of hidden layer neurons (n), value of learning rate (lr), momentum constant (mc) and number of epochs (ep). To determine them efficiently, ten levels of n, nine levels of mc and ten levels of ep are tested in the parameter setting experiments. Initially, value of lr is fixed to 0.1. These parameters and their values which are tested are

13 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET23 summarized in Table 3.5. These settings of parameters yield a total of = 900 treatments for ANN for one stock. Considering two indices and two stocks, total of 3600 treatments for ANN are carried out. The top three parameter combinations that resulted in the best average of training and holdout performance are selected as the top three ANN models for comparison experiments on comparison data set. For these top performing models, learning rate lr is varied in the interval of [0.1, 0.9]. Table 3.5: ANN parameters and their values tested in parameter setting experiments Parameters Value(s) Number of Hidden Layer Neurons (n) 10,20,, 100 Epochs (ep) 1000, 2000,, Momentum Constant (mc) 0.1, 0.2,, 0.9 Learning Rate (lr) SVM Model SVM was first introduced by (Vapnik). There are two main categories for SVMs: Support Vector Classification (SVC) and Support Vector Regression (SVR). SVM is a learning system using a high dimensional feature space. (Khemchandani, Chandra, et al.) stated that in SVM, points are classified by means of assigning them to one of the two disjoint half spaces, either in the pattern space or in a higher-dimensional feature space. The main objective of SVM is to identify maximum margin hyper plane. The idea is that the margin of separation between positive and negative examples is maximized (Xu, Zhou, and Wang). SVM finds maximum margin hyper plane as the final decision boundary. Assume that x i R d, i = 1, 2,, N forms a set of input vectors with corresponding class labels y i {+1, 1}, i = 1, 2,, N. SVM can map the input vectors x i R d into a high dimensional feature space Φ(x i ) H. A kernel function K(x i, x j ) performs the mapping φ( ). The resulting decision boundary is defined in Equation 3.1.

14 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET24 N f(x) = sgn( y i α i K(x, x i ) + b) (3.1) i=1 To get the values of α i, involved in Equation 3.1, quadratic programming problem shown in Equation 3.2 is solved. Maximize N α i 1 2 i=1 N N α i α j y i y j K(x i, x j ) i=1 j=1 Subject to 0 α i c (3.2) N α i y i = 0, i = 1, 2,..., N i=1 The trade-off between margin and misclassification error is controlled by the regularization parameter c. The polynomial and radial basis kernel functions are used in this study and they are shown in Equations 3.3 and 3.4 respectively. P olynomial F unction : K(x i, x j ) = (x i x j + 1) d (3.3) Radial Basis F unction : K(x i, x j ) = exp( γ x i x j 2 ) (3.4) where d is the degree of polynomial function and γ is the constant of radial basis function. Choice of kernel function, degree of kernel function (d) in case of polynomial kernel, gamma in kernel function (γ) in case of radial basis kernel and regularization constant (c) are considered as the parameters of SVM in this study. To determine them efficiently, four levels on d, ten levels of γ and four to five levels of c are tested in the parameter setting experiments. These parameters and their values which are tested are summarized in Table 3.6. For one stock, these settings of parameters yield a total of 20 and 40 treatments for SVM employing polynomial and radial basis kernel functions respectively. Considering two indices and two stocks, total of 240 treatments for SVM are carried out. One parameter combination for each of the polynomial kernel SVM and radial basis kernel SVM, that resulted in the best average of training and holdout performance is selected as the top SVM model for comparison experiments.

15 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET25 Table 3.6: SVM parameters and their values tested in parameter setting experiments Parameters Values (polynomial) Values (radial basis) Degree of Kernel Function (d) 1, 2, 3, 4 - Gamma in Kernel Function (γ) - 0.5, 1.0, 1.5,, 5.0, 10.0 Regularization Parameter (c) 0.5, 1, 5, 10, , 1, 5, Random Forest Decision tree learning is one of the most popular techniques for classification. Its classification accuracy is comparable with other classification methods, and it is very efficient. The classification model learnt through these techniques is represented as a tree and is known as a decision tree. ID3 (Quinlan, Induction of decision trees ), C4.5 (Quinlan, C4. 5: programs for machine learning) and CART (Breiman et al.) are decision tree learning algorithms. Details can be found in (Han, Kamber, and Pei). Random Forest belongs to the category of ensemble learning algorithms (Breiman). It uses decision tree as the base learner of the ensemble. The idea of ensemble learning is that a single classifier is not sufficient for determining class of test data. Reason being, based on sample data, classifier is not able to distinguish between noise and pattern. So, it performs sampling with replacement, such that, given n trees that are to be learnt, are based on these data set samples. In the experiments performed in this study, each tree is learnt using 3 features selected randomly. After creation of n trees, when testing data is used, the decision which majority of trees come up with is considered as the final output. This also avoids problem of over-fitting. Implementation of random forest algorithm in this study is summarized in the Algorithm 1. Number of trees (ntrees) in the ensemble is considered as the parameter of Random Forest. To determine it efficiently, it is varied from 10 to 200 with increment of 10 each time during the parameter setting experiments. For one stock, these settings of parameter yield a total of 20 treatments. Considering two indices and two

16 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET26 stocks, total of 80 treatments are carried out. The top three parameter values that resulted in the best average of training and holdout performance are selected as the top three Random Forest models for the comparison experiments. Algorithm 1 Random Forest Input: training set D, number of trees in the ensemble k Output: a composite model M 1: for i = 1 to k do 2: Create bootstrap sample D i by sampling D with replacement 3: Select 3 features randomly 4: Use D i and randomly selected three features to derive tree M i 5: end for 6: return M Naive Bayes Classifier Naive Bayes classifier assumes class conditional independence (Han, Kamber, and Pei Markov and Larose). Given test data, Bayesian classifier predicts the probability of data belonging to a particular class. To predict probability it uses concept of Bayes theorem. Bayes theorem is useful in calculating the posterior probability, P (C X), from P (C), P (X C), and P (X). Bayes theorem states that P (C X) = P (X C)P (C) P (X) (3.5) Here, P (C X) is the posterior probability which tells us the probability of hypothesis C being true given that event X has occurred. In this work, hypothesis C is the probability of belonging to class Up/Down and event X is our test data. P (X C) is a conditional probability of occurrence of event X, given hypothesis C is true. It can be estimated from the training data. The working of naive Bayesian classifier, or simple Bayesian classifier, is summarized as follows. Assume that, m classes C 1, C 2,..., C m and event of occurrence of test data, X, is given. Bayesian classifier classifies the test data into a class with the highest probability. By Bayes theorem (Equation (3.5)),

17 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET27 P (C i X) = P (X C i)p (C i ), i = 1, 2,, m (3.6) P (X) Given data sets with many attributes (A 1, A 2,..., A n ), it would be extremely computationally expensive to compute P (X C i ). In order to reduce computation in evaluating P (X C i ), the naive assumption of class conditional independence is made. This presumes that the values of the attributes are conditionally independent of one another, given the class label of the tuple (i.e. that there are no dependence relationships among the attributes). Therefore, P (X C i ) = n P (x k C i ) (3.7) k=1 = P (x 1 C i ) P (x 2 C i ) P (x n C i ) Here, x k denotes the value of attribute A k for tuple X. Computation of P (x k C i ) depends on whether it is categorical or continuous. If A k is categorical, P (x k C i ) is the number of observations of class C i in training set having the value x k for A k divided by the number of observations of class C i in the training set. If A k is continuous-valued, Gaussian distribution is fitted to the data and the value of P (x k C i ) is calculated based on Equation 3.8. f(x, µ, σ) = 1 σ 2 2π e (x µ) /2σ 2 so that, P (x k C i ) = f(x k, µ Ci, σ Ci ) (3.8) Here, µ Ci and σ Ci are the mean (i.e., average) and standard deviation, respectively, of the values of attribute A k for training tuples of class C i. These two quantities are then plugged into Equation 3.8 together with x k, in order to estimate P (x k C i ). P (X C i )P (C i ) is evaluated for each class C i in order to predict the class label of X. The class label of observation X is predicted as class C i, if and only if P (X C i )P (C i ) > P (X C j )P (C j ) for 1 j m; j i (3.9) Bayesian classifiers also serve as a theoretical justification for other classifiers that do not explicitly use Bayes theorem. For example, under specific assumptions, it

18 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET28 can be demonstrated that many neural networks and curve-fitting algorithms output the maximum posteriori hypothesis, as does the naive Bayesian classifier. 3.5 Experimental Evaluation This section discusses about evaluation measures, experimental methodology and results of the experimentations Evaluation Measures Accuracy and F-measure are used to evaluate the performance of proposed models. Computation of these evaluation measures require estimating Precision and Recall which are evaluated from True Positives (TP), False Positives (FP), True Negatives (TN) and False Negatives (FN) (Han, Kamber, and Pei Markov and Larose). These parameters are defined in Equations 3.10, 3.11, 3.12 and P recision positive = P recision negative = Recall positive = Recall negative = T P T P + F P T N T N + F N T P T P + F N T N T N + F P (3.10) (3.11) (3.12) (3.13) Precision is the weighted average of precision positive and negative while Recall is the weighted average of recall positive and negative. Accuracy and F-measure are estimated using Equations 3.14 and 3.15 respectively. It is to be noticed that, the value range of these measures is between 0 and 1, where 0 indicates the worst performance while 1 indicates the best performance. Accuracy = F measure = T P + T N T P + F P + T N + F N 2 P recision Recall P recision + Recall (3.14) (3.15) Accuracy of a classifier on a given test set is the percentage of test set observations that are correctly classified by the classifier. It is also to be mentioned that F-measure used throughout the thesis is traditional/balanced F-measure or F 1 -score.

19 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET Experimental Methodology & Results Experiments are carried out in two phases. First phase of experimentation considers input as the continuous-valued data. The best parameter combinations are identified by means of experiments on parameter setting data set for each of the prediction models. These parameter combinations with corresponding accuracy and f-measure during parameter setting experiments are reported in Tables 3.7, 3.8 and 3.9. It is to be noted that there are no parameters to be tuned for naive Bayes classifier. Comparison data set (which is a whole data set) is used to compare the performance of various prediction models. 50% observations of these data set are used to learn various prediction models along with the parameters identified during parameter tuning experiments. Remaining 50% observations are used as holdout observations and accuracy reported in the chapter shows the accuracy on these holdout observations. Table 3.10 reports average accuracy and f-measure of each of the models during comparison experiment. Average accuracy and f-measure reported are averaged over the top performing models. It can be seen that naive Bayes with the Gaussian process is the least accurate while Random Forest is the most accurate with average accuracy of nearly 84%. Figure 3.2 depicts the prediction process when data is continuous-valued. Second phase of experimentation is identical to the first one except that the input to the models is trend deterministic data. The idea is depicted in Figure 3.3. Tables 3.11, 3.12 and 3.13 show result of best performing combinations for ANN, SVM and Random Forest respectively during parameter setting experiments. It is to be noted that when data is represented as trend deterministic data, naive Bayes classifier is learnt by fitting multivariate Bernoulli distribution to the data. Results on comparison data set for all the proposed models are reported in Table Final comparison shows that all the models perform well with discrete data input but SVM, random forest and naive Bayes performance better than ANN. The accuracy of SVM, Random Forest and naive Bayes is nearly 90%.

20 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET Discussions Stock market data is an example of non-stationary data. At particular time there can be trends, cycles, random walks or combinations of the three. It is desired that if a particular year is part of a cycle, say a bullish one, the proposed models should follow this pattern for trend prediction. Same can be considered for a trending year. However, usually stock values of a particular year are not isolated and there are days with random walks. Stock values are also affected by external factors creating trends and state of the country s economy. Political scenarios are also the influencing factors which may result in cycles. Table 3.7: Best three parameter combinations of ANN model and their performance on continuous-valued parameter setting data set CNX Nifty ep:n:mc & lr= :20: :10: :10:0.9 Accuracy F-measure S&P BSE Sensex ep:n:mc & lr= :80: :40: :100:0.1 Accuracy F-measure Infosys Ltd. ep:n:mc & lr= :70: :150: :10:0.3 Accuracy F-measure Reliance Industries ep:n:mc & lr= :50: :40: :20:0.5 Accuracy F-measure

21 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET31 It can be seen from the results that all the models perform well when they are learnt from continuous-valued inputs but the performance of each of the models is further improved when they are learnt using trend deterministic data. The reason behind the improved performance is justified in the remainder of this section. Table 3.8: Best two parameter combinations (one for each type of kernel) of SVM model and their performance on continuous-valued parameter setting data set CNX Nifty Kernel:Polynomial c=100,d=1 Kernel:RBF c=0.5,γ=5 Accuracy F-measure S&P BSE Sensex Kernel:Polynomial c=100,d=1 Kernel:RBF c=0.5,γ=5 Accuracy F-measure Infosys Ltd. Kernel:Polynomial c=0.5,d=1 Kernel:RBF c=0.5,γ=5 Accuracy F-measure Reliance Industries Kernel:Polynomial c=0.5,d=1 Kernel:RBF c=1,γ=5 Accuracy F-measure Trend deterministic data is prepared by discretizing the continuous-valued data. The idea is based on the intuition that each continuous-valued parameters when

22 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET32 compared with its previous day s value indicates the future up or down trend. The data is discretized based on these heuristics. When this data is given as the input to the model, we are already inputting the trend based on each input parameters. It is actually the situation where each of the input parameters signifies about the probable future trend and we have the actual future trend to identify the transformation from probable trends to the correct trend. This is a step forward because original dataset is converted to trend deterministic data set. Now prediction models have to determine co-relation between the input trends and output trend. Though it is non-linear, it is easy to create a model which can transform input trends to the output trend. Table 3.9: Best three parameter combinations of random forest model and their performance on continuous-valued parameter setting data set CNX Nifty ntrees Accuracy F-measure S&P BSE Sensex ntrees Accuracy F-measure Infosys Ltd. ntrees Accuracy F-measure Reliance Industries ntrees Accuracy F-measure

23 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET33 Figure 3.2: Predicting with continuous-valued data Table 3.10: Performance of prediction models on continuous-valued comparison data set Prediction Models Stock/Index ANN (Kara, Acar Boyacioglu, and Baykan) SVM Accuracy F-measure Accuracy F-measure S&P BSE Sensex CNX Nifty Reliance Industries Infosys Ltd Average Random Forest Naive Bayes(Gaussian) Stock/Index Accuracy F-measure Accuracy F-measure S&P BSE Sensex CNX Nifty Reliance Industries Infosys Ltd Average

24 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET34 Figure 3.3: Predicting with trend deterministic data Further to notice is that for any stock or index, there are scenarios, when a stock or index is trading at some value, say 200, then due to some external factors, it may start trading at higher price, say 400, and then stabilize at that higher value. If prediction model is given direct continuous-valued input, it is possible that it tries to establish relation between the values in 200 and that in 400, which is not required as far as predicting future trend is considered. Each parameter is relative while signifying future trend. It means that the important thing is how its value has changed with respect to previous days rather than the absolute value of change. Therefore, trend deterministic data which is discrete in nature is basically the statistical indication of whether the stocks are overbought or over-sold and is value independent. Hence, these input parameters, when represented as probable future trends serve as a better measure of stocks condition rather than the scenario, when they are represented as continuous-valued. 3.7 Conclusions The task focused in this study is to predict direction of movement for stocks and stock price indices. Prediction performance of four models namely ANN, SVM, Random Forest and Naive Bayes is compared based on ten years ( ) of historical data

25 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET35 of CNX Nifty, S&P BSE Sensex, Infosys Ltd. and Reliance Industries from Indian stock markets. Ten technical parameters reflecting the condition of stock and stock price index are used to learn each of these models. Table 3.11: Best three parameter combinations of ANN model and their performance on discrete-valued parameter setting data set CNX Nifty ep:n:mc & lr= :50: :100: :70:0.3 Accuracy F-measure S&P BSE Sensex ep:n:mc & lr= :100: :30: :90:0.1 Accuracy F-measure Infosys Ltd. ep:n:mc & lr= :50: :70: :80:0.4 Accuracy F-measure Reliance Industries ep:n:mc & lr= :100: :90: :100:0.5 Accuracy F-measure

26 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET36 Table 3.12: Best two parameter combinations (one for each type of kernel) of SVM model and their performance on discrete-valued parameter setting data set CNX Nifty Kernel:Polynomial c=1,d=1 Kernel:RBF c=1,γ=4 Accuracy F-measure S&P BSE Sensex Kernel:Polynomial c=1,d=1 Kernel:RBF c=5,γ=1.5 Accuracy F-measure Infosys Ltd. Kernel:Polynomial c=0.5,d=1 Kernel:RBF cc=1,γ=3 Accuracy F-measure Reliance Industries Kernel:Polynomial c=1,d=1 Kernel:RBF c=0.5,γ=4 Accuracy F-measure A Trend Deterministic Data Preparation Layer is proposed on the basis of the fact that each technical indicator has its own inherent property through which traders generally predict the stocks up or down movement. Utilizing these heuristics, the trend deterministic data preparation layer converts each of the technical parameter s

27 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET37 continuous value to +1 or 1, indicating probable future up or down movement respectively. Table 3.13: Best three parameter combinations of random forest model and their performance on discrete-valued parameter setting data set CNX Nifty ntreess Accuracy F-measure S&P BSE Sensex ntreess Accuracy F-measure Infosys Ltd. ntreess Accuracy F-measure Reliance Industries ntreess Accuracy F-measure Experiments with continuous-valued data show that naive Bayes (Gaussian Process) model exhibits least performance with 73.3% accuracy and random forest with highest performance of 83.56% accuracy. Performance of all these models is improved

28 CHAPTER 3. PREDICTING DIRECTION OF MOVEMENT IN STOCK MARKET38 significantly when they are learnt through trend deterministic data. ANN is slightly less accurate in terms of prediction accuracy compare to other three models which perform almost identically. The accuracy of 86.69%, 89.33%, 89.98% and 90.19% is achieved by ANN, SVM, Random Forest and Naive Bayes (Multivariate Bernoulli Process) respectively. Table 3.14: Performance of prediction models on discrete-valued comparison data set Prediction Models Stock/Index ANN SVM Accuracy F-measure Accuracy F-measure S&P BSE Sensex CNX Nifty Reliance Industries Infosys Ltd Average Random Forest Naive Bayes Stock/Index Accuracy F-measure Accuracy F-measure S&P BSE Sensex CNX Nifty Reliance Industries Infosys Ltd Average Trend Deterministic Data Preparation Layer proposed in this chapter exploits inherent opinion of each of the technical indicators about stock price movement. The layer exploits these opinions in the same way as the stock market s experts. In earlier researches, the technical indicators were used directly for prediction while this study first extracts trend related information from each of the technical indicators and then utilizes the same for prediction, resulting in significant improvement in accuracy. The proposal of this Trend Deterministic Data Preparation Layer is a distinct contribution to the research. Owing to the noteworthy improvement in the prediction accuracy, the proposed system can be deployed in real time for stocks trend prediction, making investments more profitable and secure.