to Combine Fundamental and Technical Analysis A thesis presented to the faculty of In partial fulfillment of the requirements for the degree

Transcription

1 Restaurant Industry Stock Price Forecasting Model Utilizing Artificial Neural Networks to Combine Fundamental and Technical Analysis A thesis presented to the faculty of the Russ College of Engineering and Technology of Ohio University In partial fulfillment of the requirements for the degree Master of Science Ronald W. Dravenstott June Ronald W. Dravenstott. All Rights Reserved.

2 2 This thesis titled Restaurant Industry Stock Price Forecasting Model Utilizing Artificial Neural Networks to Combine Fundamental and Technical Analysis by RONALD W. DRAVENSTOTT has been approved for the Department of Industrial and Systems Engineering and the Russ College of Engineering and Technology by Gary R. Weckman Associate Professor of Industrial and Systems Engineering Dennis Irwin Dean, Russ College of Engineering and Technology

3 3 ABSTRACT DRAVENSTOTT, RONALD W., M.S., June 2012, Industrial and Systems Engineering Restaurant Industry Stock Price Forecasting Model Utilizing Artificial Neural Networks to Combine Fundamental and Technical Analysis Director of Thesis: Gary R. Weckman Stock price forecasting is a classic problem facing analysts. Forcasting models have been developed for predicting individual stocks and stock indices around the world and in numerous industries. According to a literature review, these models have yet to be applied to the restaurant industry. Strategies for forecasting typically include fundamental and technical variables. In this thesis, fundamental and technical inputs were combined into an Artificial Neural Network stock prediction model for the restaurant industry. Models were designed to forecast 1 week, 4 weeks, and 13 weeks into the future. The model performed better than the benchmarks. The prediction accuracy of the model reached as high as 60%. The model with the most success was a Multilayer Perceptron Artificial Neural Network with 2 hidden layers having 40 and 20 processing elements in those layers using the hyperbolic tangent transfer function and Delta Bar Delta learning algorithm. Approved: Gary R. Weckman Associate Professor of Industrial and Systems Engineering

4 4 ACKNOWLEDGMENTS Thank you to those who helped and supported me through my educational journey. I would especially like to thank Kelley, my parents, John, and Dr. Weckman. I could not have done this without you.

5 5 TABLE OF CONTENTS Page Abstract... 3 Acknowledgments... 4 List of Tables... 7 List of Figures Introduction Approaches to Stock Market Prediction Literature Review Data Set Description Methodology Classification Models Function Approximation Models Plan of Action Building and Training the ANN Models Testing Different Network Architectures Single-Company Models All-Company Models Testing Different Network Sizes Testing Different Learning Algorithms Best Performing Model Parameters Results Classification Networks Function Approximation Models Performing Sensitivity Analysis Constructing Models Based on Sensitivity Analysis Evaluating Sensitivity Analysis Further Evaluating Initial Models Development of the Maintenance Approach... 41

6 6 6.4 Maintenance Approach Results Classification Networks Class Prediction Models Class Prediction Models Class Prediction Models Benchmark Results Stepwise Multilinear Regression Results Analyst Results Buy and Hold Results Summary and Discussion Conclusion Future Research References... 63

7 7 LIST OF TABLES Page Table 1: Data Inputs and Lag Lengths Table 2: Source Legend Table 3: Company List Table 4: Model Evaluation Examples Table 5: Model Characteristics Table Table 6: Model Evaluation Example Table 7: Initial Function Approximation Model Results Table 8: 13-week Prediction Model Confusion Matrix Table 9: 4-week Prediction Model Confusion Matrix Table 10: 4-week Prediction Model Results with Differing Test Period Lengths Table 11: Maintenance ANN Prediction Model Results Table 12: Maintenance ANN Prediction Model Confusion Matrix Table 13: 2-Class Prediction Model Confusion Matrix Table 14: 3-class Prediction Model Confusion Matrix Table 15: 5-class Prediction Model Confusion Matrix Table 16: Results of 5-class Prediction Models Table 17: Stepwise Multilinear Regression Results Table 18: Stepwise Multilinear Regression Confusion Matrix Table 19: Stepwise Multilinear Regression Price Coefficients Table 20: Analyst Confusion Matrix Table 21: Buy and Hold Confusion Matrix Table 22: Maintenance ANN Network and Benchmark Results... 57

8 8 LIST OF FIGURES Page Figure 1: A Snapshot of the S&P 500 Index Over the Last Ten Years Figure 2: A Snapshot of McDonalds Corporation and Brinker Intl., Inc. Compared to the S&P 500 Over the Past Ten Years Figure 3: A Head and Shoulders Pattern [6] Figure 4: Project Flowchart Figure 5: Network Architecture Figure 6: Analyst Hit Rate Versus Decision Point Figure 7: 8-Week Period Returns and Plot of S&P 500 Index for Test Period... 56

9 9 1 INTRODUCTION The stock market is among the most difficult time series to predict. The volatile behavior of the stock market causes it to suddenly rise and fall without much discernable pattern to the point that many still argue that it cannot be predicted. Mandelbrot [27] argues that the stock market moves with, wild randomness exemplified by distributions with infinite variance. Mandelbrot [27] goes on to conclude that the stock market cannot be more accurately predicted by inventing better statistical methods. This conclusion partially illustrates the opposition to the possibility of stock price prediction. Something that varies so wildly is exceedingly difficult to analyze and predict. The efficient market hypothesis (EMH), which has three forms, also illustrates the opposition to the possibility of stock market prediction [14]. The weak form of the EMH assumes an investor will not make profits consistently [14] The semi-strong form of the EMH assumes the market makes a decision on the price based on all public information [14] The strong form of the EMH states that the current market price fully reflects all available information, implying that the price movement history has no impact on the future price movement [14] As information is random, or unpredictable in general, so is the stock price movement, which is then classified a random walk pattern. If the efficient market hypothesis is true,

10 10 then any attempts to predict the series should fail. If this thesis is able to reliably predict the stock market, it will present an argument against the efficient market hypothesis. The volatile behavior exhibited by the stock market make the time series of the stock prices a non-linear series. Measures have been taken to try and remove some of the volatility of the stock market, making it more stable and predictable. Stock market indices reduce some of the volatile behavior of the stock market by grouping a large number of stocks together into one number. The S&P 500 Index includes 500 of the largest companies in the United States equities market. Even an index such as the S&P 500 has non-linear behavior, as demonstrated by Figure 1 plotting the index over the past 10 years. Figure 1: A Snapshot of the S&P 500 Index Over the Last Ten Years (Source: Google finance Over the last 10 years the index has moved a great deal, but if an investor simply bought and held an index fund based on the S&P 500 as many recommend, their portfolio would be down 3.88% over the period (excluding dividends, fees, inflation, and taxes). If an investor was able to buy during the lower periods and sell during the higher periods,

11 11 they could make a significant profit. This index containing 500 large cap US stocks has considerable volatility and non-linear behavior. Predicting individual stocks is significantly more difficult than predicting stock indices because individual stocks are considerably more volatile than stock market indices. They tend to generally follow stock market indices, such as the S&P 500, but with more volatility. McDonalds Corporation is the only company in the data set of 16 companies compiled for this thesis that is contained in the S&P 500 Index. Figure 2: A Snapshot of McDonalds Corporation and Brinker Intl., Inc. Compared to the S&P 500 Over the Past Ten Years (Source: Google finance By comparison, the S&P 500 (blue) looks tame compared to McDonalds Corporation (red) and Brinker Intl., Inc. (yellow) over the past 10 years. Figure 2 clearly demonstrates that individual stocks express more volatility than the S&P 500 Index. By indexing hundreds of stocks into an index, the volatility of the stock market is partially removed, as are some of the opportunities for making a profit.

12 12 2 APPROACHES TO STOCK MARKET PREDICTION Analysts have employed a wide variety of tools to predict the stock market. They are vastly different in calculation and application. Fundamental analysis and technical analysis are the most popular ways to predict price movements of individual companies. Traditionally they are viewed as separate tools to solve the similar problems of evaluating a company s value and predicting a company s future stock price movement. Behavioral analysis has also been employed to analyze the behavior of traders in the marketplace, and their decision-making tendencies. Dart Board Theory is a technique utilizing random selection of stocks. Dart Board Theory, like behavioral analysis, is less accepted. Nonlinear modeling techniques such as Artificial Neural Networks have been employed as well. Fundamental analysis uses data to determine the fundamental value of a company. The focus is on long-term prediction with the assumption that the stock price of a company will eventually move toward its fundamental value. A company that is undervalued by the stock market is considered a Buy situation, and a company that is overvalued by the stock market is considered a Sell situation. Fundamental values are assumed to be long-term estimates of where the individual company stock price is moving, as it will eventually reflect fundamental value in the next month, quarter or year. Various kinds of data can be used for fundamental analysis such as company sales, number of locations, etc. Fundamental analysis will not use historical stock price data.

13 13 Technical analysis uses time-delayed historic price data to predict where the stock price is moving [26]. The focus is on the current price movement and trends, and the future stock price of the company. Technical analysis employs a broad array of indicators that are calculated and used by traders. Charts are also analyzed and widely used by technical traders. Technical analysis is a short-term stock price prediction method used for forecasting where the stock will be the next day, the next hour, or the next minute [31]. Another approach to stock market prediction is behavioral analysis. Similar to technical analysis, behavioral analysis predicts where the stock price is moving, though the prediction is based on analyzing behavioral factors. Bollen [9] used people s general mood via Twitter to predict where stock price was moving. Lee [25] uses volume and price momentum to determine where the stock price is moving based on investors behavior. This approach is not widely accepted, though it could be incorporated into a model predicting stock price prediction. According to Shaikh [38], Dart Board Theory of Stock Selection is implemented by throwing a dart at the Wall Street Journal and selecting the stock that the dart hit. The chance of success following this strategy is higher than when following expert advice [38]. Traditionally when discussing stock price prediction techniques, fundamental and technical analysis are the only two techniques mentioned. Artificial Neural Networks (ANNs) have also been used to predict the stock market. An ANN is a non-linear statistical data modeling tool that functions similar to

14 14 biological neural networks such as the human brain [25]. ANNs process information and adapt during a training phase to create the model [25]. Multi-Layer Perceptron ANNs allow for prediction of non-linear data, but present the problem of selecting the number of layers and processing elements contained in each layer [25]. The trained network can be examined with sensitivity analysis to extract knowledge about the complex relationships between the inputs and outputs of the model, as well as predict outputs for given inputs to the model. ANNs have been considered ideal for stock market prediction by Qian [35] due to the fact that they can learn hidden patterns from data and the composition of an ANN is complex enough to be very difficult for other traders to create a copy. Similarly, Elliman [15] suggests that if a predictive model could produce excess returns, the model itself may change the market it was modeling. A baseline comparison to an ANN is traditionally multi-linear regression [33]. Similar to technical analysis, ANNs predict the future based on the past, and assume that past trends predict future values. ANNs can incorporate the inputs used by fundamental analysis as well. According to Leigh [26] there is a new style of decision support system for stock market prediction. The support system involves using computing power to combine previously uncombined techniques, including: pattern recognition, neural networks, and genetic algorithms [26]. This research will attempt to combine technical analysis, fundamental analysis, and artificial neural networks together in a similar way.

15 15 3 LITERATURE REVIEW Some authors have concluded that the stock market can be predicted, rejecting the idea that stock price is a random walk. Qian [35] had prediction results up to 65% correct using a model implementing ANN, k-nearest neighbor, and a decision tree. Gallagher [18] concluded that stock prices are not pure random walks, supporting the meanreversion hypothesis. Atsalakis [5] concluded that ANNs and neuro-fuzzy models can forecast the stock market accurately, function in trading strategies effectively, and usually outperform conventional models. Many authors have supported fundamental or technical analysis in equity valuation. Ohlson [32] supports fundamental analysis using empirical examples. Alvarez- Ramirez [1] supports technical analysis using an empirical example. Lo [28] uses goodness-of-fit test to test technical analysis results and concludes that technical analysis can add value to the investment process. Bettman [7] had success integrating fundamental and technical analysis in an empirical example. Bettman [7] obtained better results with a model containing both fundamental and technical analysis than in either analysis used in isolation. Also, Lam [24] successfully integrated fundamental and technical analysis in a neural network. Wittkemper [42] used fundamental variables in a neural network to predict beta values for German corporations. Authors predicted individual stocks and stock indices for varying periods of time. Bao [6] predicted the S&P 500 Index from 2002 to Arujo [3] tested indices and individual stocks from 1998 to Cheng [13] tested a stock index from 2000 to 2005.

16 16 Lo [28] tested various technical indicators on NYSE, AMEX, and NASDAQ stocks from 1962 to 1996 where available. Mitsuo [31] used NASDAQ data from 2000 to Wittkemper [42] analyzed the financial statements of German corporations from 1967 to Atsalakis [5] surveyed more than 100 journal articles and conference papers that used ANNs to predict the stock market. A vast majority of these articles predicted the movement of stock market indices, composed of many individual stocks. Of the papers that were accessible, the following predicted individual stock prices: Weckman [41], Lakshminarayanan [23], Hui [21], Arujo [3], Araujo [4], Hadavandi [19], Quah [36], and Lam [24]. Various baseline performance measures were found in the literature. Qi [34] compared the performance of a model using neural networks to a model based on Buy and Hold, though the validity of the results were later questioned by Racine [37]. Quah [36] evaluated the performance of a model by calculating excess returns generated by a neural network portfolio. Atsalakis [5] cited works using Multilinear Regression, ARMA and ARIMA models, genetic algorithms, random walk, and Buy and Hold strategy. Chavarnakul [12] compared neural network results to volume adjusted moving averages without neural networks, simple moving averages, and the Buy and Hold strategy. Akinwale [2] compared the results of a Neural Network to Regression analysis using Nigeria Stock Market prices. Vaisla [40] compared a Neural Network prediction to

17 17 regression and found that the Neural Network model vastly outperformed the Regression model. Bao [6] tried to identify the turning point in the series, or the point where the time series would switch from a generally positive to a generally negative direction, or vice versa. Different patterns were identified including the Head and Shoulders pattern in Figure 3 [6]. Chang [11] predicted turning points using a backpropagation neural network. Huang [20] used Support Vector Machine to predict the weekly movement direction of the NIKKEI 225 Index. Figure 3: A Head and Shoulders Pattern [6]

18 18 Chakravarty [10] used neural networks to predict stock market indices, though only using two input variables, and concluded that performance could be improved if more input variables were included in the model. Chavarnakul [12] had success using neural networks to predict price of stocks and volume traded. Chenoweth [14] combined neural networks with decision rules to predict the S&P 500 index, retraining the network after each prediction was made to use earlier testing data as additional training data. Wu [43] used neural networks combined with Fuzzy Logic to predict the S&P 500 index. Hui [21] combined a neural network, which are supervised learning, with a Kohonen network, which is unsupervised learning, to predict individual stocks on the Kuala Lumpur stock exchange index. Leigh [26] combined neural networks, pattern recognition, genetic algorithms, and technical analysis to predict the NYSE composite index. Cheng [13] combined neural networks, rough set theory, and C4.5 to predict the Taiwan Stock Exchange Index. Ferreira [17] forecasted the Dow Jones Industrial Average Index and the Nasdaq Index using neural networks and genetic algorithms. Hadavandi [19] forecasted the daily stock price of British Airlines, Ryanair Airlines, IBM and Dell Corporation using genetic fuzzy systems and artificial neural networks. Kohara [22] combines prior knowledge and Artificial Neural Networks to predict the daily stock price of TOPIX, a weighted average of select stocks on the Tokyo Stock Exchange. Mitsuo [31] used Artificial Neural Networks and Candle technicians to predict daily NASDAQ returns. Quah [36] used a neural network with 7 financial inputs to select stocks for a portfolio, testing the implementation by using a moving-window for training and testing to retrain

19 19 the model. Araujo [4] combined an Artificial Neural Network with a Particle Swarm Optimizer to predict stock prices for four companies. Enke [16] suggests that the data to be used for stock market prediction will have to be lagged appropriately to represent realistic situations. For example, Inflation was reported on January 14, 2011 for the month of December A prediction generated during December 2010 therefore cannot use the information reported on January 14 th. Therefore, inflation will need to be lagged at least 6 weeks to accurately represent available data at any given week. This approach simulates using the model in the present for each of the instances in the dataset. Not only does it make testing the model more similar to real-life, the approach also provides more useful data for training purposes to create a more accurate network. Not all inputs needed to be lagged, as they were current as reported. Input names and the associated lagging for each input are shown in Table 1 below.

20 20 Table 1: Data Inputs and Lag Lengths Input Name Frequency Source Lag Type Dow Jones Industrial Average Weekly [H] None Technical EV Weekly [A] None Technical Price Weekly [A] None Technical S&P 500 Weekly [H] None Fundamental ALL URBAN SAMPLE: ALL ITEMS - ANNUAL INFLATION RATE NADJ (CPI) Monthly [B] 6 Weeks Fundamental Common Shares Outstanding Monthly [A] None Fundamental EPS FYR1 Monthly [A] None Fundamental EPS NTM Monthly [A] None Fundamental Inflation Rate Monthly [F] 6 Weeks Fundamental New Home Sales Seasonally Adjusted Annual Rate Monthly [D] 8 Weeks Fundamental Sales FYR1 Monthly [A] None Fundamental Unemployment Rate Seasonally Adjusted Monthly [C] 6 Weeks Fundamental UNIV OF MICHIGAN CONSUMER SENTIMENT - EXPECTATIONS VOLN Monthly [B] 4 Weeks Fundamental US CONSUMER CONFIDENCE INDEX - PRESENT SITUATION SADJ (PSI) Monthly [B] 4 Weeks Fundamental US CONSUMER CONFIDENCE INDEX SADJ (CCI) Monthly [B] 4 Weeks Fundamental Cash Quarterly [A] 19 Weeks Fundamental EBITDA LTM Quarterly [A] 19 Weeks Fundamental EPS LTM Quarterly [A] 19 Weeks Fundamental LTM Gross Margin ($ amt) Quarterly [A] 19 Weeks Fundamental LTM Net Income Quarterly [A] 19 Weeks Fundamental MRQ Total Assets Quarterly [A] 19 Weeks Fundamental Number of Locations Quarterly [G] 19 Weeks Fundamental Sales LTM Quarterly [A] 19 Weeks Fundamental Same Store Sales Quarterly [G] 19 Weeks Fundamental Totalcommonequity Quarterly [A] 19 Weeks Fundamental LTM Capital expenditures Yearly [A] 10 Weeks Fundamental LTM CFO Yearly [A] 10 Weeks Fundamental Aroon Down Daily [H], [I] None Fundamental Aroon Up Daily [H], [I] None Fundamental Bollinger Band (bottom) Daily [H], [J] None Fundamental Chaikin Oscillator Daily [H], [K] None Fundamental Moving Average Convergence Divergence Daily [H], [L] None Fundamental Mass Index Daily [H], [M] None Fundamental Money Flow Index Daily [H], [N] None Fundamental Relative Momentum Index Daily [H], [O] None Fundamental Relative Strength Index Daily [H], [P] None Fundamental Stochastic Oscillator (14 day) Daily [H], [Q] None Fundamental Trading Band (Top) Daily [H], [R] None Fundamental TRIX Indicator Daily [H], [S] None Fundamental Williams' %R Daily [H], [T] None Fundamental Exponential Moving Average (10 day) Daily [H] None Fundamental Simple Moving Average (10 Day) Daily [H] None Fundamental Exponential Moving Average (26 day) Daily [H] None Fundamental

21 21 Table 2: Source Legend [A] [B] [C] [D] [E] [F] [G] [H] Source Legend Thomson-Reuters Datastream SEC Filings 10K 10Q and Press Releases Commodity Systems, Inc. via The literature as a whole suggests that stock market prices are predictable, though not everyone agreed [30] [37]. There are many individual methods used to predict the stock market, but a reoccurring theme utilizing neural networks is to design a decisionmaking algorithm to decide on an action based on network output [14] [21] [26] [39] [43]. No articles that were available focused on or even mentioned the restaurant industry. The purpose of this thesis is to investigate the prediction capability of ANNs in the restaurant industry, using data relevant to the restaurant industry.

22 22 4 DATA SET DESCRIPTION The current data set obtained contains numerous inputs of varying frequency. The data set contains 15 companies in the restaurant industry. Individual companies are listed in a table below. Most companies have data from the past ten years. Caribou Coffee Company, Inc. had an initial public offering toward the end of 2005, and only has five years of data available. The data set includes 43 inputs, including raw data and calculated inputs. From the raw data, moving averages and technical indicators can be calculated to supplement the raw data and become additional inputs. Data is reported weekly, monthly, quarterly, and yearly for the various inputs. Some inputs for various companies contain missing rows, sometimes in sizeable groups. The data came in various forms and required significant cleaning and organization to create a single sheet to use with an ANN. An ANN requires data to be complete with no missing data in a two-dimensional table, with columns as inputs and rows as instances. The raw data was organized and expanded into rows of weekly entries. Any quarterly data is the same for each week of the typically 13 weeks in that particular quarter. Missing values were either approximated by carrying over the previous value, or the row or the column was deleted because it lacked sufficient data to be useful. Typically a column missing more than 20% of the data was automatically eliminated. The economic data was also matched to each of the weekly entries for each of the companies. The data consisted of 15 companies from the restaurant industry. It included a mix of large and small market capitalization stocks. It also included restaurants serving

23 23 different consumer needs. Data includes casual dining restaurants, such as Cheesecake Factory. Data also includes quick serve restaurants such as McDonald s and Panera Bread. Finally, data contains family restaurants, such as Cracker Barrel and Bob Evans. A complete list of companies is listed below in Table 3. Table 3: Company List Company Symbol Exchange BJ's Restaurants, Inc. BJRI NASDAQ Bob Evans Farms, Inc. BOBE NASDAQ Buffalo Wild Wings BWLD NASDAQ The Cheesecake Factory Incorporated CAKE NASDAQ Caribou Coffee Company, Inc. CBOU NASDAQ Cracker Barrel Old Country Store, Inc. CBRL NASDAQ CEC Entertainment, Inc. CEC NYSE Darden Restaurants, In. DRI NYSE Brinker Intl., Inc. EAT NYSE McDonald's Corporation MCD NYSE P.F. Chang's China Bistro PFCB NASDAQ Panera Bread Company PNRA NASDAQ Papa John's Int'l, Inc. PZZA NASDAQ Red Robin Gourmet Burgers, Inc. RRGB NASDAQ Ruby Tuesday, Inc. RT NYSE Sonic Corporation SONC NASDAQ

24 24 5 METHODOLOGY The goal of this thesis is to determine a methodology that combines the powers of fundamental and technical analysis with an ANN. Ideally this model will accurately predict future stock price movement for the restaurant industry and be developed into an effective trading strategy. After the model is created it will be analyzed for key relationships using sensitivity analysis. Knowledge will also be extracted from the neural network itself. Prediction models were constructed for each company individually to predict the future stock price, as well as all companies combined. Predictions will range from one week to one quarter (about 13 weeks) in advance, which is the time range between fundamental and technical analysis predictions. To construct the ANN prediction models, data was divided into training, cross-validation, and testing sets for each company. The training and cross validation data was the earlier data for a given company. Data from 1/4/2002 3/21/2008 was used for training and cross validation, and data from 3/28/2008 3/19/2010 was used for testing data. Separating testing data from the data used for training and validating the model guarantees a more realistic evaluation of model performance. Training and cross validation data was randomized between the two. This data set was used to create the neural network and the baseline comparison models. Two types of models will be created to attempt to predict the stock market: a classification network and a function approximation network. A classification network will be used to classify the instances into categories, such as Buy and Sell. A function

25 25 approximation network will be used to generate a point estimate of the future stock price. Both of these approaches can be implemented into a stock trading strategy in similar fashions. If the ANN model predicts a Buy situation or a stock price higher than the current stock price, the stock trading strategy will implement the Buy move and so forth. Both types of models will be used to predict one week, one month, and one quarter into the future. 5.1 Classification Models The data was classified in multiple ways to attempt to implement a successful classification model. The simplest classification scheme had 2 classes: Buy and Sell. If a future stock price was greater than or equal to the current stock price, it was classified as a Buy. Otherwise, it was a Sell. Another classification scheme included 3 classes: Buy, Hold, and Sell. If the future stock price was up more than 10% annualized, it was classified as a Buy. If the future stock price was down more than 10% annualized, it was classified as a Sell. Otherwise it was classified as a Hold. The most complex classification scheme had 5 classes: Strong Buy, Buy, Hold, Sell, and Strong Sell. If a future stock price was up more than 20% annualized, it was classified as a Strong Buy. If a future stock price was up more than 10% annualized, it was classified as a Buy, and so forth. The classification rules are detailed in Table 4.

26 26 Table 4: Model Evaluation Examples 2-class prediction Prediction Percentage Model Action >= 0% Buy <0% Sell 3-class prediction Prediction Percentage (annualized) Model Action > 10% Buy (-10%) - (10%) Hold < -10% Sell 5-class prediction Prediction Percentage (annualized) Model Action >20% Strong Buy (10%) - (20%) Buy (-10%) - (10%) Hold (-20%) - (-10%) Sell <-20% Strong Sell For each of the classification schemes, different network variations were tested. The size of the network was tested from 1 hidden layer to 2 hidden layers. The number of processing elements was varied from 10 to 80 in each layer. The transfer functions were varied between tanh and sigmoid. The learning algorithm was varied between momentum and delta bar delta. The length of prediction was varied as well, predicting 1 week, 4 weeks, and 13 weeks into the future. Varying each of these parameters allows a model to be created to best fit the dataset. Also, the results will show which time period prediction is appropriate for this methodology. These various network structures are listed in Table 4 below.

27 27 Table 5: Model Characteristics Table Prediction Interval Learning Algorithm Transfer Function Hidden Layers Processing Elements 1 Week Delta Bar Delta Tanh 1 and 2 10 to 80 in each layer 1 Week Delta Bar Delta Sigmoid 1 and 2 10 to 80 in each layer 1 Week Momentum Tanh 1 and 2 10 to 80 in each layer 1 Week Momentum Sigmoid 1 and 2 10 to 80 in each layer 1 Month Delta Bar Delta Tanh 1 and 2 10 to 80 in each layer 1 Month Delta Bar Delta Sigmoid 1 and 2 10 to 80 in each layer 1 Month Momentum Tanh 1 and 2 10 to 80 in each layer 1 Month Momentum Sigmoid 1 and 2 10 to 80 in each layer 1 Quarter Delta Bar Delta Tanh 1 and 2 10 to 80 in each layer 1 Quarter Delta Bar Delta Sigmoid 1 and 2 10 to 80 in each layer 1 Quarter Momentum Tanh 1 and 2 10 to 80 in each layer 1 Quarter Momentum Sigmoid 1 and 2 10 to 80 in each layer 5.2 Function Approximation Models Function Approximation Models for this thesis were created to take in 43 inputs, and output a future stock price point estimate. This point estimate can then be used to determine the predicted direction of the stock price. This direction then determines the decision the model suggests. If the model predicts the stock price to increase, the decision is a Buy. If the model predicts a stock price to decrease, it is a Sell. Function approximation models were used as an alternative to classification networks due to the lack of success of classification. It was difficult to get the model to understand that an increase in stock price warrants a buy situation. In building function approximation models, users can change transfer functions, network architecture, and learning algorithms. By varying these parameters, a model was catered fit the dataset. Preparing the data for the function approximation models was much simpler than preparing the data for the classification model. Other than pulling in the future stock price

28 28 for the appropriate prediction period (1, 4, or 13 weeks), the compiled data could be put directly into the software. ANNs are designed to take in numerical inputs and produce numerical outputs. 5.3 Plan of Action An ANN will be trained and tested for each of the 15 companies. Once trained, the ANN will be put into an investment strategy for the individual company. The ANN will be used to make predictions from 1 week, 4 weeks, and 13 weeks in advance. For example, an evaluation of the performance of a classification one week forecasting model is found below: Table 6: Model Evaluation Example Week ANN Prediction Current Stock Price Stock Price in 1 Week Correct? Return from Period 1 Buy Yes 9% 2 Buy Yes 12% 3 Buy No -7% 4 Sell Yes 8% 5 Buy No -11% Based on the example, this model will average a weekly return of 2%. The model was correct 60% of the time, which is labeled the hit rate in most literature [5]. As opposed to a classification model, a function approximation model will also be created. This function approximation model will predict the actual stock price in the future. This numerical point estimate allows for additional performance measures to be calculated for model evaluation. The magnitude of the prediction direction conveys the

29 29 confidence of the model. Also, the numerical prediction can also be classified into categories the same as the classification model described above. A baseline model will be constructed to compare with performance of the neural network. The baseline model will be constructed using (stepwise) multi-linear regression. Also, the performance of the neural network investment model will be compared to the Buy and Hold strategy as implemented by Qi [34], which is to buy the first week of testing and sell the final week of testing. The Buy and Hold strategy will be used on the individual stock being predicted, as well as for the S&P 500 index during the testing period. The S&P 500 has been used as a benchmark to compare the performance of professional managers of mutual funds by Bogle [8]. A visual plan of action is illustrated in Figure 4.

30 30 Figure 4: Project Flowchart Models will be analyzed with statistical and non-statistical measures as those conveniently calculated by the neural network software, as well as those that have been used as performance measures in stock market prediction [5]. Statistical measures will be: MSE and Correlation Coefficient (R^2). MSE is a measure of the difference between the predicted and actual value, which is squared to include positive and negative differences as additive errors, rather than have them cancel each other out. R^2 is a

31 31 measure of how much of the variation in the data is explained by the model. An R^2 value of 0.82 would show that the model accounts for 82% of the variation in the data. Non-statistical measures used will be hit rate and ROR. Hit rate will offer a measure of the contribution of a model above a random walk strategy. If the hit rate is calculated at 55%, then the model is contributing 5% accuracy above what can be expected from a long-term random choice. According to Qian [35] a hit rate of 56% is a good result. ROR will offer a measure of the return that can be generated using the model. The performance measures used for this thesis will allow the most accurate and effective predictive models to be chosen. A model that predicts the large movements in stock price effectively will produce a higher ROR without necessarily a higher hit rate. All of the possible performance measures will be taken into account when selecting the best-performing, which can then be compared to the established benchmarks. These performance measures will also be used to compare the performance of the models utilizing neural networks to all the other benchmarks. The ROR can be compared between analyst predictions, the model s predictions, Buy and Hold strategy, and other benchmarks. Risk free rates for Treasury Bills are known, and any effective trading strategy must exceed the risk free rate. Otherwise, the model, which has more risk than the Treasury bills, is not a viable investment option. Investors would be able to make more money with less risk.

32 32 Choosing a model by combining all the performance measures gives confidence to the model s ability to perform in the real world. An ideal model would have a high ROR, high hit rate, high R^2, and low MSE. The bottom line in investing strategies is the ROR. However, a model with a high ROR, a low hit rate, a low R^2, and a high MSE will be looked at with scrutiny. The model may simply have gotten lucky. For this reason, all performance measures will be considered when choosing the best model. 5.4 Building and Training the ANN Models All experiments trained and tested the networks by using the data from 1/4/2002 to 3/19/2010 with the future price 1 week, 4 weeks, and 13 weeks into the future. The data used for testing was the most recent 25% of the data, which spanned from 3/28/2008 3/19/2010. The data before 3/28/2008 was then randomized and separated into 80% training and 20% cross validation. This approach left 3206 training instances (60% overall), 801 cross validation instances (15%), and 1336 testing instances (25%). The number of testing instances was eventually reduced to 1262 entries; after 12/18/2009, the dataset only had data for 3 of the 15 companies, so those entries were pruned. All results published in this thesis reflect the 1262 test entries. Training data was used to build the ANN models. Cross validation was used to determine when the model had finished training and started memorizing. When models begin memorizing the input data, they are said to be over-trained. An over-trained model is not desirable because it is only useful in predicting the input data, but any additional data points will have low accuracy. The model will have memorized specific data points

33 33 rather than discovered trends in the data. Using cross validation prevents models from becoming over-trained, and creates a better model in general. Models that use cross validation in general provide better predictions of future values. 5.5 Testing Different Network Architectures Single-Company Models Initially, models were constructed for individual companies. A model was trained, cross validated, and tested on data coming from the same company. This approach was attempted multiple times on three companies. The three companies that had the most available data, MCD, RT, and RRGB, were selected for this approach. The companies had around 400 instances each, which is a relatively small amount of data to construct an ANN from. Data was divided into training (70%), cross validation (15%), and testing (15%). The performance of these initial models was so poor that the single-company models were scrapped and the approach shifted to models that would contain all the available data for restaurants. The additional data points provided by the additional companies will ideally improve model performance All-Company Models In creating prediction models, different network architectures were varied to determine the best composition for this specific application. The different architectures that were utilized were: Multilayer Perceptron and Generalized Feed Forward. Multilayer Perceptron outperformed Generalized Feed Forward in all performance measures: r^2, MSE, Hit Rate, and ROR.

34 34 Transfer functions were also experimented with to determine whether hyperbolic tangent sigmoid function was better suited for the application. Tanh performed better than sigmoid Testing Different Network Sizes Various network sizes were compared, utilizing 1 or 2 hidden layers. Networks with 2 hidden layers worked better. Various networks varying the number of processing elements in each layer from 5 to 80 were tested. Networks with 40 processing elements in the first hidden layer and 20 processing elements in the second hidden layer typically had the best results Testing Different Learning Algorithms Different learning algorithms were also tested to determine which would perform the best for this particular application. Beyond the typical performance measures, the time taken to train the model, the size of the network required, and the ability of the algorithm to train the model within 20,000 iterations were also taken into consideration. Numerous learning algorithms were implemented: momentum, conjugate gradient, delta bar delta, Levenberg Marquardt, and quickprop. Delta bar delta was clearly the best performing learning algorithm. Delta bar delta trained significantly faster than all the other learning algorithms, taking a few hours to train, rather than a few days for the next fastest, momentum. Delta bar delta also overall had better networks, with a higher R^2 value, lower MSE, higher hit rate, and smaller network architecture required to train the model. Delta bar delta typically trained the model in less than 2000 epochs.

35 Best Performing Model Parameters By varying the ANN parameters, the best architecture for the given application and data set was discovered. This network architecture was used to train, test, and run a sensitivity analysis on four models for each of the three time period predictions: 1 week, 4 weeks, and 13 weeks. The optimal network architecture is illustrated in Figure 5 and is as follows: Two hidden layers 40 processing elements in the first hidden layer 20 processing elements in the second hidden layer Tanh transfer function in the first two layers A linear transfer function in the output layer Delta Bar Delta learning algorithm Input Layer 1 st Hidden Layer 2 nd Hidden Layer Output Layer Linear Tanh Tanh 20 Figure 5: Network Architecture

36 36 6 RESULTS Results published in this section reflect the performance of models trained with the dataset that combined all companies. All experiments trained and tested the networks by using the data from 1/4/2002 to 12/18/2009 with the future price 1 week, 4 weeks, and 13 weeks into the future. The most recent instances, which spanned from 3/28/ /18/2008, were used to test the data. This test data set contained 1262 test entries. 6.1 Classification Networks Initial attempts at creating classification networks were unsuccessful. Network architecture was varied to create numerous 2-class ANN classification models, but none of them were successful. The fully trained ANN models only predicted one of the two classes nearly 100% of the time, even though the data was fairly balanced. This was true of all prediction lengths (1 week, 4 weeks, and 13 weeks), as well as learning algorithms, and transfer functions. No variation of the number of hidden layers or number of processing elements produced a model that predicted both classes. The models simply would guess Buy or Sell, never a mixture of both. For this reason, function approximation models were created to predict a point estimate of stock price. 6.2 Function Approximation Models Four ANN predictive models were created using the best-performing model parameters for each prediction length of time. The network architecture is as follows: Multilayer Perceptron, 2 hidden layers, 40/20 processing elements, Hyperbolic Tangent transfer function, and Delta Bar Delta learning algorithm. Models were created, trained,

37 37 and tested. Once four good models were found for each time period, they had sensitivity analysis performed on them. The results from the first run of models are as follows: Table 7: Initial Function Approximation Model Results Prediction Length 1w 1w 1w 1w 4w 4w 4w 4w 13w 13w 13w 13w Model Number Hit Rate Annualized Return Perfect Strategy Return % of Perfect Strategy R^ MSE % Buy Predicted 64.07% 80.39% 81.89% 61.60% 63.62% 91.92% 56.81% 56.21% 92.22% 71.03% 75.00% 81.96% Correct Buy % 50.22% 50.22% 50.22% 50.22% 52.99% 52.99% 52.99% 52.99% 58.31% 58.31% 58.31% 58.31% *Best model for each time period highlighted in gray Analyzing the results, different implications are apparent. The 1 week prediction models have the highest R^2 value, averaging This is followed by the 4 week prediction models average of 0.81 and the 13 week prediction models average of The 1 week prediction models also have the lowest average MSE at 20.9, compared to 69.2 and for the 4-week and 13-week models, respectively. The hit rate results are the opposite, with the 13-week predictions averaging a hit rate of 0.59, 4-week predictions averaging 0.55, and the 1-week predictions averaging The Annualized Return results are close between all prediction lengths: 0.54 for 13-week predictions, 0.49 for 4-week predictions, and 0.58 for 1-week predictions. The annualized return is calculated as the return earned by investments alone. It does not take into account taxes, transaction costs, and any other fees associated with trading. As such, the 1-week prediction models will most likely have the highest

38 38 transaction costs, and the 13-week prediction models would have the lowest transaction costs. These costs are not reflected in the above figures. Based on the results, the 1-week predictions were better statistically, but the 13- week predictions had higher accuracy and more return. This phenomenon is accounted for by a few explanations. In shorter time intervals, typically the stock market moves with less magnitude than it does over longer time periods. A perfect strategy for the 1-week periods would only yield 6.8% where a perfect strategy for the 13-week periods yielded 27%. The 1-week models had less change to predict, so they usually predicted small amounts of change, and therefore the prediction was usually close. Further analysis of the results shows that the 13-week predictions had higher accuracy. This higher relative accuracy was due to the fact that, in general, stock prices increase quarterly. These 13-week predictions are very similar to the Buy and Hold baseline strategy. Comparing the 13-week prediction hit rate to the 1-week prediction hit rate, the 13-week prediction was 8% better, but that 8% was not significant because the annualized rate of return for the 13-week prediction was within 1% of the 1-week prediction. Taking into account all of the performance measures, the following models were selected for each prediction period as the best: Model 1 for the 1 week predictions, Model 3 for the 4 week predictions, and Model 4 for the 13 week predictions. These models are highlighted in gray in Table 7.

39 Performing Sensitivity Analysis In an attempt to simplify the model by reducing the number of inputs, sensitivity analysis was conducted on the 12 models created in the previous section. For each of the 43 inputs, the input being studied was varied 2 standard deviations, while the other 42 inputs were held at their average. The effect on the predicted price was then recorded. From the sensitivity results, the most sensitive variables were ranked for each time period from 1 to 43 with 43 being the least sensitive. A list of inputs was created using the top 10 from each time period (14 total inputs), and a second list was created using the top 15 inputs from each time period (24 inputs total) Constructing Models Based on Sensitivity Analysis In running the same experiment, building and training 4 separate models for each of the three time periods, performance significantly decreased for all performance measures for all time periods. In most cases, the hit rate was below 50%. Also, the annualized return also fell below zero for nearly all models. The variables that had been eliminated, even though their contribution was small relative to the other variables, still significantly contributed to the accuracy of the predictive models Evaluating Sensitivity Analysis Considering who has access to the data used to create the models, implications about the necessity to reduce inputs were discovered. If an analyst in the field has access to one of the input variables in the data, they can probably readily access all of them. Reducing the number of inputs to the model would not necessarily help the end user. The

40 40 only benefit to reducing inputs would be to simplify the model for its internal analysis and construction. A simpler model would be easier to analyze. Also, a simpler model should train in less time. Though it would be optimal to have a model with fewer inputs, it does not work in this situation. Reduction in input variables hurts the performance of the models to the point that they become inaccurate and unprofitable Further Evaluating Initial Models Further analysis was then performed on the results from the initial 12 model results. Though the 13-week predictions had a higher overall hit rate, they usually erred on the side of predicting an upward trend in the data. The point estimates from the 13- week model were converted into Buy and Sell Classifications to analyze the predictions similar to a classification network. The confusion matrix for the best 13-week models is as follows: Table 8: 13-week Prediction Model Confusion Matrix Output/Desired Buy Sell Buy Sell Hit Rate The best 13-week model predicted a very large number of Buy situations. The model was correct a majority of the time, but the trend for 13-week stock prices is generally upward. The model predicted an upward trend 82% of the time, though a perfect model would have predicted a Buy situation 58% of the time.

41 41 The 4-week model with the highest hit rate predicted the upward and downward trends in the data, as demonstrated by the following confusion matrix. Table 9: 4-week Prediction Model Confusion Matrix Output/Desired Buy Sell Buy Sell Hit Rate It was also observed that after a few months, the predictions strayed from the actual values, sometimes predicting negative stock prices. To remedy this problem, a maintenance schedule was developed to retrain the model in set increments. Once a certain number of weeks were tested, those testing weeks would be randomly inserted into the cross validation and training data. This maintenance approach is also more realistic because an end user using a model like this would prefer a model trained within the last couple of months, rather than a couple of years. This approach is similar to the approach proposed by Chenoweth [14], though the model in this thesis will not be retrained after each individual prediction. Also, the model trained with newer and additional data should be able to make more accurate predictions. 6.3 Development of the Maintenance Approach To determine the time interval for retraining, the model results were sorted by date and tested between 6 and 12 weeks for model performance. The results are as follows:

42 42 Table 10: 4-week Prediction Model Results with Differing Test Period Lengths Length of Test Period MSE r Squared Hit Rate 6 Weeks Weeks Weeks Weeks Weeks Weeks Weeks According to the results, MSE is fairly stable throughout the different lengths of testing. The R^2 value slowly declined as the number of weeks increased. The hit rate peaked at 0.63 at the 8-week mark. Based on the results, a maintenance approach was developed to retrain the model every 8 weeks, with the data that had previously been tested randomly added to training and cross validation data sets. The network architecture would be the same as was used to create the original function approximation models. 6.4 Maintenance Approach Results The results of the 8-week maintenance approach are as follows: Table 11: Maintenance ANN Prediction Model Results Maintenance Model Best 4w Model Avg Monthly Return Yearly Return R Squared MSE Hit Rate

43 43 Table 12: Maintenance ANN Prediction Model Confusion Matrix Output/Desired Buy Sell Buy Sell Hit Rate Although the best 4w model produced a better return and hit rate, the maintenance model performed very well. Statistically the maintenance model outperformed the other 4-week models previously created. The maintenance model combined the statistical performance of the 1-week models with the high hit rate and annual returns of the longerperiod models. 6.5 Classification Networks Based on the success of function approximation models, additional classification models were created. To compare the classification models to the best of the function approximation models, a prediction length of 4 weeks was more closely examined. Additional classification variations were attempted to replicate the success of the function approximation networks. The initial 2-class prediction models performed similar to the initial classification models, but the variations produced some encouraging results Class Prediction Models Similar to the performance of the initial classification attempts in this thesis, the 2-classification models performed poorly. Of the more than 10 models trained, none of them achieved above 50% accuracy or made a positive annual return. Network architectures were varied in all of the models, but one could not be found to accurately

44 44 predict the two classes. An example of a confusion matrix from a 2-class prediction models is shown below: Table 13: 2-Class Prediction Model Confusion Matrix Output / Desired 4w Decision(Buy) 4w Decision(Sell) 4w Decision(Buy) w Decision(Sell) This particular model performed similarly to the 2-class prediction models as a whole. With an overall hit rate below 50%, it would be better to bet against this model, rather than for it. The 2-class prediction models generally predicted a Buy situation, but still had trouble distinguishing between Buy and Sell. When the model predicted a Buy situation, it was only correct 51.25% of the time. The model predicted a Buy situation 71.85% of the time. A perfect model would have predicted a Buy situation only 53% of the time. The model was also wrong on a vast majority of the Sell situations, with a hit rate well below 50%. Because of the poor predictions of sell situations, only 3 of the 2- class prediction models had positive returns Class Prediction Models The 3-class prediction models performed poorly overall as well. The 3 predicted classes were: Buy, Hold, and Sell. 10 models with varying parameters were constructed, trained, and tested. None of the 10 models achieved above 50% accuracy. Also, none of

45 45 the 10 models predicted a single Hold situation. The models generally predicted Buy situations, though again they had trouble distinguishing between Buy and Sell situations. Below is the confusion matrix of the only 3-classification model to have a positive return: Table 14: 3-class Prediction Model Confusion Matrix 4w Decision(Buy) 4w Decision(Hold) Output / Desired 4w Decision(Buy) w Decision(Hold) w Decision(Sell) 4w Decision(Sell) From the above confusion matrix, it is clear that the model had a hit rate below 50%. The model was incorrect a majority of the time whichever class it predicted. The actual hit rate of this model is 44.08%. Surprisingly, out of all the 3-class prediction models created, this individual model is the only one to have earned a positive return. The model correctly predicted the instances with higher magnitude movements than the smaller movements, which allowed it to produce a positive return even though the majority of predictions were incorrect. Overall the 3-class prediction models were unsuccessful in predicting the stock market in terms of hit rate and ROR Class Prediction Models Overall the 5-class prediction models performed fairly well. The 5-class prediction models better than the 2-class and 3-class models, though they exhibited some

46 46 of the same issues. Of the 10 5-class models, only half of them had hit rates at or above 50%. This hit rate is slightly modified for the 5-class prediction models to include predictions in the right direction as correct predictions. For example a Buy prediction would be deemed correct if the true situation was a Strong Buy situation. This was decided because, for the rate of return calculation, if the model predicted a Buy situation for a company, and the stock price increased enough to create a Strong Buy situation, the model still made a profitable decision. Even though the magnitude of the decision was incorrect, the direction was correct and should be rewarded. A confusion matrix for Model 3 of the 5-class prediction models is in Table 15. Table 15: 5-class Prediction Model Confusion Matrix Output / Desired 4w Decision(Strong Buy) 4w Decision(Buy) 4w Decision(Hold) 4w Decision(Sell) 4w Decision(Strong Sell) 4w Decision(Strong Buy) w Decision(Buy) w Decision(Hold) w Decision(Sell) w Decision(Strong Sell) For this model, the % correct would be ( ) / 1336 = By including the correct sign predictions, the modified hit rate is ( ) / 1336 = This particular model performed the best of the 10 5-class predictive models, in

47 47 terms of modified hit rate and ROR. The network architecture for this model was a Multilayer Perceptron with two hidden layers with 40 and 20 processing elements in the hidden layers, Hyperbolic Tangent transfer functions, and Delta Bar Delta was the learning algorithm. Again, as evidenced by the above confusion matrix, the model failed to predict any hold situations. Of the 10 5-class prediction models created, only 3 models predicted any Hold situations. Models also predicted significantly more Strong Buy and Strong Sell situations than Buy and Sell situations. This is due to imbalanced data. Although when training the networks, For Classification problems, make classes evenly weighted was enabled, the networks did not necessarily train with that in mind. The full results of the 5- class prediction models are in Table 16. Table 16: Results of 5-class Prediction Models Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7 Model 8 Model 9 Model 10 Exactly Right Correct Sign w ROR w ROR Model Bias Sell Buy Balanced Balanced Buy Sell Buy Buy Sell Buy The ROR was positive for all of the models, with a couple exceptions. Both Model 6 and 9 had a Sell bias, meaning they predicted a Sell situation significantly more than a Buy situation. Model 6 and 9 predicted a Sell situation 74.25% and 97.31%, respectively. A perfect model would have predicted a sell situation only 45.21% of the

48 48 time. The other model with a sell bias, Model 1, had a hit rate only 0.07% above 50% and a ROR of only 3.4%, the third lowest of the 10 5-class predictive models. Of all the classification models created, Models 3 and 4 of the 5-class predictive models were clearly the best. Models 3 and 4 were the best-performing models in terms of adjusted hit rate, ROR, and bias. Both models were able to predict the stock going up and down effectively, while also earning a profit twice as high as any of the other models. 6.6 Benchmark Results Stepwise Multilinear Regression Results Stepwise Multilinear Regression was performed on the same data used for training and cross validation data sets for the ANN predictive models. The stepwise Multilinear Regression model was also retrained in the same way as the maintenance ANN model. A stepwise Multilinear Regression model was created from the training and cross validation data, which set the structure for the model using 20 inputs. The model was tested for the first 8 weeks of test data. Then the test data for the first 8 weeks was added to the training data, and the model was retrained for the second 8 weeks of test data. This retraining procedure happened 10 times, using the same data and retraining times as used in the ANN maintenance model. The point estimate predictions of the Multilinear Regression model were classified in the same way as the ANN predictive models: a predicted price above the current price was a Buy situation, where a predicted price below the current price was a Sell situation. Results are as follows:

49 49 Table 17: Stepwise Multilinear Regression Results Multilinear Regression R^ MSE Hit Rate w Return Annualized Return Table 18: Stepwise Multilinear Regression Confusion Matrix Output/Desired Buy Sell Buy Sell Hit Rate The Multilinear Regression model performed very well. The hit rate was above 50%, and the model made a positive ROR. The Multilinear Regression model predicted a large percentage of Buy situations, accounting for 78% of the predictions. A perfect model would have predicted a Buy situation only 53% of the time. The model made very conservative estimates, weighted heavily toward current price. A table containing the price coefficients for each retrain is below:

50 50 Table 19: Stepwise Multilinear Regression Price Coefficients Retrain Price Coefficient Generally the current price coefficient accounted for more than 75% of the future price estimate in the Multilinear Regression models. This is understandable because current price is the most sensitive input in every one of the ANN models as well. The R^2 value for the prediction and the current price is , meaning that more than 99% in the variation in the predictions is explained by the variation in the stock price. Comparatively, the R^2 value for the prediction and current price of the maintenance ANN model was By conservatively predicting a change from the current stock price, the Multilinear Regression models made fairly close predictions to the actual future price, though they did not capture the directional movements of the stock price in those predictions, evidenced by the low hit rate Analyst Results A compilation of analyst ratings of the restaurant stocks was obtained from Thompson-Reuters. The analyst ratings were rated from 1 (Strong Sell) to 5 (Strong

51 51 Buy), then an average was taken across the board of all analysts that had updated recently. These raw analyst ratings needed to be converted into a trading strategy. The analyst ratings were paired with the appropriate company and date for each instance in the dataset. It is understood that analysts do not necessarily expect a stock price to move toward their prediction exactly in 4 weeks when they publish a rating. However, analysts expect the long-term prospects of purchasing a company s stock with a Buy recommendation to be favorable. With this in mind, a strategy that would create the best possible predictions from the analyst ratings was created. A point was chosen to separate the Buy predictions from the Sell predictions based on the average analyst rating. This point, according to the data set description, should lie near 3. Multiple scenarios were tried to maximize the analyst hit rate. The results from the scenarios are in Figure 6.

52 52 Figure 6: Analyst Hit Rate Versus Decision Point For the test, any average analyst rating greater than or equal to the decision point was classified as a Buy situation, while any rating less than the decision point was classified as a Sell situation. At the decision point of 3, the model had a hit rate of 51.12%. At the decision point of 3.4 the model had a hit rate of 51.65%. As the numbers decreased below 2.8, the analyst rating always became a Buy situation. The average analyst rating had a minimum of 2.33, and that only happened 17 out of 1262 instances. The vast majority of analyst ratings were above 3, at 1095 out of 1262 instances. To determine the best decision point, 3 and 3.4 were chosen to be compared because of their relatively high hit rate. The monthly rate of return was negative for both 3 and 3.4. In fact, any decision point above 2.8 produced a negative monthly return. The ROR was -

53 53 3.8% and -14.1% for the decision points of 3 and 3.4, respectively. As such, 3 was chosen as the decision point for the analysts, and the average predictions were classified accordingly. The results are contained in Table 20. Table 20: Analyst Confusion Matrix Output/Desired Buy Sell Buy Sell From the results table, there is a clear bias towards a Buy decision. The analysts as a whole were correct slightly above 50%. The analysts predicted a Buy situation 87% of the time. Again, a perfect model would have predicted a Buy situation only 53% of the time Buy and Hold Results A final benchmark was constructed using a Buy and Hold strategy. The Buy and Hold strategy is simple: buy at the beginning of a period and hold the stock indefinitely. This strategy minimizes transaction costs and depends on the general upward trend in the stock market to make money. For the restaurant stock model comparison, the Buy and Hold strategy was evaluated and implemented similar to the analyst predictions. The Buy and Hold strategy prediction was simply to always have a Buy decision. For the Buy and Hold strategy, the confusion matrix is as follows:

54 54 Table 21: Buy and Hold Confusion Matrix Output/Desired Buy Sell Buy Sell 0 0 The Buy and Hold strategy had a hit rate of 52.61%. Generally during this test period the restaurant stock prices increased. As such, the annualized return for the Buy and Hold strategy was 43.13%. The Buy and Hold strategy predicted a Buy situation 100% of the time, where a perfect model would have predicted a Buy situation 53% of the time.

55 55 7 SUMMARY AND DISCUSSION The three baselines calculated in this thesis are compared to the best ANN model in the following tables and charts. The three baseline comparisons are the Analyst Predictions strategy, Multilinear Regression Predictions Trading strategy, and the Buy and Hold trading strategy. The S&P 500 index is also plotted along the same time windows to show how the different methods performed in different market environments. The ANN model chosen as the best for comparison was the model with the maintenance approach of retraining every 8 weeks. This approach is the most realistic method of utilizing an ANN model in the actual stock market. Based on the experiments conducted before developing the maintenance ANN model, the maintenance ANN model should perform the best of the ANN models. The maintenance ANN model combines success in both statistical and non-statistical categories, as evidenced by the charts and tables below. The time window of each set of entries on Figure 7 is 8 weeks, to capture the returns of the different retrains of the maintenance ANN model and Multilinear Regression model. The results of the baselines and ANN model are below:

56 56 40% 8-Week Period Returns 30% 20% 10% Maintenance Model Monthly Return Buy And Hold Monthly Return 0% -10% Analyst Monthly Return -20% -30% Date S&P 500 Index *Time scale same as above chart Multilinear Regression Monthly Return Figure 7: Eight-Week Period Returns and Plot of S&P 500 Index for Test Period

57 57 According to Figure 7, the maintenance ANN model outperformed the baseline methods in many ways. The ANN model had a negative return in only 1 of 11 8-week periods. The Buy and Hold strategy, the Analyst strategy, and the Stepwise Multilinear Regression strategy would have had 5, 3, and 4 periods of negative returns, respectively. The most significant period of comparison is the period starting 9/19/2008, at the outset of the housing crisis where the S&P 500 and most stocks began to decrease rapidly. The ANN maintenance model predicted that this would happen for the most part and made a positive return by producing a short sell recommendation. The analyst model was also able to see the fall in the stock prices coming and also produced a positive return during the period. Overall, the models behaved similarly to each other statistically and some performed differently using non-statistical measures. A summary of the statistical and non-statistical performance measured are in Table 22. Table 22: Maintenance ANN Network and Benchmark Results Maintenance Buy and Analyst Multilinear Measurement\Model ANN Model Hold Predictions Regression R^ N/A N/A MSE N/A N/A Hit Rate w Return Annualized Return Statistically, the maintenance ANN model performed at least comparatively if not better than the comparative Multilinear Regression model. The maintenance ANN model

58 58 had a slightly lower R^2 value, meaning the Multilinear Regression model explained the variance slightly better than the maintenance ANN model. The maintenance ANN model had a lower MSE value, meaning that the maintenance ANN model predictions were generally closer to the actual values than those predictions made by the Multilinear Regression model. Both numbers, R^2 and MSE, must be taken into account when evaluating the statistical performance of a model. The R^2 values are within 1.55% of each other, where the MSE values are only within 14.79% of each other. Because the maintenance ANN model performed 14.79% better in one measure, and only 1.55% worse in the other measure, the maintenance ANN model performed at least as good as the Multilinear Regression benchmark if not better. Comparing the non-statistical measures, the maintenance ANN model performed better than the comparative models. The maintenance ANN model had a hit rate of 56.1%, compared to 52.61%, 51.12%, and 51.82% for the Buy and Hold strategy, Analyst Prediction strategy, and the Multilinear Regression model predictions. The ANN models were the only models tested that exceeded Qian s benchmark of 56% hit rate [35]. The maintenance ANN model had an annualized return of 56.43%, compared to 43.13%, -3.8%, and 35.57% for the Buy and Hold strategy, Analyst Prediction strategy, and the Multilinear Regression model predictions.

59 59 8 CONCLUSION This thesis demonstrates that utilizing an ANN to combine fundamental and technical analysis to predict restaurant stock prices can be successful. The strategy used to implement ANN models demonstrated their predictive power and minimized weaknesses of ANNs, namely training time. Because ANN models take time to train, this implementation was designed to allow for training the ANNs while the stock market was closed and thus prices were not changing. All stock prices used in this thesis were the closing prices on Friday. As the stock market is only open Monday through Friday, a user has the entire weekend to train a new predictive ANN and make a prediction for the coming weeks. The network size and learning algorithm were also chosen to not only minimize training time but also maintain prediction accuracy. The maintenance ANN model performed better than the comparative models using statistical and non-statistical measures. Statistically, the maintenance ANN model performed at least as well as the Multilinear Regression model. Using non-statistical measures the maintenance ANN model was correct more often and had a higher return than any of the other models. The different prediction lengths affected the performance of the ANN. The 13- week predictions generally produced a Buy decision. These predictions generally made a profit, but statistically they struggled. The shorter time periods had the opposite problem. Weekly prediction models had strong statistical results, but poor non-statistical results. The weekly predictions were closer statistically, but did not make as much profit. The

60 60 compromise between the two time periods was a 4-week prediction model that combined the strengths of the 1-week and 13-week models. The final maintenance ANN model, as a 4-week model, balanced statistical and non-statistical performance. It must be noted how effectively a Buy and Hold strategy performed. By definition, a Buy and Hold strategy will have fewer transactions and therefore the lower transaction costs than any other strategy. With that in mind, the Buy and Hold annualized returns were only 13.3% lower than the best ANN model. Transaction costs must be kept in mind when comparing that small margin of returns between the different investment strategies. As with similar research done by Chenoweth [14], no statement can be made on the superiority of the maintenance ANN model to Buy and Hold strategy without more analysis into transaction costs associated with the maintenance ANN trading strategy. This thesis proves that ANN models can produce excess returns, but those returns may not necessarily cover transaction costs. It must also be noted that the investor subscribing to the Buy and Hold strategy, as well as any investment strategy, takes risks. Recent plunges in the stock market such as the tech bubble and the housing crisis demonstrate why investors would not want to subscribe only to a Buy and Hold strategy. In general, the Buy and Hold strategy works very well because the US stock market has an upward trend. The effectiveness in predictive models comes from their ability to predict when a shift in trend is coming. The reason the maintenance ANN model performed better than the Buy and Hold strategy was that it predicted the housing crisis well enough to have a positive return while the

61 61 Buy and Hold strategy was taking huge losses. There will always be rises and falls in the stock market, and avoiding the falls such as the housing crisis are why alternatives to the Buy and Hold strategy can be viable. When trading between the small rises and falls in the stock market, it is incredibly difficult to make a profit. The small gains made by buying and selling a stock can easily be cancelled out by a few wrong decisions or the transaction costs associated with trading stocks. Typically it is better to hold on to the stocks you have already purchased, as evidenced by the success of the Buy and Hold strategy. Every once in a while, though, opportunity comes in the form of a downward plunge in the stock market. When these can be effectively predicted, an opportunity for excess profits exists. Generally the Multilinear Regression model performed similar to the Buy and Hold strategy. According to Figure 7, the Multilinear Regression model and the Buy and Hold strategy had returns within 5% in each of the 2-month periods. However, this is not because the two strategies always made the same decisions. The Multilinear Regression model made a Sell decision 22% of the time, where the Buy and Hold strategy did not make a Sell decision. Generally the analyst predictions were similar to a Buy and Hold strategy. Analysts predicted a Buy situation 87% of the time, though they did not predict Sell situations correctly. As such, the analysts had a negative return over the test period, where all other strategies made a positive return. It is worth noting, however, that the analysts also partially predicted the housing crisis, earning a positive return during the

62 62 biggest fall in the stock market during the test period. The Multilinear Regression and Buy and Hold strategies both lost more than 20% in the same 8-week period. So although analysts failed to net a positive return during the testing period, they displayed some of the same promise the maintenance ANN model showed during the housing crisis. 8.1 Future Research Based on the results of this thesis, future research can be conducted. Because of the success of the Bollinger Band (bottom), future models should also attempt to incorporate Bollinger Band (top) to see if it also contributes to the accuracy of predictions. Bollinger Band (bottom) consistently appeared among the most sensitive variables in the sensitivity analysis conducted in this thesis. Behind current price, Bollinger Band (bottom) was the second most sensitive input. Because of the dataset used for this thesis, 4-week predictions most effectively predicted the stock prices for the 15 restaurants contained in the dataset. For a different dataset, other time periods may be more effective. Weekly predictions probably were not effective for this dataset because only 20 of the 43 inputs had unique values for each date. For an input reported monthly or quarterly, 4 to 13 entries had the exact same value in the dataset. Check to see if 20 regression inputs lined up with 20 updated weekly. This is a limitation because corporations are only required to file some of these inputs quarterly, so they will not report them more frequently than required.

63 63 REFERENCES [1] J. Alvarez-Ramirez, "Some Issues on the Stability of Trading Based on Technical Analysis," Physica A: Statistical Mechanics and its Applications, vol , pp , [2] A. T. Akinwale, O. T. Arogundade, and A. F. Adekoya, Translated Nigeria Stock Market Prices Using Artificial Neural Network for Effective Prediction, Journal of Theoretical and Applied Information Technology, vol. 9 pp 36-43, [3] R. A. Araujo and T. A. E. Ferreira, An Intelligent Hybrid Morphological-Rank- Linear Method for Financial Time Series Prediction, Neurocomputing, vol. 72, pp , 2009 [4] R. A. Araujo, Swarm-Based hybrid Intelligent Forecasting Method for Financial Time Series Prediction, Learning and Nonlinear Models, vol. 5, pp , [5] G. S. Atsalakis, "Surveying Stock Market Forecasting Techniques - Part II: Soft Computing Methods." Expert Systems with Applications, vol. 36.3, pp , [6] D. Bao and Z. Yang, Intelligent Stock Trading System by Turning Point Confirming and Probabilistic Reasoning, Expert Systems with Applications, vol. 34, pp , [7] J. Bettman, S. Sault, and E. Schultz, "Fundamental and Technical Analysis: Substitutes or Complements?" Accounting and Finance, vol. 49.1, pp , [8] J. C. Bogle, The Little Book of Common Sense Investing: The Only Way to Guarantee Your Fair Share of Stock Market Returns, Hoboken, NJ: Wiley, [9] J. Bollen, H. Mao, Z. Huina, and Xiao-Jun. Twitter Mood Predicts the Stock Market, eprint arxiv: [10] S. Chakravarty, Forecasting Stock Market Indices Using Hybrid Network, World Congress on Nature & Biologically Inspired Computing, pp , [11] P. Chang, C. Fan, and C. Liu, Integrating a Piecewise Linear Representation Method and a Neural Network Model for Stock Trading Points Prediction, IEEE Transactions on Systems, Man, and Cybernetics Part C: Applications and Reviews, vol. 39, pp 80-92, 2009 [12] T. Chavarnakul, "Intelligent Technical Analysis Based Equivolume Charting for Stock Trading Using Neural Networks," Expert Systems with Applications vol. 34.2, pp , [13] J. Cheng, H. Chen, and Y. Lin, A Hybrid Forecast Marketing Timing Model Based on Probabilistic Neural Network, Rough Set and C4.5, Expert Systems with Applications, vol. 37, pp , [14] T. Chenoweth, "A Multi-Component Nonlinear Prediction System for the S and P 500 Index." Neurocomputing, vol. 10.3, pp , [15] D. Elliman, Pattern Recognition and Financial Time-Series, Intelligent Systems in Accounting, Finance and Management, vol. 14.1, pp , 2006.

64 [16] D. Enke, "The Use of Data Mining and Neural Networks for Forecasting Stock Market Returns." Expert Systems with Applications, vol. 29.4, pp , [17] T. Ferreira, G. Vasconcelos, P. Adeodato, A New Intelligent System Methodology for Time Series Forecasting with Artificial Neural Networks, Neural Processing Letters, vol. 28, pp , [18] L. Gallagher and M. Taylor, "Permanent and Temporary Components of Stock Prices: Evidence from Assessing Macroeconomic Shocks,"Southern Economic Journal, vol. 69.2, pp , [19] E. Hadavandi, H. Shavandi, and A. Ghanbari, Integration of Genetic Fuzzy Systems and Artificial Neural Networks for Stock Price Forecasting, Knowledge- Based Systems, vol. 23, pp , [20] W. Huang, Y. Nakamori, and S. Wang, Forecasting Stock Market Movement Direction with Support Vector Machine, Computers & Operations Research, vol. 32, pp , [21] S. C. Hui, M.T. Yap, and P. Prakash, A Hybrid Time Lagged Network for Predicting Stock Prices, International Journal of the Computer, the Internet, and Management, vol 8.3, [22] K. Kohara, T. Ishikawa, Y. Fukuhara, and Y. Nakamura, Stock Price Prediction Using Prior Knowledge and Neural Networks, Intelligent Systems in Accounting, Finance, and Management, vol. 6, pp 11-22, [23] S. Lakshminarayanan, An Integrated Stock Market Forecasting Model Using Neural Networks, Thesis, Ohio University, [24] M. Lam, "Neural Network Techniques for Financial Performance Prediction: Integrating Fundamental and Technical Analysis," Decision Support Systems, vol. 37.4, pp , [25] C. Lee, Fusion Investing, AIMR conference proceedings, pp.15-23, [26] W. Leigh, R. Purvis, and J. Ragusa, "Forecasting the NYSE Composite Index with Technical Analysis, Pattern Recognizer, Neural Network, and Genetic Algorithm: A Case Study in Romantic Decision Support," Decision Support Systems, vol. 32.4, pp , [27] R. Lippman, An Introduction to Computing with Neural Nets, IEEE ASP Magazine, pp 4-22, [28] A. Lo, H. Mamaysky, and J. Wang, Foundations of Technical Analysis: Computational Algorithms, Statistical Inference, and Empirical Implementation, The Journal of Finance, vol. 40, pp , [29] B. B. Mandelbrot, "A Fractals and Scaling in Finance," New York, NY: Springer Verlag, [30] B. R. Marshall, R. H. Cahan, and J. M. Cahan, Does Intraday Technical Analysis in the U.S. Equity Market Have Value? Journal of Empirical Finance, vol. 15, pp ,

65 [31] A. Mitsuo, Applying Neural Networks to the Extraction of Available Investment Information from the Previous Day s Stock Market, NUCB Journal of Economics and Information Science, vol. 48, pp 13-22, [32] J. A. Ohlson, "Earnings, Book Values, and Dividends in Equity Valuation: An Empirical Perspective," Contemporary Accounting Research, vol. 18.1, pp , [33] T. Poddig and H. Rehkugler, A World Model of Integrated Financial Markets Using Artificial Neural Networks, Neurocomputing, vol. 10, pp , [34] M. Qi, Nonlinear Predictability of Stock Returns Using Financial and Economic Variables, Journal of Business & Economic Statistics, vol. 17, pp , [35] B. Qian, "Stock market prediction with multiple classifiers," Applied Intelligence, vol. 26.1, pp , [36] T. Quah and B. Srinivasan, Improving Returns on Stock Investment Through Neural Network Selection, Expert Systems with Applications, vol. 17, pp , [37] J. Racine, On the Nonlinear Predictability of Stock Returns Using Financial and Economic Variables, Journal of Business & Economic Statistics, vol. 19, pp , [38] F. M. Shaikh, A. Latif, A. Shah, and M. S. Nazar, Forecasting the Stock Market through Dart Board Theory, International Journal of Economics and Finance, vol. 4, pp , [39] S. Thawornwong and D. Enke, "The Adaptive Selection of Financial and Economic Variables for Use with Artificial Neural Networks," Neurocomputing, vol. 56, pp , [40] K. S. Vaisla and A. K. Bhatt, An Analysis of the Performance of Artificial Neural Network Technique for Stock Market Forecasting, International Journal on Computer Science and Engineering, vol. 2, , [41] G. Weckman, S. Lakshminarayanan, A. Snow, and J. H. Marvel, An Integrated Stock Market Forecasting Model Using Neural Networks, International Journal of Business Forecasting and Marketing Intelligence, vol. 1.1, [42] H. Wittkemper and M. Steiner, Using Neural Networks to Forecast the Systematic Risk of Stocks, European Journal of Operational Research, vol. 90, pp , [43] X. Wu, M. Fung, and A. Flitman, Forecasting Stock Market Performance Using Hybrid Intelligent System, Lecture notes in Computer Science, vol. 2074, pp ,

66 Thesis and Dissertation Services