Design and Application of Artificial Neural Networks for Predicting the Values of Indexes on the Bulgarian Stock Market

Transcription

1 Design and Application of Artificial Neural Networks for Predicting the Values of Indexes on the Bulgarian Stock Market Veselin L. Shahpazov Institute of Information and Communication Technologies, Bulgarian Academy of Sciences 3 Sofia, Bulgaria veselin_georgiev@abv.bg Vladimir B. Velev Allianz Bulgaria 202 Sofia, Bulgaria vladimir.velev@allianz.bg Lyubka A. Doukovska Institute of Information and Communication Technologies, Bulgarian Academy of Sciences 3 Sofia, Bulgaria doukovska@iit.bas.bg Abstract The Artificial Neural Networks are flexible computing frameworks and universal approximators that can be applied to a wide range of time series forecasting problems with a high degree of accuracy. They are an artificial intelligence method for modeling complex target functions. For certain types of problems, such as learning to interpret complex real-world sensor data, Artificial Neural Networks are among the most effective learning methods currently know. During the last decade they have been widely applied to the domain of financial time series prediction and their importance in this field is growing. In this paper our aim will be to analyze different neural networks for financial time series forecasting. Specifically their ability to predict future values of The Bulgarian Stock exchange Sofia and the respective representative indexes. In order to yield better results Artificial Neural Networks need to have an optimal architecture and be trained in a suitable way. This will be the main challenge for the authors of this paper. Conclusions made by multiple authors that Artificial Neural Networks do have the capability to forecast the stock markets studied and, if properly trained, can improve the robustness according to the network structure are put to the test in this paper by constructing and applying three different models that will be tested in the environment of the Bulgarian capital market. Keywords Artificial Neural Networks, Finance, Forecasting, Economic Forecasting, Stock Markets I. INTRODUCTION The ability of neural networks to closely approximate unknown functions to any degree of desired accuracy has generated considerable demand for neural network research in Business. The attractiveness of neural network research stems from researchers need to approximate models within the business environment without having a priori knowledge about the true underlying function. However, despite all advantages cited for artificial neural networks, their performance for some real time series is not satisfactory. Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing forecasters. Both theoretical and empirical findings have indicated that integration of different models can be an effective way of improving upon their This work has been partially supported by FP7 grant AComIn and partially supported by grant BG05PO predictive performance, especially when the models in the ensemble are quite different. Artificial Neural Networks are flexible computing frameworks and universal approximators that can be applied to a wide range of time series forecasting problems with a high degree of accuracy. They are an artificial intelligence method for modeling complex target functions. For certain types of problems, such as learning to interpret complex real-world sensor data, artificial neural networks are among the most effective learning methods currently know. During the last decade they have been widely applied to the domain of financial time series prediction and their importance in this field is growing. In this paper our aim will be to analyze different neural networks for financial time series forecasting. Specifically their ability to predict future trends of the Bulgarian Stock Exchange Sofia and the respective representative indexes. Our basic intention is to use Back-propagation and conjugate gradient descent algorithms for training multilayer feed-forward networks as a network model for predicting the price movements of indexes on the Bulgarian Stock Market. But other structures like general regression neural networks radial basis neural networks will also be discussed and analyzed if they were to be more suitable for the case. It must be said that gradient techniques, such as back-propagation, are currently the most widely used methods for neural network optimization. ed feed-forward neural networks possess a number of properties which make them particularly suited to complex pattern classification problems. However, their application to some real-world problems has been hampered by the lack of a training algorithm which reliably finds a nearly globally optimal set of weights in a relatively short time. Genetic algorithms are a class of optimization procedures which are good at exploring a large and complex space in an intelligent way to find values close to the global optimum. Hence, they are well suited to the problem of training feed-forward networks. Supervised neural networks can be used for building the forecasting model. Supervised learning is by far the most common type of training in artificial neural networks. It requires many samples to serve as examples. Each sample of

2 this training set contains input values with corresponding desired output values (also called target values). Then, the network will attempt to compute the desired output from the set of given inputs of each sample by minimizing the error of the model output to the desired output. It attempts to do this by continuously adjusting the weights of its connection through an iterative learning process called training. Conclusions made by multiple authors that artificial neural networks do have the capability to forecast the stock markets studied and, if properly trained, can improve the robustness according to the network structure are put to the test in this paper by the authors by constructing and applying a model that will be tested in the environment of the Bulgarian capital market. II. BRIEF HISTORY OF NEURAL NEURAL NETWORKS APPLICATION IN FINANCIAL FORECASTING Dealing with uncertainty in finance primarily involves recognition of patterns in data and using these patterns to predict future events. Accurate prediction of economic events, such as interest rate changes and currency movements currently ranks as one of the most difficult exercises in finance; it also ranks as one of the most critical for financial survival. ANNs handle these problems better than other artificial intelligence techniques because they deal well with large noisy data sets. Unlike expert systems, however, artificial neural nets are not transparent, thus making them difficult to interpret []. Artificial neural networks have been used in stock market prediction during the last couple of decades. One of the first projects were by Kimoto, Asakawa, Takeoka and Yoda [2] who had used artificial neural nets for the prediction of Tokyo stock exchange index. Mizuno, Kosaka, Yajima, and Komoda [3] applied ANN again to Tokyo stock exchange to predict buying and selling signals with an overall prediction rate of 63%. Phua, Ming and Lin [4] applied neural network with genetic algorithm to the stock exchange market of Singapore and predicted the market direction with an accuracy of 8%. Although most of the papers in this avenue of research are related to the financial markets in developed economies, several recent articles do show that return predictability also exists in those less-developed financial markets. Ferson and Harvey [5] examine 8 international equity markets, some of which are found in developing economies. The study provides evidence of returns predictability. Harvey [6] focuses on emerging markets by looking at the returns of more than 800 equities from 20 emerging markets including Taiwan. He finds that the degree of predictability in the emerging markets is greater than that found in the developed markets. In addition, local information plays a much more important role in predicting returns in the emerging markets than in the developed markets. This characteristic helps explaining the difference in predictability between the two kinds of markets. At present artificial neural nets are being used in various fields of application including business forecasting, credit scoring, bond rating, business failure prediction, medicine, pattern recognition, image processing, speech processing, computer vision and control systems. In the context of financial forecasting, Kuan and Liu [7] discuss forecasting of foreign exchange rates using ANNs. They show that properly designed artificial neural nets have lower out of sample mean squared prediction error relative to the random walk model. Jasic and Wood [8] discuss the profitability of trading signals generated from the out-of-sample short-term predictions for daily returns of S&P 500, DAX, TOPIX and FTSE stock market indices evaluated over the period The out of sample prediction performance of neural networks is compared against a benchmark linear autoregressive model. They find that the buy and sell signals derived from neural network predictions are significantly different from unconditional one day mean return and are likely to provide significant net profits for reasonable decision rules and transaction cost assumptions. Cao et al. [9] provide a comparison between the Fama and French model and the artificial neural network model in the context of prediction of the Chinese stock market. They report that artificial neural nets outperform the linear models from financial forecasting literature in terms of its predictive power. According to Nelson and Illingworth [0, ], there are infinitely many ways to organize a neural network although perhaps only two dozen models are in common usage. A neural network organization can be described in terms of its neurodynamics and architecture. Neurodynamics refer to the properties of an individual artificial neuron that consist of combination of input, production of output, type of transfer (activation) functions, weighting schemes, i.e. weight initialization and weight learning algorithms. Network architecture (also sometimes referred to as network topology) defines the network characteristics like types of interconnections among artificial neurons, number of neurons and number of layers. III. PROBLEM FORMULATION The Bulgarian capital market is characterized by its relatively short history and its low liquidity, especially in recent years. Yet the companies listed represent different economic segments from the heavy production industry to pharmaceuticals and local banking. The most representative index of the local market Sofix is an index based on the market capitalization of the included issues of common shares, adjusted with the free-float of each of them. The index covers the 5 issues of shares complying with the general requirements for selection of constituent issues that have the greatest market value of the free-float. BG 40 is an index based on the price performance of the issues and shall cover 40 issues of common shares of the companies with the greatest number of transactions and the highest median value of the daily turnover during the last 6 months as the two criteria shall have equal weight. The data used for the study is from the Bulgarian stock exchange official website, and consists of the following: last price, open, high, low. The period covered is between and The training data is split into three parts, with the major part of 70% of the data is treated as actual training data, and the rest are treated as a testing data (5%) and validation data (5%).

3 Three different networks are tested starting with most widely used multi-layer perceptron, a feed-forward artificial neural network model that maps sets of input data onto a set of appropriate outputs. A multi-layer perceptron consists of multiple layers of nodes in a directed graph, with each layer fully connected to the next one the network utilizes a supervised learning technique called back-propagation for training the network it is the most popular algorithm and is extremely simple to program but tends to converge slowly [2]. It calculates the local gradient of each weight with respect to each case during training. Weights are updated once per training case. The formula is: Δω ( t) = ηδ ο + αδω ( t ) () where: η - the learning rate; δ- the local error gradient; α- the momentum coefficient; o i - the output of the i'th unit. ij j The activation functions, used for the output layer are the sigmoid and hyperbolic functions. In this paper, the sigmoid transfer function is employed and is given by: i ij E( t) = (2) + e t The second type is the radial basis function neural network. It is a network that uses radial basis functions as activation functions. The output of the network is a linear combination of radial basis functions of the inputs and neuron parameters. A typical radial function is the Gaussian, their characteristic feature is that their response decreases (or increases) monotonically with distance from a central point. Radial basis function networks have a number of advantages over MLPs. First, as previously stated, they can model any nonlinear function using a single hidden layer, which removes some design-decisions about numbers of layers. Second, the simple linear transformation in the output layer can be optimized fully using traditional linear modeling techniques, which are fast and do not suffer from problems such as local minima which plague multilayer perceptrons training techniques. Radial basis functions networks can therefore be trained extremely quickly. The third network type used for the forecasting procedure is the general regression neural network. The general regression neural network performs regression where the target variable is continuous as opposed to the probabilistic neural networks (they both have similar architectures), which performs classification where the target variable is categorical. The main advantages over the multilayer perceptron networks are: It is usually much faster to train a general regression neural network than a multilayer perceptron network. neural networks often are more accurate than multilayer perceptron networks. neural networks are relatively insensitive to outliers (wild points). neural networks generate accurate predicted target probability scores. The respective shortfalls of General regression neural networks are: neural networks are slower than multilayer perceptron networks at classifying new cases. neural networks require more memory space to store the model. IV. PRODUCED RESULTS The authors decided not to perform any manipulation of the generated data sets for Sofix and BG40, because of the fact that initial results from all three types of networks showed no improvement after converting the values into daily change. The criteria which will evaluate the networks performance will be the error of the network on the subsets used during training (Root Mean Square-RMS). RMS = n ( ˆ δi δ 2 i ) i= n The subsequently conducted experiment produced the following results: For the forecasting of Sofix index the best results, measured in the least amount of test error were produced by the general regression neural networks, the structure of the most effective for the task model is 2 inputs, 533 nodes in the first hidden layer, 2 nodes in the second hidden layer and output. The error generated was for the first network and for the second best general regression network. The network used a subsampling algorithm for training. The radial basis neural networks put to the test showed consistently good results with the two best models showing an error of and respectively. Structured with and 4 inputs, both had 42 nodes in the hidden layer and a single output, both trained using K-Means, K-Nearest Neighbor and pseudo-inverse algorithms. The K-Means algorithm in [3] tries to select an optimal set of points that are placed at the centroids of clusters of training data. Given K radial units, it adjusts the positions of the centers so that each training point belongs to a cluster center, and is nearer to this center than to (3)

4 any other center and each cluster center is the centroid of the training points that belong to it. Once centers are assigned, deviations are set. The size of the deviation (also known as a smoothing factor) determines how spiky the Gaussian functions are. With the K-Nearest Neighbor each unit's deviation is individually set to the mean distance to its K nearest neighbors [3]. Hence, deviations are smaller in tightly packed areas of space, preserving detail, and higher in sparse areas of space (interpolating where necessary). Once centers and deviations have been set, the output layer is optimized using the standard linear optimization technique - the pseudoinverse (singular value decomposition) algorithm [4]. Multi-layer perceptron networks turned out to be the worst performing out of the three models. The two best results were and , produced by three layer networks with input 3 nodes in the hidden layer and a single output, for the first one and 5 nodes in the hidden layer for the second. Networks were trained back-propagation algorithm and the conjugate gradient descent algorithms. In back propagation, the gradient vector of the error surface is calculated. This vector points in the direction of steepest descent from the current point, so we know that if we move along it a "short" distance, we will decrease the error. A sequence of such moves (slowing as we near the bottom) will eventually find a minimum of some sort. The idea behind the conjugate gradient descent algorithms is once the algorithm has minimized along a particular direction, the second derivative along that direction should be kept at zero. Conjugate directions are selected to maintain this zero second derivative on the assumption that the surface is parabolic (smooth). If this condition holds, N epochs are sufficient to reach a minimum. On a complex error surface the conjugacy deteriorates, but the algorithm still typically requires far less epochs than back propagation, and also converges to a better minimum. The next two tables - Table and Table 2 show the results of the two best networks out of each different type. Figure and Figure 2 show the values of the index and the predicted values by the three different models (two of each) for the last 50 days of the observed period Neural Network name Perceptron Perceptron 2 Function NN Function NN 2 Neural Network Neural Network 2 TABLE I. Forecasting Sofix Index Training Algorithm Inputs Hidden () Hidden (2) Test Error Subsampling Algorithm Subsampling Algorithm Neural Network name Perceptron Perceptron 2 Function NN Function NN 2 Neural Network Neural Network 2 TABLE II. Forecasting BG 40 Index Training Algorithm Inputs Hidden () Hidden (2) Test Error Subsampling Algorithm Subsampling Algorithm

5 When forecasting the BG 40 index identical results were produced. Once again the general regression neural networks produced the lowest error only this time of a slightly greater magnitude. The lowest error was accomplished with a 2 input, two hidden layer consisting of 550 nodes in the firs and 2 in the second and output all trained with subsampling algorithm. V. CONCLUSIONS The results obtained with the above experiment reveal that for the case of predicting the value of a low liquidity indexes like the Sofix and BG40 out of the three models analyzed the most successful is the general regression neural networks, producing the lowest test errors. Surprisingly multi-layer perceptrons do not produce as good results as the above mentioned. The lack of any data prior data manipulation like smoothing as well as the fact that the networks were fed with values of the indexes and not with changes as done in many other studies would be the obvious explanation. The authors intend to investigate further the causes of this and test more different structures of neural networks in the environment of low liquidity. The following conclusion must be made: although the overall results produced by all network structures was good, a closer look at their ability of the models to forecast in advance the values of the Bulgarian Stock Exchange indexes shows that all of them lag in the majority of time. This does not permit us to make the ruling these models are ready to be implemented successfully in the real life trading processes and at this point more studies are needed. Figure. Second best performance was again for the radial basis neural networks, their best two results were and and employed 2 and 4 inputs, 30 nodes in the hidden layer and a single output. The networks were trained using K- Means, K-Nearest Neighbor and pseudo-inverse algorithms. ACKNOWLEDGMENT The research work reported in the paper is partly supported by the project AComIn Advanced Computing for Innovation, grant 36087, funded by the FP7 Capacity Programme (Research Potential of Convergence Regions) and partially supported by the European Social Fund and Republic of Bulgaria, Operational Programme Development of Human Resources , Grant BG05PO Figure 2. Worst performance once more was for the results obtained by the multilayer perceptrons. The differences in produced test error were substantial and the best result multi-layer perceptrons could produce was and Structured in the following way: input and just and for the latter 2 nodes in the respective hidden layer and output. Both networks were trained with back-propagation and conjugate gradient algorithms. REFERENCES [] Lawrence R., Using Neural Networks to Forecast Stock Market Prices, Department of Computer Science University of Manitoba, December 2, 997. [2] Kimoto T., K. Asakawa, M. Yoda, M. Takeoka, Stock market prediction system with modular neural network, Proceedings of the International Joint Conference on Neural Networks, -6, 990. [3] Mizuno H., M. Kosaka, H. Yajima, N. Komoda, Application of Neural Network to Technical Analysis of Stock Market Prediction, Studies ininformatic and Control, Vol.7, No.3, pp.-20, 998. [4] Phua P., D. Ming, W. Lin, Neural Network With Genetic Algorithms For Stocks Prediction, Proc. of the fifth Conference of the Association of Asian-Pacific Operations Research Societies, 5th - 7th July, Singapore, [5] Ferson W., C. Harvey, The risk and predictability of international equity returns, Review of Financial Studies, 6:527 66, 993. [6] Harvey C., Predictable risk and returns in emerging markets, Review of Financial Studies, 8:773 86, 995. [7] Kuan C., T. Liu, Forecasting Exchange Rates Using Feedforward and Recurrent Neural Networks, Journal of Applied Econometrics, 0(4): , 995. [8] Jasic T., D. Wood, The Profitability of Daily Stock Market Indices Trades Based on Neural Network Predictions: Case Study for the S&P 500, the DAX, the TOPIX and the FTSE in the Period , Applied Financial Economics, 4(4): , [9] Cao Q., K. Leggio, M. Schniederjans, A Comparison Between Fama and French s Model and Artificial Neural Networks in Predicting the Chinese Stock Market, Computers and Operations Research, 32: , 2005.

6 [0] Hect-Nielsen R., Neurocomputing, Addison-Wesley, Menlo Park, CA, USA, ISBN: , 990. [] Nelson M., W. Illingworth, A Practical Guide to Neural Nets, Addison-Wesley Publishing Company, Inc., USA, ISBN: / , 99. [2] Rumelhart D., G. Hinton, R. Williams, Learning Internal Representations by Error Propagation, Parallel Distributed Processing Explorations in the Microstructure of Cognition Foundations, , MIT Press, Cambridge, Massachusetts, 986. [3] Bishop C., Neural Networks for Pattern Recognition, Oxford, 995. [4] Haykin S., Neural Networks: A Comprehensive Foundation, 2nd Edition, Prentice-Hall, 999.