Applying Artificial Neural Network and Chinese News Classification Techniques to Taiwan Stock Market

Transcription

1 Tamsui Oxford Journal of Mathematical Sciences 19(2) (2003) Aletheia University Applying Artificial Neural Network and Chinese News Classification Techniques to Taiwan Stock Market Deng-Yiv Chiu Department of Information Management, Chung Hua University, Hsinchu, Taiwan and Kun-Po Chuang APEX International Financial Engineering Res. & Tec. Co., Ltd., Hsinchu, Taiwan Received May 6, 2003, Accepted October 7, Abstract In this research, we combine neural network with document classification technique to predict the tendency of stock market to find out how 911-terror attack influenced investors in Taiwan. We integrate quantitative factors with qualitative factors. In qualitative aspect, we use document classification techniques to classify them and quantitate the classified information. Finally, we apply neural network to them and the other quantitative factors. In the experiments, we only focus on Chinese document classification and target the Taiwan stock market. Keywords and Phrases: Neural Network, Document Classification, Stock Market Prediction.

2 202 Deng-Yiv Chiu and Kun-Po Chuang 1. Introduction The 911-terror attack occurred in New Yew, USA in 2001 is an important issue to stock market world-wise. Especially, The Taiwan stock market is a kind of light-tray stock market [21]. The turnover rate is higher than most of stock market. About ninety percent of investors are individual person in Taiwan. The others are legal person or corporate person. The legal person or corporate person can obtain adequate information such as financial statements, professional analysis, and so on. They also have much fund to invest in the stock market. Some researches had tried to find out the relationship between the Taiwan stock market and the macroeconomic activities. Many researchers studied the return of stock and the macroeconomic variables. They might focus on the researches about single economic variable and the return of stock. But these factors are not enough to influence the fluctuation of stock prices. Many foreigner researches had focused on this perspective of the stock market [2]. From the view of economic, the stock price is decided by the supply and the demand. It made up from the behaviors of the investors. Messages related to stock market influence the behavior of the investors rather than messages influence the stock market. And, the behaviors of investors influence the price [3]. In fact, we should consider economic factors, technical factors, qualitative factors, and mentalities of investors. For example, in Taiwan stock market, there are many individual investors, and these individual investors just consider the information about investment market. So we propose this architecture to deal with quantitative and qualitative factors and want to provide more accurate prediction. According to some empirical researches, the importance of qualitative factors in stock market was discovered. We also can find some evidences by reviewing the historical news and exploring some important relevance among them. For this reason, the qualitative factors about the stock market should be concerned if we want to predict the tendency of stock market with satisfactory results. In this research, we use classification technique and clustering method under the framework of Artificial Neural Network to predict the tendency of Taiwan stock market. By using these approaches, we want to improve the accuracy. Chinese news is collected from Internet as the source of our knowledge base. The format of the qualitative factors in knowledge base design are caption and content since those factors could be found in news.

3 Artificial Neural Network and Chinese News Classification Techniques 203 We also have to observe these assumptions as followings. 1. We focus on processing Chinese document containing a caption and a content. (This type of documents includes newspapers, magazines and some reports.) 2. Each classification keyword contains at least two characters and the main keywords of a document appear frequently in the document. 2. Methodology and Architecture As discussed above, many researches just concerned the quantitative factors, such as index in open or volume, instead of qualitative, such as political effect. However, the latter always plays a very important role in the stock market environment. And, knowledge management is a topical subject nowadays. So we use this concept to deal with the qualitative factors. In this research, we are based on the automatic classification of Chinese documents proposed [14] and apply the clustering method to handle the Chinese news. In this research, we will construct a framework to predict the Taiwan Weighted Stock Price Index (TWSPI). The framework consists of the following components shown in figure The information collection: this component is responsible for the system to search information, including quantitative and qualitative data everyday through the WWW. And, the information is the content of our knowledge base. The information consists of three parts as following: a. Qualitative Data: this part is responsible to search daily Chinese news, including internal political situations, internal economic situations, international political situations, international economic situations, company situations, cross-strait relationships, general investment environments, and so on. Thus, we would collect the Chinese news from several websites.

4 204 Deng-Yiv Chiu and Kun-Po Chuang World Wide Web Collecting daily quantitative and qualitative data Quantitative data Information Collection Qualitative data Information Classification The value of the classified information Main Process Weight Training Figure 1. System Architecture b. Macroeconomic Variables: this part is responsible to search several economic indicators, consisting of daily change in exchange rate, daily change in interest rate, monthly change in unemployment rate, annual change in economic growth rate, annual change in consumer price index, annual change in Gross Domestic Product (GDP), monthly change in inflation rate, and monthly change in money supply rate. These data can be collected from several major countries, including USA, Japan, Hong Kong, Singapore, Korea, England, France, and Germany. c. Technological Variables: this part is responsible to search Taiwan Weighted Stock Price Index (TWSPI) and to compute several technological indicators, including Twenty-Four Days Moving Average (MA), Relative Strength Index (RSI), Bias, Moving Average Convergence/Divergence (MACD), Volume Moving Average, and Twenty-Four Days On Balance Volume (OBV). 2. The Information Classification: as shown in figure 2, this component is responsible to classify the Chinese news. The Chinese news collected from the WWW is complex. And we must classify the Chinese news to several categories first. Therefore we use the classified framework that has been

5 Artificial Neural Network and Chinese News Classification Techniques 205 established by news websites. In order to obtain better result, we also use the news clustering method to reclassify the Chinese news of each category under the classified framework. This component can be divided into two parts, including news clustering and news classification. The news clustering is the major method to deal with Chinese news on the training period. After clustering, it would produce several keywords for each Chinese news and group similar Chinese news into the same type. And then, the news classification can just deal with Chinese news according to the keywords of each type the results of the training period. Then the news classification would classify the Chinese news into the similar type. Finally, we can use the classified Chinese news to find the relative change rate of TWSPI. Then we can compute the raising or falling probability and the expected value for each type. If Chinese news is classified into certain type, we can use the raising or falling probability and the expected value of the type to quantify it. Chinese News 1. Record the Chinese news according to the categories created by the News-Website Category 1 Category 2 Category n Type 1 Type 2 Type m Type 1 Type 2 Type k 5. Create types News Clustering 7. Keywords and Chinese news News Classification 3. Chinese news of each category 4. Keywords of each Chinese news 6. Record information of the clustered Chinese news Training Period 8. Record result of the classified Chinese news Testing Period Knowledge Base 2. Create categories of Chinese news according to the categories created by the News-Website Outline of the Classification Result Figure 2. Classification Mechanism

6 206 Deng-Yiv Chiu and Kun-Po Chuang 3. The Weight Training: in this component, we use the quantitative data and the value of the classified Chinese news as input of ANN, and the daily change rate of the TWSPI as output. In this research, we use backpropagation algorithm to increase accuracy of prediction. And we use the three-layers architecture, namely, input-layer, hidden layer, and output layer. The initial value of learning rate parameter is 0.1, and we would find the best value by experiment We will construct the main process of our model with the following steps as shown in figure 3. Quantitative Data Qualitative Data Quantitative data The value of Classified Chinese news Knowledge Base Chinese news or keywords of each type Keywords of each Chinese news and Classified Chinese news The probability and the expected value of each type of the Classified Chinese news Classification Mechanism Transformation Process Classified Chinese news Neural Network Information Classification Output Figure 3. Main Process of System Step one: Searching the required information from Internet. We would collect Chinese history news from several fixed websites. Because these websites collected a lot of Chinese history news, we can use these websites

7 Artificial Neural Network and Chinese News Classification Techniques 207 Chinese history news to form our knowledge base. Chinese history news has been classified to several fixed categories that are created by these websites. And we would record and classify the Chinese history news according to these categories. This step also collects the Taiwan Weighted Stock Price Index (TWSPI) and computes the raising or falling rate every day. And we could map this result to the Chinese history news. This step also collects the quantitative data mentioned above from National Statistics of Taiwan, the Republic of China. The routine procedure will search and collect the Chinese history news from the websites automatically daily. Step two: Clustering the Chinese historical news We would use the method of keyword extraction to segment Chinese sentences[14]. We use the method to handle each Chinese news. After handling every Chinese news, we would record the keywords extracted from every Chinese news and compute the weight of every keyword. The local keywords are extracted from one Chinese news, and the weight of local keyword is computed from keywords of this Chinese news. The global keywords are collected from the Chinese news of the same type, and the weight of global keyword must consider the relationship of any two Chinese news. Step three: Classifying obtained information In this component, we use document classification techniques to classify the Chinese news used for testing according to the global keywords of each type of a category. This research would extract keywords from the testing Chinese news. Then we utilize these keywords extracted from testing Chinese news to compute the distance for all types for the same category. Finally, we classify the Chinese news into a type that has the smallest distance between the Chinese news and that type. Of course, we also save these keywords extracting from that Chinese news in order to re-compute the global weight and to re-train our framework future. Step four: Quantitating the information by transformation process We could use the rate of raising or falling to be the value of the Chinese news. And the importance of some Chinese news would be emerged by accumulating a huge mass of Chinese news, as shown in equation 1.

8 208 Deng-Yiv Chiu and Kun-Po Chuang Degree of Influence = Average Raising Rate * (Times of Raising / Total Times) + Average Falling Rate * (Times of Falling / Total Times) Equation 1. Transformation Equation for Each Type Step five: feeding the quantitative data and qualitative data into neural network Finally, we feed the quantitative data and qualitative data into neural network and use artificial neural network to train the weights as the degree of influence During the training period, we provide the value of the classified information, the variables of macroeconomic, the technical variables, and the expected result for ANN. The weight between one neuron and another neuron could be adjusted according to the expected result. And, the weight between any two neurons is the degree of influence between each type. During the testing period, we provide the Chinese news and quantitative data to our system. Then we use the result of training data to obtain the value of each Chinese news. Finally, we can use the value of Chinese news and quantitative data to predict the tendency of stock market. 3. Experiment In our experiment, we collected relative Chinese news for nine months, from 2001/2/11 to 2001/11/11. There are totally 6 categories and 5411 Chinese news. And, the Chinese news were collected from UDNNEWS. The news website, UDNNEWS, collects a lot of Chinese history news and the news website is very famous in Taiwan. We use the news of first three months to form our knowledge base, and then we could use last six months to train and test the knowledge base. We also collected quantitative variables for six months, from 2001/5/11 to 2001/11/11. We show these variables used in table 1. In first three experiments, we only used technological variables to predict the tendency of the Taiwan stock market. So we performed three different experiments to find the appropriate architecture of ANN. Then, we divided the experiment into two

9 Artificial Neural Network and Chinese News Classification Techniques 209 parts, because a terror event happened in USA, on 11 September The period of first part is from 2001/8/12 to 2001/9/9, and the second part is from 2001/9/10 to 2001/11/11. Many researchers found that suitable number of neurons for hidden layer was between the number of neurons of input layer and the number of neurons of output layer. And we must perform some experiments to decide the number of neurons of hidden layer first. Table 1. Variables Used In This Research Variable Type Variable Name Technological Variables 5-Days Moving Average, 10-Days Moving Average, 20-Days Moving Average, 6-Days Relative Strength Index, 12-Days Relative Strength Index, 9-Days RSV, 10-Days Bias, 20-Days Bias, 9-Days KD(9-Days K, 9-Days D), 9-Days J, 12-Days WMS, Daily Transaction Change Rate, Daily Volume Change Rate, Stock Price Change Rate Macroeconomic Variables Daily Interest Change Rate, Daily Exchange Change Rate, Monthly Unemployment Change Rate, Monthly Change in Inflation Rate, Monthly Change in Money Supply Rate, Taiwan Daily Change Rate of Stock Price, American Daily Change Rate of Stock Price, Japanese Daily Change Rate of Stock Price Qualitative Data Economical News, News of Financial Market, International Economical News, News of Fund and Futures, News of Stock Market, News of American Stock Market The ANN architectures used for experiment one, experiment two, and experiment three are listed in table 2. And, we used the fifteen technological variables

10 210 Deng-Yiv Chiu and Kun-Po Chuang to predict the tendency of stock market. In experiment one, we used fifteen neurons of input layer to test. Because we wanted to speed up the convergence of ANN, we initialize the learning rate to 0.1. The confidence value is used to decide when ANN could stop training and it is set to And the learning rate and the confidence value were set by referring to several previous researches [10,15,19]. Table 2. The ANN Architecture for Experiment One, Two, and Three Experiment Input Neuron Hidden Neuron Output Neuron Learning Rate The error of ANN during training period is shown in figure 4. The square error of ANN went down to 0.01 after completing about 5500 cycles. And the result could be acceptable Square Error Times Figure 4. The Error of Experiment One in Training Period To compare the difference among experiments, we would stay the same architecture. So the only difference between the experiment one and the experiment two is the neuron number of the ANN hidden layer. The experiment two has ten neurons. The error of experiment two in training period is shown in figure 5.

11 Artificial Neural Network and Chinese News Classification Techniques Square Error Times Figure 5. The Error of Experiment Two in Training Period In experiment three, the hidden layer of ANN only has five neurons. The situation of convergence of ANN in experiment three is shown in figure 6. First, comparing the speed of convergence with previous experiments, the convergence speed of experiment three is the fastest among these three experiments. Secondly, the curve is smoother than the curves of other two experiments, so we could believe that the situation of convergence is the best among these three experiments. Moreover, experiment three outperforms others and it was used for following experiments Square Error Times Figure 6. The Error of Experiment Three in Training Period

12 212 Deng-Yiv Chiu and Kun-Po Chuang In experiment four, we used not only the technological variables but also macroeconomic variables shown in table 1 to predict the tendency of stock market. The ANN architectures and experiment results for experiment four and five are listed in table 3. And, then we could find the speed of convergence is obvious faster than previous experiments as shown in figure 7. Although it has less training cycles among these experiments, the curve of square error is not smoother than experiment three. Table 3. The ANN Architectures for Experiment Four and Five Experiment Input Neuron Hidden Neuron Output Neuron Learning Rate Accuracy Rate Square Error Times Figure 7. The Error of Experiment Four in Training Period In experiment five, we combined the quantitative and qualitative factors listed in table 1 to predict the tendency of stock market. The error of convergence of ANN is shown in figure 8. Because the error of ANN reaches 0.01 quickly and the curve of square error is smooth, we decided to continue to train until the convergence of ANN. So the confidence value of experiment five is 1.0E-7. As shown in figure 8, we could find that curve of square error becomes smooth much faster than others. Also, the accuracy rate of experiment five is the best among accuracy rates of all other experiments done in this research as a whole.

13 Artificial Neural Network and Chinese News Classification Techniques Square Error Times Figure 8. The Error of Experiment Five in Training Period 4. Conclusion In these experiments, we observed that, if a major event happened, the quantitative factors might not respond adequately. But, the qualitative factors showed significant influence on unstable situations. So we would use methods to quantitate news and utilize the result to predict the tendency of stock market. It shows that news plays an important role to stock investor s decision making in Taiwan. Especially, a horrible issue, such as 911-terror attack, occurs. Reference Chinese Reference: [1] 張喬富, 類神經網路股市投資決策支援系統 -- 總體經濟變數之再探討, 國立台灣大學資訊管理研究所碩士論文,1998 [2] 楊晴華, 影響股市波動因素之研究 -- 以台灣股市為例, 國立中正大學企業管理研究所碩士論文,2000

14 214 Deng-Yiv Chiu and Kun-Po Chuang [3] 謝榮記, 台中地區個別投資人對訊息心理反應與加權股價指數關聯性之研究, 私立朝陽科技大學財務金融研究所碩士論文, 2000 English Reference: [4] A. K. Jain and R. C. Dubes, Algorithms for Clustering Data, Prentice-Hall Advanced Reference Series. Prentice-Hall, Inc., Upper Saddle River, NJ, [5] A. K. Jain, M. N. Murty and P. J. Flynn, ACM Computing Surveys, 31 (1999) 3, [6] Atiya and Talaat, An Efficient Stock Market Forecasting Model Using Neural Networks, IEEE, 1997, [7] B. King, Step-Wise Clustering Procedures, J. Am. Stat. Assoc. 69 (1967), [8] C. J. van Rijsbergen, Information Retrieval, Buttersworth, London, second edition, [9] D. F. Bassi, Utilization of Neural Network for Constructing a User Friendly Decision Support System to Deal Stocks, IEEE, 1995, [10] D. W. Patterson, Artificial Neural Networks Theory and Applications, 1996, [11] F. Murtagh, A Survey of Recent Advances in Hierarchical Clustering Algorithms Which Use Cluster Centers, Comput. J. 26 (1984), [12] Gerald Kowalski, Information Retrieval Systems Theory and Implementation, Kluwer Academic Publishers, [13] H. Iba and T. Sasaki, Using Genetic Programming to Predict Financial Data, Evolutionary Computation, CEC 99, Proceedings of the 1999 Congress, 1999, [14] J. C. Lin and S. Y. Huang, Automatic Classification of Chinese Documents, Master Thesis, Institute of Computer and Information Science, National Chiao Tung University, [15] J. A. Freeman and D. M. Skapura, Neural Networks Algorithms, Application, and Programming Techniques, 1997,

15 Artificial Neural Network and Chinese News Classification Techniques 215 [16] J. H. J. Ward, Hierarchical Grouping to Optimize an Objective Function, J. Am. Stat. Assoc. 58 (1963), [17] J. Roman and A. Jameel, Backpropagation and Recurrent Neural Networks in Financial Analysis of Multiple Stock Market Returns, System Sciences, Proceedings of the Twenty-Ninth Hawaii International Conference, 2 (1996), [18] J. H. Wang and J. Y. Leu, Dynamic Trading Decision Support System Using Rule Selector Based on Genetic Algorithms, Neural Networks for Signal Processing VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop, 1996, [19] M. J. A. Berry and Gordon Linoff, Data Mining Techniques For Marketing, Sales, and Customer Support, 1997, [20] P. H. A. Sneath and R. R. Sokal, Numerical Taxonomy, Freeman, London, UK, [21] R. J. Kuo, L. C. Lee, and C. F. Lee, Integration of Artificial Neural Networks and Fuzzy Delphi for Stock Market Forecasting, Man and Cybernetics, IEEE International Conference, 2 (1996), [22] R. J. Kuo, A Decision Support System for the Stock Market Through Integration of Fuzzy Neural Networks and Fuzzy Delphi, Applied Artificial Intelligence, 12 (1998), [23] V. K. Sagar and C. K. Lee, A Neural Stock Predictor Using Qualitative and Quantitative Data, Neural Information Processing, Proceedings, ICONIP '99, 6th International Conference, 2 (1999),