BEHAVIOR VISUALIZATION OF AUTONOMOUS TRADING AGENTS

Transcription

1 BEHAVIOR VISUALIZATIO OF AUTOOMOUS TRADIG AGETS Tomoharu akashima, Hiroko Kiano, Hisao Ishibuchi College of Engineering Osaka Prefecure Universiy Gakuen-cho 1-1, Sakai, Osaka , Japan {nakashi, kiano, KEYWORDS Fuzzy rule, black-box analysis, fuures rading ABSTRACT In his paper, we visualize he behavior of a fuures rading agen by using fuzzy if-hen rules. The main aim of his paper is o graphically inerpre how rading agens make decisions such as o buy or o sell a fuures sock. Fuzzy ifhen rules are used for his aim because heir aneceden par specifies he feaures of ime series. The procedure for visualizing he behavior of rading agen is firs o rain a rading agen ha consiss of a se of fuzzy if-hen rules. Bycarefully examining, we selec a small number of prominen fuzzy if-hen rules ha mos represen he behavior of rading agens. Finally hose seleced fuzzy ifhen rules are represened in a graphically undersandable manner. ITRODUCTIO Fuzzy rule-based sysems have been successfully applied o various problems such as conrol problems (Sugeno 1985, Lee 1990). The advanage of he fuzzy rule-based sysems is is inerpreabiliy. Since fuzzy if-hen rules in a fuzzy rule-based sysem are wrien using linguisic values, human users can linguisically undersand he meaning of fuzzy ifhen rules. The validiy of fuzzy if-hen rules can be measured using wo well-known crieria in he field of daa mining: confidence and suppor (Agrawal e al. 1996, Agrawal and Srikan 1994). The fuzzy version of hese measures are also inroduced in (Hong e al. 2001, Ishibuchi e al. 2001). Alhough here are numerous researches on exracing paerns or rules from a large daa base, he number of researches on how such exraced rules are effecive for pracical use in a paricular domain is no large. The purpose of his paper is o examine he effeciveness of he exraced rules for supporing decision making of human users. In his paper, we consider a virual fuures marke as a problem domain. The virual marke allows a number of auonomous agens o ake par in he fuures marke. Human beings are also allowed o rade in he fuures sock index in he virual marke. Auonomous agens and human beings are required o deermine wheher hey buy he fuures sock index or sell, he limi price, and he quaniy of he fuures rade. We have developed an auonomous agen ha rades in fuures sock index in a virual marke (akashima e al. 2002). An adapive fuzzy rule-based sysem was used in he auonomous agen. The adapive fuzzy rule-based sysem consiss of a number of fuzzy if-hen rules ha linguisically provide he decision making on he rading acion (i.e., buy he fuures index or sell) for differen condiions. The evaluaion (i.e., successful rade or no) of he rade acion is performed afer he new spo price and he new fuures price are obained. According o he evaluaion, he agen aduss he weighs of fuzzy if-hen rules in he adapive fuzzy rule-based sysem. Tha is, we increase he weighs of fuzzy if-hen rules if he agen's decision making in he previous ime sep is successful. On he oher hand, he weighs of fuzzy if-hen rules are decreased if he decision making is no successful. Since he weigh updae can be performed on-line, i is expeced ha he performance of he auonomous agen is gradually improved during he course of he rade. Thus he resulan fuzzy rule-based sysem afer he enough number of rade can be viewed as a knowledge base for he virual fuures rade. In his paper, we ry o linguisically inerpre he behavior of he auonomous agen by examining he weighs of fuzzy if-hen rules in he adapive fuzzy rule-based sysem. We selec a small number of fuzzy if-hen rules ha have a conras beween he weighs associaed wih Buy he fuures index and. Afer he seleced fuzzy if-hen rules are graphically ransformed ino he rading knowledge base, he rading knowledge base is shown o a human rader who paricipaes in he virual fuures rade. The human being can consul he rading knowledge hroughou he fuures rade. Saisical evidence shows ha he rading knowledge base exraced from he adapive fuzzy rule-based sysem improves he performance of human raders. A VIRTUAL FUTURES MARKET U-MART Recenly, virual economic markes have araced a grea deal of aenion for analyzing economic sysems and developing auonomous agens. From he view poin of he economics, he advanages of he virual economic markes are as follows. Firs, one can analyze paerns of humans' rading behavior wih respec o he rading acion. Secondly, one can examine how o avoid he speculaive acion such as violen flucuaions of socks. On he oher hand, from he engineering poin of view, we can examine he acual effeciveness of he use of learning mehods, evoluionary mehods, and muli-agen echniques in economic sysems. The U-MART (Unreal Marke as Arificial Research Tes-bed) proec is one of such virual markes where muliple players including human beings can simulaneously rade in a fuures sock index (Fig. 1). In he U-MART, a machine (i.e., an auonomous agen) or a human being is called a U-MART clien and is given marke informaion such as ime series daa of spo prices and fuures prices. Cliens also have is own curren informaion such as is posiion (i.e., a balancing amoun of

2 U-MART Server Exernal informaion ( S, F ) Trade order ( BS, P, Q) U-MART Clien Fig. 1 Snapsho of U-Mar he fuures index rade), remaining cash, and ime o he final selemen. Based on he above informaion, each clien has o make a decision on wheher i buys or sells he fuures index, he limi price of he fuures rading, and he quaniy of he fuures rade. Thus, a clien in U-MART can be viewed as an inpu-oupu sysem A as follows: A( S, F, Pos, Cash, ) ( BS, P, Q), (1) where S and F is he ime series of he spo prices and he fuures prices (called he U-MART prices), respecively, Pos is he posiion of a clien, Cash is he remaining cash, is he remaining ime o he final selemen, BS represens he clien's rading decision on he fuures sock index, P is he limi price, and Q is he quaniy of he rade. Each U- MART clien ineracs wih he U-MART server for rading in a fuures sock index hrough he TCP/IP proocol. Among he inpu variables o he U-MART clien, he wo ime series S and F are exernally provided by he U-MART server and he oher inpu variables are inernally held by he U-MART clien. In Fig. 2, we show a general view of he rade beween a U-MART clien and he U-MART server. The U-MART server deermines he fuures index price by a mehod called Iayose. In Iayose, he U-MART server firs collecs an order from each U-MART clien such as buy or sell of he fuures index, he limi price, and he quaniy of he rade. Then i compares he buy orders wih he sell orders. The fuures price is deermined a he poin where he price and quaniy of buy orders are mached by hose of sell orders. The goal of he U-MART cliens is o maximize he profi caused by he difference beween he selling fuures prices and he buying fuures prices. ADAPTIVE FUZZY RULE-BASED SYSTEM We have already developed a learning U-MART clien for he virual fuures marke (akashima e al. 2002). In (akashima e al. 2002), he U-MART clien mainains an adapive fuzzy rule-based sysem ha deermines wheher he clien should buy or sell a fuures sock index. The weighs of fuzzy if-hen rules correspond o he suppor for buying or selling he fuures sock index. During he course of he fuures rades, weighs of fuzzy if-hen rules are updaed on-line according o he evaluaion of he rade a he previous ime sep. The rade is evaluaed a each ime sep by he high-and-low relaion beween he spo prices and he fuures prices (i.e., U-MART prices). The following subsecions describe he adapive fuzzy rule-based sysem used for he auonomous agen in deail. Problem Formulaion and Fuzzy If-Then Rules In his subsecion, we explain he fuzzy rule-based sysem ha was applied o he learning U-MART clien in (akashima e al. 2002). In he fuures marke, ime series daa of boh he spo and he fuures prices are available o he adapive fuzzy rule-based sysem. Le us assume ha $n$ pieces of informaion are used by he agen for deermining wheher o buy or sell he fuures sock index. In his case, he problem of he fuures marke for our fuures rade agen can be viewed as a wo class paern classificaion problem wih $n$-dimensional inpus. A fuzzy rule-based sysem is applied o his n-dimensional wo-class paern classificaion problem. The adapive fuzzy rule-based sysem in he learning U-MART clien consiss of fuzzy if-hen rules of he following ype: R : If x1 is A1 and and xn is An hen Buy wih b and wih s, 1,,, (2) where R is a label of -h fuzzy if-hen rule, x ( x 1,, x n ) is an inpu vecor o he fuzzy rule-based sysem, A 1,, An are aneceden fuzzy ses, and b and s are real values of he fuzzy if-hen rule R corresponding o buying and selling he fuures sock index, respecively. In he implemenaion of he learning U-MART clien in his paper, we use he difference beween he spo price a he curren ime sep and hose a he hree differen ime seps as hree inpu values x1, x2, x3 o he fuzzy rule-based sysem (Fig. 3). Tha is, he learning clien deermines wheher i buys or sells he fuures sock index from an inpu vecor x ( x1, x2, x3). Thus, in his paper we deal wih he decision making problem as an hree-dimensional wo-class paern classificaion problem. The fuzzy if-hen rules can be wrien as follows: R : If is and is and is Inernal informaion ( Pos, Cash, ) Fig. 2 The general view of he rade in U-MART x1 A1 x2 A2 x3 A 3 hen Buy wih b and wih s, 1,,, (3)

3 Price In Fig. 3, S ( s1,, s ) is he ime series of spo prices from he begining of he rade (i.e., s1 ) unil ime sep (i.e., s ) where s k is a spo price a ime sep k. We use he following hree pieces of informaion as inpu variables for he fuzzy rule-based sysem: x s s 1 1, x x s - 5 s s 2 2 s s 3 3 Spo price: S s - 4 s - 3 x 3 s - 2 From he above explanaion, we can see ha he fuzzy rulebased sysem performs a mapping from a hree dimensional sae vecor x ( x1, x2, x3) o a single binary value corresponding o eiher Buy or. (4), (5). (6) Fuzzy Inference and Decision Making s - 1 Curren ime sep Fig. 3 Inpu variables in our fuzzy rule-based sysem Le us consider ha a a paricular ime we have already calculaed hree inpu variables x 1, x2, and x 3 for he fuzzy rule-based sysem. In his subsecion, we show how an agen makes a decision on wheher i buys or sells he fuures sock index. Assume ha here are fuzzy if-hen rules in he fuzzy rule-based sysem. In his paper, we divide each axis of inpu variables ino hree fuzzy ses as in Fig. 4. In Fig. 4,, Z, and P represen linguisic erms negaive, zero, and posiive, respecively. Since here are hree inpu variables in our fuzzy rule-based sysem and hree fuzzy ses for each inpu variable, he oal number of fuzzy if-hen rules involved in he fuzzy rule-based 3 sysem is oe ha each fuzzy if-hen rule has wo weigh values associaed wih buying and selling he fuures index, respecively. Afer he calculaion of hree inpu values, Membership x 2 Z x P Spo price () - Spo price ( - k) Fig. 4 Membership funcions s x 1 x2, and x 3, QBuy and Q are calculaed using fuzzy inference as follows: 1 Buy Q Q ( x) b 1 1 ( x) ( x) s 1 ( x), (7), (8) where x ( x1, x 2, x3) is an inpu vecor o he fuzzy rulebased sysem, () is he compaibiliy of he inpu vecor x wih he fuzzy if-hen rule R. The compaibiliy ( ) of he inpu vecor x wih he fuzzy if-hen rule R is calculaed by he following produc operaor: ( x ) 1( x1) 2( x2) 3( x3), (9) where i () is he membership funcion of an aneceden fuzzy se A in he -h fuzzy if-hen rule R (see Fig. 4). i Afer calculaing Q Buy and Q, he agen makes a decision on wheher he agen buys or sells he fuures index based on he following decision rule: [Decision Rule] If QBuy Q, he agen buys he fuures index, Else if Q Buy Q, he agen sells he fuures index, Oherwise, he agen's rade is he same as he decision a he previous ime sep. oe ha he auonomous agen does no make a decision for he firs hree ime seps since here are no enough informaion o calculae an inpu vecor for he firs hree ime seps. Tha is, he agen only collecs he informaion for he decision making for he firs hree ime seps. In his paper, we assume ha he agen ries o opimize he decision making only on wheher he agen buys or sells he fuures index under various condiions of he ime series daa of spo prices. Thus fuzzy if-hen rules in he adapive fuzzy rule-based sysem have only wo weighs corresponding o Buy and. There are, however, wo more hings o be deermined in order o rade he fuures index. One is he limi price and he oher is he quaniy of he rade. The oher wo pieces of informaion such as he limi price and he quaniy of he rade order are deermined as follows. Firs, he limi price P in (1) is deermined as follows: sm 5, P sm 5, if decision is Buy, oherwise. (10)

4 Tha is, he limi price is deerminisically decided based on he spo price a he curren ime sep. On he oher hand, he quaniy Q of he rade in (1) is specified as Q 200. Tha is, we don' adapively deermine he quaniy of he rade according o any informaion bu fixed o a prespecified value. On-Line Learning of Fuzzy If-Then Rules b s new new b b s s (1 b b s (1 s ) ( x), ( x), ( x), if successful, oherwise, ) ( x), if successful, oherwise, (11) (12) In his subsecion, we show how weighs of fuzzy if-hen rules in our fuzzy rule-based sysem are adused so ha he agen can maximize he profi hrough he rading. Le us assume ha he agen has already made a decision on wheher he agen buys or sells he fuures index. A he nex ime sep, he agen is given anoher informaion on he ime series daa of spo prices (i.e., s m1 ) and he fuures prices (i.e., fm 1 ). We evaluae he agen's rade decision (Buy or ) according o he high-and-low relaion beween and as follows: sm1 f m 1 [Evaluaion Crierion] If QBuy Q and s m 1 f m 1, hen he decision is evaluaed as successful, Else If QBuy Q and s m 1 f m 1, hen he decision is evaluaed as successful, Oherwise he decision is evaluaed as unsuccessful. Tha is, we evaluae he decision making based on he absolue price difference beween he spo price and he fuures price a he following ime sep. If he agen's decision is Buy, he evaluaion for he decision is successful only if he spo price is higher han he fuures price. On he oher hand, if is chosen, he evaluaion for he decision is successful only if he spo price is lower han he fuures prices. This crierion is derived from he observaion ha he spo price and he fuures price mus be coincide a he final selemen (see Fig. 5). This evaluaion is used for updaing he weighs of fuzzy if-hen rules. Price Spo price Fuures price Price where is a posiive learning rae and b and s are weigh values of he -h fuzzy if-hen rule R ha are associaed wih buying and selling he fuures sock index, respecively. oe ha only he weighs corresponding o he seleced acion are updaed by he above equaions. We do no updae he weighs corresponding o he acion ha is no seleced. For example, when we selec o Buy he fuures sock index, he weighs b, 1, 2,,, are updaed and we do no modify he weigh s ha are associaed wih he acion. To summarize, he procedure of our learning clien for he fuures rade is described as follows: [Procedure of on-line learning for fuures rading] Sep 1: Iniializaion. Se iniial weighs of he fuzzy ifhen rules o eiher some prespecified values or random values. Sep 2: Decision making. Using he ime series of spo prices, calculae he value corresponding o Buy or as in (7) and (8). Make a decision of Buy or according o he decision rule described in he previous subsecion. The limi price and he quaniy of he rade are also deermined. Sep 3: Evaluaion. Given he fuures price and he spo price a he following ime sep, evaluae he decision making of Buy or as successful or no according o he evaluaion crierion described in his subsecion. Sep 4: Weigh updae. Updae he weighs of fuzzy if-hen rules involved in he fuzzy rule-based sysem. oe ha only hose weighs corresponding o he decision making (i.e., Buy or ) a he curren ime sep are updaed. These above seps are ieraed unil he conrac monh (i.e., he final selemen) is reached in he fuures rading. VISUALIZATIO Selemen Selemen Procedure of Knowledge Exracion Fig. 5 Final selemen. The main idea of he weigh updae is ha he weighs of he fuzzy if-hen rules ha conribue o he successful decision making are increased while we decrease he weighs of hose fuzzy if-hen rules ha are responsible for unsuccessful decision making. Thus he updae rule of he weighs is described as follows: The learning U-MART clien in he las secion can be used as a knowledge acquisiion ool since he adapive fuzzy rule-based sysem in he U-MART clien can be seen as a knowledge base for he virual fuures rade. In his secion, we examine such possibiliy hrough laboraory experimens. The knowledge exracion procedure consiss of wo phases: uning of fuzzy if-hen rules in he adapive fuzzy rulebased sysem and selecing a small number of fuzzy if-hen rules wih a large conras beween consequen weighs. In he following subsecions, each phase of he knowledge exracion is explained.

5 Tuning and Inerpreing he Fuzzy Rule-Based Sysem Firs, he learning U-MART clien wih he adapive fuzzy rule-based sysem is ieraively applied o he virual fuures marke. Since he learning clien needs a number of ieraions for learning he weighs of fuzzy if-hen rules, we repeaed he virual fuures rade several imes. Afer he fuures rade, i is expeced ha hose fuzzy if-hen rules ha are relaed o he criical inpu saes have a conras beween weighs for Buy and. For example, he weigh of Buy is larger han for a fuzzy if-hen rule if he U- MART clien has made a number of successful decision making of Buy in a siuaion compaible o he aneceden par of he fuzzy if-hen rule. Such a fuzzy if-hen rule is likely o sugges Buy in he corresponding siuaion. Anoher example is ha if he U-MART clien has made a number of unsuccessful decision making in a siuaion compaible o he aneceden par of a fuzzy if-hen rule, he weigh corresponding o he decision making becomes smaller han he weigh corresponding o he oher decision making. In his case, he suggesion by such a fuzzy if-hen rules is no o perform he rade acion (eiher Buy or ) wih he smaller weigh. Selecing a Small umber of Fuzzy If-Then Rules We examined he weighs of each fuzzy if-hen rule o selec a small number of fuzzy if-hen rules wih a srong conras beween weighs for Buy and. From he oal 3 number of 27 ( 3 ) fuzzy if-hen rules, we manually seleced five such fuzzy if-hen rules. Table 1 shows he seleced fuzzy if-hen rules wih a srong conras in he weighs. (e) correspond o he seleced rules o. 1-5 in Table 1, respecively. The visualizaion is done such ha he aneceden linguisic values (, Z, and P) are inerpreed as he relaive posiion beween he presen spo price s and he previous spo prices s3, s2, and s 1, and he recommended acion is deermined according o he exreme value of he consequen value (0 or 1) for each seleced fuzzy if-hen rule (oe ha he value 0 means ha he corresponding acion is no recommended and i is recommended when he value is 1). In our experimens, six human users separaely paricipaed in he virual fuures rade wice. One experimen is done wih he presenaion of he seleced fuzzy if-hen rules, and he oher wihou he presenaion of he seleced fuzzy ifhen rules. We performed his experimen for six differen human users. For hree human users, he firs experimen was done wih he presenaion of he seleced fuzzy if-hen rules and he second experimen wihou he presenaion. On he oher hand, he experimens for he oher hree human users were done wih he seleced fuzzy if-hen rules presened in he firs experimen and wihou he presenaion in he second experimen. This is because we need o minimize he effec of he ordering condiion in he experimens Tha is, he bias of presening he seleced fuzzy if-hen rules in he firs experimen for he firs hree human users is offse by he bias of presening hem in he second experimen for he oher hree. We show he experimenal resuls in Table 2. Table 2 shows he remaining asses afer he final selemen. Table 1 Seleced Fuzzy If-Then Rules o x3 x2 x1 q 1 q Z P P P Z Z P P P Table 2 Experimenal resuls Human ID Wihou guide Wih guide A 258,687,000 1,449,355,000 B 878,280,000 2,093,095,000 C 961,393,000 2,335,901,000 D * 1,675,616,000 E 926,983,000 1,220,221,000 oe ha hese fuzzy if-hen rules were seleced manually and subecively according o he difference in he weighs of fuzzy if-hen rules. Alhough i is possible o sysemaically selec a small number of fuzzy if-hen rules using some saisical echnique, i is beyond our scope of his paper. I will be invesigaed in our fuure research. Experimens wih Human Users In his subsecion, we show he experimenal resuls where human users are provided wih a small number of he seleced fuzzy if-hen rules when hey paricipae in he virual fuures rade. In order o make he seleced fuzzy ifhen rule more undersandable, we visualized he seleced fuzzy if-hen rules as shown in Fig. 6. In Fig. 6, graphs (a)- F 740,654, ,363,000 * shows he human user wen bankrupcy. From Table 2, we can see ha almos all he human users could performed beer wih he presenaion of he seleced fuzzy if-hen rules han wihou hem. Thus we can expec ha our learning clien could become a human decision suppor ool. In order o confirm his observaion saisically, we perform he Wilcoxon's rank-sum es. The Wilcoxon's rank-sum es is a nonparameric es ha is a sample -es based solely on he order in which he observaions from he wo samples fall. In he Wilcoxon's es, we order he resuls of human users in he descending

6 o Buy Price ( ) (a),, Price ( ) (b), Z, P Price ( ) (c), P, Price ( ) (d) P, Z, Z Buy Price( ) (e) P, P, P Fig. 6 Visualizaion of he seleced fuzzy if-hen rules order of he final remaining asses in Table 2. The order is used as a rank for each human user. The sum of he rank is used as he saisic for one-sided es. In he es ha compares he null hypohesis (here is no difference beween he resul wih and wihou he presenaion of he seleced rules) agains he alernaive hypohesis (here is difference), he null hypohesis is reeced wih a 0.05 level. Thus, we can saisically say ha he human users can perform beer wih he help of he seleced fuzzy if-hen rules. COCLUSIOS In his paper, we presened how an auonomous agen in a virual fuures marke is graphically analyzed. The auonomous rading agen has a learning mechanism during he course of he rade. A se of fuzzy if-hen rules were used whose aneceden par represens he ups-and-downs of ime series daa. The consequen par of fuzzy if-hen rules is he rading decision of he agen, i.e., Buy or. Firs we applied a learning mehod o a virual fuures marke in order o obain he possible fuures rading knowledge. Then we manually seleced a small number of fuzzy if-hen rules wih a srong conras in weighs for decision making opions. The seleced fuzzy if-hen rules were presened o human users afer we visualized hose fuzzy if-hen rules in order for human users o easily undersand hem. Experimenal resuls showed ha here was a posiive effec of presening he seleced fuzzy if-hen rules. Tha is, human users could achieve beer remaining asses by using he exraced knowledge for he fuures rade. REFERECES M. Sugeno. 1985, An Inroducory Survey of Fuzzy Conrol, Informaion Science, Vol. 30, o. 1/2: C. C. Lee. 1990, Fuzzy logic in conrol sysems: fuzzy logic conroller Par I and Par II, IEEE Trans. Sys., Man, Cybern., Vol. 20: R. Agrawal, H. Mannila, R. Srikan, H. Toivonen, and A. I. Verkamo, 1996, Fas discovery of associaion rules', Advances in Knowledge Discovery \& Daa Mining, , AAAI Press, Menlo Park, CA. R. Agrawal and R. Srikan, 1994, Fas algorihms for mining associaion rules, Proc. of 20h Inernaional Conference on Very Large Daa Bases, Expanded version is available as IBM Research Repor RJ9839, T.-P. Hong, C.-S. Kuo, and S.-C. Chi, 2001, Trade-off beween compuaion ime and number of rules for fuzzy mining from quaniaive daa, Inernaional Journal of Uncerainy Fuzziness and Knowledge- Based Sysems, vol. 9, no. 5: H. Ishibuchi, T. Yamamoo, and T. akashima, 2001, Fuzzy daa mining: Effec of fuzzy discreizaion, Proc. of 1s IEEE Inernaional Conference on Daa Mining, T. akashima, T. Ariyama, and H. Ishibuchi, 2002, Online learning of a fuzzy sysem for a fuures marke, 2002 Inernaional Conference on Fuzzy Sysems and Knowledge Discovery,