A Self-adaptive Predictive Policy for Pursuit-evasion Game

Transcription

1 JOURNAL OF INFORMATION SCIENCE AND ENGINEERING 24, (2008) A Self-adaptive Predictive Policy for Pursuit-evasio Game ZHEN LUO, QI-XIN CAO AND YAN-ZHENG ZHAO Research Istitute of Robotics Shaghai Jiaotog Uiversity Shaghai , P.R. Chia The proposed self-adaptive predictive pursuig policy cosists of a actio decisio-makig procedure ad a procedure of adjustig the estimatio of evader s actio preferece. Sice correct estimatio of oppoet s itetio would do good to wi adversarial games, it itroduces the coceptio of actio preferece to model oppoet s decisio-makig. Because evader ofte has differet actio preferece i differet situatio, to model evader s decisio-makig, pursuer has to divide the situatio space ito may categories ad provide a set of estimatio of evader s actio preferece for each kid of situatio. Pursuer adjusts the estimatio of evader s actio preferece i certai situatio by observig evader s actio. Actio decisio-makig procedure cosists of situatio sortig, possible future states computatio, payoff evaluatio ad actio selectio. Actio decisio-makig is based o the decisio tree costructed by expected payoffs. Expected payoffs are itegrated from sigle payoffs. Sigle payoffs are evaluated by gais of features reflectig adversarial situatio. A simulatio of middle size soccer robots has bee carried out ad illustrated that the proposed policy is effective. Keywords: actio preferece, payoff fuctio, predictive, pursuit-evasio games, selfadaptive. INTRODUCTION Pursuit-evasio games are importat issues. There are may types of pursuit-evasio games, for examples visibility-based pursuit-evasio game [], the game of multipursuers capturig oe evader [2], pursuit-evasio game with safety-zoe [3, 4], etc. It ca also be cosidered that the RoboCup Soccer Robot game is costituted with some pursuit-evasio scearios. The sceario of a robot team iterceptig a oppoet dribblig is a typical pursuit-evasio game with safety-zoe. Today, RoboCup is a popular game to foster itelliget robotics research by providig a stadard problem [5]. Correct speculatio of oppoet s itetio is a key to wi a adversarial game. To estimate oppoets itetio, some researchers assumed that game players are ratioal [6, 7]. I such works, the assumptio I KNOW THAT HE KNOWS THAT I KNOW is grated by the game players. But i may adversarial games, especially today s adversarial games of robot, players are ofte short of iformatio of their oppoets. For example, i the typical pursuit-evasio game of missile iterceptio problem [3, 4], evader is ofte thought as blid, i.e. it is impossible that players of this kid game are ratioal. As the state-of-the-art of RoboCup soccer robot, ratioal assumptio is ot always i accord with the fact of soccer robots. For examples, robots of EIGEN MSL team 2006 use Fuzzy Potetial Method (FPM) to geerate actio [8], robots of Philips MSL Team 2006 are reactive [9] ad they could ot be ratioal. I fact, eve whe professioal hu- Received November 24, 2006; revised March, 2007; accepted Jue, Commuicated by Takeshi Tokuyama. 397

2 398 ZHEN-LUO, QI-XIN CAO AND YAN-ZHENG ZHAO ma athletes play adversarial games requirig quick decisio-makig, such as basketball game, they are ofte ot ratioal ad traied to follow if X, the execute actio Y rules. To atagoize with irratioal oppoets, a short-term predictio based pursuig policy for pursuit-evasio games is proposed i [0]. If it is possible, it is useful to predict other s actio []. The policy proposed i [0] supposes that player assumes that the probability of the oppoet executig each actio is equal. The oppoet s preferece for certai actio i certai situatio is igored. I real world, aget ofte follows some give actio decisio-makig rules, i.e., there is a kow or potetial mappig relatioship betwee situatios ad actios. For example, robots with reactive itelliget system architecture [2-4] have actio preferece for each situatio. Obviously, to beat the oppoet with actio preferece, it is helpful to take the actio preferece of oppoet ito accout. Thus, a self-adaptive predictive pursuig policy is preseted. The self-adaptatio of it is realized by modifyig the estimatio of oppoets actio prefereces i differet situatios. The preseted self-adaptive method is ot same with reiforcemet learig. The coceptio of reiforcemet learig is first suggested by Misky [5]. The basic idea of reiforcemet learig is as follows: if a actio leads to positive reward, the the possibility of the aget executig the actio will be icreased; otherwise, if a actio leads to egative reward, the the possibility of executig the actio would be decreased. As the paper [6] said, a drawback of reiforcemet learig is that the learig procedure would be log. O the cotrary, the estimatio of oppoet s actio preferece i certai situatio ca be adjusted quickly. For may adversarial games, such as soccer game, it is ot a good idea to use the try ad adjust procedure of reiforcemet learig (although reiforcemet learig is useful i traiig soccer robot). Furthermore, i may adversarial games, situatio is ot so simple as the evet of shootig/defedig i most time. It is a problem to desig a perfect reward evaluatio. O the other had, if situatios ca be evaluated perfectly, why ot use the method that decisio is made by evaluatig the result of actios directly? With the preseted method, desiger ca focus o how to improve the reward evaluatio. For the great maeuverability of itelliget players i may games, their possible state spaces of log-term ruig would be extremely huge. So i such games, it is ofte impractical to do log-term predictio before selectig a actio to execute. Furthermore, it is difficult to work out a o-greedy GLOBAL-MAX solutio i such pursuitevasio games. Thus, the preseted method oly uses short-term iformatio for decisiomakig. The basic actio decisio-makig procedure is described i sectio 2. Sectio 3 expatiates o the method of actio preferece estimatio. A simulatio is preseted i sectio 4. Sectio 5 cocludes. 2. ACTION DECISION-MAKING PROCEDURE I practice, it is difficult to work out a optimal actio solutio i pursuit-evasio games with itelliget ad highly maeuverable oppoets. Player ofte has to execute a actio that seems to be reasoable. Thus, it is reasoable to assume that a aget P has

3 SELF-ADAPTIVE SHORT-TERM PREDICTIVE PURSUING POLICY 399 a discrete ad fiite actio space ad it justly selects a ratioal actio withi the limited actio space to execute. I fact, for may real systems, their actios or cotrols are restricted to be a limited umber of classes. For example, i several versios of middle size soccer robot itelliget system developed by us, there are oly several levels of power i use. Thus, if W is the actio space of P, W = {w i i =,, N}, where w i deotes actio i ad N is the umber of actios, the the result of P s decisio-makig is that it selects oe actio w j (or actio sequece) to execute. For the proposed predictive policy, the selected actio w j should lead to a maximal reward or miimal payoff for certai future. Payoff (or reward) is used to idicate the vatage grade of players i a adversarial situatio. Sice a aget usually does ot kow whether its oppoet is ratioal or ot, it is reasoable to use the aggregate values of payoffs at some future time to make actio decisio. Here, the aggregate value of payoff is called as expected payoff. Expected payoff is itegrated with sigle payoff. Sigle payoff ca be defied as i curret situatio (player P is i state s 0 ad its oppoet E is i state s 0,), the payoff J(s 0, s 0, w i, u j, t) of a player P executig actio w i with time t agaist its oppoet E executig actio u j. Thus, after time t, expected payoff E[J(s 0, s 0, w i, u, t)] of a player P executig actio w i i curret situatio satisfies the followig equatio: EJs [ (, s, w, ut, )] = pujs ( ) (, s, w, utdu, ), () i U where U is the actio space of player E, p(u) is the probability of E executig actio u i the situatio (costituted by P state s 0 ad E state s 0 ). It satisfies the equatio: i pudu= ( ). (2) U Defiitio The probability or probability desity p(u) of a aget executig a actio u i certai situatio is the actio preferece of the aget executig u i that situatio. I this paper, the actio preferece p(u) is P s subjective estimatio of the probability of E executig actio u. The payoff is evaluated with the result of executig certai actios, i.e. it is evaluated by evaluated the situatio as result of executig certai actios. If i curret situatio, P will be i state S it by executig actio w i with time t ad E will be i sate S ut by executig actio u with time t, the J ( s, s, w, u, t) = ψ ( s, s ). (3) i it ut Thus, Eq. () ca be rewritte as follows: E[ J( s, s, w, u, t)] = p( u) ψ ( s, s ) du. (4) i it ut U Accordig to Eq. (4), to evaluate its expected payoff of executig each kid actio with time t i curret situatio, a player eeds to kow the actio preferece of its oppo-

4 400 ZHEN-LUO, QI-XIN CAO AND YAN-ZHENG ZHAO et ad the future states of players. Because a player usually has differet actio preferece i differet situatios, it eeds estimate the type ofr the curret situatio firstly. Thus, the proposed actio decisio-makig procedure cosists of curret situatio sortig, possible short-term future states computatio for all players, payoff evaluatio ad actio selectio, as show i Fig.. Possible future states computatio Situatio sortig Tree of states Payoff evaluatio Decisio tree Actio selectio Fig.. Basic decisio-makig procedure. 2. Payoff Evaluatio ad Decisio-makig Payoff evaluatio procedure is divided ito two steps: at first, evaluates sigle payoff; the itegrates sigle payoffs ito expected payoffs. Actio decisio is made based o a decisio tree costructed from expected payoffs. 2.. Sigle payoff evaluatio I pursuit-evasio games, sigle payoff is ofte scaled with the distace amog game players. For may adversarial pursuit-evasio games, the vatage of situatio is ot oly reflected by the parameters of distace. With the coceptio of feature that reflects profile of the situatio vatage, a sigle payoff evaluatio method is proposed as follows: () At first, works out values of features those reflect the situatio costituted by P state s it agaist E state s ut ; (2) Evaluates each feature respectively; (3) Itegrates gais of all features ito a sigle payoff. For pursuit-evasio games with safety-zoe, such as typical sceario of iterceptig a robot E dribblig i middle size soccer robots game, several features could be extracted: the distace betwee P ad E at that momet, the distace betwee P ad the possible shootig lie of E, the related approachig speed betwee P ad E, etc. Payoff fuctio used to evaluate certai feature ca be desiged based o experiece kowledge ad adjusted by experimets. For example, i soccer robots game, it is show with experiece kowledge that the shorter the distace betwee P ad the possible shootig lie of E is, the safer P is. So the payoff fuctio of feature the distace betwee P ad the possible shootig lie of E ca be i followig form: f = e -d/k (5) where d is the distace ad k is a positive tuable costat. Assumig that feature payoff fuctios are f, f 2,, f respectively, the sigle payoff fuctio of P state s it agaist E state s ut ca be a liear weighted sum:

5 SELF-ADAPTIVE SHORT-TERM PREDICTIVE PURSUING POLICY 40 J ( s, s, w, u, t) = ψ( s, s ) = α f (6) i it ut j j j= where α j is the weight coefficiet Expected payoff evaluatio As Eq. (4) show i the above, the expected payoff E[J(s 0, s 0, w i, u, t)] of a player P executig actio w i i curret situatio ca be evaluated. Sice i adversarial game, players usually caot kow their oppoets actios i detail ad they usually ca oly gai a estimatio of their oppoets actio space boud values. Thus it is ot bad for a player P to assume its oppoet has a cotiuous actio space. It meas that the payoff evaluatio method as Eq. (4) is reasoable. But it is difficult to compute expected payoff E[J(s 0, s 0, w i, u, t)] with Eq. (4) directly. Thus, approximately evaluatio method is eeded. Accordig to the priciple of Quasi-Mote Carlo, the whole characteristic of somethig ca be reflected approximately by itegratig characteristics of evely distributed sample poits. Sice the expected payoff E[J(s 0, s 0, w i, u, t)] of P executig actio w i i some situatio is a fuctio of u as Eq. (4), P ca select evely distributed virtual actios u of E to evaluate E[J(s 0, s 0, w i, u, t)] approximately. For example, select agular velocities ± jω emax /m as values of E s 2m + actios u j, where j = 0,,, m. Thus, the expected payoff of P executig actio w i i the situatio ca be evaluated approximately as follows: 2m+ i i j j j= E[ J( s, s, w, u, t)] J( s, s, w, u, t) p( u ) (7) where p(u j ) is the estimated actio preferece of E executig actio u j i the situatio. After all expected payoffs have bee evaluated, P establishes a decisio tree show as Fig. 2, supposig that P has choices of actios. I Fig. 2,,, k 0 represet the idexes of predictig step. Based o the decisio tree, P determies which actio to execute i ext time. k 0 : ' ' : E [ J ( s, s, w, u, T )] E[ J ( s,,,, )] 0 s0 w u T ' ' E J ( s, s, w, u, k )] J ( s, s, w, u, k )] x [ T 0 Fig. 2. Decisio tree of P. E[ T z States Predictig As metioed i the above, payoff evaluatio is based o the predicted situatios; the policy executer P should work out possible future states of all players.

6 402 ZHEN-LUO, QI-XIN CAO AND YAN-ZHENG ZHAO P ca do multiple steps predictig. Here, a sigle predictig step is correspodig to time T: for P calculatig future states of a game player, it is assumed that the game player executes certai actio cotiuously with time T. If P has worked out all possible states of all players at time T, P fiishes the first step predictio; if P has worked out all possible states of all players at time 2T, P fiishes the secod step predictig;. I geeral terms, if P has worked out all possible states of all players at time T, 2T,, T, P fiishes steps predictio. P predicts its ow future states as follows: with curret state s 0, it calculates the future state s it as the result of executig actio w i with time T. After it obtaiig all possible states at time T, it fiishes the first step predictio of itself. Similarly, at the secod predictio step, it works out all possible states at time 2T with the states predicted at the first step. I the same way, P ca fiish steps predictio. The procedure of P predictig E s future states is similar with predictig its ow future states. The differece is that: whe P predicts its ow states, it uses real executed actios; whe P predicts E s future states, it is based o the virtual selected actios that are evely distributed i E s virtual cotiuous actio space. The time of each predictio step should ot be too log. Obviously, if it is too log, it will lead to isufferable error for decisio-makig. For example, there may exist two actios A ad B, at time T, executig A seems to be better tha executig B; but at some time i (0, T), executig A meas lose ad executig B meas wi. O the other had, the time T of each predictio step should ot be too short either. As the work show i [0], it is better to let the total predictio time legth reach some value to gai a reasoable decisio. Thus, to achieve a reasoable decisio, the less the time of each predictio step is, the bigger the predictio step umber will be. With a big predictio steps umber, the computatio complexity will be eormous. The computig complexity may affect the capability to react i real-time. The computig complexity ca be estimated with the times of computig sigle payoff. Suppose that P has kids of actios ad P assumes that E has m kids of actios, k k i i the after k predictio steps, the predicted states umber of P ad E are ad m k i= i= i i respectively, the umber of calculatig sigle payoffs is m. i= Thus, the growth of predictio step umber leads to great icreasig of computig complexity. For example, if = m = 0, icreasig oe predictio step would lead to 00 times augmet of computig complexity; if the predictio step umber k = 5, it takes more tha 5s to calculatig sigle payoffs usig a 2GHz computer; o the cotrary, if k = 3, the time used is about ms quatity. The time of each predictio step ca be determied accordig to the maeuverability of players. 3. ACTION PREFERENCE ESTIMATION A game player usually executes differet actios i differet situatios, i.e. there are differet types of actio preferece of the player for differet situatios. Thus, situatio should be divided ito may categories accordig to some stadards. P has to estimate actio prefereces of E for all kids of situatios. I essece, the procedure of estimatig

7 SELF-ADAPTIVE SHORT-TERM PREDICTIVE PURSUING POLICY 403 player E s actio prefereces is a procedure of modelig E s decisio-makig. Obviously, it is ideal for player P that P divides the situatio space i the same way as E does. But it is impractical sice P usually does t kow how E divides its situatio space. If P wats to capture E s itetio correctly, size of situatios i P should be less tha that i E. O the other had, P is usually uable to kow the size of situatios i E. I order to achieve the best estimatio, what P ca do is to divide the situatio space ito as may classes as possible with the limitatio of computig complexity ad the requiremets for real-time decisio-makig. The estimatio of a player s actio preferece i certai situatio is self-adaptive. I each decisio-makig cycle, the estimatio of E s actio preferece is adjusted. As poited i Defiitio, the estimatio of E s actio preferece i certai situatio is defied as the estimatio of the probability of E executig each kid of actio. Accordig to the priciples of probability, the probability of evet u i ca be approximately estimated as follows: pu ( ) x x, (8) i i j j= where x j is the occurrece cout of evet u j, is the total evet umber of the sample space. Thus, E s actio preferece i certai situatio ca be estimated by coutig executed times of each actio i the situatio. Iitially, P may supposes that the probability of E executig each actio is equal, i.e. the iitial executed times of all actios are set to be equal. If x mi is the executed times of E executig actio u i i certai situatio m, the iitially, all x mi are set to be c 0, where c 0 is a o-egative costat. I each decisio-makig cycle, P estimates the parameters of actio that E executed ad selects the virtual actio u i, that is the least differece with the estimated parameters, as the observed actio. The, P adjusts the executed times x mi of the virtual actio u i for the last situatio m as follows: x mi + c x mi, (9) where c is a costat, such as. Fially, recalculate actio preferece of E i the situatio m: p ( u ) x x. = (0) m i mi mj j= 4. SIMULATION A simulatio is carried out based o the practice of the autoomous soccer robots game. I order to illustrate the policy more explicitly, it uses Oe VS Oe iterceptig sceario: aget P is resposible for iterceptig the football dribbled or shot by E. For coveiece, E is used to deote the robot dribblig the football or the shot football: if the football is shot, E is the football; otherwise, E is the robot dribblig the football. The victory criterio of E is that the ball is set ito the goal. The victory criterio

8 404 ZHEN-LUO, QI-XIN CAO AND YAN-ZHENG ZHAO (a) The first time simulatio result. (b) The fifth time simulatio result. Fig. 3. Simulatio results. of P is that oe of followig evets happes: (a) P eter ito a zoe with size 0.5m 0.6m i frot of E; (b) The football is out of the field. I the simulatio, the field size is 0m 5m. The goal (safety-zoe of E) is located at (0m,.5m-3.5m). P s liear velocity is.5m/s ad maximum agular velocity is 2 rad/s. E s liear velocity is.5m/s ad maximum agular velocity is rad/s. The iitial velocity of football shot by robot is 5m/s, ad its acceleratio is 2m/s 2. The system istructio cycles of P ad E are both 0.s. The horizotal orietatio to goal side is 0rad as show i Fig. 3. E s policy is as follows: if the shootig coditio is satisfied, the it shoots the football; otherwise, it tries to dribble the football to goal; if P appears at certai positio i frot of it, it turs aroud to avoid P; E does ot predict. I the simulatio of P usig the proposed self-adaptive predictive pursuig policy, it divides the adversarial situatio ito 08 categories: () Accordig to the X coordiatio of E, it is divided ito 4 types: x 5m, 5m < x 6.5m, 6.5m < x 8m ad x > 8m; (2) Accordig to the Y coordiatio of E, it is divided ito 3 types: y <.5m,.5m y < 3.5m ad y 3.5m; (3) Accordig to the relative positio betwee P ad E, it is divided ito 9 types: (a) The distace betwee P ad E is greater tha 4m; (b) The distace betwee P ad E is greater tha 2m ad ot greater tha 4m. Accordig to the agle α betwee the orietatio of E ad the lie betwee P ad E, it is divided ito 4 types furthermore: π/6 α < π/2, π/2 α 3π/2, 3π/2 α π/6 ad otherwise; (c) The distace betwee P ad E is ot greater tha 2m. Accordig to the agle α betwee the orietatio of E ad the lie betwee P ad E, it is divided ito 4 types furthermore: π/6 α <π/2, π/2 α 3π/2, 3π/2 α π/6 ad otherwise. I the simulatio, P just uses 2 features to evaluate sigle payoff. The sigle payoff fuctio of P executig actio w i agaist E executig certai actio u j is as follows: d/0 d2/0 J( s, s, w, u, t) = ψ ( s, s ) = 0.5e + 0.5e () i j it jt

9 SELF-ADAPTIVE SHORT-TERM PREDICTIVE PURSUING POLICY 405 where d is the distace from P to the iterceptig poit; d 2 is the distace from P to possible shootig lie of E. Note: The sigle payoff fuctio of P executig actio w i has ot bee optimized. The uit of distace i above equatios is m. I the simulatio, the iitial coordiatio of P is (7.5m, 2.5m, 3.42 rad). The time of each predictio step is s. P does oe-step predictio. P has actios ad assumes that E has actios. The iitial positio of E is (5m, 2.5m) ad its start agle varies. For the proposed self-adaptive predictive pursuig policy, it carries o 50 times simulatio i each iitial coditio (icludig players iitial positios, orietatios): () Iitially, P assumes that the actio preferece of E is equal ad adjusts the estimatio of E s actio preferece durig simulatio; (2) With the same iitial coditio as the first time simulatio ad P s adjusted estimatio of E s actio preferece as the result of last time simulatio, the secod time simulatio is carried out ad P adjusts the E s actio preferece estimatio furthermore; (3) I the same way, fifty times of simulatios are carried out. As a cotrast, i the same iitial coditios, with the assumptio that E s actio preferece is equal; a simulatio of short-term predictive pursuig policy is carried out. The simulatio result is show i Table. Table. Result of simulatio. E s start Equal actio preferece Self-adaptive pursuig policy agle assumptio 0 Wi all Wi 0.2 Wi all Wi 0.4 Wi all Wi 0.6 Wi the first 2 times Wi 0.8 Wi the times of st, 2 d ad 4 th Wi 0.9 Wi the first 6 times, ad the times of 8 th, 9 th ad 0 th Wi.0 Wi all Lose.05 Wi all Lose.5 Wi the first 8 times Lose.25 Wi the times of from 2 d to 7 th ad 5 th to 50 th Lose Result of Simulatio: () Usig the self-adaptive predictive pursuig policy, it is possible that P improve the effect of pursuig. A simulatio result with E s origial positio (m, 3m) ad start agle 0 rad is show i Fig. 3. (2) The proposed policy is effective as illustrated i Table, where Wi or Lose is about P. Although i some cases, the effectiveess of the proposed self-adaptive policy is t better tha the method with equal actio preferece assumptio. There may exist several reasos. For example, the situatio space divisio maer of P does ot match with which of E.

10 406 ZHEN-LUO, QI-XIN CAO AND YAN-ZHENG ZHAO I Fig. 3, + represets the pursuer, ο represets the evader. 5. CONCLUSION A self-adaptive predictive policy for pursuit-evasio games is preseted, which takes actio preferece of its oppoet ito accout. The estimatio of a oppoet s actio preferece i some situatio would be adjusted accordig to the observatio of the oppoet. Thus, the predictive policy is self-adaptive. I essece, the adjustmet procedure of the estimatio of a oppoet s actio preferece is a procedure of modelig the oppoet s decisio-makig. Aget usually has differet actio prefereces i differet situatios. To model a oppoet more precisely, the player should divide the situatio space ito may categories. The policy ca be used i real-time adversarial games. I such games, it may be ukow whether players are irratioal or ot. Based o the model of RoboCup middle size soccer robots, a simulatio has bee carried out. The proposed policy has bee illustrated to be effective. I future, we would try to do further research ad apply it i real soccer robots. It requires precise world modelig, icludig localizatio, motio estimatio, etc. REFERENCES. S. M. Lavalle ad J. E. Hirichse, Visibility-based pursuit-evasio: the class of curved eviromets, IEEE Trasactios o Robotics ad Automatio, Vol. 7, 200, pp R. Vidal, O. Shakeria, H. J. Kim, et al. Probabilistic pursuit-evasio games: Theory, implemetatio, ad experimetal evaluatio, IEEE Trasactios o Robotics ad Automatio, Vol. 8, 2002, pp J. Shiar, Y. Lipma, ad M. Zarkh, Mixed strategies i missile versus missile iterceptio scearios, i Proceedigs of the America Cotrol Coferece, 996, pp V. Turetsky ad J. Shiar, Missile guidace laws based o pursuit-evasio game formulatios, Automatica, Vol. 39, 2003, pp H. Kitao, M. Asada, I. Noda, ad H. Matsubara, RoboCup: robot world cup, IEEE Robotics & Automatio Magazie, 998, pp J. P. Hespaha, M. Pradii, ad S. Sastry, Probabilistic pursuit-evasio games: a oe-step ash approach, i Proceedigs of the 39th IEEE coferece o Decisio ad Cotrol, 2000, pp P. C. Zhou ad B. R. Hog, Group robot pursuit-evasio problem based o game theory, Joural of Harbi Istitute of Techology, Vol. 35, 2003, pp H. Fujii, M. Kawashima, E. Sugiyama, et al., EIGEN Keio Uiversity Team Descriptio, i RoboCup 2006, Breme, Germay. 9. B. Dirkx, Philips RoboCup Team Descriptio, i RoboCup 2006, Breme, Germay. 0. Z. Luo ad Q. X. Cao, Pursuig method based o short-term predictio, Joural of Shaghai Jiaotog Uiversity, Vol. 40, 2006, pp , P. Stoe ad M. Veloso, Multiaget systems: a survey from a machie learig perspective, Autoomous Robots, Vol. 8, 2000, pp

11 SELF-ADAPTIVE SHORT-TERM PREDICTIVE PURSUING POLICY R. A. Brooks, A robust layered cotrol system for a mobile robot, IEEE Joural of Robotics ad Automatio, Vol. RA-2, 986, pp B. L. Zhog, Q. Zhag, ad Y. M. Yag, Real time reactive strategies based o potetial fields for robot soccer, i Proceedigs of the IEEE Iteratioal Coferece o Robotics, Itelliget Systems ad Sigal Processig, 2003, pp D. Camacho, F. Ferádez, ad M. A. Rodelgo, Roboskeleto: a architecture for coordiatig robot soccer agets, Egieerig Applicatios of Artificial Itelligece, Vol. 9, 2006, pp M. L. Misky, Theory of eural aalog reiforcemet systems ad its applicatio to the brai model problem, Ph.D. Thesis, Priceto Uiversity, New Jersey, X. C. Wag, R. B. Zhag, ad G. C. Gu, Research o multi-aget team formatio based o reiforcemet learig, Computer Egieerig, Vol. 28, 2002, pp Zhe Luo ( 罗 ) received his B.S. degree from Northwester Poly-techology Uiversity, Chia, i 996, received his M.S. degree i Cyberetics ad Automatio Egieerig from Shaghai Uiversity i 999. He is pursuig his Ph.D. degree i Shaghai Jiaotog Uiversity. His research iterest is o autoomous robot. Qi-Xi Cao ( ) received the M.S. degree from Uiversity of Miyazaki, Japa, i 994, the Ph.D. degree from Uiversity of Kagoshima, Japa, i 997. He is a professor of Research Istitute of Robotics, Shaghai Jiaotog Uiversity, Chia. His research iterests iclude autoomous robot, machie visio, eural etwork, ad patter recogitio. Ya-Zheg Zhao ( 赵 ) received his M.S. ad Ph.D. degrees i Mechaical Egieerig from Harbi Istitute of Techology, Harbi, Chia, i 988 ad 999. He is a professor of Research Istitute of Robotics, Shaghai Jiaotog Uiversity, Chia. His research iterests iclude special type robots ad itelliget cotrol, bioics robot, etc.