Belief-Based Packet Forwarding in Self-Organized Mobile Ad Hoc Networks with Noise and Imperfect Observation

Transcription

1 Belef-Based Packet Forwardng n Self-Organzed Moble Ad Hoc Networks wth Nose and Imperfect Observaton Zhu (James) J, We Yu, and K. J. Ray Lu Electrcal and Computer Engneerng Department and Insttute for Systems Research Unversty of Maryland, College Park, MD emal: zhuj, weyu, kjrlu@umd.edu Abstract In self-organzed moble ad hoc networks (MANET) where each user s ts own authorty, fully cooperatve behavors, such as uncondtonally forwardng packets for each other, cannot be drectly assumed. In ths paper, we focus on cooperaton enforcement n the self-organzed moble ad hoc networks wth nose and mperfect observaton and study the basc packet-forwardng functon usng the repeated game models wth mperfect nformaton. A belef-based packet forwardng framework s proposed to obtan cooperaton-enforcement strateges solely based on each node s own past actons and ts prvate mperfect observaton of other nodes nformaton. The smulaton results llustrate that the proposed belef-based packet forwardng approach can enforce the cooperaton wth only a small performance degradaton compared to the uncondtonally cooperatve outcomes n the ad hoc networks wth nose and mperfect observaton. I. INTRODUCTION Moble ad hoc networks (MANET) have drawn extensve attenton n recent years due to the ncreasng demands of ts potental applcatons [1], [2]. In tradtonal emergency or mltary stuatons, the nodes n a MANET usually belong to the same authorty and act cooperatvely for the common goals. Recently, emergng applcatons of MANETs are also envsoned n cvlan usage [3] [5], where nodes typcally do not belong to a sngle authorty and may not pursue a common goal. We refer to such networks as self-organzed (self-organzed) MANETs. Before MANETs can be successfully deployed n a selforganzed way, the ssue of cooperaton stmulaton must be resolved frst. In the lterature, two types of schemes have been proposed to stmulate cooperaton among selfsh nodes: reputaton-based schemes and payment-based schemes. In reputaton schemes, such as [3], [4], [6], a node determnes whether t should forward packets for other nodes or request other nodes to forward packets for t based on ther past behavors. In the payment-based schemes, such as [5], [7], a selfsh node wll forward packets for other nodes only f t can get some payment from those requesters as compensaton. Recently, some efforts have been made towards mathematcally analyzng the cooperaton n self-organzed ad hoc networks usng game theory, such as [8], [9]. In these exstng game theoretc approaches, all of them have assumed perfect observaton and most of them have not consdered the effect of nose on the strategy desgn. However, n self-organzed ad hoc networks, even when a node has decded to forward a packet for another node, ths packet may stll be dropped due to lnk breakage or transmsson errors. Further, snce central montorng s n general not avalable n self-organzed ad hoc networks, perfect publc observaton s ether mpossble or too expensve to be employed. Each node makes ts decsons only based on ts own past actons and mperfectly observed prvate nformaton of ts opponents. In ths paper we study the cooperaton enforcement for self-organzed moble ad hoc networks wth both nose and mperfect observaton and focus on the most basc networkng functonng, namely packet forwardng, n ad hoc networks. A belef-based packet forwardng approach s proposed to stmulate the packet-forwardng among nodes and maxmze the expected payoff of each selfsh node. Specfcally, the repeated game model s appled to analyze the nteractons among nodes. A formal belef system based on Bayes rule s developed to assgn and update belefs of other nodes contnuaton strateges for each node based on ts prvate mperfect nformaton. Further, we not only show that the packet forwardng strategy obtaned from the proposed belefbased framework acheves a sequental equlbrum that guarantees the strategy to be cheat-proof but also develop ts performance bounds. The smulaton results llustrate that the proposed belef-based packet forwardng approach can enforce the cooperaton n the ad hoc networks wth nose and mperfect observaton wth only a small performance degradaton compared to the uncondtonally cooperatve outcomes. The rest of ths paper s organzed as follows. The system model of self-organzed ad hoc networks wth nose and mperfect observaton s presented n Secton II. In Secton III, we propose the belef-based packet forwardng framework and carry out the equlbrum and effcency analyss. In Secton IV, the belef-based mult-hop mult-node packet forwardng approach s developed. The smulaton studes are provded n Secton V. Fnally, Secton VI concludes ths paper. II. SYSTEM MODEL We consder self-organzed ad hoc networks where nodes belong to dfferent authortes and have dfferent goals. Assume all nodes are selfsh and ratonal, that s, ther objectve are to maxmze ther own payoff, not to cause damage to other nodes. Each node may act as a servce provder: packets are scheduled to be generated and delvered to certan destnatons; or act as a relay: forward packets for other nodes. The sender wll get some payoffs f the packets are successfully delvered to the destnaton and the forwardng effort of relay nodes wll also ntroduce certan costs /06/$20.00 (c)2006 IEEE Ths full text paper was peer revewed at the drecton of IEEE Communcatons Socety subject matter experts for publcaton n the WCNC 2006 proceedngs.

2 Fg. 1: Two-player packet forwardng game n strategc form. In ths paper we assume that some necessary traffc montorng mechansms, such as those descrbed n [3], [5], [6], [10], wll be launched by each node to keep trackng of ts neghbors actons. However, t s worth mentonng that we do not assume any publc or perfect observaton, where a publc observaton means that when an acton happens, a group of nodes n the network wll have the same observaton, and perfect observaton means all actons can be perfectly observed wthout any mstake. In ad hoc networks, due to ts mult-hop nature and the lack of central montorng mechansm, publc observaton s usually not possble. Meanwhle, to our best knowledge, these exst no such montorng mechansms n ad hoc networks whch can acheve perfect observaton. Instead, n ths paper, we study the cooperaton-enforcement strateges based on mperfect prvate observaton. Here, prvate means that the observaton of each node s only known to tself and won t or cannot be revealed to others. We focus on two scenaros causng mperfect observaton n ad hoc networks. One scenaro s that the outcome of the forwardng acton may be packet droppng due to lnk breakage or transmsson errors. The other scenaro s that a node has dropped a packet but was observed as forwardng the packet, whch may happen when the watchdog mechansm [3] s used and the node wants to cheat ts prevous node on the route. III. BELIEF-BASED PACKET FORWARDING FRAMEWORK Two-player packet forwardng game s studed n ths secton n attempt to shed lght on the solutons to the more complcated mult-player case. A. Statc and Repeated Packet-Forwardng Game Model We model the process of routng and packet-forwardng between a source node and a relay node as a game. The players of the game are the network nodes. There are two players n ths game, denoted by I = {1, 2}. Each player s able to serve as the relay for the other player and needs the other player to forward packets for hm based on current routng selecton and topology. Each player chooses hs acton,.e., strategy, a from the acton set A = {F, D}, where F and D are packet forwardng and droppng actons, respectvely. Also, each player observes a prvate sgnal ω of the opponent s acton from the set Ω={f,d}, where f and d are the cooperaton and non-cooperaton observatons, respectvely. Snce the player s observaton can not be perfect, the forwardng acton F of one player may be observed as d by the other player due to lnk breakage or transmsson error. We let such probablty be p f. Also, the noncooperaton acton D may be observed as the cooperaton sgnal f under certan crcumstances. Wthout loss of generalty, let the observaton error probablty be p e n our system, whch s usually caused by cheatng behavors and the packet s actually dropped though forwardng sgnal f s observed. For each node, the cost of forwardng a group of packets for the other node durng one play s l, and the gan t can get for the packets that the other node has forwarded for t s g. We frst consder the packet forwardng as a statc game, whch s only played once. Gven any acton profle a = (a 1,a 2 ), we refer to u(a) =(u 1 (a),u 2 (a)) as the expected payoff profle. Let a and Prob(ω a ) be the acton of the th player s opponent and the probablty of havng observaton ω gven a, respectvely. Then, u (a) can be obtaned as follows. u (a) = ω Ω ũ (a,ω,a ) Prob(ω a ), (1) where ũ s the th player s payoff dependng on the acton profle and hs own observaton. Then, calculatng u(a) for dfferent strategy pars, we have the strategc form of the statc packet forwardng game as a matrx n Fgure 1, where g = (1 p f ) g. To analyze the outcome of a statc game, the Nash Equlbrum [11] s a well-known concept, whch s the strategy, one for each other, such that no player has ncentve to unlaterally change hs acton. The only Nash equlbrum of our two-player packet-forwardng game s the acton profle a = (D, D). But, the better cooperaton payoff outcome (g l, g l) of the cooperaton acton profle {F, F} wll not be practcally realzed n the statc packet-forwardng game due to the greedness of the players. However, generally speakng, the above packet forwardng game wll be played many tmes n real ad hoc networks. It s natural to extend the above statc game model to a repeated game model. Bascally, n the repeated games, the players face the same statc game n every perod, and the player s overall payoff s a weghted average of the payoffs n each stage over tme. Let ω t be the prvately observed sgnal of the th player n perod t. Suppose that the game begns n perod 0 wth the null hstory h 0. In ths game, a prvate hstory for player at perod t, denoted by h t, s a sequence of player s past actons and sgnals,.e., h t = {aτ,ωτ }t 1 τ=1. Denote the nfnte packetforwardng repeated game wth mperfect prvate hstores by G(p, δ), where p =(p f,p e ), δ (0, 1) s the dscount factor. Assume that p f < 1/2 and p e < 1/2. Then, the overall dscounted payoff for player I s defned as follows [11]. U (δ) =(1 δ) δ t u t (a t 1(h t 1),a t 2(h t 2)). (2) t=0 Folk Theorems for nfnte repeated games [11] assert that there exsts a ˆδ <1 such that any feasble and ndvdually ratonal payoff can be enforced by an equlbrum for all δ (ˆδ, 1) based on the publc nformaton shared by players. However, one crucal assumpton for the Folk Theorems s that 344 Ths full text paper was peer revewed at the drecton of IEEE Communcatons Socety subject matter experts for publcaton n the WCNC 2006 proceedngs.

3 players share common nformaton about each other s actons. In contrast, the nature of our repeated packet forwardng game for self-organzed ad hoc networks determnes that the nodes behavoral strateges can only rely on the prvate nformaton hstores ncludng ther own past actons and mperfectly observed sgnals. A seemngly mnor game-settng change from the publc observaton to the prvate observaton due to nose and mperfect observaton wll make a substantal dfference n analyzng the effcency of the packet-forwardng game. In the stuatons of mperfect prvate observaton, each node must conduct statstcal nference to detect potental devatons and estmate what others are gong to do next, whch can become extremely complex due to the mperfect observaton [12], [13]. B. Belef-Based Packet Forwardng Approach In order to have an effcent and robust forwardng strategy based on each node s own observaton and actons, we propose a belef-based packet forwardng approach enlghtened by [13]. Frst, we defne two strateges,.e., σ F and σ D.Letσ F be the trgger cooperaton strategy, whch means that the player forwards packets at current stage, and at the next stage the player wll contnue to forward packets only f t observes the other player s forwardng sgnal f. Letσ D be the defecton strategy, whch means that the player always drops packets regardless of ts observaton hstory. Such strateges are also called contnuaton strateges [13]. Snce both of the two strateges also determne the player s followng actons at every prvate hstory, the strategy path and expected future payoffs caused by any par of the two strateges are fully specfed. Let V α,β (p, δ), α, β {F, D} denote the repeated game payoff of σ α aganst σ β, whch can be llustrated by the followng Bellman equatons [14] for dfferent pars of contnuaton strateges. V FF =(1 δ)(g l)+δ((1 p f ) 2 V FF + p f (1 p f )V FD + p f (1 p f )V DF + p 2 f V DD ), (3) V FD = (1 δ)l + δ((1 p f )(1 p e )V DD + p f (1 p e )V DD + p e (1 p f )V FD + p f p e V FD ), (4) V DF =(1 δ)g + δ((1 p f )(1 p e )V DD + p e (1 p f )V DF + p f (1 p e )V DD + p e p f V DF ), (5) V DD =(1 δ) 0+δ((1 p e ) 2 V DD + p e (1 p e )V DD + p e (1 p e )V DD + p 2 e V DD ). (6) Note that the frst term n the above equatons represents the normalzed payoffs of current perod, whle the second term llustrates the expected contnuaton payoffs consderng four possble outcomes due to the mperfect observaton. By solvng the above equatons, V α,β (p, δ) can be easly obtaned. Then, we have V DD > V FD, for any δ, p. Furthermore, f δ>δ 0, then V FF >V DF, where δ 0 can be calculated as δ 0 = l (1 p f p e )g [p f (1 p f ) p e ]l, (7) TABLE I: Belef-based Two-player Packet Forwardng Algorthm 1. Intalze usng system parameter confguraton (δ, p e,p f ): Node ntalzes hs belef µ 1 of the other node as π(δ, p) and chooses the forwardng acton n perod 1 wth probablty π(δ, p). 2. Belef update based on the prvate hstory: Update each node s belef µ t 1 nto µ t usng (10-13) accordng to dfferent realzatons of prvate hstory. 3. Optmal Decson of the player s next move: If the contnuaton belef µ t >π, node plays σ F ; If the contnuaton belef µ t <π, node plays σ D; If the contnuaton belef µ t = π, node plays ether σ F or σ D. 4. Iteraton: Let t = t +1, then go back to Step 2. Note that ths constrant of δ wll prevent trval defecton outcomes. Suppose that player beleves that hs opponent s playng ether σ F or σ D, and s playng σ F wth probablty µ. Then the dfference between hs payoff of playng σ F and the payoff of playng σ D s gven by V (µ; δ, p) =µ (V FF V DF ) (1 µ) (V DD V FD ). (8) Hence V (µ) s ncreasng and lnear n µ and there s a unque value π(p, δ) to make t zero, whch can be obtaned as follows. π(δ, p) = V FD (δ, p) V FF (δ, p) V DF (δ, p) V FD (δ, p), (9) where π(p, δ) s defned so that player has no preference n choosng σ F or σ D when player j plays σ F wth probablty π(δ, p) and σ D wth probablty 1 π(δ, p). For smplcty, π(δ, p) may be denoted as π n the followng parts under the crcumstances wth no confuson. Intutvely, f node holds the belef that the other node wll help hm to forward the packets wth a probablty smaller than 1/2, node s nclned not to forward packets for the other node. Consderng such stuaton, we let δ be such that π(δ, p) > 1/2. It s worth mentonng that equaton (8) s applcable to any perod. Thus, f a node s belef of an opponent s contnuaton strategy beng σ F s µ, n order to maxmze ts expected contnuaton payoff, the node prefers σ F to σ D f µ>πand prefers σ D to σ F f µ<π. Gven any ntal belef µ, the th player s new belef when he takes acton a and receves sgnal ω can be defned usng the Bayes rule [11] as follows., (F, f)) =, (F, d)) =, (D, f)) =, (D, d)) = )(1 p f ) 2 )(1 p f )+p e (1 )(1 p f ) p f ) p f +(1 p e) (1 )), (10) )), (11) )(1 p f ) p e ) (1 p f )+p e (1 )p f p e ) p f +(1 p e) (1 )), (12) )). (13) Based on the above dscusson, we propose a two-player belef-based packet forwardng algorthm shown n Table I. 345 Ths full text paper was peer revewed at the drecton of IEEE Communcatons Socety subject matter experts for publcaton n the WCNC 2006 proceedngs.

4 C. Effcency Analyss of the Belef-Based Packet Forwardng Framework In ths part, we show that the behavoral strategy obtaned from the proposed algorthm wth well-defned belef systems s a sequental equlbrum and further analyze ts performance bounds. Frst, we brefly ntroduce the equlbrum concepts of the repeated games wth mperfect nformaton. As for the nfntely repeated game wth perfect nformaton, the Nash Equlbrum concept s a useful concept for analyzng the game outcomes. But, snce the threats n Nash equlbra may not be credble and become empty threats, the subgame perfect equlbrum [11] s defned to elmnate those equlbra n whch the players threats are ncredble. However, the above equlbrum crtera for the nfntely repeated game requre that perfect nformaton can be obtaned for each player. In our packet forwardng game, each node s only able to have hs own strategy hstory and form the belefs of other nodes future actons through mperfect observaton. Sequental Equlbrum [11] s a well-defned counterpart of subgame perfect equlbrum under such crcumstance, whch guarantees that any devatons wll be unproftable. In our packet-forwardng game wth prvate hstory and observaton, the proposed strategy wth belef-system can be denoted as (σ,µ), where µ = {µ } I and σ = {σ } I. By studyng (10), we fnd that there exsts a pont φ such that, (F, f)) < ) as ) > φ whle, (F, f)) > ) as ) < φ. Here, φ can be calculated as φ = [(1 p f ) 2 p e ]/(1 p f p e ).Its easy to show that, (a,ω )) < ) when (F, d), (D, f) and (D, d) are reached. Snce we ntalze the belef wth π, wehaveµ t φ after any belef-updatng operaton f π<φ. Consderng the belef updatng n the scenaro that π φ becomes trval, we assume π<φ, thus µ t [0,φ] and φ>1/2. Then, let the proposed packet-forwardng strategy profle σ be defned as: σ (µ ) = σ F f µ > π and σ (µ )=σ D f µ <π;ifµ = π, the node forwards packets wth probablty π and drops them wth probablty 1 π. Smlar to [13], we have the followng two theorems. Theorem 1: The proposed strategy profle σ wth the belef system µ from Table I s a sequental equlbrum for π (1/2,φ). Theorem 2: Gven g and l, there exst δ (0, 1) and p for any small postve τ such that the average payoff of the proposed strategy σ n the packet-forwardng repeated game G(p, δ) s greater than g l τ f δ> δ and p e,p f < p. Theorem 1 shows that the strategy profle σ and the belef system µ obtaned from the proposed algorthm s a sequental equlbrum, whch not only responds optmally at every hstory but also has consstency on zero-probablty hstores. Thus, the cooperaton can be enforced usng our proposed algorthm snce the devaton wll not ncrease the players payoffs. Then, Theorem 2 addresses the effcency of the equlbrum and shows that when the p e and p f are small enough, our proposed strategy approaches the cooperatve payoff g l. However, n real ad hoc networks, a more useful and mportant measurement s the performance bounds of the proposed strategy gven some fxed p e and p f values. We further develop the followng theorem studyng the lower bound and upper bound of our strategy to provde a performance gudelne. In order to model the prevalent data applcaton n current ad hoc networks, we assume that the game dscount factor s very close to 1. Theorem 3: Gven the fxed (p e,p f ) and dscount factor of the repeated game δ G close to 1, the payoff of the proposed algorthm n Table I s upper bounded by where Ū =(1 κ) (g l), (14) κ = p f [g(1 p f )+l] (1 p f p e )(g l). (15) The lower bound of the performance wll approach the upper bound when the dscount factor of the repeated game δ G approaches 1 f the packet forwardng game s dvded nto N sub-games as follows: the frst sub-game s played n perod 1,N +1, 2N +1,... and the second sub-game s played n perod 2,N +2, 2N +2,..., and so on. The optmal N s N = log δ/ log δ G, (16) where δ = l/{[(1 p f ) 2 p e ] g + l p e }. The proposed strategy s played n each sub-game wth equvalent dscount factor δ N G. In the above theorem, by ntroducng the dea of dvdng the orgnal game nto several sub-games, even f the outcomes of some sub-games become the non-cooperaton case due to the observaton errors, cooperaton plays can stll contnue n other sub-games to ncrease the total payoff. Note that due to the lmted space, the proofs of theorems are omtted here, whch can be found n [15]. IV. BELIEF-BASED MULTI-NODE MULTI-HOP PACKET FORWARDING A. Mult-Node Mult-Hop Game Model In ths secton, we consder self-organzed ad hoc networks where nodes can move freely nsde a certan area. For each node, packets are scheduled to be generated and sent to certan destnatons. Dfferent from the two-player packet forwardng game, the mult-player packet forwardng game studes multhop packet forwardng whch nvolves the nteractons and belefs of all the nodes on the route. Before studyng the belefbased strategy n ths scenaro, we frst model the mult-player packet forwardng game as follows: There are M players n the game, whch represent M nodes n the network. Denote the player set as I M = {1, 2,..., M}. For each player I M, he has groups of packets to be delvered to certan destnatons at dfferent tme. For each player I M, forwardng a packet for another player wll ncur some cost l. 346 Ths full text paper was peer revewed at the drecton of IEEE Communcatons Socety subject matter experts for publcaton n the WCNC 2006 proceedngs.

5 p e =0.01 p e =0.1 p e =0.2 Average Node Payoff Payoff rato Fully cooperatve strategy The payoff lower bound of our strategy The average payoff of our strategy The payoff upper bound of our strategy P f Fg. 2: The average payoffs of the cooperatve strategy and proposed strategy. Due to the mult-hop nature of ad hoc networks, the destnaton player j may not le n the sendng player s drect transmsson range. Player needs to not only fnd the possble routes leadng to the destnaton (.e., route dscovery), but also choose an optmal route from the routng canddates (.e., route selecton). Each player only knows hs own past acton and mperfect observaton of other players acton. Note that the nformaton hstory consstng of the above two parts s prvate to each player. Smlar to [8], we assume the network operates n dscrete tme. In each tme slot, one node s randomly selected from the M nodes as the sender. The probablty that the sender fnds r possble routes s gven by q r (r) and the probablty that each route needs h hops s gven by q h ( h) (assume at lease one hop s requred n each tme slot). Note that the h relays on each route are selected from the rest of nodes wth equal probablty and h g/l. Assume each routng sesson lasts for one slot and the routes reman unchanged wthn each tme slot. In our study, we consder that delcate traffc montorng mechansms such as recept-submsson approaches [5] are n place, hence the sender s able to have the observaton of each node on the forwardng route. B. Belef-Based Strategy for Mult-hop Packet Forwardng In ths part, we develop effcent belef-based strateges for mult-hop packet forwardng games based on the proposed two-player approach. Let ω j, µ j and h j denote the sendng player s observaton, belef value and the prvate hstory on the forwardng player j, respectvely. The proposed forwardng strategy for the mult-player case s llustrated as follows. Belef-Based Mult-hop Packet Forwardng (BMPF) Strategy: In the mult-node mult-hop packet forwardng game, gven the dscount factor δ G and p =(p e,p f ), the sender and relay nodes act as follows durng dfferent phases of routng process. Game partton and belef ntalzaton: Partton the orgnal game nto N sub-games accordng to (16). Then, each node ntalzes ts belef of other nodes as π(δ N G,p) and forwards packets wth probablty π(δ N G,p) Fg. 3: Payoff ratos of the proposed strategy to the cooperatve strategy. Route partcpaton: The selected relay node on each route partcpates n the routng f and only f ts belef of the sender s greater than π. Route selecton: The sender selects the route wth the largest µ = j R µ j from the route canddates. Packet forwardng: The sender updates ts belef of each relay node s contnuaton strategy usng (10)-(13) and decde the followng actons based on ts belef. Based on Theorem 1, t s straght-forward to show that BMPF Strategy desgned for mult-node mult-hop packet forwardng games s also a sequental equlbrum. Here, n order to calculate the equvalent two-player expected gan g n Table I, we need to consder the routng statstcs such as q r (r) and q h ( h) to deal wth the error propagaton and routng dversty. V. SIMULATION STUDIES In ths secton, we nvestgate the cooperaton enforcement results of our proposed belef-based packet forwardng approach by smulaton. We frst focus our smulaton studes on one-hop packet forwardng scenaros n ad hoc networks, where the twoplayer belef-based packet forwardng approach can be drectly appled to. Let M = 100, g = 1 and l = 0.2 n our smulaton. For comparson, we defne the cooperatve strategy, whch assumes every node wll uncondtonally forwardng packets wth no regard to other nodes past behavors. Such cooperatve strategy s not mplementable n self-organzed ad hoc networks. But t can serve as a loose upper bound of the performance of the proposed strategy and determne the performance loss of cooperaton enforcement due to nose and mperfect observaton. Fgure 2 shows the average payoff and performance bounds of the proposed belef-based strategy for dfferent p f by comparng them wth the cooperatve payoff. Note that p e = 0.01 and δ G =0.99. It can be seen from Fgure 2 that our proposed approach can enforce cooperaton wth only small performance loss compared to the uncondtonally cooperatve payoff. Further, ths fgure shows that the average payoff of our proposed strategy satsfes the theoretcal payoff bounds p f 347 Ths full text paper was peer revewed at the drecton of IEEE Communcatons Socety subject matter experts for publcaton n the WCNC 2006 proceedngs.

6 Average payoff Average payoff our strategy wthout devaton our strategy wth devaton p d =0.1 our strategy wth devaton p d =0.2 Cooperatve payoff 0.1 Cooperatve payoff The proposed strategy wth p e =0.2 The proposed strategy wth p e =0.1 The proposed strategy wth p e = Fg. 4: Payoff comparson of the proposed strategy and devatng strateges. developed n Theorem 3. The fluctuaton of the payoff curve of our strategy s because that only nteger number of sub-games can be parttoned from the orgnal game. Fgure 3 shows the rato of the payoffs of our strategy to those of the cooperatve strategy for dfferent p e and p f.hereweletδ G =0.999 to approach the payoff upper bound. It can be seen from Fgure 3 that even f p f s as large as 0.1 due to lnk breakage or transmsson error, our cooperaton enforcement strategy can stll acheve as hgh as 80% of the cooperatve payoff. In order to show that the proposed strategy s cheat-proof among selfsh users, we defne the devaton strateges for comparson. The devaton strateges dffer from the proposed strategy only when the contnuaton strategy σ F and observaton F are reached. The devaton strateges wll play σ D wth devatng probablty p d nstead of playng σ F as the proposed strategy specfes. Fgure 4 compares the nodes average payoffs of the proposed strategy, cooperatve strategy and devaton strateges wth dfferent devatng probabltes. Note that δ G =0.999 and p e =0.1. Ths fgure shows that the proposed strategy has much better payoffs than the devatng strateges. Then, we study the performance of the proposed mult-hop mult-node packet forwardng approach. Let q r (1) = q r (2) = 1/2, q h (2) = q h (3) = 1/2 and δ G = We compare the payoff of our approach wth that of the cooperatve one n Fgure 5. Note that mult-hop forwardng wll ncur more costs to the nodes snce one successful packet delvery nvolves the packet forwardng efforts of many relay nodes. Also, the nose and mperfect observaton wll have more mpact on the performance as each node s ncorrect observaton wll affect the payoffs of all other nodes on the selected route. We can see from Fgure 5 that our proposed strategy stll mantans hgh payoffs even when the envronment s nosy and the observaton error s large. p f VI. CONCLUSION In ths paper, we study the cooperaton enforcement n self-organzed ad hoc networks wth nose and mperfect observaton. By modelng the basc network functon, packet forwardng, as a repeated game wth mperfect nformaton, Fg. 5: Average payoffs of the proposed strategy and devatng strateges n mult-node mult-hop scenaros. we develop the belef-based packet forwardng framework to enforce cooperaton n the scenaros of nose and mperfect observaton. We show that the behavoral strategy wth welldefned belef system from the proposed approach can not only acheve the sequental equlbrum but also mantan hgh payoffs for both two-player and mult-player cases. The smulaton results llustrate that the proposed belef-based packet forwardng approach acheves stable and near-optmal equlbrum n the ad hoc networks wth mperfect observaton. p f REFERENCES [1] C. Perkns, Ad Hoc Networkng, Addson-Wesley, [2] C.K.Toh, Ad Hoc Moble Wreless Networks: Protocols and Systems, Prentce Hall PTR, [3] S. Mart, T. J. Gul, K. La, and M. Baker, Mtgatng Routng Msbehavor n Moble Ad Hoc Networks, n ACM Mobcom 2000, August 2000, pp [4] P. Mchard and R. Molva, CORE: a COllaboratve REputaton Mechansm to Enforce Node Cooperaton n Moble Ad Hoc Networks, n IFIP - Communcatons and Multmeda Securty Conference, [5] S. Zhong, J. Chen, and Y. R. Yang, SPRITE: A Smple, Cheat- Proof, Credt-Based System for Moble Ad-Hoc Networks, n IEEE INFOCOM, [6] W. Yu and K. J. R. Lu, Attack-Resstant Cooperaton Stmulaton n Autonomous Ad Hoc Networks, to appear n IEEE Journal on Selected Areas n Communcatons, specal ssue n Autonomc Communcaton Systems, Dec [7] L. Buttyan and J. P. Hubaux, Enforcng Servce avalablty n moble Ad-hoc Network, n ACM MobHOC 00, August [8] V. Srnvasan, P. Nuggehall, C. F. Chassern, and R. R. Rao, Cooperaton n Wreless Ad Hoc Networks, n IEEE INFOCOM, [9] M. Felegyhaz, J.-P. Hubaux, and L. Buttyan, Nash Equlbra of Packet Forwardng Strateges n Wreless Ad Hoc Networks, to appear n IEEE Transactons on Moble Computng, [10] W. Yu, Y. Sun, and K. J. R. Lu, HADOF: Defense aganst Routng Dsruptons n Moble Ad Hoc Networks, n Proc. of IEEE INFOCOM 05, [11] M. J. Osborne and A. Rubnsten, A Course n Game Theory, TheMIT Press, Cambrdge, Massachusetts, [12] T. Sekguch, Effcency n repeated prsoner s dlemma wth prvate montorng, Journal of Economc Theory, vol. 76, pp , [13] V. Bhaskar and I. Obara, Belef-based equlbra n the repeated prsoners dlemma wth prvate montorng, Journal of Economc Theory, vol. 102, pp , [14] D. Bertsekas, Dynamc Programmng and Optmal Control, vol. 1,2, Athena Scentfc, Belmont, MA, Second edton, [15] Z. J, W. Yu, and K. J. R. Lu, Cooperaton enforcement n autonomous moble ad hoc networks under nosy and mperfect observatons, submtted to IEEE Transactons on Moble Computng, Ths full text paper was peer revewed at the drecton of IEEE Communcatons Socety subject matter experts for publcaton n the WCNC 2006 proceedngs.