PRELIMINARIES. Introductions. Luiz DaSilva 7/27/2010

Transcription

1 Game Theory for Wreless Nt Networks PRELIMINARIES Luz DaSlva I 2 R Sngapore August 23 26, 2010 Introductons M.S., Ph.D. from The Unversty of Kansas Faculty member n ECE at Vrgna Tech snce 1998 Currently at Trnty College, Dubln My research and teachng focus on wreless networks, cogntve and self organzng networks CTVR s the Telecommuncatons Research Centre n Ireland 1

2 Acknowledgement Ths short course expands on two tutorals that were gven by myself andmy colleague and frequent collaborator, Prof. AllenMacKenze, from Vrgna Tech Much of the materal here was adapted from (or reuses) materal orgnally prepared by Prof. MacKenze and me What s game theory? A set of analytcal tools from economcs and mathematcs to predctthe the outcome of complex nteractonsamongamong ratonal enttes n the context of cogntve rado Models of nteractons among adaptatons performed by cogntve rados n a network Economc models of spectrum markets Desgn of ncentve structures for effcent resource sharng n a cogntve network ( ) 2

3 Cogntve adaptatons as a game Cogntve Rado Cogntve rados n network Avalable waveforms (modulaton, codng, operatng frequency) Objectve functon (e.g., ncreasng, concave functon of SINR) Game Player set Acton set Utlty functon Dstrbuted channel assgnment as a game Cogntve Rado Cogntve rados n network Avalable channels Game Player set Acton set Objectve functon (e.g., derved from network connectvty graph and conflct graph) Utlty functon 3

4 Dynamc spectrum access as a game Cogntve Rado Secondary users n a network Potentally avalable channels Game Player set Acton set Objectve functon (e.g., 0 f any conflcts wth prmary user, ncreasng w/ # of channels used otherwse) Utlty functon Prcng of spectrum as a game Cogntve Rado Secondary users n a network (bandwdth, bd) Game Player set Acton set Objectve functon (e.g., consumer surplus) Utlty functon 4

5 Objectves Provde an ntroducton to game theory, ncludng fundamental concepts such as Nash equlbrum and some common cooperatve and non cooperatve game models Enable you to read and evaluate research work that apples game theory to the analyss and desgn of wreless communcatons and networks Provde a crtcal understandng of some of the assumptons, promse, and lmtatons of game theory when appled to telecommuncatons problems Equp you to apply game theory to model problems of nterest n your research Pre requstes Typcal Engneerng level mathematcal background Functons (real analyss great, but not a requrement) Set theory Probablty and random varables General understandng of telecommuncatons and networks Most examples come from wreless communcatons and networks 5

6 Tentatve Agenda Day 1 Prelmnares Introducton to game theory Day 3 Channel Assgnment and Topology Control Cooperatve games Coaltonal games Day 2 Power control games Repeated games Potental games Day 4 Mechansm desgn Real tme spectrum markets Wrap up Luz DaSlva Trnty College Dubln ldaslva@vt.edu 6

7 7/26/2010 Game Theory for Wreless Nt Networks INTRODUCTION TO GAME THEORY CONCEPTS Luz DaSlva I 2 R Sngapore August 23 26, 2010 Objectves Introduce extensve form and strategc form games Defne Nash equlbrum and fnd the Nash equlbrum for some fnte games State some of the theorems on the exstence of a Nash equlbrum Defne Pareto optmalty and determne whether a Nash equlbrum s Pareto optmal Dscuss lmtatons t t of the Nash equlbrum concept Introduce preference relatons and dscuss how utlty functons can represent these relatons 1

8 7/26/2010 Game theory A bag of analytcal tools desgned to help us understand the phenomena that we observe when decson makersnteract nteract Osborne and Rubnsten The study of mathematcal models of conflct and cooperaton between ntellgent ratonal decson makers Myerson Components of a game A set of 2 or more players A set of actons for each player A set of preference relatonshps for each player for each possble acton tuple usually expressed as a utlty functon 2

9 7/26/2010 Extensve form game representaton (full nformaton) II X Y I (2,3) a c d e b (3,4) (4,5) (1,3) ¼ ¾ (0,20) (3,0) (0,5) (4,2) Z II P I Q Players: I and II Players take turns selectng actons (X, d, P, etc.) Payoffs: (x,y) Players am to maxmze ther (expected) payoffs Comments on game trees The symbol represents a move by nature, whle the symbol represents a decson node where one of the players chooses an acton Sometmes t s possble to elmnate rratonal strateges by a process of backward nducton A player s strategy specfes a move for each of that player s decson nodes A strategy profle s a tuple of strateges, wth one strategy for each player 3

10 7/26/2010 From extensve to strategc form T I B I II L R (1,3) L II R T B (1,3) (1,3) (0,0) (2,2) (0,0) (2,2) extensve form strategc (normal) form Normal form games G = N A { u },, N Set of players A Set of actons avalable to player A = A A L A 1 2 n A Acton space {u } Set of ndvdual payoff (utlty) functons 4

11 7/26/2010 Peer to peer fle sharng example There s a cost to sharng, but f all refuse to share all wll suffer What s the expected outcome of ths stuaton? Ths s a three player varant of the famous Prsoner s s Dlemma Source: V. Srvastava et al., Quck notaton comment... Consder the vector: s = ( s, s, K, s ( 1 2 N e.g., the strategy profle vector contanng the strateges of all players ) Then we wll denote by s = ( s, s2, K, s 1, s+ 1, K, s 1 N the vector contanng all elements of s except the th element ) 5

12 7/26/2010 Domnated strateges Let t and s be strateges for player j Strategy t strctly domnates strategy s f for any fxed strateges for the other players, player j s payoff from adoptng strategy t s hgher than from strategy s Strategy t weakly domnates strategy s f for any fxed strateges for the other players, payer j s payoff from adoptng strategy t s at least as hgh as from strategy s, and at least n one case strctly hgher Domnated strateges, wth mathematcal notaton Strategy s s strctly domnated for player f there exsts t S such that u ( t, s ) u ( s, s > ) s S Note: consstently wth our prevous notaton, we have S S S S S K S = 1 2 K N 6

13 7/26/2010 Iterated deleton of domnated strateges I II L R U M D (3,0.5) (1,1) (2,1.5) (0.5,2) (1.5,3) (0,1.5) Nash equlbrum A pont from whch no user can beneft by unlaterally devatng An acton tuple a s a Nash equlbrum f, for every player n N and every acton b n A, u ( a) u ( b, a ) 7

14 7/26/2010 Example: battle of sexes I II C S C S (1,2) (0,0) (0,0) (2,1) A couple decdng whether to go to a concert (C) or a sports event (S) One of them prefers C, the other prefers S, but they would both lke to go to the same event Note that ths game cannot be solved by terated deleton of domnated strateges Example: rock, paper, scssors I II (0,0) (1, 1) ( 1,1) ( 1,1) (0,0) (1, 1) (1, 1) ( 1,1) (0,0) 8

15 7/26/2010 Mxed strateges A player can randomze over her strategy set Denote by σ a mxed strategy avalable to player And σ (a ) s the probablty that the mxed strategy assgns to acton a The expected utlty to player under a jont mxed strategy s u ( σ ) = ( σ ( a N a A j= 1 j j )) u ( a) Exstence of Nash equlbrum [Nash, 1951] Every fnte game n strategc form has a Nash equlbrumnpure n or mxedstrateges The exstence of the Nash equlbrum can also be determned for some classes of games wth nfnte strategy spaces Proof of exstence usually reles on fxed pont theorems 9

16 7/26/2010 Exstence theorems (1) [Glcksberg, 1952] Consder a strategc form game whose strategy spaces S are non empty compact subsets of a metrc space. If the payoffs u are contnuous there exsts a Nash equlbrum Note that a pure strategy Nash equlbrum s not guaranteed to exst Math remnder: compact sets A S set of real numbers s called compact s every sequence n S contans a sub sequence sequence that converges to an element of S Proposton: A set S R n s compact ff t s closed and bounded 10

17 7/26/2010 Exstence theorems (2) [Debreu, Glcksberg, Fan, 1952] Consder a strategc form game whose strategy spaces S are non empty compact convex subsets of an Eucldean space. If the payoffs u are contnuous n s and quasconcave n s there exsts a Nash equlbrum Math remnder: convex sets A set S s convex f, for any two ponts x and y n S, f I draw a straght lne between x and y all the ponts on the lne wll le n S Note: obvously, ths s not the formal defnton of convexty, but t wll do for our purposes 11

18 7/26/2010 Math remnder: quas concavty A sngle varable functon f(x) s quas concave ff ether f s non ncreasng, f s non decreasng, or there exsts y such that f s non decreasng for x < y and t s non ncreasng for x > y Note: ths s not the formal defnton of quas concavty and ths result does not hold for mult varate functons, but t s enough to provde some ntuton about quas concavty Pareto optmalty A resource allocaton soluton s Pareto optmal f no player can be made happer wthout sacrfcng the welfare of at least one other player A measure of effcency n resource allocaton In mult objectve optmzaton, the Pareto fronter s often sought Nash equlbra are not necessarly Pareto optmal The prsoner s dlemma s a famous example of unque Nash equlbrum that s not Pareto optmal 12

19 7/26/2010 Pareto optmalty examples The peer to peer fle sharng example has a unque Nash equlbrum. Is t Pareto optmal? Are the Nash equlbra for the battle of the sexes Pareto optmal? Is the (mxed strategy) Nash equlbrum for the rock/paper/scssors game Pareto optmal? Predctve power of Nash equlbrum A consstent predcton of the outcome of the game If all players predct the NE, t s reasonable to assume that they wll play t Once reached, there s no reason to beleve any player wll devate, and the system wll reman n equlbrum untl condtons change But not wthout ts ssues If players start from an acton profle that s not an NE, are we sure they eventually reach the NE? (Convergence) What f there are multple NEs? Is one more lkely than the others? (Refnements to the concept of NE) Vulnerable to devatons by a coalton of players 13

20 7/26/2010 Intellgent decson makng Games model cooperaton and competton between ntellgent decson makers Game theory s mult agent decson theory Must model preference relatons for these decson makers, and from those derve utlty functons The utlty functon can be the weakest lnk n the model There s no fxed recpe for how to determne the approprate utlty functon but for partcular applcatons there may be certan propertes that the functon s expected to have Preference relatons Let X be any set, called the set of alternatves or outcomes Let f be a bnary relaton on X % f s sad to be complete f for all x,y X ether x f y or % % y f x % f s sad to be transtve f x f y and y f z mply x f z % % % % The bnary relaton f % transtve s a preference relaton f t s complete and 14

21 7/26/2010 Types of preference relatons Preference relatons express a player s desrablty for one outcome over another f Weak preference relatonshp % x f y means that outcome x s preferred at least as much by player % as outcome y f Strct preference relatonshp x f y ff x f y but not y f x ~ % % Indfference preference relatonshp x ~ y ff x f y and y f x % % Examples of preferences Applcaton layer: users prefer hgh qualty to low qualty vdeo Network layer: nodes prefer robust, relable paths over transent, unrelable paths Data lnk layer: nodes prefer short medum access delay Physcal layer: nodes prefer hgh SINR and low BER Of course, thngs get complcated when there are tradeoffs to be made 15

22 7/26/2010 Utlty representaton We would lke to represent preferences usng a real valued functon A preference relaton f s sad to be represented by a utlty % functon u : X R when x f y u (x) u (y) % Can we always construct utlty representatons? If X s fnte or countably nfnte, we can always construct a utlty representatonfor any preferencerelaton relaton If X s uncontably nfnte, there are some preference relatons for whch a utlty representaton s not possble Example: lexcographc preferences Let X = [0,1] x [0,1] and consder the preference relaton (x 1, x 2 ) f (y 1, y 2 ) f x 1 > y 1 or (x 1 = y 1 and x 2 y 2 ) % 16

23 7/26/2010 Preferences over lotteres Sometmes we need to choose between probablstc outcomes ( lotteres ), (lotteres), rather than guaranteed outcomes Whch of the followng would be preferable? Hgh data rate WLAN connectvty wth 70% probablty, but 30% probablty of no connectvty at all Low data rate 3G cellular connectvty, wth close to 100% probablty Representng uncertanty Let Z denote the set of outcomes Let X denote the set of choce objects, whch are probablty dstrbutons over Z Example: Z = {WLAN, 3G, no_coverage} X = {(p WLAN, p 3G, 1 p WLAN p 3G )} How do we express, wth utlty functons, a preference relaton over X? 17

24 7/26/2010 Expected utlty representaton A bnary relaton f over X s sad to have an expected utlty % representaton f there exsts a functon u : Z R such that p f q E p [u(z)] E q [u(z)] % E p denotes the expected value wth respect to the probablty dstrbuton p The von Neumann Morgenstern axoms are key to the exstence of expected utlty representatons Readng lst D. Fudenberg and J. Trole, Game Theory, MIT, 1991 A. B. MacKenze and L. A. DaSlva, Game Theory for Wreless Engneers, Morgan and Claypool Publshers, 2006 V. Srvastava, J. Neel, A. MacKenze, R. Menon, L.A. DaSlva, J. Hcks, J.H. Reed and R. Glles, Usng Game Theory to Analyze Wreless Ad Hoc Networks, IEEE Communcatons Surveys and Tutorals, vol. 7, no. 4, pp , 4 th quarter

25 7/26/2010 Summary Game theory models competton and cooperaton among ratonal decson makers Nash equlbrum s often used to predct the outcome of a game It always exsts for fnte games Pareto optmalty s a desrable property for the effcency of an outcome Utlty functons express players preferences over a set of outcomes Exercses These smple exercses n modelng smplfed wreless networkng problemsas fnte games am to renforce the concepts of game models, Nasth equlbrum, and Pareto optmalty These examples are adapted from Felegyhaz and Hubaux, Game Theory n Wreless Networks: a Tutoral 19

26 7/26/2010 Exercse I: the forwarder s dlemma (1) In a mult hop wreless network, node s A requres node s B to forward ts packets so that they can reach destnaton d A the stuaton s symmetrc for node s B Whenever one of the nodes forwards a packet for the other, t ncurs a fxed cost 0 < C << 1 Each tme a packet successfully reaches ts destnaton, ts sender gets a beneft of 1 The utlty/payoff s the dfference between beneft and cost Exercse I: the forwarder s dlemma (2) Each of the two nodes s tempted to drop packets t s asked to forward, to conserve resources and reduce cost If both reason ths way, no packets wll get through Both wll be better off f both forward each other s packets But s ths a feasble outcome? d B s A s B d A 20

27 7/26/2010 Exercse I: the forwarder s dlemma (3) Model the forwarder s dlemma as a game Defne the players, acton sets, and payoffs Represent the game n strategc form, usng a table/matrx as we dd n the rock/paper/scssors example Fnd all Nash equlbra and determne whether they are Pareto optmal Exercse II: jont packet forwardng (1) In a mult hop wreless network, two nodes A and B need to decde whether to forward packets between source S and destnaton D Whenever one of the nodes forwards a packet, t ncurs a fxed cost 0 < C << 1 Each tme a packet successfully reaches ts destnaton (.e., both A and B decde to forward), each node n the path s rewarded by a beneft of 1 (e.g., the sender or the recever may pay the reward) The utlty/payoff s the dfference between beneft and cost 21

28 7/26/2010 Exercse II: jont packet forwardng (2) Each of the two nodes s tempted to drop packets t s asked to forward, to conserve resources and reduce cost If both reason ths way, no packets wll get through Both wll be better off f both forward each other s packets But s ths a feasble outcome? S A B D Exercse II: jont packet forwardng (3) Model jont packet forwardng as a game Defne the players, acton sets, and payoffs Represent the game n strategc form, usng a table/matrx as we dd n the rock/paper/scssors example Fnd all Nash equlbra and determne whether they are Pareto optmal 22

29 7/26/2010 Exercse III: multple access game (1) Node s A wshes to transmt to destnaton d A, whle node s B wshes to transmt to destnaton d B All four nodes wthn range of one another, so all nterfere In each tme slot, the sendng nodes need to decde whether to transmt a frame or wat If both transmt smultaneously, a collson occurs s A d A d B s B Exercse III: multple access game (2) Whenever one of the sendng nodes transmts a frame, t ncurs a cost 0 < C << 1 If a frame s successfully transmtted, ts sender gets a beneft of 1 The utlty/payoff s the dfference between beneft and cost Perform the same steps as for the prevous two exercses 23

30 Game Theory for Wreless Nt Networks POWER CONTROL GAMES Luz DaSlva I 2 R Sngapore August 23 26, 2010 Objectves Dscuss desrable propertes of utlty functons, wth examples for game theoretc models of power control Dfferentate between postve and normatve models of utlty Present equlbrum and Pareto effcency results for selected power control games Enumerate approaches to deal wth neffcences n Nash equlbra 1

31 Power control In a cellular or ad hoc network wth multple nodes ndependently makngpower control decsons, power control can be naturally modeled as a game Each node sets ts transmt power p [ 0, ) A node s utlty s a functon of ts selected power, p, and ts SIR, whch we denote by γ Note that the SIR (or SINR) also depends on the power levels selected by other nodes,.e., the vector p Propertes of the utlty functon: lmt cases The utlty functon s a non negatve functon of SIR, and U ( p,0) 0, p > 0 = As the transmt power tends ether to zero or nfnty, the node s utlty tends to zero lm 0 U ( p, γ ) = 0 p lm U ( p, γ ) = 0 p 2

32 Propertes of the utlty functon: monotoncty n SIR The utlty functon s a monotoncally ncreasng functon of the node s SIR, for a fxed transmt power U ( p, γ ) γ > 0, γ, p > 0 (There s an mplct assumpton here that the functon s dfferentable) Propertes of the utlty functon: monotoncty n transmt power The utlty functon s a monotoncally decreasng functon of the node s transmt power, for a fxed SIR U ( p, γ ) p < 0, γ, p > 0 3

33 Propertes of the utlty functon: dmnshng margnal utlty The margnal utlty tends to zero for hgh values of SIR U ( p, γ ) lm γ = 0, p > 0 γ A canddate utlty functon Let E be the energy avalable n node s battery (n Joules), R be the rate at whch nformaton s transmtted, and L be the length of a frame n bts.5 U ( p, ) (1 0 γ ) γ ER = p e L Does ths functon meet the propertes n the prevous sldes? Source: Shah, Mandayam, and D. Goodman,

34 Exstence of Nash equlbrum The strategy spaces are closedandbounded (fwe consder power wthn some range [0, p max ]) The utlty functon s contnuous n p The utlty functon s quasconcave n p Source: Shah, Mandayam, and D. Goodman, Property of the Nash equlbrum If the problem s formulated consderng rados communcatng wth a common base staton (sngle cell), at equlbrum p the power receved at the base staton wll be the same for all players hy p = hjy p j, j h y s the channel gan from rado to the base staton 5

35 Pareto (n)effcency of the Nash equlbrum The equlbrum p for ths game s Pareto neffcent At equlbrum, there s a value α < 1 such that f all users reduce (multply) ther power by that factor, then all users wll obtan hgher utlty Ths arms race or shoutng match result occurs n several smlarly formulated games Dealng wth neffcency: a dfferent utlty functon What f we consder nstead the followng utlty functon? ) U ( p, γ ) = ( γ γ ) 2 where γ ) s the target SINR for rado The resultng game has an equlbrum that s Pareto effcent Smple adaptaton algorthms can be shown to converge to the Nash equlbrum 6

36 Two ways of thnkng about utlty Is t cheatng to change the utlty functon? Postve model of utlty the functon attempts to represent what reasonable players would value n cooperaton and competton Normatve model of utlty the functon represents how we desgn the players to behave In wreless networks applcatons, ether can be used, as long as you are clear on what your utlty functon captures Dealng wth neffcency: prcng Suppose we attach a prce c per unt of transmt power ER 0.5γ L U ( p, γ ) = (1 e ) p cp Users are effectvely charged for the nterference they create on others n the network In ths case the Nash equlbrum s stll not Pareto optmal, but t gets much closer than n the orgnal formulaton 7

37 Dealng wth neffcency: repetton and reputaton Another way of dealng wth neffcent s through repeated games Ths can be used to establsh a self enforcng enforcng mechansm A target operatng pont s selected (e.g., the equal receved power Pareto optmal pont) If any user exceeds the target receved power, the user s punshed by revertng to the one shot NE for several rounds Other technques n ths category may more explctly track reputaton Users wth poor reputaton may be dened servce or offered a lower grade of servce Readng lst D. Famolar, N. Mandayam, D. Goodman, V. Shah, A new framework for power control nwreless data networks: games, utlty, and prcng, Wreless Multmeda Network Technologes, pp , 1999 V. Shah, N. Mandayam, and D. Goodman, Power control for wreless data based on utlty and prcng, Proc. IEEE Internatonal Symposum on Personal, Indoor, and Moble Rado Communcatons, vol. 3, pp ,

38 Summary It s sometmes possble to postulate reasonable propertes for utlty functons (monotoncty, dmnshng margnalutlty, etc.) A formulaton of a dstrbuted power control game may yeld a Nash equlbrum that s not effcent (Pareto optmal) Approaches to deal wth the neffcency nclude defnng a dfferent utlty functon, ncludng prcng consderatons, and formulatng the problem as a repeated game It s crucal to dstngush between normatve and postve models of utlty functon (both are found n wreless network applcatons) 9

39 Game Theory for Wreless Nt Networks REPEATED GAMES Luz DaSlva I 2 R Sngapore August 23 26, 2010 Objectves Defne subgame, and subgame perfecton Introduce repeated games models and notaton Introduce repeated games models and notaton Descrbe dscount factor and dscounted payoffs Defne mn max payoffs and feasble ndvdually ratonal payoffs State the folk theorem for repeated games 1

40 Revstng extensve form games The game s represented as a tree Each vertex represents a decson pont for one of the players Edges from a vertex represent possble actons by a player At the leaves, we specfy payoffs to the player by followng that path from the root Extensve form games can account for dfferent nformaton sets Descrbe how much a player knows when asked to select an acton Extensve and normal forms Any game n extensve form can be represented n normal form And vce versa Extensve form games do not necessarly model sequental actons But t s a convenent form to represent games that nvolve sequental actons 2

41 Peer to peer fle sharng game, revsted Unque Nash equlbrum s not Pareto optmal Ths s a normal form representaton of the game What does the game look lke represented n strategc form? Source: V. Srvastava et al., Peer to peer fle sharng game: strategc form nformaton set 3

42 Another strategc form game example What are the Nash equlbra for ths game? Are all the equlbra equally lkely? A subgame Take a vertex x n an extensve form game Let F(x) represent the set of vertces and edges that follow x, ncludng x A subgame s a subset of the orgnal game such that It s rooted at vertex x, whch s the only vertex of that nformaton set The game contans all vertces n F(x) If a vertex n a partcular nformaton set s contaned n the subgame, then all vertces n that nformaton set are also contaned 4

43 Subgame perfecton A proper subgame of a game Γ s a subgame whose root s not the root of Γ A subgame perfect equlbrum of game Γ s a Nash equlbrum of Γ that s also a Nash equlbrum of every proper subgame of Γ Repeated games Players nteract repeatedly wthn a potentally nfnte tme horzon Used to model deas of reputaton and punshment n games, wth applcatons to wreless networks 5

44 Repeated games: the settng A strategc form game, known as the stage game, s played repeatedly In each stage, all players know the past actons taken by all other players Players strve to maxmze ther expected payoff over multple rounds of the game, usng a dscounted sum of payoffs The dscount rate, 0 δ <11, expresses how much players value the present over the future u = (1 δ) (δ) k g (a k ) k= 0 Notaton The vector of players actons n stage k of the game s denoted by a k Player s payoff n the k th stage of the game s A hstory s a record of all actons played by all players n the past h k = (a 0,a 1,a 2,...,a k ) g ( k a A player s strategy s a mappng from hstores to actons a k = f (h k 1 ) ) 6

45 Repeated games: equlbrum The defnton of Nash Equlbrum stll apples to repeated games A Nash Equlbrum strategy profle s one such that, for each player, her chosen strategy maxmzes her expected payoff, gven the chosen strateges of the other players Often for repeated games, the NE s refned to the subgame perfect equlbrum Ths refnement rules out equlbra whch contan empty threats Peer to peer fle sharng as a repeated game Users nteract repeatedly n sharng fles Grm trgger strategy: user wll select acton share ; however, f at any stage one of the other users selects not share, then user wll retalate by playng not share for ever more All players playng the grm strategy s a Nash equlbrum And, unlke n the sngle stage verson of the game, Pareto optmalty s achevable But there are nfntely many Nash equlbra 7

46 Folk theorems A folk theorem consders a subclass of games and dentfes a set of payoffs that are feasble under some equlbrum strategy profle There are many subclasses of games, so many folk theorems Feasble payoff vector The convex hull of a set U s the smallest convex set that contans U The stage game payoff vector v=(v 1, v 2,, v N ) s feasble f t s an element of the convex hull of the pure strategy payoffs for the game 8

47 Mn max payoff The mn max (or reservaton) payoff establshes the best payoff that each player can guarantee for herself, regardless of others actons The mn max payoff for player s v = mn ( A ) max ( A ) g (, α α Δ α Δ α ) Feasble ndvdually ratonal payoffs In any Nash equlbrum of the repeated game, player s payoff s at least her reservaton payoff The set of feasble strctly ndvdually ratonal payoffs s { v V v > v N} 9

48 Folk theorem In a repeated game, any combnaton of payoffs such that each player gets at least her mn max max payoff s sustanable, provded that each player beleves the game wll be repeated wth hgh probablty Thm: For every feasble strctly ndvdually ratonal payoff vector v, there exsts δ <1 such that for all δ (δ,1) there s a Nash equlbrum of the game wth payoffs v Summary A subgame perfect equlbrum s a Nash equlbrum for the game that s also an equlbrum for every proper subgame In repeated game models, a player s strategy can take nto account players past hstory of actons These models can ncorporate deas of reputaton and punshment Provded that each player gets at least her mn max payoff, the folk theorem states that any combnaton of payoffs s sustanable f the players attach a hgh enough probablty that the game wll be repeated further 10

49 Exercse I: multple access wth retransmssons (1) Let us revst the multple access exercse we dd n a prevous lecture Agan, there are two rados (A and B) whch may access the medum The cost of transmsson s 0 < C << 1 The beneft of a successful transmsson s 1 Collsons occur when both rados transmt smultaneousl Now, let us look at two stages If a collson occurs n the frst stage, agan n the second stage each rado can choose to transmt or wat Exercse I: multple access wth retransmssons (2) Suppose the game s represented n extensve form as below (note the asymmetry n nformaton) A S W S B B A W S W (, ) S W B (, ) (, ) S W S W (, ) (, ) (, ) (, ) 11

50 Exercse I: multple access wth retransmssons (3) We wll denote node A s strateges as a 2 tuple For nstance, strategy (S,W) means: send n the frst stage and wat n the second state We wll denote node B s strateges as a 3 tuple For nstance, strategy (W,S,W) means: wat n the frst stage f node A sends, send n the frst state f node A wats, and wat n the second stage In the prevous slde, fll out the payoffs from each leaf of the tree Exercse I: multple access wth retransmssons (4) Fnd the Nash equlbra for the subgame rooted at the begnnng of the second stage For each of these equlbra, redraw the tree, substtutng the entre subgame above by the payoffs at equlbrum Fnd all subgame perfect Nash equlbra for the game of multple access wth retransmssons 12

51 Game Theory for Wreless Nt Networks POTENTIAL GAMES Luz DaSlva I 2 R Sngapore August 23 26, 2010 Objectves Defne best reply, better reply,andrandom better reply dynamcs Defne exact and ordnal potental games Dscuss propertes of potental games, ncludng convergence propertes of best and better response dynamcs Dscuss how to dentfy a potental game Model dstrbuted nterference avodance n a CDMA network as a potental t game 1

52 Why do we care about potental games? Potental games are a class of games wth desrable propertes regardng the Nash equlbrum Smple adaptaton strateges are guaranteed to lead to a Nash equlbrum Under some condtons, the equlbrum can also be shown to be effcent Best reply (response) dynamc Consder a normal form game played repeatedly At each stage, exactly one player s offered the opportunty to change her acton from the prevous stage Wth the best reply strategy, a player wll always swtch to an acton whch s a best reply to the current actons of other players 2

53 Better reply (response) dynamc Wth the better reply strategy, a player wll always swtch to an acton whch gves a hgher payoff than her current acton, gven the current actons of other players Wth the random better reply strategy, a player wll choose an acton at random from those whch yeld a hgher payoff than her current acton, gven the current actons of other players Exact potental games An exact potental functon, s a functon any player,, and any acton tuples V : A R such that for V (a,a ) V (b,a ) = u (a,a ) u (b,a ) A game for whch an exact potental functon exsts s called an exact potental game 3

54 Peer to peer fle sharng game as an exact potental game exact potental functon Ordnal potental games An ordnal potental functon s a functon for any player,, and any acton tuples V : A R such that V (a,a ) V (b,a ) > 0 u (a,a ) u (b,a ) > 0 A game for whch an ordnal potental functon exsts s called an ordnal potental game Every exact potental game s an ordnal potental game 4

55 Propertes of potental games Potental functon maxmzers are guaranteed to be Nash equlbra If a pont maxmzes an exact or ordnal potental functon, then a unlateral devaton cannot possbly ncrease a player s utlty Under smple condtons, all potental games have at least one pure strategy NE Exstence of Nash equlbrum for potental games All fnte potental games have at least one pure strategy Nash equlbrum If the strategy space S s compact and the potental functon s contnuous, then the game must have at least one pure strategy Nash equlbrum 5

56 Fnte mprovement path An mprovement path s a sequence of strategy profles {s 1,s 2,s 3, } such that only one player, n, changes strategy from s n 1 to s n and u n (s n 1 ) < u n (s n ) For fnte potental games, all mprovement paths are of fnte length Is rock/paper/scssors a potental game? 0,0 1,-1-1,1-1,1 0,0 1,-1 1,-1-1,1 0,0 6

57 Best response convergence For potental games wth pure strategy NE, the best response dynamc wll converge to a NE For fnte games, ths can be proven usng the fnte mprovement path property For nfnte games (wth compact acton spaces) ths can be proven usng theorems from nonlnear programmng Better response convergence For fnte potental games, the better response dynamc wll converge to a NE Agan, ths can be proven usng the fnte mprovement path property For nfnte potental games wth NE, the random better response dynamc wll converge to a NE 7

58 How do we dentfy a potental game? Unfortunately, gven an arbtrary game, t can be dffcult to determne whether or not t s a potental game Ths problem s especally dffcult for ordnal potental games We wll provde some hnts for exact potental games Coordnaton and dummy games A coordnaton game s a game n whch all users have the same utlty functon I.e., u (s) = C(s) Obvously, a coordnaton game s an exact potental game A dummy game s a game n whch each player s payoff s a functon of only the actons of other players I.e., for each player, u (s) = D (s ) Dummy games are also exact potental games Every exact potental game can be wrtten as the sum of a coordnaton game and a dummy game 8

59 Example ,0.5,0.5 2,0.5, 2 0.5, 2, 2 2, 2, = , 2, , 2, , 4.5, 2 4.5, 4.5, 4.5 coordnaton game dummy game potental game Potental games: nterpretaton In some cases, the potental functon may also serve a socal welfare functon In such cases, the NE whch are potental functon maxmzers are effcent and can result from smple adaptaton processes 9

60 Applcaton: nterference avodance In a CDMA system, users are dfferentated by ther choce of spreadng codes A user s spreadng code can be vewed as a strategy The payoff to the strategy wll be related to ts orthogonalty to (or, rather, lack of correlaton wth) the spreadng codes of other users Interference avodance model (1) Suppose that each player chooses a spreadng code, s, from S M = {s R s =1} Let S be an M x N matrx whose th column s s Also part of the model s addtve M dmensonal whte Gaussan nose, assumed to have an M x M covarance matrx R z 10

61 Interference avodance model (2) One possble utlty functon s the SINR for a correlaton recever u ( s, s ) = s T 1 R s Here, R s the autocorrelaton matrx of nterference plus nose R + R T = S S z A potental functon for the nterference game It turns out that the negated generalzed total squared correlaton functon s an ordnal potental functon for ths game V (S) = SS T + R z F 2 Note: we denote the Frobenus norm of a matrx as A F = a, j 2, j 11

62 Readng lst D. Famolar, N. Mandayam, D. Goodman, V. Shah, A new framework for power control nwreless data networks: games, utlty, and prcng, Wreless Multmeda Network Technologes, pp , 1999 R. Menon, A. MacKenze, J. Hcks, R. M. Buehrer, and J. H. Reed, A game theoretc framework for nterference avodance, IEEE Trans. Comm., vol. 57, no. 4, pp , Apr Summary Roughly: potental games possess a (potental) functon that s affected by changes n a user s strategy n the same drecton as the effects on the user s utlty functon Potental games have desrable propertes If the strategy space s compact and the potental functon contnuous, all potental games have a pure strategy NE The potental functon maxmzer s a NE Best reply strateges guarantee convergence to a NE We can model nterference avodance n CDMA networks where rados ndependently select spreadng codes as a potental game 12

63 Game Theory for Wreless Nt Networks CHANNEL ASSIGNMENT AND TOPOLOGY CONTROL Luz DaSlva I 2 R Sngapore August 23 26, 2010 Objectves Dscuss the objectves of topology control and channel assgnment schemes g Model channel assgnment as a potental game and dscuss results Model topology control as a potental game and dscuss results 1

64 Objectves Topology control Typcally, control power to acheve some network goal, such as connectvty More broadly, control any communcatons parameter to nduce a better network topology Channel assgnment Allocate channels n such a way to mnmze nterference, maxmze reuse, spectral effcency, etc. Can be part of topology control Channel assgnment problem Gven: A set of transmtter recever pars A set of channels Fnd the optmum allocaton of channels to lnks Typcal nterference model: SIR j = k p k p G G kj j I( k, j) Transmtter Node Recever Node 2

65 Game theoretc model Players: N = set of transmtter/recever pars Acton: c = channel selected by the th player Utlty: U ( c) = p jg j 1( c = c j ) pgj 1( c = c j ) j j nterference suffered from others nterference mposed on others Source: Ne and Comancu, Result Ths s a potental game: an ncrease n ndvdual utlty contrbutes to network utlty Potental functon N ( 1 2 p G 1( c = c ) 1 2 j j j = 1 j j p G 1( c j = c j )) A best response strategy leads to the Nash equlbrum The paper proposes a dstrbuted best response strategy 3

66 Channel assgnment wth topology control In multple channel networks, channel selecton affects connectvty and nterference Problem formulatons For a fxed topology and fxed number of channels, assgn channels to lnks so as to mnmze nterference Assgn channels and power levels to acheve a connected topology that mnmzes nterference In a network of mult transcever rados, assgn channels to acheve a good balance between nterference mnmzaton and connectvty objectves 4

67 Game theoretc model (1) The players: N = set of nodes n the network Node s equpped wth k rados The acton set: c = (c 1, c 2,, c k ) C = set of avalable channels Channel 0 denotes no channel assgned to the nterface The utlty functon: Source: Komal and MacKenze, Game theoretc model (2) The utlty functon: f (c) = number of other nodes that node can reach (drectly or through multple hops) χ (c) = sum of nterference weghts over all nterferers on all channels node s operatng on 5

68 Results (1) If the number of orthogonal channels s large enough, nodes wll tend to assgn channels to all ther transcevers Can keep the nterference to zero If the number of orthogonal channels s small nodes wll tend to assgn channels to only one of the transcevers The lnk experencng mnmal nterference For extreme versons of the utlty functon, the game s an exact potental game Results (2) For α = 0, the game s an Exact Potental Game wth potental functon For α χ max, the game s an Exact Potental Game wth potental functon 6

69 Numercal results (1) Intal topology 1 connected wth 85% probablty Path loss model, wth exponent 2 Random better response algorthm 25 node network, 4 transcevers/node Numercal results (2) 10 node network, 2 rados per node 7

70 Readng lst N. Ne and C. Comancu, Adaptve Channel Allocaton Spectrum Etquette for Cogntve Rado Networks, Moble Networks and Applcatons, vol. 11, no. 16, Dec R. Komal and A. B. MacKenze, Analyzng Selfsh Topology Control n Mult Rado Mult Hop Wreless Networks, Proc. of IEEE ICC, 2009 Summary Topology control deals wth settng rado parameters (transmt power, channel) to acheve a network topology wth good connectvty propertes Channel assgnment deals wth the selecton of channels for rados to operate on so as to mnmze nterference and/or maxmze spectral effcency Both problems have been modeled as potental games 8

71 Game Theory for Wreless Nt Networks COOPERATIVE GAMES Luz DaSlva I 2 R Sngapore August 23 26, 2010 Objectves Defne barganng soluton Enumerate the axoms that hold for varous barganng solutons Compare the outcomes of the varous barganng solutons and ther Pareto extensons for convex and non convex utlty spaces Model spectrum sharng as a cooperatve game and determne how rados can arrve at a Nash bargan soluton n a dstrbuted fashon 1

72 Cooperatve vs. non cooperatve game theory Non cooperatve users compete for resources wthout the ablty to negotate wth one another Nash Equlbrum often used to predct the outcome Pareto optmalty used to assess effcency Cooperatve users can bargan to acheve a mutually benefcal resource allocaton Nash Barganng Soluton often used to predct the outcome of the barganng process Strateges agreed upon must be enforceable by an external entty Barganng To understand the outcome of a barganng process, we should not focus on tryng to model the process tself, but nstead we should lst the propertes, p or axoms, that we expect the outcome of the barganng process to exhbt. (Nash) Agreement Pont A possble outcome of the barganng game Barganng Soluton A map from a barganng game to a soluton Dsagreement Pont The expected outcome of the game f players do not come to an agreement 2

73 Barganng soluton A barganng soluton φ, defned on a class of barganng problems 0 Σ, s a map that assocates wth each problem ( U, u ) Σ a unque pont n U Note: u 0 = u(a 0 ) s the utlty acheved at the dsagreement pont a 0 Pareto optmalty, revsted Utlty vector u U s Pareto optmal f u u (and u' u ) mples u' U We ll denote the set of Pareto optmal vectors of U as PO(U) The vector nequalty s wth respect to the partal orderng of R N Utlty vector u U s weak Pareto optmal f u < u mples u' U We ll denote the set of weak Pareto optmal vectors of U as WPO(U) PO( U ) WPO( U ) 3

74 Math remnder: partal orderng A partal orderng of A x A s defned by an operator wth the followng propertes Reflexvty: a a for all a A Antsymmetry: for all a, b A, f a b and b a, then a = b Transtvty: for any a, b, c A, f a b and b c, then a c Barganng solutons Barganng solutons have been proposed by Nash and others A barganng soluton s a possble outcome of the barganng process, and rests on a set of assumptons Generally, we make the followng assumptons regardng the utlty functons N U R s upper bounded, closed and convex There exsts u U such that u 0 < u 4

75 Nash s axoms (1) Indvdual ratonalty (IR): φ(u, 0 0 u ) > u Pareto optmalty (PO): φ( U, u 0 ) PO( U ) Invarance to affne transformatons (INV): N N Take any ψ : R R wth let ψ ( u ) = u', wth u ' = c u + d, c, d R, c > 0, 0 0 Then φ ( ψ ( U ), ψ ( u )) = ψ ( φ( U, u )) Nash s axoms (2) Independence of rrelevant alternatves (IIR): f 0 u' = φ ( U, u ), then φ(v, u 0 ) = u' u' V U and Symmetry (SYM): f U s symmetrc wth respect to and j, u =, and u' = φ( U, u ), then u ' = u ' u j j 5

76 Nash barganng soluton 0 We call φ( U, u ) the Nash barganng soluton ff t satsfes the axoms IR, PO, INV, IIR and SYM The unque NBS s the maxmzer of the Nash product (NP): 0 arg max 0 ( ( u ) > u u u N = 1 Addtonal axoms Strong monotoncty (SMON): f 0 0 V U, then φ( U, u ) φ( V, u ) 0 0 Restrcted monotonty (RMON): f V U and h ( U, u ) = h( V, u ), 0 wth h( U, u ), called the utopa pont, defned below, then 0 0 φ( U, u ) φ( V, u ) 0 h( U, ) = (max u1( u), K, max u ( u)) u 0 u> u u > u 0 N Weak Pareto optmalty (WPO): φ( U, u 0 ) WPO( U ) 6

77 Kala Smorodnsky soluton 0 We call φ( U, u ) the Kala Smorodnsky soluton ff t satsfes the axoms IR, PO, INV, SYM and RMON Let Λ be the set of ponts n the lne contanng u 0 0 and h( U, u ). The unque KSS s WPO(U ) Λ, whch can be expressed as max u > u ( u u ) = ( u j u j ),, j θ θ j where θ = h 0 0 ( U, u ) u Egaltaran soluton 0 We call φ( U, u ) the egaltaran soluton ff t satsfes the axoms IR, WPO, SYM and SMON Consder u such that 0 u ' = u + b, b R Let Λ be the set of ponts n the lne contanng u 0 and u The unque ES s WPO(U ) Λ, whch can be expressed as max 0 0 { u > u 0 u u = u u, j} j j, 7

78 KSS and ES: nterpretaton The KSS assgns as the barganng soluton the pont n the Pareto set that ntersects the lne connectng the dsagreement pont and the utopa pont The ES assgns as the barganng soluton the pont n the weak Pareto set where all players acheve equal ncrease n utlty relatve to the dsagreement pont NBS result for non convex utlty spaces The convex hull U c of a set U s the smallest convex set that contans U If U has a unque Nash product maxmzer, u*, whch concdes wth the Nash product maxmzer for U c, then u* s the unque NBS for U 8

79 Example (non convex utlty space) A NBS(unque maxmzer of NP for both U and ts convex hull) B KSS(WPO) A PO extenson of KSS C ES D PO extenson of ES Suppose: u 0 = 0 A channel assgnment model N set of communcaton lnks M set of channels L bnary channel avalablty matrx B channel throughput matrx C bnary nterference matrx 9

80 Conflct free assgnments An assgnment, represented by a bnary assgnment matrx A, s conflct free f usersareonly assgnedavalablechannelsandno avalable and no two conflctng users are assgned the same channel User utltes are gven by total throughput acheved When the assgnments are conflct free, Nash barganng soluton The Nash Barganng Soluton, also referred to as proportonal farness, maxmzesthe Nash product or the socalwelfare functon Naïve attempts to mplement NBS fals due to communcaton and computatonal complexty Instead, focus on neghbor barganng 10

81 Neghbor barganng One on one barganng, or one buyermult seller barganng Focus on self contaned barganng scenaros Restrct channels under negotaton to those not conflctng wth neghbors outsde the barganng group At any pont n tme, dvde the network nto solated barganng groups Results (1) Cao and Zheng analyzed a scheme that allows any benefcal oneto one one barganng One buyer mult seller barganng s only permtted n specal cases referred to as feed poverty barganng In feed poverty barganng, a starved node ssues a request to neghbors to ntate the barganng process 11

82 Results (2) Greedy n these plots s a centralzed attempt to fnd the NBS based on a greedy algorthm Barganng s barganng wth the two restrcted types of barganng dscussed prevously Barganng w/o feed poverty s one to one barganng only Spectrum sharng as a cooperatve game N users (transmtter/recever pars) K channels over total bandwdth B Users allocate power over a subset of K channels, subject to a power constrant u log K k k K (p) B p h k = p Pmax K n k = 1 0 k k k = 1 + p j h j K j 12

83 Mxed and pure strateges If we take any mxed strategy that assocates ratonal probabltes to actons, then that strategy can be replcated wth a pure strategy Implcaton: the utlty space becomes closer to convex as we partton the bandwdth nto an ncreasng number of subchannels Mxed and pure strateges: llustraton 13

84 Achevng the NBS For the spectrum sharng problem, t s possble to devse a dstrbuted algorthm for ndvdual node pars to reach the NBS See Surs et al. Assumes nodes can exchange nformaton wth ther twohop neghborhood Convergence of the algorthm can be proved But s the NBS a good outcome? Compare aganst other cooperatve soluton as well as other approaches to resource management Comparson (1) R a (m) sde of the square area smulated N = 10 K = 10 Capacty n bps/hz 14

85 Comparson (2) ES s best at mnmzng the varance of capacty acheved by all nodes, but does so at the expense of lower overall (average) capacty MaxSum gets good average capacty but performs poorly on farness NBS has the edge over KSS n terms of average capacty as densty decreases and acheves hgher mnmum capacty Readng lst J. E. Surs, L. A. DaSlva, Z. Han, A. B. MacKenze, and R. S. Komal, AsymptotcOptmalty for DstrbutedSpectrumSharngUsng Sharng Barganng Solutons, IEEE Trans. on Wreless Communcatons, vol. 8, no. 10, Oct. 2009, pp L. Cao and H. Zheng, Dstrbuted spectrum allocaton va local barganng, Sensor and Ad Hoc Communcatons and Networks, pp , September

86 Summary Cooperatve games model the outcome of barganng by players who cooperate n maxmzng ther own utlty The Nash barganng soluton, the Kala Smorodnsky soluton, and the egaltaran soluton are three models for the outcome of such cooperaton Each s based on a set of axoms that the outcome of barganng s expected to obey It s possble to model spectrum sharng n a spectrum commons as a cooperatve game among nterferng transmtter/recever pars 16

87 Game Theory for Wreless Nt Networks COALITIONAL GAME THEORY Luz DaSlva I 2 R Sngapore August 23 26, 2010 Objectves Defne coaltonal games and mputatons Characterze the core of a coaltonal game Characterze the core of a coaltonal game Dscuss nternal and external stablty and mult coalton equlbra Model nterference management as a coaltonal game 1

88 Coaltons and payoffs Coaltonal game theory studes the ncentves for players to form coaltons An mportant aspect s how to partton the payoffs among players Players can communcate before or durng the game and can redstrbute the payoff (drectly or va sde payments) The worth of the coalton s the utlty that players belongng to the coalton can guarantee for themselves Mathematcal model Set of players: N = {1, 2,, N} A coalton s any subset of N The coalton of all players (N tself) s called the grand coalton The set of all possble coaltons s 2 N A coalton game s a mappng v : 2 N R such that v( )=0 For any coalton A N, v(a) s the worth of the coalton 2

89 Imputaton Players n a coalton A have a total amount v(a) to dvde among themselves No ratonal player wll accept less than she could get by herself (outsde of any coalton) An mputaton n a game v s a vector x = ( x 1, K, x n ) R x = v(n) N x v({}) for every N n such that Denote by E(v) the set of all mputatons of game v Exstence of mputatons For every game v, E(v) f and only f v ( N) v({ }) N 3

90 Smple example: sellng spectrum A lcensed spectrum user (player 1) has excess spectrum that t s wllng to sell to secondary users (players 2 and 3), whch value the spectrum at 90 and 100 $ unts, respectvely v({1}) = v({2}) = v({3}) = v({2,3}) = 0 v({1,2}) = 90, v({1,3}) = 100, v(n) = 100 The grand coalton can form The spectrum s assgned to player 3, who possbly makes a sde payment ( 0) to player 2 Superaddtve game A game s superaddtve f for every par of coaltons A, B N, wth A B Unty makes strength v ( A B) v( A) + v( B) 4

91 Preference between mputatons Gven mputatons x, y E(v), whch one are players lkely to choose? Coalton S N prefers x to y (denoted x x > y for every S, and x v(s) S y ) f An mputaton x s preferred to y (denoted x f y ) f there exsts a coalton S N wth x f y S f S The core of a game Let v be a game. The core of v s the set Core(v) R n such that { y E( v) x E( v), x f y does not hold} Core( v) = Some coaltonal games have an empty core 5

92 Characterzng the core Let n C( v) = x R x = v( N), x v( S), S 2 N S N Then For every game v, C(v) Core(v) For a superaddtve game v, C(v) () = Core(v) () Spectrum example: characterzng the core What s the core for the spectrum sellng example? The vector x must meet the followng requrements x 0, = 1, 2, 3 x 1 + x 2 90, x 1 + x 3 100, x 1 + x 2 + x 3 = {( t,0,100 t) R } C( v) = t Interpretaton: player 1 sells the spectrum to player 3, for a prce between 90 and 100; player 2 may bd up to 90 but eventually s prced out of the market 6

93 Coalton structure A coalton structure s a partton of N nto dsjont and exhaustve subsets. The set of all coalton structures s C = {C 1, C 2,, C C } The number of possble coalton structures (the cardnalty of C) s gven by the Bell number B N = B 0 N k = = 1 1 N 1 Bk, N 0 k 1 Mult coalton equlbrum If every coalton n a coalton structure s nternally and externallystable, than that coalton structure s an equlbrum Notce the smlarty wth the tradtonal concept of Nash equlbrum Wth nternal and external stablty, there s no ncentve for coaltons to merge or for players to splnter from current coaltons 7

94 Game objectve The formaton of the grand coalton s not necessarly the outcome of a coaltonal game We are nterested n whch coalton structure(s) provde the maxmum worth: arg max C C j v ( Al ) A C l j Stablty A coalton S Ns nternally stable f none of ts current members hasany any ncentve to leave and become a sngleton coalton,.e., v ( S) v({ }) S A coalton S Ns externally stable f no other coalton has an ncentve to jon t,.e., v( T ) > v( S T ) v( S) T c S 8

95 Interference tolerance as a coaltonal game N wreless lnks (transmtter/recever pars) operatng n a channel wthbandwdth W Rados can transmt over the entre band or they can enter nto an agreement wth other rados to partton the band h j s the channel gan from the transmtter of lnk to the recever of lnk j For smplcty, assume transmt power s the same on all lnks Worth of a coalton Consder a coalton S whose members have agreed to share the bandwdth on a non nterferng bass 0 η 1 s the fracton of the band that lnk s allowed to use 1 S η The worth of a coalton S N s then 9

96 Externaltes A coaltonal game has externaltes f the worth of a coalton depends on what other coaltons form A coaltonal game has postve externaltes f for any mutually dsjont coaltons S 1, S 2, S 3 N v( S3;{ S1 S2, S3}) v( S3;{ S1 S2 S3}) The coaltonal game that models coexstence n an nterference channel has postve externaltes Interference management: coalton dynamcs Wth probablty p, any exstng coalton can propose a merger wth another coalton 10

97 Interference management: Markov chan The ndcator functons 1 (X) ndcates whether h a merger would beneft members of both coaltons We can calculate the average and varance of the tme untl equlbrum s reached Interference management: comments In some cases, the core s non empty but there s no way to reach the grand coalton Due to the myopc nature of lnks n the coalton formaton process, even when the grand coalton s advantageous to all, there s no need to reach t because ntermedate stages are not sustanable Canderve a condton basedon anndvdual lnkscontrbuton Can derve a condton, based on an ndvdual lnks contrbuton to the coalton, for the grand coalton to be reachable 11

98 Results (1) Average gan n bps/hz due to coalton formaton as compared to non cooperaton d s the average dstance between a transmtter and ts recever Results (2) Average coalton szes d s the average dstance between a transmtter and ts recever 12

99 Readng lst Z. Khan, S. Glsc, L. DaSlva, J. Lehtomakk, Modelng the Dynamcs of Coalton Formaton Gamesfor Cooperatve SpectrumSharng Sharng n an Interference Channel, under revew, 2010 Summary Coaltonal game theory studes ncentves for players to form coaltons and obtan a payoff that s hgher than f they acted alone Model must consder how players n a coalton wll partton the coalton s payoff At equlbrum, a game reaches a coalton structure that s nternally and externally stable Recent work apples coaltonal game theory to spectrum sharng and nterference management 13

100 Game Theory for Wreless Nt Networks MECHANISM DESIGN Luz DaSlva I 2 R Sngapore August 23 26, 2010 Objectves Defne a Bayesan game and Bayesan equlbrum Defne mechansm and socal choce functon and dscuss the objectves of mechansm desgn Provde examples of mpossblty theorems Defne ncentve compatblty and state the revelaton prncple Dscuss challenges n applyng mechansm desgn to wreless communcatons and networks problems 1

101 Analyss versus desgn How do I use game theory to desgn a protocol? Well, game theory s prmarly an analyss tool Once the protocol s desgned, game theory can help you understand ts behavor And analyss can provde nsght nto desgn But there s an area of game theory that s devoted to mechansm desgn Bayesan game (1) Set of players A set of actons avalable to player A = A 1 x A 2 x x A n Θ set of possble types for player Θ = Θ 1 x Θ 2 x x Θ n p probablty dstrbuton over player s types 2

102 Bayesan game (2) Set of outcomes Mappng from actons to outcomes g: A O {u } set of ndvdual utlty functons u : Θ x O R Bayesan equlbrum A strategy of player s a functon s : Θ A A strategy s s (weakly) domnant f t maxmzes an agent s payoff for all possble actons of other agents A profle of strateges s 1, s 2,, s n s a Bayesan Nash equlbrum f s maxmzes the expected utlty under p 3

103 Mechansms A mechansm,, s an acton space, A, and a mappng from acton profles to outcomes, g We assume that the player set, N; type space and dstrbuton, Θ and p; outcome space, O; and utlty functons, {u }, are fxed From ths pont forward, t s probably easest to thnk of a player s acton space, A, as the set of messages that the player can send Socal choce functons A socal choce functon, f, specfes the desred outcome gven a type profle A mechansm, functon f f, s sad to mplement socal choce where s* s an equlbrum of the game nduced by M 4

104 Impossblty theorem: prelmnares (1) Suppose that each player s type determnes her rankng over a set of canddates for offce The outcome s a group rankng of the canddates Suppose we want to desgn a votng scheme (the mechansm n ths case) A socal welfare functon satsfes unanmty f, when all players have dentcal rankngs, the rankng gven by the socal welfare functon s the same Impossblty theorem: prelmnares (2) A socal welfare functon satsfes ndependence of rrelevant alternatves f, when no players change ther relatve preferences between a and b, the relatve socal rankng of a and b also does not change A socal welfare functon s a dctatorshp f the rankng gven by the socal welfare functon s always dentcal to a partcular player s rankng 5

105 Arrow s mpossblty theorem Every socal welfare functon over a set of more than two canddates that satsfes unanmty and ndependence of rrelevant alternatves s a dctatorshp (Kenneth Arrow, Ph.D. thess) Drect revelaton and ncentve compatblty A mechansm M s sad to be a drect revelaton mechansm f A = Θ A strategy s (under a drect revelaton mechansm) s truth revealng f A drect revelaton mechansm s ncentve compatble f truth revelaton s an equlbrum strategy 6

106 Revelaton prncple Suppose that there exsts any mechansm M that mplements a partcularsocalchocefuncton choce f Then f s mplementable va an ncentve compatble drect revelaton mechansm Quas lnear preferences For many results n mechansm desgn, we need addtonal structure on the outcome space and preferences (utltyfunctons) One such structure s quas lnear preferences Outcomes n ths case consst of a dscrete set of alternatves plus a payment receved from (or, f negatve, pad to) each player Each player s utlty s then 7

107 Propertes wth quas lnear preferences (1) Under quas lnear preferences, a socal choce functon maps a type profle nto one of the alternatves and payments for each player The socal choce functon s sad to be budget balanced f Propertes wth quas lnear preferences (2) The socal choce functon s sad to be weakly budget balanced f The socal choce functon s sad to be ndvdually ratonal f 8

108 The VCG mechansm A Vckrey Clarke Groves (VCG) mechansm n a quas lnear preferences settng s a drectrevelaton revelaton mechansmwth The VCG mechansm s ncentve compatble Truth tellng s a domnant strategy n general the VCG mechansm can be ether weakly budget balancng OR ncentve compatble Impossblty Results (Hurwcz, 1972) It s mpossble to mplement an effcent and budget balanced mechansm wth domnant strateges n a smple exchange economy wth quas lnear preferences VCG mechansm can acheve effcency wth weak budget balance, but the Hurwcz results says that strong budget balance s mpossble (Myerson Satterthwate Myerson1983) It s mpossble to (Myerson Satterthwate, Myerson1983) It s mpossble to mplement an effcent, budget balanced mechansm wth a Bayesan Nash ncentve compatble mechansm, even wth quaslnear preferences 9

109 Applcaton challenges (1) Computatonal Complexty: Whle the revelaton prncple smplfesthe conceptual understandng of mechansms, revelatory mechansms may not be computatonally effcent Centralzed Decson Makng: In tradtonal mechansm desgn, the mechansm s completely centralzed Ths s napproprate for many wreless applcatons Applcaton challenges (2) Poor Preference Assumptons: Many mechansm desgn results rely on quas lnear preferences The assumpton of monetary transfers s often napproprate n engneerng applcatons Impossblty: The strongest results n mechansm desgn are often the mpossblty results 10

110 Readng lst N. Nsan, T. Roughgarden, E. Tardos, and V. V. Vazran, eds., AlgorthmcGame Theory, Cambrdge Unversty Press, 2007 In partcular, the chapter by N. Nsan, Introducton to Mechansm Desgn (for Computer Scentsts) Summary Bayesan games dffer from games we have studed so far due to the ntroducton of player types and utltes that depend on player type and outcome A mechansm s an acton space and a mappng from acton profles to outcomes A drect revelaton mechansm s ncentve compatble f truth revelaton s an equlbrum strategy Challenges n applyng mechansm desgn to wreless problems nclude computatonal complexty, centralzed decson makng and poor preference assumptons 11

111 Game Theory for Wreless Nt Networks REAL TIME SPECTRUM MARKETS Luz DaSlva I 2 R Sngapore August 23 26, 2010 Objectves Dscuss the applcaton of economc prncples to resource allocaton n wreless networks Descrbe a VCG aucton and ts propertes Dscuss examples of power and SINR auctons and summarze ther results 1

112 In the news... Economc prncples Applyng economc prncples to the allocaton of resources n a wreless network The market as an effcent allocator of resources, when these resources are scarce Hghly decentralzed processng of nformaton Buyers and sellers nteract to fnd the prce pont where demand equals supply 2

113 Dynamc spectrum access (DSA) Rado adapts to select operatng spectrum avalable (n local tmefrequency space) spectrum holes wth lmted usage rghts Prmary user has a lcense (spectrum access rght) protected from nterference Secondary user accessng a band for whch someone else has PU rghts, and must avod nterferng wth those Cooperatve DSA: SU accesses the band wth permsson from the PU Non cooperatve DSA: SU accesses the band wthout requrng permsson from the PU Spectrum markets Tradng of spectrum access rghts In DSA context, we re nterested n tradng after the ntal assgnment of rghts by regulators Applcatons: Tradng among spectrum lcense holders Tradng between SUs and PUs n cooperatve DSA Modelng of supply/demand behavors n non cooperatve DSA (opportunstc, so no actual tradng) 3

114 An aucton A mechansm to assess market prces A good s sold to (or a resource s allocated to) the submtter of the most favorable bd Frst prce aucton hghest bdder gets the good, pays the amount bd Second prce aucton hghest bdder gets the good, pays the second hghest bd Bddng one s true value s a domnant strategy Vckrey Clark Groves (VCG) auctons Sealed bd, second prce auctons It s a domnant strategy for users to bd truthfully Auctoneer decdes whch player(s) wn(s) based on utlty maxmzaton Wnnng player(s) pay(s) the opportunty cost The cost ther presence mposes on all other players 4

115 VCG aucton example Ann : 1 apple, $5 Bella : 1 apple, $2 Carla : 2 apples, $6 (not nterested n a sngle apple) VCG aucton example: results Wnners: Ann and Bella How much does Ann pay? Currently, Bella has a utlty of $2. If Ann hadn t been here, Carla would have won and would have had a utlty of $6. So Ann pays $6 $2 = $4 How much does Bella pay? Ann has a utlty of $5. If Bella hadn t been here, Carla would have won and would have had a utlty of $6. So Bella pays $6 $5 = $1 5

116 An SINR aucton mechansm All partcpants (e.g., secondary users n a spectrum market) submt ther demand curves Auctoneer (e.g., a holder of spectrum lcense or a spectrum broker) calculates the market clearng prce Example: n an SINR aucton, users announce ther demand curves for a certan level of receved SINR Problem: user s demand curves are nter dependent due to nterference A power/sinr aucton (1) Proposed by Huang, Berry and Hong, 2006 A spectrum manager manages bandwdth B M users (tx/rcv pars) share ths bandwdth Users bd for power/sinr Power evenly spread across the entre band Reduces the power bandwdth allocaton problem to a receved power allocaton problem U ( γ ) = θ lnγ user dependent parameter receved SINR 6

117 A power/sinr aucton (2) Interference temperature constrant at a specfed pont For any Pareto optmal soluton, constrant must be tght γ ph = 1 M p h 0 n + p j h 0 j = 1 B j P VCG spectrum sharng aucton 1. Users submt ther utltes U ( γ )} { 2. Spectrum manager calculates the power allocaton p = ( pˆ K pˆ ) ˆ 1 M that maxmzes the total utlty M U = = U j ( γ (ˆ p 1 and allocates power accordngly gy max )) 3. Spectrum manager also calculates the maxmum total utlty f user s excluded from the aucton, to determne prces 7

118 Practcal ssues wth ths aucton Spectrum manager must solve M+1 optmzaton problems Typcally non convex Computatonally ntensve for large M Channel gans h j for all and j must be measured and reported to the manager Agan, may not be feasble for large M One dmensonal SINR aucton 1. Manager announces reserve bd β > 0 and prce π > 0 (reserve bd can be made arbtrarly small) 2. Each user submts bd b 0 3. Manager reserves power p 0 and allocates to each user receved power proportonal to her bd b β ph 0 = P M p0 = P M b j = + β b 1 j = + β 1 4. User pays = πγ C 8

119 Results (1) Reserve bd guarantees a unque desrable outcome of the aucton β > 0 guarantees a unque Nash Equlbrum above a certan threshold prce Wth co located recevers and logarthmc utltes, closed form soluton for threshold prce as a functon of users utlty parameters Power allocatons are arbtrarly close to those obtaned from a VCG aucton Results (2) Authors also present a dstrbuted algorthm that can acheve a NE wth lmted nformaton wth lmted nformaton Each user needs only to know n 0, the SINR at ts own recever, and the channel gan rato h /h 0 9

120 Stackelberg games Stackelberg (leader/follower) games have also been proposed for dynamc leasng of spectrum Prmary user (leader) sets prces for leasng spectrum, takng nto account mpact of secondary user (SU) transmsson on the performance experenced by the PU SU decdes whether to take advantage of lcensed bands at the prce advertsed Readng lst J.M. Peha, "Approaches to Spectrum Sharng," IEEE Communcatons Magazne, vol. 43, no. 2, February 2005, pp J. Huang, R. Berry, and M. Hong, Aucton Based Spectrum Sharng, Moble Networks and Applcatons, 11, pp ,

121 Summary Game theory can also be used when economc prncples are appled to the allocaton of scarce resources n a wreless network Auctons are mechansms to determne what value a user (or applcaton) places on the resources t needs Second prce auctons can be appled to the real tme leasng of spectrum Formulated drectly as an aucton for spectrum or ndrectly as an aucton for power or SINR Exercse I: a problem wth VCG auctons Consder the aucton of two spectrum lcenses and suppose there are three bdders Bdder 1 wants only the package of 2 lcenses and s wllng to pay $2 mllon for both Bdder 2 wants a sngle lcense and s wllng to pay $2 mllon Bdder 3 has the same bd as bdder 2 Who s/are the wnner(s) and how much do they pay? Suppose bdder 1 nstead bds for each lcense separately, each at $1 mllon. How does ths change the results? 11

122 Game Theory for Wreless Nt Networks SUMMARY Luz DaSlva I 2 R Sngapore August 23 26, 2010 Revew (1) We have ntroduced a number of concepts from game theory: Non cooperatve versus cooperatve games Pure versus mxed strateges Repeated games Nash equlbrum (and some of ts varatons) Potental games and best/better response dynamcs Nash barganng gsoluton Coalton games Aucton mechansms Bayesan games and mechansm desgn 1

123 Revew (2) We have also dscussed a number of applcatons of game theory to wreless networks: Power control Interference management Real tme spectrum (SINR, power) auctons Cooperatve spectrum sharng Channel assgnment and topology control Cauton: f game theory s a hammer, we must be careful not to make every problem look lke a nal (Very) rough gudelnes Are decsons made by a central unt? No Are decsons made autonomously? Yes Canplayersreach reach agreements? Yes No Yes Consder classcal optmzaton Consder dstrbuted optmzaton Canplayerssplt splt off nto coaltons? No Non cooperatve game theory No Cooperatve game theory Yes Coaltonal game theory 2

124 Partng words Increased complexty, greater autonomy and ntellgence n wrelessnodes, trendstowards towards decentralzedor opportunstc resource management make game theoretc models attractve n wreless networks Game theoretc models are very powerful, but you must also understand the lmtatons of your model Best wshes and stay n touch! 3