A Unified Distributed Algorithm for Non-Games Non-cooperative, Non-convex, and Non-differentiable. Jong-Shi Pang and Meisam Razaviyayn.

Transcription

1 A Unfed Dstrbuted Algorthm for Non-Games Non-cooperatve, Non-convex, and Non-dfferentable Jong-Sh Pang and Mesam Razavyayn presented at Workshop on Optmzaton for Modern Computaton Pekng Unversty, Bejng, Chna 10:10 10:45 PM, Thursday September 04, 2014 Department of Industral and Systems Engneerng, Unversty of Southern Calforna, Los Angeles Vstng Research Assstant, Department of Electrcal and Computer Engneerng, Unversty of Mnnesota, Mnneapols 1

2 The Non-cooperatve Game G An n-player non-cooperatve game G wheren each player = 1,, n, antcpatng the rvals strategy tuple x ( x j) n n j=1 X X j, solves the optmzaton problem: X R n s a closed convex set mnmze x X θ (x, x ) j=1 θ : Ω R s a locally Lpschtz contnuous and drectonally dfferentable n functon defned on Ω Ω where each Ω s an open convex set contanng X =1 A key structural assumpton for convergence of dstrbuted algorthm: each θ (x) = f (x)+g (x ), wth f (x), dependent on all players strategy profle x ( x ) n, beng twce contnuously dfferentable but not necessarly convex, =1 and g (x ), dependent on player s strategy profle x only s convex but not necessarly dfferentable. 2

3 Quas-Nash equlbrum: Defnton and exstence Defnton. A player profle x ( x,) n =1 θ (, x, ) (x, ; x x, ) 0, x X. s a QNE f for every = 1,, n: Exstence. Suppose each θ (x) = f (x) + g (x) wth x f contnuously dfferentable on Ω and g (, x ) convex on X that s compact and convex. Proof by a fxed-pont argument appled to the map: Φ : x ( x ) n =1 X n X Φ(x) (Φ (x)) n =1 X, where, for = 1,, n, =1 [ Φ (x) argmn f (z, x ) + g (z, x ) + α ] z X 2 z x 2, wth α > 0 such that the mnmand s strongly convex n z for fxed x. Remark. For exstence, g (x) can be fully dependent on the player profle x; but for convergence of dstrbuted algorthm, g (x ) s only player dependent. 3

4 The unfed algorthm The man dea: Employng player-convex surrogate objectve functons and the nformaton from the most current terate, non-overlappng groups of players update, n parallel, ther strateges from the soluton of sub-games. Thus the algorthm s a mxture of the classcal block Gauss-Sedel and Jacob teratons, appled n a way consstent wth the game-theoretc settng of the problem. Two key famles: The player groups: σ ν { σ ν 1,, σν κ ν } conssts of κν parwse dsjont subsets of the players labels, for some nteger κ ν > 0. Players n each group σ ν k solve a sub-game; all such sub-games n teraton ν are solved n parallel. N ν κ ν k=1 σ ν k not necessarly equal to {1,, n};.e., some players may not update n an teraton. 4

5 Gven x ν X, the bvarate surrogate objectves: n leu of the orgnal objectves { } κν {θ } σ ν k k=1. The subgames, denoted G σν k ν the players n σk ν are mnmze x X θ σν k { σ θ ν k (x σν k; x ν ) : σk ν } κν k=1 for k = 1, κ ν : the optmzaton problems of x, x σν k ; subgame varables x σν k The new terate for a step sze τ σ ν k x ν+1;σν k x ν;σν k + τσ ν k (0, 1] ; x ν nput to subgame at teraton ν x ν;σν k soluton to subgame x ν;σν k. σ ν k Need drectonal dervatve consstency at lmt x of generated sequence: θ (, x, ) (x, ; x x, ) θ σ t k (, x,σt k ; ; x ) (x, ; x x, ), x X.. 5

6 An llustraton. A 10-player game wth the groupng: σ ν = { {1, 2}, {3, 4, 5}, {6, 7, 8, 9} } so that κ ν = 3 and N ν = {1,, 9}, leavng out the 10th-player. Players 1 and 2 update ther strateges by solvng a subgame G ν {1,2} {1,2} by the surrogate objectve functons θ 1 ( ; x ν {1,2} ) and θ 2 ( ; x ν ). defned In parallel, players 3, 4, and 5 update ther strateges by solvng a subgame G ν {3,4,5} usng the surrogate objectve functons { } {3,4,5} θ 3 ( ; x ν {3,4,5} ), θ 4 ( ; x ν {3,4,5} ), θ 5 ( ; x ν ) ; smlarly for players 6 through 9. The 10th player s not performng an update n the current teraton ν accordng to the gven groupng. 6

7 Specal cases: player groups Block Jacob N ν = {1,, n} and σ ν k may contans multple elements. Pont Jacob κ ν = n; thus σk ν optmzaton problem: = {k} for k = 1, n: each player solves an mnmze x X θ (x ; x ν ). Block Gauss-Sedel κ ν = 1 for all ν: only the players n the block σ ν 1 update ther strateges that mmedately become the nputs to the new terate x ν+1 whle all other players j σ ν 1 keep ther strateges at the current terate xν,j. Pont Gauss-Sedel κ ν = 1 and σ ν 1 s a sngleton. Above are determnstc player groups; also consder randomzed player groups: Let {σ 1, σ K } be a partton of {1,, n}. At teraton ν, the subset σ ν {σ 1,..., σ K } of player groups s chosen randomly and ndependently from the prevous teratons, so that Pr(σ σ ν ) = p σ > 0, There s a postve probablty p σ, same at all teratons ν, for the subset σ of players to be chosen to update ther strateges. 7

8 Specal cases: surrogate objectves Standard convex case Suppose θ (, x ) s convex. For σ ν k, let θ σν k (x σν k ; z) θ (x σν k, z σ α ν k ) + 2 x z 2, for some postve scalar α regularzaton Mxed convexty and dfferentablty Suppose θ (, x ) = g (, x )+f (, x ), where g (, x ) s convex and f (, x ) s dfferentable. Let θ σν k (x σν k ; z) g (x σν k, z σ ν k ) + f (z) + z jf (z) T ( x j z j ) j σ ν k + α 2 x z 2 partal lnearzaton convex n x σν k for fxed z Newton-type quadratc approxmaton Suppose x θ (, x ) exsts. Let θ σν k (x σν k ; z) θ (z) + x jθ (z) T ( x j z j ) + 1 ( x j z j ) T B σν;j,j k ( x j z j ), 2 j σ ν k j,j σ ν k quadratc n x σν k for fxed z B σν k ;j,j approxmates mxed partal dervatves of θ (, z σν k) w.r.t. x j and x j. 8

9 Convergence analyss Two approaches Contracton showng that the sequence {x ν } ν=1 contracts n the vector sense by means of the assumpton of a spectral radus condton of a key matrx Potental relyng on the exstence of a potental functon that decreases at each teraton. Thnk about a system of lnear equatons: Ax = b (Generalzed) dagonal domnance yelds convergence under contracton. Symmetry of A yelds the potental functon: P (x) 1 2 xt Ax b T x. 9

10 Contracton approach An nteger T > 0 and a fxed famly { σ t { σ1 t,, }} T σt κ t t=1 of the players labels that parttons {1,, n}. of ndex subsets Famles of bvarate surrogate functons θ t { } σ = θ t k : σk t κt, for t = 1,, T, such that for every par (x σt; σ k ; z), the functon θ t k (, x σt; k ; z) s convex. k=1 For each set σk t, let Gσt k t denote the subgame consstng of the players σk t σ wth objectve functons θ t k ( ; z) for certan (known) terate z to be specfed. Let κ ν = κ t and σ ν k = σt k for ν t modulo T and for all k = 1,, κ ν; thus, σ for each = 1,, n, θ ν k ndex set contanng. = θ σ t k where ν t modulo T and σ t k s the unque Thus, each player and the members n σk t wll update ther strategy tuple exactly once every T teratons through the soluton of the subgame G σt k t. Fnally, we take each step sze τ σ ν k = 1. 10

11 A further llustraton Consder a 12-player game wth T = 3 and wth σ 1 = {{1, 2}, {3, 5, 6}}, σ 2 = {{4, 7, 8}}, and σ 3 = {{9}, {10, 11, 12}}. Startng wth x 0 = ( x 0;) 12 =1 = x(0), we obtan after one teraton ( ) x 1 = x 1;{1,2}, x 1;{3,5,6}, x 0;4, x 0;{7thru12}. The sub-vectors x 1;{1,2} and x 1;{3,5,6} mean that the players 1 and 2 update ther strateges by solvng a 2-player subgame and smultaneously the players 3, 5, and 6 update ther strateges by solvng a 3-player subgame. The remanng players 4, 7 through 12 do not update ther strategy n ths frst teraton. The next two teratons yeld, respectvely, x 2 = ( x 1;{1,2}, x 1;{3,5,6}, x 2;{4,7,8}, x 0;{9thru12} ) x 3 = ( x 1;{1,2}, x 1;{3,5,6}, x 2;{4,7,8}, x 3;{9}, x 3;{10,11,12} ). The update of x 2 employs x 1 {4,7,8} n defnng the player objectves θ 4 ( ; x 1 ), θ {4,7,8} 7 ( ; x 1 {4,7,8} ), and θ 8 ( ; x 1 ). Smlarly, the update of x 3 employs x 2. 11

12 After three teratons, we have completed a full cycle where all players have updated ther strateges exactly once, obtanng the new terate x (1) = x 3. The next cycle of updates s then ntated accordng to the same partton { σ 1, σ 2, σ 3} and employs the same famly of bvarate surrogate functons. {1,2} {1,2} {3,5,6} {3,5,6} {3,5,6} group 1: θ 1, θ }{{ 2, θ } 3, θ 5, θ 6 ; parallel 2-person subgame 3-person subgame 2 subgames solve n parallel {4,7,8} {4,7,8} {4,7,8} group 2: θ 4, θ 7, θ }{{ 8 ; sngle game } 3-person subgame {9} {10,11,12} {10,11,12} {10,11,12} group 3: θ 9, θ 10, θ 11, θ }{{ 12 } sngle-player opt 3-person subgame 2 subgames solved n parallel parallel: sngle opt. + game group 1 sequental > group 2: sequental > group 3 12

13 Assume θ σ t k objectve) f σ t k Set-up for assumptons (x σt k; z) = g (x σ )+ f t k (x σt k; z), where g s convex and the (surrogate ( ; z) s twce contnuously dfferentable. σ f t k (x σt k; z) s strongly convex n x σt k unformly n z;.e., γk; t > 0 such that for all x X, all u σt k X σt k and all z X, ( x u ) T 2 f ( σt u u k (u σt k ; z) x u ) γk; t x u 2. σ Further assume that each functon u f t k s contnuously dfferentable n both arguments wth bounded dervatves. Let γk;j t sup 2 f σt u j u k (u σt k ; z) <, j n σk t u σt k X σt k; z X γ k;l t sup 2 f σt z l u k (u σt k ; z) <, σk t and l = 1,, n. u σt k X σt k; z X Let Γ blkdag [ Γ t ] T t=1, where each Γ t blkdag [ Γ t k Let Γ [ Γ ts ] T t,s=1, where each Γ ts [ Γ ts k,k ] (κt,κ s ) (k,k )=(1,1) ] κt k=1 and Γ t k [ γ t k;j wth Γ ts k,k ],j σ t k. [ ( ) ] j σ s γ k;j t k. 13 σ t k

14 [ ] The comparson matrx: Γ blkdag Γ t T [, where each Γ t blkdag t=1 [ ] ( ) { and Γ t k γk;j t, where Γ t γ t k k; f = j γk;j t otherwse for, j σt k.,j σ t k j Γ t k ] κt k=1 Key assumpton: The matrx Γ Γ, whch has all off-dagonal entres nonpostve (thus a Z-matrx), s also a P-matrx (thus a Mnkowsk matrx). Wrtng Γ = L + D + Ũ as the sum of the strctly lower trangular, dagonal, and strctly upper trangular parts, respectvely, we have Γ L s nvertble and has a nonnegatve nverse, [ the spectral radus of the (nonnegatve) matrx than unty, or equvalently, Γ L ] 1 ( ) D + Ũ s less postve scalars d t k;j and d t k;l such that γ t k; d t k; > j σ t k γ t k;j d t k;j + n γ k;l t d k;l t, t = 1,, T, k = 1,, κ t, σk t. l=1 14

15 Potental Games Defnton. A famly of functons {θ (x)} n =1 on the set X admts an exact potental functon P : Ω R f P s contnuous such that for all, all x Ω, and all y and z Ω, P (y, x ) P (z, x ) = θ (y, x ) θ (z, x ); a generalzed potental functon P : Ω R f P s contnuous such that for all, all x Ω, and all y and z Ω, θ (y, x ) > θ (z, x ) P (y, x ) P (z, x ) ξ (θ (y, x ) θ (z, x )), for some forcng functons ξ : R + R +,.e., lm ν ξ (t ν ) = 0 lm ν t ν = 0. Example Generalzed exact: mnmze θ 1 (x 1, x 2 ) x 1 mnmze θ 2 (x 1, x 2 ) x 1 x 2 + x 2 x 1 R x 2 R subject to 2 x 1 2 subject to 1 x 2 3. Generalzed potental functon: P (x 1, x 2 ) = x 1 x 2 + x 2. The potental functon, f t exsts, s employed to gauge the progress of the algorthm. 15

16 How to recognze the exstence of a potental? The convex case. Suppose that θ (, x ) s convex. Recallng ts subdfferental, x θ (, x ), we defne the multfuncton n Θ(x) x θ (x), x X. =1 Among the followng four statements, t holds that (a) (b) (c) (d): n (a) Θ(x) s maxmally cyclcally monotone on Ω Ω ; (b) a convex functon ψ(x) such that ψ(x) = Θ(x) for all x Ω; (c) a convex functon ψ(x) on Ω and contnuous functons A (x ) on Ω such that θ (x) = ψ(x) + A (x ) for all x Ω and all = 1,, n; (d) the famly {θ (x)} n =1 =1 admts a convex exact potental functon P (x). If x θ (x) s dfferentable, the exstence of a (dfferentable) potental s related to the symmetry of the Jacoban of the vector functon ( x θ (x)) n =1. 16

17 Player selecton rule s Essentally coverng f an nteger T 1 such that N ν N ν+1... N ν+t 1 = { 1, 2,..., n }, ν = 1, 2,..., so that wthn every T teratons, all players wll have updated ther strateges at least once. [Unlke parttonng, the above ndex sets may overlap, resultng n some players updatng ther strateges more than once durng these T teratons. ] Randomzed f the players are chosen randomly, dentcally, and ndependently from the prevous teratons so that Pr(j N ν ) = p j p mn > 0, j = 1, 2,..., n, ν = 1, 2,.... Postulates on objectves and ther surrogates: Each θ (x) = f (x) + g (x ) for some dfferentable functon f and convex functon g. Correspondngly, θ σ ν k (x σν k; z) = g (x σ )+ f ν k (x σν k; z), where the famly admts an exact potental functon fσ ν k ( ; xν ) satsfyng { } σ f ν k ( ; x ν ) σ ν k 17

18 Strong convexty: there exsts a constant η > 0 such that f σ ( x ν σν k k ; y) fσ ν k (xσν k ; y) + x σ f ( ) ν k σ ν k (xσν k ; y) T x σν k x σ η ν k + σν k x x σ ν k 2 2 for all x, x σν k X σν k, and y n X. Gradent consstency: x f (x) T (u x ) = for all u, x X, x X and σ ν k. ( x f σ ν k (, x σν k ; ; x) x ) T (u x ) 18

19 Convergence wth constant step-sze. Assume an exact potental functon P exsts; a scalar L > 0 exsts such that f (x) f (x ) L x x for all x, x X and all = 1,, n; a constant step-sze τ (0, 2η/L) s employed. Then, for an essentally coverng player selecton rule, every lmt pont of the terates generated by the unfed algorthm s a QNE of the game G. Same holds wth probablty one for the randomzed player selecton rule. Generalzed potental games: 2 more restrctons: Pont Gauss-Sedel,.e., each σ ν k s a sngleton; Tght upper-bound assumpton: θ σ ν(x σν ; y) θ σ ν(x σν ; y σν ) and θσ ν(x σν ; x) = θ σ ν(x σν ; x σν ), x, y X. 19

20 Concludng remarks We have ntroduced and analyzed the convergence of a unfed dstrbuted algorthm for computng a QNE of a mult-player game wth non-smooth, non-convex player objectve functons and wth decoupled convex constrants. The algorthm employs a famly of surrogate objectve functons to deal wth the non-convexty and non-dfferentablty of the orgnal objectve functons and solves subgames n parallel nvolvng determnstc or randomzed choce of non-overlappng groups of players. The convergence analyss s based on two approaches: contracton and potental; the former reles on a spectral condton whle the latter assumes the exstence of a potental functon. Extenson of the algorthm and analyss to games wth coupled convex constrants can be done by ntroducng multplers (or prces) of such constrants that are updated n an outer teraton. Non-convex constrants are presently beng researched. Thank you! 20