arxiv: v1 [cs.ds] 16 Jul 2015

Transcription

1 The Complexty of All-swtches Strategy Improvement John Fearnley and Rahul Savan Unversty of Lverpool arxv: v1 [cs.ds] 16 Jul 2015 Abstract. Strategy mprovement s a wdely-used and well-studed class of algorthms for solvng graph-based nfnte games. These algorthms are parametrzed by a swtchng rule, and one of the most natural rules s all swtches whch swtches as many edges as possble n each teraton. Contnung a recent lne of work, we study all-swtches strategy mprovement from the perspectve of computatonal complexty. We consder two natural decson problems, both of whch have as nput a game G, a startng strategy s, and an edge e. The problems are: 1. The edge swtch problem, namely, s the edge e, ever swtched by all-swtches strategy mprovement when t s started from s on game G? 2. The optmal strategy problem, namely, s the edge e used n the fnal strategy that s found by strategy mprovement when t s started from s on game G? We show PSPACE-completeness of the edge swtch problem and optmal strategy problem for the followng settngs: Party games wth the dscrete strategy mprovement algorthm of Vöge and Jurdzńsk; mean-payoff games wth the ganbas algorthm [11, 32]; and dscounted-payoff games and smple stochastc games wth ther standard strategy mprovement algorthms. We also show PSPACE-completeness of an analogous problem to edge swtch for the bottom-antpodal algorthm for Acyclc Unque Snk Orentatons on Cubes.

2 1 Introducton In ths paper we study strategy mprovement algorthms for solvng two-player games such as party games, mean-payoff games, dscounted games, and smple stochastc games [20,32,38]. These games are nterestng both because of ther mportant applcatons and ther unusual complexty status. Party games, for example, arse n several areas of theoretcal computer scence, for example, n relaton to the emptness problem for tree automata [5, 18] and as an algorthmc formulaton of model checkng for the modal µ-calculus [6,36]. Moreover, all of these problems are n NP conp, andevenup coup[2,22],sotheyareunlkelytobenp-complete. However, desptemucheffortfrom thecommunty, noneof theseproblemsare knownto benp, andwhether thereexsts apolynomaltme algorthm to solve these games s a very mportant and long-standng open problem. Strategy mprovement s a well-studed method for solvng these games [20,32,38]. It s an extenson of the well-known polcy teraton algorthms for Markov decson processes. The algorthm selects one of the two players to be the strategy mprover. Each strategy of the mprover has a set of swtchable edges, and swtchng any subset of these edges produces a strctly better strategy. So, the algorthm proceeds by frst choosng an arbtrary startng strategy, and then n each teraton, swtchng some subset of the swtchable edges. Eventually ths process wll fnd a strategy wth no swtchable edges, and t can be shown that ths strategy s an optmal strategy for the mprover. To completely specfy the algorthm, a swtchng rule s needed to pck the subset of swtchable edges n each teraton (ths s analogous to the pvot rule used n the smplex method). Many swtchng rules have been proposed and studed [8,19,25,26,33]. One of the most natural rules, and the one that we consder n ths paper, s the all-swtches rule, whch always swtches a vertex f t has a swtchable edge. In partcular, we consder greedy all-swtches, whch chooses the best edge whenever more than one edge s swtchable at a vertex (tes are broken arbtrarly). For a long tme, the all-swtches varant of the dscrete strategy mprovement algorthm of Vöge and Jurdznsk [38], was consdered the best canddate for a polynomal-tme algorthm to solve party games. Indeed no example was known that requred a super-lnear number of teratons. However, these hopes were dashed when Fredmann showed an exponental lower bound for greedy all-swtches strategy mprovement for party games [12]. In the same paper, he showed that hs result extends to strategy mprovement algorthms for dscounted games, and smple stochastc games. The computatonal power of pvot algorthms. In ths paper, we follow a recent lne of work that seeks to explan the poor theoretcal performance of pvotng algorthms usng a complexty theoretc pont of vew. The frst results n ths drecton were proved for problems n PPAD and PLS. It s know that, f a problem s tght-pls-complete then computng the soluton found by the natural mprovement algorthm s PSPACE-complete [21]. Smlarly, for the canoncal PPAD-complete problem end-of-the-lne, computng the soluton that s found by the natural lne followng algorthm s PSPACE-complete [30]. Ths has recently been extended to show that s PSPACE-complete to compute any of the solutons that can be found by the Lemke-Howson algorthm [17], a pvotng algorthm that solves the PPAD-complete problem of fndng a Nash equlbrum of a bmatrx game. Untl recently, results of ths type were only known for algorthms for problems that are unlkely tole np. However, arecent seres ofpapershasshownthat smlar resultsholdeven forthesmplex method for lnear programmng! Adler, Papadmtrou and Rubnsten [1] showed that there exsts an artfcally contrved pvot rule for whch s s PSPACE-complete to decde f a gven bass s on the path of bases vsted by the smplex algorthm wth ths pvot rule. Dsser and Skutella[4] studed the natural pvot rule that Dantzg proposed when he ntroduced the smplex method, and they showed that t s NP-hard to decde whether a gven varable enters the bass when the smplex method s run wth ths pvot rule. Fnally, Fearnley and Savan strengthened both these results by showng

3 that the decson problem that Dsser and Skutella consdered s actually PSPACE-complete [10], and they also showed that determnng f a gven varable s used n the fnal optmal soluton found by the algorthm s PSPACE-complete. Ths result exploted a known connecton between between sngle-swtch polcy teraton for Markov Decson Processes (MDPs) and the smplex method for a correspondng LP. The result was frst proved for a greedy varant of sngle-swtch polcy teraton, whch then mpled the result for the smplex method wth Dantzg s pvot rule. All of theresults on lnear programmngaremotvated by thequest to fndastrongly polynomal algorthm for lnear programmng, whch was ncluded n Smale s lst of great unsolved problems of the 21st century [35]. One way of resolvng ths problem would be to desgn a polynomal-tme pvot rule for the smplex method, and f we are to do ths, then t s crucal to understand why exstng pvot rules fal. The PSPACE-completeness ndcate that they fal not because they are stupd, but n fact because they are capable of solvng any problem that can be computed n polynomal space. We face a smlar quest to fnd a polynomal-tme algorthm for the games studed n ths paper. Strategy mprovement s a promnent algorthm for solvng these games, and ndeed t s one of the only algorthms for solvng dscounted and smple stochastc games. So, devsng a polynomaltme swtchng rule s an obvous drecton for further study. It may n fact be easer to devse a polynomal tme swtchng rule, because there s a lot more freedom n each step of the algorthm: smplex pvot rules correspond to swtchng rules that can only swtch a sngle edge, whereas strategy mprovement rules can swtch any subset of edges. Indeed, t may be the case that the polynomal Hrsch conjecture fals, rulng out a strongly polynomal smplex method, but the analogue of the Hrsch conjecture for strategy mprovement s known to be true: one can always reach an optmal strategy n at most n strategy mprovement teratons, where n s the number of vertces. Our contrbuton. Our man results are that, for greedy all-swtches strategy mprovement, determnng whether the algorthm swtches a gven edge s PSPACE-complete, and determnng whether the optmal strategy found by the algorthm uses a partcular edge s PSPACE-complete. One of the key features that strategy mprovement has s the ablty to swtch multple swtchable edges at the same tme, rather than just one as n the smplex method. Our results show that navely usng ths power does not help to avod the PSPACE-completeness results that now seem to be endemc among pvotng algorthms. The proof prmarly focuses on the strategy mprovement algorthm of Vöge and Jurdzńsk for solvng party games [38]. The followng defnton formalses the problem that we are nterested n. Defnton 1. Let G be a game, and let e be an edge and σ be a strategy profle of G. The problem EdgeSwtch(G,σ,e) s to decde f the edge e s ever swtched by greedy all-swtches strategy mprovement when t s appled to G startng from σ. The man techncal contrbuton of the paper s to show the followng theorem. Theorem 1. EdgeSwtch for Vöge and Jurdzńsk s algorthm s PSPACE-complete. We use ths theorem to show smlar results for other games. For mean-payoff games, our results apply to the gan-bas algorthm [11]; and for dscounted and smple stochastc games our results apply to the standard strategy mprovement algorthms [3, 31]. We utlse the well-known polynomal-tme reductons from party games to mean-payoff games [31], mean-payoff games to dscounted games, and from dscounted games to smple stochastc games [39]. The party games we construct have the property that when they are reduced to the other games mentoned above strategy mprovement wll behave n the same way (for dscounted and smple stochastc games ths was already observed by Fredmann [12]), so we get the followng corollary of Theorem 1.

4 Corollary 1. EdgeSwtch for the gan-bas algorthm, and the standard strategy mprovement algorthms for dscounted-payoff and smple stochastc games s PSPACE-complete. Theorem 1 proves a property about the path taken by strategy mprovement durng ts computaton. Prevous results have also studed the complexty of fndng the optmal strategy that s produced by strategy mprovement, whch we encode n the followng problem. Defnton 2. Let G be a game, and let e be an edge and σ be a strategy profle of G. The problem OptmalStrategy(G,σ,e) s to decde f the edge e s used n the optmal strategy that s found by greedy all-swtches strategy mprovement when appled to G startng from σ. Augmentng our constructon for party games wth an extra gadget gves the followng theorem. Theorem 2. OptmalStrategy for Vöge and Jurdzńsk s algorthm s PSPACE-complete. Ths result requres that the party games has multple optmal solutons because otherwse the PSPACE hardness of OptmalStrategy would mply NP conp = PSPACE. Wth further modfcatons, we can agan extend ths result to strategy mprovement algorthms for other games. Corollary 2. OptmalStrategy for the gan-bas algorthm, and the standard strategy mprovement algorthms for dscounted-payoff and smple stochastc games s PSPACE-complete. Our results can also be appled to unque snk orentatons (USOs.) A USO of a n-dmensonal hypercube s an orentaton of ts edges such that every face has a unque snk [37]. The fundamental algorthmc problem for USOs s to fnd the global snk assumng oracle access to the edge orentaton. The desgn and analyss of algorthms for ths problem s an actve research area [13,14,16,19,28,34], n partcular for acyclc orentatons (AUSOs). The BottomAntpodal algorthm [34] for AUSOs on cubes starts at an arbtrary vertex and n each teraton jumps to the antpodal vertex n the sub-cube spanned by the outgong edges at the current vertex. For bnary games, where vertces have outdegree at most two, and no two strateges have the same value, the valuaton functons used by strategy mprovement nduce an AUSO on a cube, and all-swtches strategy mprovement corresponds to BottomAntpodal on ths AUSO. For nonbnary games we nstead get an AUSO on a grd [15], so our results mmedately gve a PSPACEhardness result for grd USOs for a problem analogous to EdgeSwtch. To get a smlar result for AUSOs on cubes we turn our constructon nto a bnary party game, and we get the followng. Corollary 3. Let C be a d-dmensonal cube AUSO, specfed by a poly(d)-szed crcut that computes the edge orentatons for each vertex of C. Gven a dmenson k {1,...,d}, and a vertex v, t s PSPACE-complete to decde f BottomAntpodal started at v, ever swtches the kth coordnate. Snce a USO has a unque soluton, by defnton, we cannot hope to get a result for AUSOs that s analogous to OptmalStrategy, snce, as noted above, PSPACE-hardness of OptmalStrategy requres multple optmal solutons under standard complexty-theoretc assumptons. Related work. The best known algorthms for these games have subexponental runnng tme: the random facet strategy mprovement algorthms combne strategy mprovement wth the randomfacetalgorthmforlps[25 27].FollowngtheworkofFredmann[12]thatwebuldonheavly nths paper, Fredmann, Hansen, and Zwck showed a sub-exponental lower bound for the random facet strategy mprovement algorthm [13]. They also used a constructon of Fearnley [7] to extend the bound to the random facet pvot rule for the smplex method [14]. For party games, a determnstc subexponental-tme algorthm s known [23].

5 Roadmap. In Secton 2, we gve a formal defnton of party games, and more specfcally the one-snk games used by Fredmann that we also use for our constructon. We then gve a hghlevel overvew of how all-swtches strategy mprovement works for one-snk games, snce t operates more smply for one-snk games than for general games; a more detaled techncal exposton of allswtches strategy mprovement for one-snk games can be found n the appendx. Our man reducton starts wth a terated crcut evaluaton problem. In Secton 3, we descrbe our man constructon of a party game that wll mplement terated crcut teraton when strategy mprovement s run on t. In Secton 4, we descrbe the sequence of strateges that all-swtches strategy mprovement wll go through as t mplements the terated crcut evaluaton. In Secton 5 we show that the constructon works as clamed and thus prove that EdgeSwtch s PSPACE-hard for party games. In Secton 6, we show how ths result for EdgeSwtch extends to strategy mprovement algorthms for other games. In Secton 7, we show how to augment our constructon wth an extra gadget to gve PSPACE-hardness results for OptmalStrategy. In Secton 8, we state some open problems. 2 Prelmnares 2.1 Party games A party game s defned by a tuple G = (V,V Even,V Odd,E,pr), where (V,E) s a drected graph. The sets V Even and V Odd partton V nto the vertces belongng to player Even, and the vertces belongng to player Odd, respectvely. The prorty functon pr : V {1,2,...} assgns a postve natural number to each vertex. We assume that there are no termnal vertces, whch means that every vertex s requred to have at least one outgong edge. The strategy mprovement algorthm of Vöge and Jurdzńsk also requres that we assume, wthout loss of generalty, that every prorty s assgned to at most one vertex. A strategy for player Even s a functon that pcks one outgong edge for each Even vertex. More formally, a determnstc postonal strategy for Even s a functon σ : V Even V such that, for each v V Even we have that (v,σ(v)) E. Determnstc postonal strateges for player Odd are defned analogously. Throughout ths paper, we wll only consder determnstc postonal strateges, and from ths pont onwards, we wll refer to them smply as strateges. We use Σ Even and Σ Odd to denote the set of strateges for players Even and Odd, respectvely. A play of the game s an nfnte path through the game. More precsely, a play s a sequence v 0,v 1,... such that for all N we have v V and (v,v +1 ) E. Gven a par of strateges σ Σ Even and τ Σ Odd, and a startng vertex v 0, there s a unque play that occurs when the game starts at v 0 and both players follow ther respectve strateges. So, we defne Play(v 0,σ,τ) = v 0,v 1,..., where for each N we have v +1 = σ(v ) f v V Even, and v +1 = τ(v ) f v V Odd. Gven a play π = v 0,v 1,... we defne MaxIo(π) = max{p : nfntely many N s.t. pr(v ) = p}, to be the maxmum prorty that occurs nfntely often along π. We say that a play π s wnnng for player Even f MaxIo(π) s even, and we say that π s wnnng for Odd f MaxIo(π) s odd. A strategy σ Σ Even s a wnnng strategy for a vertex v V f, for every strategy τ Σ Odd, we have that Play(v,σ,τ) s wnnng for player Even. Lkewse, a strategy τ Σ Odd s a wnnng strategy for v f, for every strategy σ Σ Even, we have that Play(v,σ,τ) s wnnng for player Odd. The followng fundamental theorem states that party games are postonally determned. Theorem 3 ( [5,29]). In every party game, the set of vertces V can be parttoned nto wnnng sets (W 0,W 1 ), where Even has a postonal wnnng strategy for all v W 0, and Odd has a postonal wnnng strategy for all v W 1.

6 The computatonal problem that we are nterested n s, gven a party game, to determne the partton (W 0,W 1 ). Prortes. As we have mentoned, the strategy mprovement algorthm that we consder requres that every prorty s assgned to at most one vertex. Ths s unfortunately a rather cumbersome requrement when desgnng more complex constructons. To help wth ths, we defne a shorthand for specfyng prortes. Let c N, let,l {1,..., V }, let j {0,1,2}, and let e {0,1}. We defne P(c,,l,j,e) = 6 V 2 c+6 V +6 l+2 j +e. The frst four parameters should be thought of as a lexcographc orderng, whch determnes how large the prorty s. The fnal number e determnes whether the prorty s odd or even. Note that P(c,,l,j,e) s an njectve functon, so f we ensure that the same set of arguments are never used twce, then we wll never assgn the same prorty to two dfferent vertces. One thng to note s that, snce ths prorty notaton s rather cumbersome, t s not possble to use t n our dagrams. Instead, when we draw parts of the constructon, we wll use representatve prortes, whch preserve the order and party of the prortes used n the gadgets, but not ther actual values. 2.2 Strategy mprovement Valuatons. We now descrbe the strategy mprovement algorthm of Vöge and Jurdzńsk [38] for solvng party games, whch wll be the prmary focus of ths paper. The algorthm assgns a valuaton to each vertex v under every par of strateges σ Σ Even and τ Σ Odd. Let p be the largest prorty that s seen nfntely often along Play(v,σ,τ). Snce every prorty s assgned to at most one vertex, there s a unque vertex u wth pr(u) = p. We use ths vertex to decompose the play: let P(v,σ,τ) be the fnte smple path that starts at v and ends at u, and let C(v,σ,τ) be the nfntely-repeated cycle that starts at u and ends at u. We can now defne the valuaton functon Val σ,τ VJ (v) = (p,s,d) where p s as above and: S s the set of prortes on the fnte path that are strctly greater than p: S = {pr(u) : u P(v,σ,τ) and pr(u) > p}. d s the length of the fnte path: d = P(v,σ,τ). We now defne an order over valuatons. Frst we defne an order over prortes: we have that p q f one of the followng holds: p s odd and q s even. p and q are both even and p < q. p and q are both odd and p > q. Furthermore, we have that p q f ether p q or p = q. Next we defne an order of the sets of prortes that are used n the second component of the valuaton. Let P,Q N. We frst defne: MaxDff(P,Q) = max ( (P \Q) (Q\P) ). If d = MaxDff(P,Q) then we defne P Q to hold f one of the followng condtons holds: d s even and d Q. d s odd and d P.

7 Furthermore, we have that P Q f ether P = Q or P Q. Fnally, we can provde an order over valuatons. We have that (p,s,d) (p,s,d ) f one of the followng condtons holds: p p. p = p and S S. p = p and S = S and p s odd and d < d. p = p and S = S and p s even and d > d. Furthermore, we have that (p,s,d) (p,s,d ) f ether (p,s,d) (p,s,d ) or (p,s,d) = (p,s,d ). Best responses. Gven a strategy σ Σ Even, a best response aganst σ s a strategy τ Σ Odd such that, for every τ Σ Odd and every vertex v we have: Val σ,τ VJ (v) Valσ,τ VJ (v). Vöge and Jurdzńsk proved the followng propertes. Lemma 1 ( [38]). For every σ Σ Even a best response τ can be computed n polynomal tme. We defne Br(σ) to be an arbtrarly chosen best response strategy aganst σ. Furthermore, we defne Val σ VJ (v) = Valσ,Br(σ) (v). VJ Swtchable edges. Let σ be a strategy and (v,u) E be an edge such that σ(v) u. We say that (v,u)sswtchable nσ fval σ VJ (σ(v)) Valσ VJ (u).furthermore,wedefneamost appealng outgong edge at a vertex v to be an edge (v,u) such that, for all edges (v,u ) we have Val σ VJ (u ) Val σ VJ (u). There are two fundamental propertes of swtchable edges that underle the strategy mprovement technque. The frst property s that swtchng any subset of the swtchable edges wll produce an mproved strategy. Let σ be a strategy, and let W E be a set of swtchable edges n σ such that, for each vertex v, there s at most one edge of the form (v,u) W. Swtchng W n σ creates a new strategy σ[w] where for all v we have: σ[w](v) = We can now formally state the frst property. { u f (v,u) W, σ(v) otherwse. Lemma 2 ( [38]). Let σ be a strategy and let W E be a set of swtchable edges n σ such that, for each vertex v, there s at most one edge of the form (v,u) W. We have: For every vertex v we have Val σ VJ (v) Valσ[W] VJ (v). There exsts a vertex v for whch Val σ VJ (v) Valσ[W] VJ (v). The second property concerns strateges wth no swtchable edges. A strategy σ Σ Even s optmal f for every vertex v and every strategy σ Σ Even we have Val σ VJ (v) Valσ VJ (v). Lemma 3 ( [38]). A strategy wth no swtchable edges s optmal. Vöge and Jurdzńsk also showed that wnnng sets for both players can be extracted from an optmal strategy. If σ s an optmal strategy, then W 0 contans every vertex v for whch the frst component of Val σ VJ (v) s even, and W 1 contans every vertex v for whch the frst component of Val σ VJ (v) s odd. Hence, to solve the party game problem, t s suffcent to fnd an optmal strategy.

8 The algorthm. The two propertes that we have just descrbed gve rse to an obvous strategy mprovement algorthm that fnds an optmal strategy. The algorthm begns by selectng an arbtrary strategy σ Σ Even. In each teraton, the algorthm performs the followng steps: 1. If there are no swtchable edges, then termnate. 2. Otherwse, select a set W E of swtchable edges n σ such that, for each vertex v, there s at most one edge of the form (v,u) W. 3. Set σ := σ[w] and go to step 1. By the frst property, each teraton of ths algorthm produces a strctly better strategy accordng to the orderng, and therefore the algorthm must eventually termnate. However, the algorthm can only termnate when there are no swtchable edges, and therefore the second property mples that the algorthm wll always fnd an optmal strategy. The algorthm gven above does not specfy a complete algorthm, because t does not specfy whch subset of swtchable edges should be chosen n each teraton. Indeed, there are many varants of the algorthm that use a varety of dfferent swtchng rules. In ths paper, we focus on the greedy all-swtches swtchng rule. Ths rule swtches every vertex that has a swtchable edge, and f there s more than one swtchable edge, t arbtrarly pcks one of the most appealng edges. One-snk games. Fredmann observed that, for the purposes of showng lower bounds, t s possble to smplfy the Vöge-Jurdzńsk algorthm by restrctng the nput to be a one-snk game [12]. Aone-snk partygamecontans asnkvertex ssuchthat pr(s) = 1. Aneven strategy σ Σ Even s called a termnatng strategy f, for every vertex v, the frst component of Val σ VJ (v) s 1. Formally, a party game s a one-snk party game f: There s a vertex s V such that pr(s) = 1, and (s,s) s the only outgong edge from s. Furthermore, there s no vertex v wth pr(v) = 0. All optmal strateges are termnatng. Now, suppose that we apply Vöge-Jurdzńsk algorthm, and furthermore suppose that the ntal strategy s termnatng. Snce the ntal and optmal strateges are both termnatng, we have that, for every strategy σ vsted by the algorthm and every vertex v, the frst component of Val σ VJ (v) = 1, and so t can be gnored. Furthermore, snce there s no vertex wth prorty 0, the second component of Val σ VJ (v) must be dfferent from the second component of Valσ VJ (u), for every v,u V. Therefore, the thrd component of the valuaton can be gnored. Thus, for a one-snk game, we can defne a smplfed verson of the Vöge-Jurdzńsk algorthm that only uses the second component. So, we defne Val σ (v) to be equal to the second component of Val σ VJ (v), and we carry out strategy mprovement usng the defntons gven above, but wth Val σ (v) substtuted for Val σ VJ (v). Note, n partcular, that n ths strategy mprovement algorthm, and edge (v,u) s swtchable n σ f Val σ (σ(v)) Val σ (u). In our proofs, we wll frequently want to determne the maxmum dfference between two valuatons. For ths reason, we ntroduce the followng notaton. For every strategy σ, and every par of vertces v,u V, we defne MaxDff σ (v,u) = MaxDff(Val σ (v),val σ (u)). 2.3 Crcut teraton problems The problems. To prove our PSPACE-completeness results, we wll reduce from crcut teraton problems, whch we now defne. A crcut teraton nstance s a trple (F,B,z), where: F : {0,1} n {0,1} n s a functon represented as a boolean crcut C,

9 B {0,1} n s an ntal bt-strng, and z s an nteger such that 1 z n. We usestandard notaton for functon teraton: gven a bt-strng B {0,1} n, we recursvely defne F 1 (B) = F(B), and F (B) = F(F 1 (B)) for all > 1. We now defne two problems that wll be used as the startng pont for our reducton. Both are decson problems that take as nput a crcut teraton nstance (F, B, z). BtSwtch(F,B,z): decde whether there exsts an even 2 n such that the z-th bt of F (B) s 1. CrcutValue(F,B,z): decde whether the z-th bt of F 2n (B) s 1. The requrement for to be even n BtSwtch s a techncal requrement that s necessary n order to make our reducton to strategy mprovement work. The fact that these problems are PSPACE-complete should not be too surprsng, because F can smulate a sngle step of a space-bounded Turng machne, so when F s terated, t smulates a run of the space-bounded Turng machne. The followng lemma was shown n [10]. Lemma 4. [10, Lemma 7] BtSwtch and CrcutValue are PSPACE-complete. Crcuts. For the purposes of our reducton, we must make some assumptons about the format of the crcuts that represent F. Let C be a boolean crcut wth n nput bts, n output bts, and k gates. We assume, w.l.o.g., that all gates are or-gates or not-gates. The crcut wll be represented as a lst of gates ndexed 1 through n+k. The ndces 1 through n represent the n nputs to the crcut. Then, for each > n, we have: If gate s an or-gate, then we defne I 1 () and I 2 () to gve the ndces of ts two nputs. If gate s a not-gate, then we defne I() to gve the ndex of ts nput. The gates k+1 through k+n correspond to the n output bts of the crcut, respectvely. For the sake of convenence, for each nput bt, we defne I() = k+, whch ndcates that, f the crcut s appled to ts own output, nput bt should copy from output bt I(). Moreover, we assume that the gate orderng s topologcal. That s, for each or-gate we assume that > I 1 () and > I 2 (), and we assume that for each not-gate we have > I(). For each gate, let d() denote the depth of gate, whch s the length of the longest path from to an nput bt. So, n partcular, the nput bts are at depth 0. Observe that we can ncrease the depth of a gate by nsertng dummy or-gates: gven a gate, we can add an or-gate j wth I 1 (j) = and I 2 (j) =, so that d(j) = d()+1. We use ths fact n order to make the followng assumptons about our crcuts: For each or-gate, we have d(i 1 ()) = d(i 2 ()). There s a constant c such that, for every output bt {k +1,k +n}, we have d() = c. From now on, we assume that all crcuts that we consder satsfy these propertes. Note that, snce all outputs gates have the same depth, we can defne d(c) = d(k+1), whch s the depth of all the output bts of the crcut. Gven an nput bt-strng B {0,1} n, the truth values of each of the gates n C are fxed. We defne Eval(B,) = 1 f gate s true for nput B, and Eval(B,) = 0 f gate s false for nput B. Gven a crcut C, we defne the negated form of C to be a transformaton of C n whch each output bt s negated. More formally, we transform C nto a crcut C usng the followng operaton: for each output bt n+ n C, we add a Not gate n+k+ wth I(n+k+) = n+.

10 3 The Constructon Our goal s to show that EdgeSwtch s PSPACE-complete by reducng from the crcut teraton problem BtSwtch. Let (F,B,z) be the nput to the crcut teraton problem, and let C be the negated formofthecrcutthat computesf.throughoutthssecton, wewllusenas thebt-length of B, and k = C as the number of gates used n C. We wll use Or, Not, and Input/Output to denote the set of or-gates, not-gates, and nput/output-gates, respectvely. We wll force greedy all-swtches strategy mprovement to compute F (B) for each value of. To do ths, we wll represent each gate n C by a gadget, and applyng strategy mprovement to these gadgets wll cause the correct output values for each gate to be computed. One complcaton s that, after computng F(B), the crcut must then copy the output values back to the nput gates. To resolve ths ssue, the constructon contans two entre copes of the crcut, numbered 0 and 1, whch take turns n computng F. Frst crcut 0 computes F(B), then crcut 1 computes F(F(B)), then crcut 0 computes F(F(F(B))), and so on. There wll be a specfc edge e n the constructon that s swtched f and only f the z-th bt of F (B) s 1 for some even. The techncal requrement for to be even s explaned by the fact that there are two copes of the crcut, and each copy only computes F (B) for ether the case where s even or the case where s odd. The constructon also uses Fredmann s exponental-tme examples as a clock. Fredmann s examples are desgned to force greedy all-swtches strategy mprovement to mmc a bnary counter. The constructon contans two entre copes of Fredmann s example, so that each copy of the crcut s equpped wth ts own clock. The fundamental dea s that, each tme the clock tcks,.e., each tme the bnary counter advances to the next bt-strng, the crcut wll start computng F. Thus the two clocks must be out of phase relatve to one another, so that the two crcuts correctly alternate. In the rest of ths secton, we descrbe the constructon. We begn by gvng an overvew of Fredmann s example, both because t plays a key role n our constructon, and because the Not gate gadgets n our crcuts are a modfcaton of the bt-gadget used by Fredmann. We then move on to descrbe the gate gadgets, and how they compute the functon F. 3.1 Fredmann s exponental-tme example In ths secton we gve an overvew of some mportant propertes of Fredmann s exponental-tme examples. In partcular, we focus on the propertes that wll be mportant for our constructon. A more detaled descrpton of the example can be found n Fredmann s orgnal paper [12]. The example works by forcng greedy all-swtches strategy mprovement to smulate an n bt bnary counter. It conssts of two components: a bt gadget that s used to store one of the bts of the counter, and a deceleraton lane that s used to ensure that the counter correctly moves from one bt-strng to the next. The deceleraton lane. Fredmann s example contans one copy of the deceleraton lane. The deceleraton lane has a specfed length m, and Fgure 1 shows an example of a deceleraton lane of length 4. Fredmann s constructon contans one copy of the deceleraton lane of length 2n. Fgure 1 shows a deceleraton lane of length 4. Remember that our dagrams use representatve prortes, whch preserve the order and party of the prortes used, but not ther values. A key property of the deceleraton lane s that greedy all-swtches strategy mprovement requres m teratons to fnd the optmal strategy. Consder an ntal strategy n whch each vertex t uses the edge to r, and that the valuaton of r s always larger than the valuaton of s. Frst note that,

11 a 4 14 a 3 12 a 2 10 a 1 8 t 4 13 t 3 11 t 2 9 t 1 7 t 0 16 r s r s r s r s r s Fg. 1: Fredmann s deceleraton lane for length 4. snce there s a large even prorty on t 0, the optmal strategy s for every vertex t, wth 1, to use the edge to t 1. However, snce the vertces t wth 1 are all assgned odd prortes, n the ntal strategy only the edge from t 1 to t 0 s swtchable. Furthermore, once ths edge has been swtched, only the edge from t 2 to t 1 s swtchable. In ths way, the gadget ensures that m teratons are requred to move from the ntal strategy to the optmal strategy for ths gadget. Another mportant property s that the gadget can be reset. Ths s acheved by havng a sngle teraton n whch the valuaton of s s much larger than the valuaton of r, followed by another teraton n whch the valuaton of r s much larger than the valuaton of s. In the frst teraton all vertces t swtch to s, and n the second teraton all vertces swtch back to r. Note that after the second teraton, we have arrved back at the ntal strategy descrbed above. The bt gadget. The bt gadget s desgned to store one bt of a bnary counter. The constructon wll contan n copes of ths gadget, whch wll be ndexed 1 through n. Fgure 2 gves a depcton of a bt gadget wth ndex. The current value of the bt for ndex s represented by the choce that the current strategy makes at d. More precsely, for every strategy σ we have: If σ(d ) = e, then bt s 1 n σ. If σ(d ) e, then bt s 0 n σ. The Odd vertex e plays a crucal role n ths gadget. If σ(d ) = e, then Odd s best response s to useedge (e,h ), to avod creatng the even cycle between d ande. Onthe other hand,f σ(d ) e, then Odd s best response s to use (e,d ), to avod seeng the large even prorty at h. One thng to note s that, n the case where σ(d ) e, the edge to e s always swtchable. To prevent d from mmedately swtchng to e, we must ensure that there s always a more appealng outgong edge from e, so that the greedy all-swtches rule wll swtch that edge nstead. The edges from d to the deceleraton lane provde ths. Once t 1 has swtched to t 0, the edge from d to a 1 becomes more appealng than the edge to e, once t 2 has swtched to t 1, the edge from d to a 2 becomes more appealng than the edge to e, and so on. In ths way, we are able to prevent d from swtchng to e for 2 teratons by provdng outgong edges to the frst 2 vertces of the deceleraton lane. The vertces s and r. The vertex s has outgong edges to every vertex f n the bt gadgets, and the vertex r has outgong edges to every vertex g n the bt gadgets. If s the ndex of the least sgnfcant 1 bt, then s chooses the edge to f and r chooses the edge to g. The prorty assgned

12 g +1. k 13 g n x h 16 g 5 s r d 3 e 4 f 15 a 2... a 2 a 1 Fg. 2: An example of Fredman s bt gadget.

13 to r s larger than the prorty assgned to s, whch ensures that the valuaton of r s usually larger than the valuaton of s, as requred to make the deceleraton lane work. When the counter moves from one bt-strng to the next, the ndex least sgnfcant 1 changes to some. The vertex s swtches to f one teraton before the vertex r swtches to g. Ths creates the sngle teraton n whch the valuaton of s s larger than the valuaton of r, whch resets the deceleraton lane. Smulatng a bnary counter. To smulate a bnary counter, we must do two thngs. Frstly, we must ensure that f the counter s currently at some bt-strng K {0,1} n, then the least sgnfcant zero n K mustbeflppedto a one. Secondly, once ths has been done, all bts whosendex s smaller than the least sgnfcant zero must be set to 0. If these two operatons are always performed, then strategy mprovement wll ndeed count through all bnary strngs. The least sgnfcant zero s always flpped because each bt has 2 edges to the deceleraton lane. Snce the purpose of the deceleraton lane s to prevent the vertex d swtchng to e, the vertex d where s the ndex of the least sgnfcant zero, s the frst to run out of edges, and subsequently swtch to e. Once ths has occurred, all bts wth ndex smaller than the least sgnfcant zero are set to 0 due to the followng chan of events. The vertex s swtches f, and then the vertex d n all bts wth ndex < wll be swtched to s. Snce d no longer uses the edge to e, the bt has now been set to 0. Our modfcatons to Fredmann s example. In order to use Fredmann s example as a clock, we make a few mnor adjustments to t. Frstly, we make the deceleraton lane longer. Fredmann s example uses a deceleraton lane of length 2n, but we use a deceleraton lane of length 2k+4n+6. Furthermore, whle the vertex d has outgong edges to each a j wth j 2 n Fredmann s verson, n our modfed verson the vertex d has outgong edges to each a j wth j 2+2k +2n+6. The reason for ths s that Fredmann s example can move from one bt-strng to the next n as lttle as four teratons, but we need more tme n order to compute the crcut F. By makng the deceleraton lane longer, we slow down the constructon, and ensure that there are at least 2k +2n+6 teratons before the clock moves from one bt-strng to the next. The second change that we make s to change the prortes, because we need to make room for the gadgets that we add later. However, we have not made any fundamental changes to the prortes: the orderng of prortes between the vertces and ther party s mantaned. We have smply added larger gaps between them. The followng table specfes the verson of the constructon that we use. Observe that two copes are specfed: one for j = 0 and the other for j = 1. Furthermore, observe that the vertex x wll be the snk n our one-snk game.

14 Vertex Condtons Edges Prorty Player t j 0 j {0,1} r j, s j P(2,0,2k +4n+4,j,0) Even t j l j {0,1}, 1 l 2k +4n+6 r j, s j, t j l 1 P(2,0,l,j,1) Even a j l j {0,1}, 1 l 2k +4n+6 t j l P(2,0,l+1,j,0) Even d j j {0,1}, 1 n e j, sj, r j, P(1,,0,j,1) Even a j l for 1 l 2k +2n+6+2 e j j {0,1}, 1 n h j, d P(1,,1,j,0) Odd g j j {0,1}, 1 n f j P(1,,2,j,1) Even k j j {0,1}, 1 n x, g j l, for < l n P(8,,0,j,1) Even f j j {0,1}, 1 n e j P(8,,1,j,1) Even h j j {0,1}, 1 n k j P(8,,2,j,0) Even s j j {0,1} x, f j l for 1 l n P(7,0,0,j,0) Even r j j {0,1} x, g j l for 1 l n P(7,0,1,j,0) Even x x P(0,0,0,0,1) Even 3.2 Our constructon Crcuts. For each gate n the constructon, we desgn a gadget that computes the output of that gate. The dea s that greedy all-swtches strategy mprovement wll compute these gates n depth order. Startng from an ntal strategy, the frst teraton wll compute the outputs for all gates of depth 1, the next teraton wll use these outputs to compute the outputs for all gates of depth 2, and so on. In ths way, after k teratons of strategy mprovement, the outputs of the crcut wll have been computed. We then use one addtonal teraton to store these outputs n an nput/output gadget. Strategy mprovement valuatons wll be used to represent the output of each gate. Each gate has a state o j, and the valuaton of ths state wll ndcate whether the gate evaluates to true or false. In partcular the followng rules wll be followed. Property 1 In every strategy σ we have the followng propertes. 1. Before the gate has been evaluated, we wll have Val σ (o j ) Valσ (r j ). 2. If the gate has been evaluated to false, we wll contnue to have Val σ (o j ) Valσ (r j ). 3. If the gate has been evaluated to true, then we wll nstead have Val σ (r j ) Val σ (o j ), and MaxDff σ (r j,o j ) wll be a large even prorty. The nput/output gadgets are connected to both crcuts, and these gadgets have two modes. 1. When crcut j s computng, the gadget s n output mode, where t reads the output of crcut j and stores t. 2. When crcut 1 j s computng, the gadget s n nput mode, where t outputs the value that was stored from the prevous computaton nto crcut 1 j. Therefore, the gates of depth 1 n crcut j read ther nputfrom the nput/output gadgets n crcut 1 j, whle the nput/output gadgets n crcut j read ther nput from the outputs of crcut j. To formalse ths, we ntroduce the followng notaton. For every Not-gate, we defne InputState(, j) as follows: { o 1 j I() f d() = 1, InputState(, j) = o j I() f d() > 1. For every Or-gate, and every l {1,2}, we defne InputState(,j,l) as follows: { o 1 j I InputState(, j, l) = l () f d() = 1, o j I l () f d() > 1.

15 The clocks. As we have mentoned, we use two copes of Fredmann s example to act as clocks n our constructon. These clocks wll be used to drve the computaton. In partcular, the vertces r j and s j wll play a crucal role n synchronsng the two crcuts. As descrbed n the prevous secton, when the clock advances,.e., when t moves from one bt-strng to the next, there s a sngle teraton n whch the valuaton of s j s much larger than the valuaton of r j. Ths event wll trgger the computaton. The teraton n whch the valuaton of s 0 s much larger than the valuaton of r 0 wll trgger the start of computaton n crcut 0. The teraton n whch the valuaton of s 1 s much larger than the valuaton of r 1 wll trgger the start of computaton n crcut 1. In order for ths approach to work, we must ensure that the two clocks are properly synchronsed. In partcular, the gap between computaton startng n crcut j and computaton startng n crcut 1 j must be at least k +3, to gve enough tme for crcut j to compute the output values, and for these values to be stored. We now defne notaton for ths purpose. Frst we defne the number of teratons that t takes for a clock to move from bt-strng K to K + 1. For every bt-strng K {0,1} n, we defne Lsz(K) to be the ndex of the least sgnfcant zero n K: that s, the smallest ndex such that K = 0. For each K {0,1} n, we defne: ( ) Length(K) = 2k +2n+6 +2Lsz(K)+5. Ths term can be understood as the length of the deceleraton lane to whch all bts n the clock have edges, plus the number of extra teratons t takes to flp the least-sgnfcant zero, plus fve extra teratons needed to transton between the two bt-strngs. Next we ntroduce the followng delay functon, whch gves the amount of tme each crcut spends computng. For each j {0,1} and each K {0,1} n, we defne: ( ) d(c)+3 +2n f j = 0, Delay(j, K) = ( ) d(c)+3 +2 Lsz(K)+5 f j = 1. Crcut 1 starts computng Delay(0,K) teratons after Crcut 0 started computng, and Crcut 0 starts computng Delay(1,K) teratons after crcut 1 started computng. Observe that Delay(0,K) +Delay(1,K) = Length(K), whch ensures that the two crcuts do not drft relatve to each other. The term d(c)+3 n each of the delays ensures that there s always enough tme to compute the crcut, before the next crcut begns the subsequent computaton. Or gates. The gadget for a gate Or s qute smple, and s shown n Fgure 3. It s not dffcult to verfy that the three rules gven n Property 1 hold for ths gate. Before both nputs have been evaluated, the best strategy at o j s to move drectly to r j, snce the valuaton of both nputs s lower than the valuaton of r j. Note that n ths confguraton the valuaton of o j s smaller than the valuaton of r j, snce o j has been assgned an odd prorty. Snce, by assumpton, both nputs have the same depth, they wll both be evaluated at the same tme. If they both evaluate to false, then nothng changes and the optmal strategy at o j wll stll be r j. Ths satsfes the second rule. On the other hand, f at least one nput evaluates to true, then the optmal strategy at o j s to swtch to the correspondng nput states. Snce the valuaton of ths nput state s now bgger than r j, the valuaton of o j wll also be bgger than rj, so the thrd rule s also satsfed.

16 s j r j o j 1 Vertex Condtons Edges Prorty Player o j j {0,1}, s j, r j, P(4,,0,j,1) Even Or InputState(, j, 1) InputState(, j, 2) o j I 1 () o j I 2 () Fg.3: The Or gate. Not gates. The constructon for a gate Not s more nvolved. The gadget s qute smlar to a bt-gadget from Fredmann s constructon. However, we use a specal modfed deceleraton lane, whch s shown n Fgure 4. a j,d()+1 a j,d() a j,d() 1... t j,d()+1 t j,d() t j,d() 1... r j s j o j I() r j s j Fg.4: Modfed deceleraton lane for a Not gate n crcut j. The modfed deceleraton lane s almost dentcal to Fredmann s deceleraton lane, except that state t j,d() s connected to the output state of the nput gate. The dea s that, for the frst d() 1 teratons the deceleraton lane behaves as normal. Then, n teraton d(), the nput gate s evaluated. If t evaluates to true then the valuaton of t j,d() wll be large, and the deceleraton lane contnues swtchng as normal. If t evaluates to false, then the valuaton of t j,d() wll be low, and the deceleraton lane wll stop swtchng. The Not gate gadget, whch s shown n Fgure 5 s a smplfed bt gadget that s connected to the modfed deceleraton lane. As n Fredmann s constructon, the strategy chosen at d j wll represent the output of the gate. In a strategy σ, the gate outputs 1 f σ(d j ) = ej, and t outputs 0 otherwse. As we know, Fredmann s bt gadget s dstracted from swtchng d j to e j by the deceleraton lane. By usng the modfed deceleraton lane, we nstead obtan a Not gate. Snce the deceleraton lane keeps on swtchng f and only f the nput gate evaluates to true, the state

17 r j h j 16 s j d j 3 e j 4 o j 15 a j,m... a j,2 a j,1 Fg.5: Not gate wth ndex n crcut j. d j wll swtch to ej n teraton d() f and only f the nput gate evaluates to false. Ths s the key property that makes the Not gate work. To see that the three rules specfed n Property 1 are respected, observe that there s a large odd prorty on the state o j, and an even larger even prorty on the state hj. Ths causes the valuaton of o j wll only be larger than the valuaton of rj f and only f d j chooses the edge to ej, whch only happens when the gate evaluates to true. Fnally, when the computaton n crcut j begns agan, the Not-gate s reset. Ths s ensured by gvng the vertex d j edges to both sj and r j. So, when the clock for crcut j advances, no matter what strategy s currently chosen, the vertex d j frst swtches to sj, and then to r j, and then begns swtchng to the deceleraton lane. The followng table formally specfes the Not gate gadgets that we use n the constructon. Vertex Condtons Edges Prorty Player t j,0 j {0,1}, Not r j, s j P(5,,2k +4n+4,j,0) Even t j,l j {0,1}, Not, r j, s j, t j,l 1 P(5,,l,j,1) Even 1 l 2k +4n+6, and l d() t j,d() j {0,1}, Not InputState(,j) P(5,,d(),j,1) Even a j,l j {0,1}, Not, t,l P(5,,l+1,j,0) Even 1 l 2k +4n+6, and l d() a j,d() j {0,1}, Not t,d() P(4,,0,j,0) Even d j j {0,1}, Not s j, r j, e j, aj,l P(4,,0,j,1) Even for 1 l 2k +4n+6 e j j {0,1}, Not h j, dj P(4,,1,j,0) Odd o j j {0,1}, Not e j P(6,,0,j,1) Even h j j {0,1}, Not r j P(6,,1,j,0) Even Input/output gates. For each nput-bt n each copy of the crcut, have an nput/output gate. Recall that these gadgets have two modes. When Crcut j s computng, the nput/output gadgets n crcut j are n output mode, n whch they store the output of the crcut, and the nput/output

18 r j s j y j : 4 z j : 2 r 1 j r j s j Fg.6: Crcut mover gadget for crcut j. Left: the edges to r j and s j n a vertex from a Not-gate. Rght: the replacement for these outgong edges. gadgets n crcut 1 j are n nput mode, n whch they output the value that was stored n the prevous computaton. At ts core, the nput/output gadget s smply another copy of the Not-gate gadget that s connected to the th output bt of crcut j. However, we modfy the Not-gate gadget by addng n extra vertces that allow t to be moved between the two crcuts. The most mportant part of ths crcut mover apparatus s shown n Fgure 6: all of the vertces n the Not-gate gadget that have edges to r j and s j are modfed so that they nstead have edges to y j and z j. Fgures 7 and 8 show the Input/Output gadget and ts assocated modfed deceleraton lane, respectvely. There are three dfferences between ths gadget and the Not gate, whch are the ncluson of the vertces h j,, the vertces pj and pj,1, and the vertces qj,. All of these vertces are nvolved n the operaton of movng the gadget between the two crcuts. When the gadget s n output mode, the vertex y j chooses the edge to r j, the vertex h j,0 chooses the edge to h j,1, and the vertex pj chooses the edge to oj I(). When these edges are chosen, the gadget s essentally the same as a Not-gate at the top of the crcut. So, once the crcut has fnshed computng, the vertex d j chooses the edge to ej (.e., the stored bt s 1) f and only f the th output from the crcut was a 0. Snce the crcut was gven n negated form, the gadget has therefore correctly stored the th bt of F(B). Throughout the computaton n crcut j, the valuaton of r j s much larger than the valuaton of r 1 j. The computaton n crcut 1 j begns when the clock n crcut 1 j advances, whch causes the valuaton of r 1 j to become much larger than the valuaton of r j. When ths occurs, the nput/output gate then transtons to nput mode. The transton nvolves the vertex y j swtchng to r 1 j, the vertex h j,0 swtchng to hj,2, and the vertex pj swtchng to pj,1. Moreover, the player Odd vertex q j,0 swtches ej. Ths vertex acts as a crcut breaker, whch makes sure that the output of the gadget s only transmtted to crcut 1 j when the gadget s n nput mode. The key thng s that all of these swtches occur smultaneously n the same teraton. Snce strategy mprovement only cares about the relatve dfference between the outgong edges from

19 r j h j,1 30 r 1 j 4 r 1 j h j,2 12 h j,0 2 q j,1 32 y j z j d j e j q j o j, a j,m... a j,2 a j,1 Fg.7: Input/Output gate wth ndex n crcut j. a j,d(c)+1 a j,d(c) a j,d(c) 1... t j,d(c)+1 t j,d(c) t j,d(c) 1... y j z j y j z j r 1 j p j,1 14 p j 2 o j I() Fg. 8: Modfed deceleraton lane for an Input/Output gate n crcut j.

20 the vertex, and snce all edges leavng the gadget swtch at the same tme, the operaton of the Not-gate s not nterrupted. So, the strategy chosen at d j wll contnue to hold the th bt of F(B), and the gadget has transtoned to nput mode. When the gadget s n nput mode, t can be vewed as a Not at the bottom of crcut 1 j that has already been computed. In partcular, the swtch from h j,1 to hj,2 ensures that, f the output s 1, then the gadget has the correct output prorty. Moreover, the deceleraton lane has enough states to ensure that, f the output s 0, then output of the gadget wll not flp from 0 to 1 whle crcut 1 j s computng. Fnally, once crcut 1 j has fnshed computng, the clock for crcut j advances, and the nput/output gadget moves back to output mode. Ths nvolves resettng the Not gate gadget back to ts ntal state. Ths occurs because, when the clock n crcut j advances, there s a sngle teraton n whch the valuaton of s j s hgher than the valuaton of r j. Ths causes z j to swtch to s j whch n turn causes a sngle teraton n whch the valuaton of z j s hgher than the valuaton of y j. Then, n the next teraton the vertex y j swtches to r j, and so the valuaton of y j s then larger than the valuaton of z j. So, the valuatons of y j and z j gve exactly the same sequence of events as r j and s j, whch allows the Not-gate to reset. The followng table specfes the nput/output gadget. Vertex Condtons Edges Prorty Player y j j {0,1} r 1 j, r j P(3,0,1,j,0) Even z j j {0,1} r j, s j P(3,0,0,j,0) Even t j,0 j {0,1}, Input/Output y j, z j P(5,,2k +4n+4,j,0) Even t j,l j {0,1}, Input/Output, y j, z j, t j,l 1 P(5,,l,j,1) Even 1 < l 2k +4n+6, and l d(c) t j,d(c) j {0,1}, Input/Output p j P(5,,d(C),j,1) Even p j j {0,1}, Input/Output o j I(), pj,1 P(3,,2,j,0) Even p j,1 j {0,1}, Input/Output r 1 j P(5,,2k +4n+5,j,0) Even a j,l j {0,1} Input/Output, t,l P(5,,l+1,j,0) Even 1 l 2k +4n+6 d j j {0,1}, Input/Output y j, z j, e j, P(4,,0,j,1) Even a j,l for 1 l 2k +4n+6 e j j {0,1}, Input/Output h j,0, dj P(4,,1,j,0) Odd q j,0 j {0,1}, Input/Output e j, qj,1 P(4,,2,j,0) Odd q j,1 j {0,1}, Input/Output r 1 j P(6,d(C)+2,0,j,0) Even o j j {0,1}, Input/Output q j,0 P(6,,0,j,1) Even h j,0 j {0,1}, Input/Output h j,1, hj,2 P(3,,3,j,0) Even h j,1 j {0,1}, Input/Output r j P(6,d(C)+1,1,j,0) Even h j,2 j {0,1}, Input/Output r 1 j P(6,0,1,j,0) Even One-snk game. If we are to use the smplfed strategy mprovement algorthm, we must frst show that ths constructon s a one-snk game. We do so n the followng lemma. Lemma 5. The constructon s a one-snk game. Proof. Inordertoshowthattheconstructonsaone-snkgame,wemustshowthatthetworequred propertes hold. Frstly, we must show that there s a vertex the satsfes the requred propertes of a snk vertex. It s not dffcult to verfy that vertex x does ndeed satsfy these propertes: the only outgong edge from x s the edge (x,x), and we have pr(x) = P(0,0,0,0,0) = 1. Furthermore, no vertex s assgned prorty 0. Secondly, we must argue that all optmal strateges are termnatng. Recall that a termnatng strategy has the property that the frst component of the Vöge-Jurdzńsk valuaton s 1, whch mples that all paths startng at all vertces eventually arrve at the snk x. So, consder a strategy