15-451/651: Design & Analysis of Algorithms January 22, 2019 Lecture #3: Amortized Analysis last changed: January 18, 2019

Transcription

1 5-45/65: Desgn & Analyss of Algorthms January, 09 Lecture #3: Amortzed Analyss last changed: January 8, 09 Introducton In ths lecture we dscuss a useful form of analyss, called amortzed analyss, for problems n whch one must perform a seres of operatons, and our goal s to analyze the tme per operaton. The motvaton for amortzed analyss s that lookng at the worst-case tme per operaton can be too pessmstc f the only way to produce an expensve operaton s to set t up wth a large number of cheap operatons beforehand. We also dscuss the use of a potental functon whch can be a useful ad to performng ths type of analyss. A potental functon s much lke a bank account: f we can take our cheap operatons (those whose cost s less than our bound) and put our savngs from them n a bank account, use our savngs to pay for expensve operatons (those whose cost s greater than our bound), and somehow guarantee that our account wll never go negatve, then we wll have proven an amortzed bound for our procedure. As n the prevous lecture, n ths lecture we wll avod use of asymptotc notaton as much as possble, and focus nstead on concrete cost models and bounds. Informally amortzed cost of an operaton n a sequence of operatons s the total cost of all of them dvded by the number of operatons (.e. the average cost of an operaton). We begn by presentng two examples: the bnary counter, and growng a table. In these examples the operatons have large worst-case cost, but constant amortzed cost. We then ntroduce potental functons and show how these same two examples can be analyzed usng potentals. Fnally we wll apply the potental functon method to analyze the problem of a table that both grows and shrnks. Frst Example: A Bnary Counter Imagne we want to store a bg bnary counter n an array A. All the entres start at 0 and at each step we wll be smply ncrementng the counter. Let s say our cost model s: whenever we ncrement the counter, we pay for every bt we need to flp. (So, thnk of the counter as an array of heavy stone tablets, each wth a 0 on one sde and a on the other.) For nstance, here s a trace of the frst few operatons and ther cost: A[m] A[m-]... A[3] A[] A[] A[0] cost In a sequence of n ncrements, the worst-case cost per ncrement s O(log n), snce at worst we flp

2 lg(n) + bts. But, what s our amortzed cost per ncrement? The answer s t s at most. Here s a proof: Proof: How often do we flp A[0]? Answer: every tme. How often do we flp A[]? Answer: every other tme. How often do we flp A[]? Answer: every 4th tme, and so on. So, the total cost spent on flppng A[0] s n, the total cost spent flppng A[] s at most n/, the total cost flppng A[] s at most n/4, etc. Summng these up, the total cost spent flppng all the postons n our n ncrements s at most n. So f we dstrbute ths cost evenly over all n operatons we see that the average cost s at most per operaton. 3 Second Example: Growng a Table A common problem n data structure desgn arses f you re usng an array (whch we ll call a table) to store somethng (a stack, a hash table, a vector n C++, etc.), and you fnd out that you need more space. The rule of thumb n these cases s to double the sze of the table. Ths s expensve because you have to allocate a lot of space, and move all the data over to the new area. Despte ths, we ll see that the amortzed cost s O(). We can set up a framework for analyzng ths problem as follows. At any pont n tme the sze of the table s denoted by n, and the number of elements used n the table s denoted by s. The followng API defnes the way the clent wll use the table. ntalze(): create a new table of sze wth nothng n t. (n = and s = 0) nsert(): add a new element to the table. (ncrement s) It wll be useful to defne the followng operaton: grow(): Double the sze of the table from n to n. The cost of ths operaton s n, the new sze of the table. (Ths cost pays for allocatng the new table and movng all of the data from the old table to the new one.) Now nsert() wll be mplemented as follows: nsert(): If s = n then grow(). Now put the new element nto the table at a cost of. (ncrement s) After an ntalze(), what s the cost of a sequence of m nserts nto the table? Let s let N denote the sze of the table at the end of ths process. The total work of all the nsert() operatons, not countng the grow() costs s m. The total work of all the grow() operatons s N < N. But N/ < m, because otherwse the table would never have grown to sze N. So the total cost s at most m + 4m = 5m. So we can say that the amortzed cost of an operaton s at most 5. 4 Potentals So far so good, but we re gong to need a better way to keep track of thngs for more complex problems. And the way to do that s wth potental functons, as we show n ths secton. But frst, here s what you mght call the Banker s Proof of the amortzed bound of on the cost of ncrementng a bnary number. These costs, and the ones that follow n the rest of these notes, are specfed here for concreteness and clarty of analyss. All of ths s somewhat arbtrary. Changes n cost by a constant factor wll only effect the fnal result by a constant factor.

3 Banker s Proof of Bnary Counter: Every tme you flp 0, pay the actual cost of $, plus put $ nto a pggy bank. So the total amount spent s $. In fact, thnk of each bt as havng ts own bank (so when you turn the stone tablet from 0 to, you put a $ con on top of t). Now, every tme you flp a 0, use the money n the bank (or on top of the tablet) to pay for the flp. Clearly, by desgn, our bank account cannot go negatve. The key pont now s that even though dfferent ncrements can have dfferent numbers of 0 flps, each ncrement has exactly one 0 flp. So, we just pay $ (amortzed) per ncrement. Equvalently, what we are dong n ths proof s usng a potental functon that equals the number of -bts n the current count. Notce how the bank-account/potental-functon allows us to smooth out our payments, makng the cost easer to analyze. Ths technque can be appled n a much more general way. The dea s to make a rule that says how much money must be kept n the bank as a functon of the state of the data structure. Then a bound s obtaned on how much money s requred to pay for an operaton and mantan the approprate amount of money n the bank. The physcst s vew of amortzaton uses dfferent termnology to descrbe the same dea. Ths s the formulaton we wll generally use n ths course. A potental functon Φ(s) s a mappng from data-structure states to the reals. (Ths takes the place of the bank account n the banker s vew.) Consder a sequence of n operatons σ, σ,..., σ n the data structure. Let the sequence of states through whch the data structure passes be s 0, s,..., s n. Notce that operaton σ changes the state from s to s. Let the cost of operaton σ be c. Defne the amortzed cost ac of operaton σ by the followng formula: ac = c + Φ(s ) Φ(s ), () or (amortzed cost) = (actual cost) + (change n potental). If we sum both sdes of ths equaton over all the operatons, we obtan the followng formula: ac = (c + Φ(s ) Φ(s )) = Φ(s n ) Φ(s 0 ) + c. Rearrangng we get c = If Φ(s 0 ) Φ(s n ) (as wll frequently be the case) we get ( ) ac + Φ(s 0 ) Φ(s n ). () c ac. (3) Thus, f we can bound the amortzed cost of each of the operatons, and the fnal potental s at least as large as the ntal potental, then the bound we obtaned for the amortzed cost apples to the actual cost. Potental Functon for Bnary Counter: We can now apply ths technque to the problem of computng the cost of bnary countng. Let the potental Φ be the number of s n the current number. Our frst goal s to show that wth ths potental the amortzed cost of an ncrement operaton s. 3

4 Consder the th ncrement operaton that changes the number from to. Let k be the number of carres that occur as a result of the ncrement. The cost of the operaton s k +. The change n potental caused by the operaton s k +. (The number of bts that change from to 0 s k and one bt changes from 0 to.) Therefore the amortzed cost of the operaton s ac = k + + ( k + ) =. Snce the fnal potental s more than the ntal potental, we can apply nequalty (3) to obtan: c ac = n. Notce that the defnton of the amortzed cost of an operaton depends on the choce of the potental functon Φ. Any choce of potental functon whatsoever defnes an amortzed cost of each operaton. However, these amortzed bounds wll not be useful unless Φ(s 0 ) Φ(s n ) s also bounded approprately. We have gven two dfferent defntons of amortzed cost, the frst n Secton, and the other n equaton (). Whch defnton apples n a dscusson wll depend on the context of the dscusson. If we dscuss amortzed cost n the context of a potental functon, then the amortzed cost s that defned by equaton (). If t s outsde the context of a potental functon, then the meanng of amortzed cost s that gven n Secton. Most of the art of dong an amortzed analyss s n choosng the rght potental functon. Once a potental functon s chosen we must do two thngs:. Prove that wth the chosen potental functon, the amortzed costs of the operatons satsfy the desred bounds.. Bound the quantty Φ(s 0 ) Φ(s n ) approprately. 5 Growng a Table Revsted Let s do the analyss of a growng table usng potentals. Here s the potental functon: { 0 f s n Φ(n, s) = 4(s n ) otherwse 4

5 The frst graph above shows Φ() as a functon of s (n s fxed). The second graph shows what happens after n doubles. Ths happens when s = n, so you can see that the value of the functon goes from n to 0 as a result. Lemma The amortzed cost of an nsert() nto a table usng the above potental functon s at most 5. Also, the total cost of a sequence of m nsert() operatons startng from n =, s = 0 s at most 5m. Proof: An ncrement s comprsed of two parts. The frst part s the condtonal grow() operaton that mght be done. The second part s puttng the new element n, and ncreasng s. We wll analyze these separately. Frst consder grow(). A grow() happens when s = n, so the value of the potental s n. After the grow, n has been doubled, so s = n/ and the new potental s 0. So Φ = n. The actual cost of grow() s n. Addng these together shows that the amortzed cost of grow() s 0. The remanng part of the nsert() costs, and causes s to ncrease by. The change n potental s 4. Thus, the amortzed cost s 5. The ntal potental s 0, and the fnal potental s 0 therefore usng equaton (3) above we get: total cost total amortzed cost = 5m. Exercse : Gve a proof for ths lemma usng the banker s method. 6 Growng and Shrnkng a Table A data structure that supports deletes can both grow and shrnk n sze. It would be nce f the sze that t occupes s not too much bgger than necessary. Ths s where shrnkng a table s useful. As before, at any gven tme the sze of the current array wll be denoted by n and the number of thngs n the table wll be denoted by s. The nterface has the followng operatons. ntalze(): create a new table of sze wth nothng n t. (n = and s = 0) nsert(): add a new element to the table. (ncrement s) delete(): delete the gven element from the table. (decrement s) (Other operatons, e.g., lookup(), are mmateral for our purposes today.) To mplement the above operatons, we ll need two other prmtves to deal wth the array: grow(): Change the sze of the table from n to n. The cost of ths operaton s n, the new sze of the table. shrnk(): Change the sze of the table from n to n/. The cost of the operaton s n. Usng these prmtves, here s how we mplement the nterface. ntalze(): create a new table of sze wth nothng n t. (n = and s = 0) nsert(): f s = n then grow(). Now nsert the element nto the table. (Cost of ths part s.) 5

6 delete(): f s = n/4 and n 4 then shrnk(). Now delete the element from the table. (Cost of ths part s.) There s a lttle bt of subtlety n ths desgn. The stuaton mmedately after a grow() or a shrnk() s that s = n/. The key thng s that rght after one of these expensve operatons, the system s very far from from havng to do another expensve operaton. Ths allows t tme to buld up ts pggy bank to pay for the next expensve operaton. Exercse : If we change delete() to shrnk when s = n/, show a sequence of operatons that ncur large amortzed cost. It also has what s known as hysteress n physcs. Ths s because the value of n s not purely a functon of s. The value of n depends on the hstory of the values of s over tme. The followng fgure shows what happens as s goes from 0 to 5 then back down to, then up to 3. The ponts t goes through (n order) are: (0, ), (, ), (, ), (3, 4), (4, 4), (5, 8), (4, 8), (3, 8), (, 8), (, 4), (, 4), (3, 4) Lemma Usng Φ(n, s) := 4 s n, the amortzed costs of nsert() and delete() are 5. Proof: To get a sense of the potental functon, consder the fgure below. It shows the value of Φ() as a functon of s (n s fxed). What s the amortzed cost of an nserton? It s the actual cost plus the change n potental. A grow() may or may not happen as a result of an nserton. If a grow() occurs, what s the amortzed cost of t? Before the grow() the potental s 4 n n = n. After the grow() the potental s 0. The actual cost of the grow s n. Thus the amortzed cost of the grow s 0. What about the rest of the nsert? 6

7 actual cost of nsert = change n potental 4 amortzed cost of nsert 5 What about delete? If a shrnk() happens, then the potental decreases by n, and the cost s n, so the amortzed cost of shrnk() s 0. What about the rest of the delete: actual cost of delete = change n potental 4. amortzed cost of delete 5. Ths completes the proof. Theorem 3 The total cost of a sequence of N nsertons and deletons s at most 5N + 4. Proof: The amortzed and real costs are related as follows: actual costs ( amortzed costs ) + ntal potental fnal potental. The ntal potental s 4, and the fnal potental s non-negatve. The amortzed costs sum to at most 5N. Actually, t s easy to replace the +4 part wth 0 n the theorem. All we have to do s change the potental for n = to the followng: { 0 f s < Φ(, s) = 4(s ) otherwse Thus, we ve zeroed the potental for the case when the ntal array of sze s not full. The proof stll goes through, because we never shrnk an array of sze, and ths part of the potental s not mportant. Ths new potental has an ntal value of 0, completng the proof. Exercse 3: Gve a proof for ths theorem usng the banker s method. 7