CMSC 441: Design & Analysis of Algorithms

Transcription

1 CMSC 441: Design & Analysis of Algorithms Hillol Kargupta

2 Today s Topics Amortized analysis April 19, 2011 CMSC 641 2

3 Amortized Analysis Aggregate Method Accounting Method Potential Method April 19, 2011 CMSC 641 3

4 Aggregate Method n operations take time T(n) Average cost of an operation = T(n)/n April 19, 2011 CMSC 641 4

5 Example: Stack Operations Operations: Push(S, x) runs in O(1) Pop(S) runs in O(1) Multi-Pop(S, k) runs in min(s, k) when the stack contains s objects Analysis of a sequence of n Push, Pop, and Multi-Pop operations Worst case behavior of Multi-Pop is O(n) So all operations costs O(n 2 )? April 19, 2011 CMSC 641 5

6 More Careful Analysis Each item can be popped only once A total of O(n) time complexity Average cost per operation is O(n)/n=O(1) No probabilistic reasoning is involved April 19, 2011 CMSC 641 6

7 Accounting Method Assign different charges to different operations: Some more or less than they actually cost Charge each operation an amortized cost Amount not used saved in bank Later operations can use saved money Balance must not go negative April 19, 2011 CMSC 641 7

8 Example: Stack Operations Cost distributions: Push: 2 Pop: 0 Multi-Pop: 0 April 19, 2011 CMSC 641 8

9 Potential Method Prepaid work as potential energy Can be released to pay for future operations Associated with the whole data structure instead of the specific objects in the data structure April 19, 2011 CMSC 641 9

10 Example: Stack Operations Adopt a scheme for defining potential energy Compute amortized cost for Push Pop Multi-Pop April 19, 2011 CMSC

11 Accounting Method Example: Dynamic Tables Implementing a table (e.g., hash table) for dynamic data, want to make it small as possible Problem: if too many items inserted, table may be too small Idea: allocate more memory as needed April 19, 2011 CMSC

12 Dynamic Tables 1. Init table size m = 1 2. Insert elements until number n > m 3. Generate new table of size 2m 4. Reinsert old elements into new table 5. (back to step 2) What is the worst-case cost of an insert? O(n)? April 19, 2011 CMSC

13 Analysis Of Dynamic Tables Let c i = cost of ith insert c i = i if i-1 is exact power of 2; 1 otherwise Example: Operation Table Size Cost Insert(1) April 19, 2011 CMSC

14 Analysis Of Dynamic Tables Let c i = cost of ith insert c i = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) Insert(2) April 19, 2011 CMSC

15 Analysis Of Dynamic Tables Let c i = cost of ith insert c i = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) Insert(2) Insert(3) April 19, 2011 CMSC

16 Analysis Of Dynamic Tables Let c i = cost of ith insert c i = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) Insert(2) Insert(3) Insert(4) April 19, 2011 CMSC

17 Analysis Of Dynamic Tables Let c i = cost of ith insert c i = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) Insert(2) Insert(3) Insert(4) Insert(5) April 19, 2011 CMSC

18 Analysis Of Dynamic Tables Let c i = cost of ith insert c i = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) Insert(2) Insert(3) Insert(4) Insert(5) Insert(6) April 19, 2011 CMSC

19 Analysis Of Dynamic Tables Let c i = cost of ith insert c i = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) Insert(2) Insert(3) Insert(4) Insert(5) Insert(6) Insert(7) April 19, 2011 CMSC

20 Analysis Of Dynamic Tables Let c i = cost of ith insert c i = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) Insert(2) Insert(3) Insert(4) Insert(5) Insert(6) Insert(7) Insert(8) April 19, 2011 CMSC

21 Analysis Of Dynamic Tables Insert(1) Insert(2) Insert(3) Insert(4) 4 1 Insert(5) Insert(6) 8 1 Insert(7) 8 1 Insert(8) 8 1 Insert(9) Let c i = cost of ith insert c i = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost April 19, 2011 CMSC

22 Aggregate Analysis n Insert() operations cost n i= 1 lg n j ci n + 2 = n + (2n 1) < 3n j = 0 Average cost of operation = (total cost)/(# operations) < 3 Asymptotically, then, a dynamic table costs the same as a fixed-size table Both O(1) per Insert operation April 19, 2011 CMSC

23 Accounting Analysis Charge each operation $3 amortized cost Use $1 to perform immediate Insert() Store $2 When table doubles $1 reinserts old item, $1 reinserts another old item Point is, we ve already paid these costs Constant (amortized) cost per operation April 19, 2011 CMSC

24 The Potential Method Same as accounting method: something prepaid is used later. Different from accounting method The prepaid work not as credit, but as potential energy, or potential. The potential is associated with the data structure as a whole rather than with specific objects within the data structure.

25 The Potential Method (cont.) Initial data structure D 0, n operations, resulting in D 0, D 1,, D n with costs c 1, c 2,, c n. A potential function Φ: {D i } R (real numbers) Φ(D i ) is called the potential of D i. Amortized cost c i ' of the ith operation is: c i ' = c i + Φ(D i ) - Φ(D i-1 ). (actual cost + potential change) i=1 n c i '= i=1 n (c i + Φ(D i ) - Φ(D i-1 )) = i=1n c i + Φ(D n ) - Φ(D 0 )

26 The Potential Method (cont.) If Φ(D n ) Φ(D 0 ), then total amortized cost is an upper bound of total actual cost. But we do not know how many operations, so Φ(D i ) Φ(D 0 ) is required for any i. It is convenient to define Φ(D 0 )=0,and so Φ(D i ) 0, for all i. If the potential change is positive (i.e., Φ(D i ) - Φ(D i- 1)>0), then c i ' is an overcharge (so store the increase as potential), otherwise, undercharge (discharge the potential to pay the actual cost).

27 Potential method: stack operation Potential for a stack is the number of objects in the stack. So Φ(D 0 )=0, and Φ(D i ) 0 Amortized cost of stack operations: PUSH: Potential change: Φ(D i )- Φ(D i-1 ) =(s+1)-s =1. Amortized cost: c i ' = c i + Φ(D i ) - Φ(D i-1 )=1+1=2. POP: Potential change: Φ(D i )- Φ(D i-1 ) =(s-1) s= -1. Amortized cost: c i ' = c i + Φ(D i ) - Φ(D i-1 )=1+(-1)=0. MULTIPOP(S,k): k'=min(s,k) Potential change: Φ(D i )- Φ(D i-1 ) = k'. Amortized cost: c i ' = c i + Φ(D i ) - Φ(D i-1 )=k'+(-k')=0. So amortized cost of each operation is O(1), and total amortized cost of n operations is O(n). Since total amortized cost is an upper bound of actual cost, the worse case cost of n operations is O(n).

28 Potential method: binary counter Define the potential of the counter after the ith INCREMENT is Φ(D i )=b i, the number of 1 s. clearly, Φ(D i ) 0. Let us compute amortized cost of an operation Suppose the ith operation resets t i bits. Actual cost c i of the operation is at most t i +1. If b i =0, then the ith operation resets all k bits, so b i-1 =t i =k. If b i >0, then b i =b i-1 -t i +1 In either case, b i b i-1 -t i +1. So potential change is Φ(D i ) - Φ(D i-1 ) b i-1 -t i +1-b i-1 =1-t i. So amortized cost is: c i ' = c i + Φ(D i ) - Φ(D i-1 ) t i +1+1-t i =2. The total amortized cost of n operations is O(n). Thus worst case cost is O(n).