Design and Analysis of Algorithms 演算法設計與分析. Lecture 9 November 19, 2014 洪國寶

Transcription

1 Design and Analysis of Algorithms 演算法設計與分析 Lecture 9 November 19, 2014 洪國寶 1

2 Outline Advanced data structures Binary heaps(review) Binomial heaps Fibonacci heaps Data structures for disjoint sets 2

3 Mergeable Heaps Support the following operations. MAKE-HEAP() creates and returns a new heap containing no elements. INSERT(H, x) inserts node x, whose key field has already been filled in, into heap H. MINIMUM(H) returns a pointer to the node in heap H whose key is minimum. EXTRACT-MIN(H) deletes the node from heap H whose key is minimum, returning a pointer to the node. UNION(Hl, H2) creates and returns a new heap that contains all the nodes of heaps H1 and H2. Heaps H1 and H2 are "destroyed" by this operation. DECREASE-KEY(H, x, k) assigns to node x within heap H the new key value k, which is assumed to be no greater than its current key value. DELETE(H, x) deletes node x from heap H. 3

4 Review: Binary Heap Binary heap. Almost complete binary tree. filled on all levels, except last, where filled from left to right Min-heap ordered. every child greater than (or equal to) parent

5 Properties. Review: Binary Heap Min element is in root. Heap with N elements has height = log 2 N N = 14 Height =

6 Review: Binary Heaps Implementing binary heaps. Use an array: no need for explicit parent or child pointers. Parent(i) = i/2 Left(i) = 2i Right(i) = 2i

7 Review: Inserting into a binary heap There are two constraints that must be met an almost complete tree the heap-order property We increase the size of the heap to make a hole in the next spot It the item can be legally inserted in the hole, then do so, otherwise move the parent value down and try again 7

8 Review: Binary Heap Decrease Key Decrease key of element x to k. Bubble up until it's heap ordered. O(log N) operations

9 Review: Binary Heap Extract-Min Delete minimum element from heap. Exchange root with rightmost leaf. Bubble root down until it's heap ordered. (heapify) power struggle principle: better subordinate is promoted

10 Union. Binary Heap: Union Combine two binary heaps H 1 and H 2 into a single heap. No easy solution. (N) operations apparently required H 1 H

11 Priority Queues Operation make-heap insert minimum extract-min union decrease-key delete Linked List 1 1 N N 1 1 N Mergeable Heaps Binary 1 Binomial 1 Fibonacci * 1 log N 1 log N N log N log N log N log N log N log N log N log N 1 1 log N 1 1 log N Coming up next 11

12 Outline Advanced data structures Binary heaps(review) Binomial heaps Fibonacci heaps Data structures for disjoint sets 12

13 Binomial Trees Binomial tree. Recursive definition: B 0 B k B k-1 B k-1 B 0 B 1 B 2 B 3 B 4 13

14 Binomial Trees Useful properties of order k binomial tree B k. Number of nodes = 2 k. Height = k. Degree of root = k. Deleting root yields binomial trees B k-1,, B 0. Proof. By induction on k. B k B k+1 B 1 B 2 B 0 B 0 B 1 B 2 B 3 B 4 14

15 Binomial Trees A property useful for naming the data structure. k B k has i nodes at depth i. depth 0 depth depth 2 depth 3 depth 4 B 4 15

16 Binomial Heap Binomial heap. Vuillemin, Sequence of binomial trees that satisfy binomial heap property. each tree is min-heap ordered 0 or 1 binomial tree of order k B 4 B 1 B 0 16

17 17

18 Binomial Heap: Properties Properties of N-node binomial heap. Min key contained in root of B 0, B 1,..., B k. (root list) Contains binomial tree B i iff b i = 1 where b n b 2 b 1 b 0 is binary representation of N. At most log 2 N + 1 binomial trees. Height log 2 N N = 19 # trees = 3 height = 4 binary = B 4 B 1 B 0 18

19 Binomial Heap Operation 1 Create a binomial heap make-binomial-heap( ) H allocate-node( ) head[h] nil return H Running time = O(1) 19

20 Minimum Binomial Heap Operation 2 Running time = O(log n) 20

21 Binomial Heap Operation 3: Union Create heap H that is union of heaps H' and H''. "Mergeable heaps." Easy if H' and H'' are each order k binomial trees. connect roots of H' and H'' choose smaller key to be root of H (Binomial-link) H' H'' 21

22 Binomial Heap: Union 22

23 Binomial Heap: Union =

24 Binomial Heap: Union

25 12 Binomial Heap: Union

26

27

28

29

30 30

31 31

32 Binomial Heap Operation 3: Union Create heap H that is union of heaps H' and H''. Analogous to binary addition. Running time = O(log N) Proportional to number of trees in root lists 2( log 2 N + 1) =

33 Binomial Heap Operation 3: Union Running time = O(log N) Binomial-heap-merge: merge two sorted linked lists Proportional to number of trees in root lists 2( log 2 N + 1). while loop (line 9- line 21) Each iteration executes one binomial-link O(1) Proportional to number of trees in root lists 2( log 2 N + 1) =

34 Binomial Heap: Extract-min Delete node with minimum key in binomial heap H. Find root x with min key in root list of H, and delete H' broken binomial trees H Union(H', H) Running time. O(log N) H H' 34

35 35

36 36

37 Binomial Heap: Extract-min Running time. O(log N) 1. Find the root with minimum key from root list: O(# of trees) = O(log n) 2. Make-binomial-heap: O(1) 3. Reverse the order of the linked list of x s children: O(degree(x)) = O(log n) 4. Binomial-heap-union: O(log n) H H' 37

38 Binomial Heap Operation 5: Insert Insert a new node x into binomial heap H. H' MakeHeap(x) H Union(H', H) Running time. O(log N) x H H'

39 Binomial Heap Operation 5: Insert 39

40 Binomial Heap: Sequence of Inserts Insert a new node x into binomial heap H. If N =...0, then only 1 steps. If N =...01, then only 2 steps. If N =...011, then only 3 steps. If N = , then only 4 steps x 37 Inserting 1 item can take (log N) time. If N = , then log 2 N steps But, inserting sequence of N items takes O(N) time! (N/2)(1) + (N/4)(2) + (N/8)(3) N Amortized analysis. Basis for getting most operations down to constant time. N n 1 n n N 2 N 2 1 N 1 40

41 Binomial Heap Operation 6: Decrease Key Decrease key of node x in binomial heap H. Suppose x is in binomial tree B k. Bubble node x up the tree if x is too small. Running time. O(log N) Proportional to depth of node x log 2 N depth = H x

42 42

43 43

44 Binomial Heap Operation 7: Delete Delete node x in binomial heap H. Decrease key of x to -. Delete min. Running time. O(log N) 44

45 Binomial Heap Operation 7: Delete 45

46 Priority Queues Operation make-heap insert minimum extract-min union decrease-key delete Linked List 1 1 N N 1 1 N Mergeable Heaps Binary 1 Binomial 1 Fibonacci * 1 log N 1 log N N log N log N log N log N log N log N log N log N 1 1 log N 1 1 log N Coming up next 46

47 Outline Advanced data structures Binary heaps(review) Binomial heaps Fibonacci heaps Data structures for disjoint sets 47

48 Fibonacci Heaps Fibonacci heap history. Fredman and Tarjan (1986) Ingenious data structure and analysis. Original motivation: O(E + V log V) shortest path algorithm. also led to faster algorithms for MST (Lecture 11), weighted bipartite matching 48

49 Fibonacci Heaps Fibonacci heap intuition. Similar to binomial heaps, but less structured. Decrease-key and union run in O(1) time. "Lazy" unions and inserts We do not attempt to consolidate trees in a Fibonacci heap when we unite two heaps or insert a new node. 49

50 Fibonacci Heaps: Structure Fibonacci heap. Set of min-heap ordered trees. min marked H

51 Fibonacci Heaps: Implementation Implementation. Represent trees using left-child, right sibling pointers and circular, doubly linked list. can quickly splice off subtrees Roots of trees connected with circular doubly linked list. fast union min[h]: Pointer to root of tree with minimum element. fast find-min min H

52 Root list Child list 52

53 Fibonacci Heaps: Potential Function Key quantities. degree[x] = degree of node x. mark[x] = mark of node x (black or gray). t(h) = # trees. m(h)= # marked nodes. (H)= t(h) + 2m(H) = potential function. t(h) = 5, m(h) = 3 (H) = 11 degree = 3 min H

54 Fibonacci Heap Operation 1: Create Create a Fibonacci heap make-fib-heap( ) H allocate-node( ) n[h] 0 min[h] nil return H Running time Amortized cost = actual cost = O(1) 54

55 Fibonacci Heap Operation 2: Minimum Minimum Fib-heap-minimum(H ) return min[h] Running time Amortized cost = actual cost = O(1) 55

56 Fibonacci Heap Operation 3: Insert Insert. Create a new singleton tree. Add to left of min pointer. Update min pointer. Update n[h] Insert min H

57 Fibonacci Heap Operation 3: Insert Insert. Create a new singleton tree. Add to left of min pointer. Update min pointer. Update n[h] Insert 21 min H

58 Fibonacci Heap Operation 3: Insert Running time. O(1) amortized Actual cost = O(1). Change in potential = +1. Amortized cost = O(1). Insert 21 min H

59 Insert. Create a new singleton tree. Add to left of min pointer. Update min pointer. Update n[h] 59

60 60

61 Fibonacci Heap Operation 4: Union Union. Concatenate two Fibonacci heaps. Root lists are circular, doubly linked lists. min min H' H''

62 Fibonacci Heap Operation 4: Union Running time. O(1) amortized Actual cost = O(1). Change in potential = 0. Amortized cost = O(1). min H' H''

63 63

64 Fibonacci Heap Operation 5: Extract Extract min. Min Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree. min

65 Fibonacci Heap Operation 5: Extract Extract min. Min Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree. min current

66 Fibonacci Heap Operation 5: Extract Extract min. Min Delete min and concatenate its children into root list. Consolidate trees so that 0 no 1 two 2 3 roots have same degree. min current

67 Fibonacci Heap Operation 5: Extract Min Extract min. Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree. current min

68 Fibonacci Heap Operation 5: Extract Min Extract min. Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree min current

69 Fibonacci Heap Operation 5: Extract Min Extract min. Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree min current Merge 17 and 23 trees. 69

70 Fibonacci Heap Operation 5: Extract Min Extract min. Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree min current Merge 7 and 17 trees. 70

71 Fibonacci Heap Operation 5: Extract Min Extract min. Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree min current Merge 7 and 24 trees. 71

72 Fibonacci Heap Operation 5: Extract Min Extract min. Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree min current

73 Fibonacci Heap Operation 5: Extract Min Extract min. Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree min current

74 Fibonacci Heap Operation 5: Extract Min Extract min. Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree min current

75 Fibonacci Heap Operation 5: Extract Min Extract min. Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree min current Merge 41 and 18 trees. 75

76 Fibonacci Heap Operation 5: Extract Min Extract min. Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree min current

77 Fibonacci Heap Operation 5: Extract Min Extract min. Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree min current

78 Fibonacci Heap Operation 5: Extract Extract min. Min Delete min and concatenate its children into root list. Consolidate trees so that no two roots have same degree. min Stop. 78

79 Delete min and concatenate its children into root list. 79

80 80

81 Running time = O(1) 81

82 82

83 Running time = O( D(n) + T(consolidate) ) where D(n): max degree of any node in n-node Fibonacci heap Note the number of nodes in root list after step 6 is O(D(n) + t(h)). 83

84 T(consolidate) simple analysis: O(D(n) + (t(h)+d(n)) D(n) + D(n)) = O( D(n) 2 + t(h)d(n) ) O(D(n)) O( t(h)+d(n) ) O( D(n) ) O(D(n)) 84

85 T(consolidate) tight analysis: the running time from step 3 to step 13 is O(# Fib-Heap-Link exec.) = O(# nodes in the root list) = O( D(n) + t(h) ) 85

86 Fibonacci Heap Extract Min Analysis Notation. D(n) = max degree of any node in Fibonacci heap with n nodes. t(h) = # trees in heap H. (H)= t(h) + 2m(H). Actual cost. O(D(n) + t(h)) O(D(n)) work adding min's children into root list and updating min. at most D(n) children of min node O(D(n) + t(h)) work consolidating trees. work is proportional to size of root list since number of roots decreases by one after each merging D(n) + t(h) - 1 root nodes at beginning of consolidation 86

87 Fibonacci Heap Extract Min Analysis Notation. D(n) = max degree of any node in Fibonacci heap with n nodes. t(h) = # trees in heap H. (H)= t(h) + 2m(H). Actual cost. O(D(n) + t(h)) Amortized cost. O(D(n)) t(h') D(n) + 1 since no two trees have same degree. (H) D(n) t(h). 87

88 Fibonacci Heap Extract Min Analysis Is amortized cost of O(D(n)) good? Yes, if only Insert, Extract-min, and Union operations supported. in this case, Fibonacci heap contains only binomial trees since we only merge trees of equal root degree this implies D(n) log 2 N Yes, if we support Decrease-key in clever way. we'll show that D(n) log N, where is golden ratio 2 = 1 + = (1 + 5) / 2 =

89 Fibonacci Heap Operation 6: Decrease Key Decrease key of element x to k. Case 0: min-heap property not violated. decrease key of x to k change heap min pointer if necessary min Decrease 46 to

90 Fibonacci Heap Operation 6 : Decrease Key Decrease key of element x to k. Case 1: parent of x is unmarked. decrease key of x to k cut off link between x and its parent mark parent add tree rooted at x to root list, updating min heap min pointer Decrease 45 to

91 Fibonacci Heap Operation 6 : Decrease Key Decrease key of element x to k. Case 1: parent of x is unmarked. decrease key of x to k cut off link between x and its parent mark parent add tree rooted at x to root list, updating min heap min pointer Decrease 45 to

92 Fibonacci Heap Operation 6 : Decrease Key Decrease key of element x to k. Case 1: parent of x is unmarked. decrease key of x to k cut off link between x and its parent mark parent 15 add tree rooted at x to root list, updating min heap min pointer Decrease 45 to

93 Fibonacci Heap Operation 6 : Decrease Key Decrease key of element x to k. Case 2: parent of x is marked. decrease key of x to k cut off link between x and its parent p[x], and add x to root list cut off link between p[x] and p[p[x]], add p[x] to root list If p[p[x]] unmarked, then mark it. 15 If p[p[x]] marked, cut off p[p[x]], unmark, and repeat. 7 min Decrease 35 to 5. 93

94 Fibonacci Heap Operation 6 : Decrease Key Decrease key of element x to k. Case 2: parent of x is marked. decrease key of x to k cut off link between x and its parent p[x], and add x to root list cut off link between p[x] and p[p[x]], add p[x] to root list If p[p[x]] unmarked, then mark it. 15 If p[p[x]] marked, cut off p[p[x]], unmark, and repeat. 5 7 min parent marked Decrease 35 to 5. 94

95 Fibonacci Heap Operation 6 : Decrease Key Decrease key of element x to k. Case 2: parent of x is marked. decrease key of x to k cut off link between x and its parent p[x], and add x to root list cut off link between p[x] and p[p[x]], add p[x] to root list If p[p[x]] unmarked, then mark it. 15 If p[p[x]] marked, cut off p[p[x]], unmark, and repeat min parent marked Decrease 35 to 5. 95

96 Fibonacci Heap Operation 6 : Decrease Key Decrease key of element x to k. Case 2: parent of x is marked. decrease key of x to k cut off link between x and its parent p[x], and add x to root list cut off link between p[x] and p[p[x]], add p[x] to root list If p[p[x]] unmarked, then mark it. If p[p[x]] marked, cut off p[p[x]], unmark, and repeat min Decrease 35 to 5. 96

97 97

98 Running time = O(1) 98

99 O(1) 99

100 C-c C-c 100

101 Fibonacci Heap Operation 6: Decrease Key Analysis Notation. t(h) = # trees in heap H. m(h) = # marked nodes in heap H. (H) = t(h) + 2m(H). Actual cost. O(c) O(1) time for decrease key. O(1) time for each of c cascading cuts, plus reinserting in root list. 101

102 Fibonacci Heap Operation 6: Decrease Key Analysis Notation. t(h) = # trees in heap H. m(h) = # marked nodes in heap H. (H) = t(h) + 2m(H). Actual cost. O(c) Amortized cost. O(1) t(h') = t(h) + c m(h') m(h) - c + 2 each cascading cut (except the last one) unmarks a node last cascading cut could potentially mark a node c + 2(-c + 2) = 4 - c. 102

103 Fibonacci Heap Operation 7: Delete 103

104 Fibonacci Heap Operation 7: Delete Delete node x. Decrease key of x to -. Extract min element in heap. Amortized cost. O(D(n)) O(1) for decrease-key. O(D(n)) for extract-min. D(n) = max degree of any node in Fibonacci heap. 104

105 Fibonacci Heaps: Bounding Max Degree Definition. D(N) = max degree in Fibonacci heap with N nodes. Key lemma. D(N) log N, where = (1 + 5) / 2. Corollary. Delete and Extract-min take O(log N) amortized time. 105

106 Fibonacci Heaps: Bounding Max Degree Lemma. Let x be a node with degree k, and let y 1,..., y k denote the children of x in the order in which they were linked to x. Then: degree ( y i ) 0 i 2 if if i 1 i 1 106

107 Fibonacci Heaps: Bounding Max Degree Lemma. Let x be a node with degree k, and let y 1,..., y k denote the children of x in the order in which they were linked to x. Then: degree ( y i Proof. When y i is linked to x, y 1,..., y i-1 already linked to x, degree(x) = i - 1 degree(y i ) = i - 1 since we only link nodes of equal degree Since then, y i has lost at most one child otherwise it would have been cut from x Thus, degree(y i ) = i - 1 or i - 2 ) 0 i 2 if if i 1 i 1 107

108 Fibonacci Heaps: Bounding Max Degree Key lemma. In a Fibonacci heap with N nodes, the maximum degree of any node is at most log N, where = (1 + 5) / 2. Proof of key lemma. For any node x, we show that size(x) degree(x). size(x) = # node in subtree rooted at x taking base logs, degree(x) log (size(x)) log N. Let s k be min size of tree rooted at any degree k node. trivial to see that s 0 = 1, s 1 = 2 s k monotonically increases with k Let x* be a degree k node of size s k, and let y 1,..., y k be children in order that they were linked to x*. Assume k 2 s k size ( x*) k size( y i 2 k i 2 k s i 2 k 2 s i 0 s deg[ y i 2 i 108 i i ] )

109 Fibonacci Facts Definition. The Fibonacci sequence is: 0,1,1, 2, 3, 5, 8, 13, 21,... recursive definition: F k 0 1 F k-1 F k-2 if if if k 0 k 1 k 2 Fact F1. F k+2 k, where = (1 + 5) / 2 = Fact F2. For k 0, 2 1 Consequence. s k F k+2 k. This implies that size(x) degree(x) for all nodes x. F k F i k i 0 s k size ( x*) k size( y i 2 k i 2 k s i 2 k 2 s i 0 s deg[ y i 2 i i i ] ) 109

110 Complexity of Mergeable Heaps Make-Heap(): creates and returns a new heap containing no elements. Minimum(H): returns a pointer to the node with the minimum key. Extract-Min(H): deletes the node with the minimum key. Decrease-Key(H, x, k): assigns to node x the new key value k, which is its current key value. Delete(H, x): deletes node x from heap H. 110

111 Heap summary Binary Binomial Fibonacci Type of trees Complete binary trees Binomial trees Any (no restriction) # of trees 1 O(log n) O(n) Important characteristics Tree height Tree height Max degree Number of trees Max degree Implementation (data structure) Array Linked lists Circular doubly linked list 111

112 Heap summary (Continued) Binary Binomial Fibonacci Heap size heap-size[h] Can be computed n[h] Minimum H[1] Computed from root list min[h] Complexity Worst case Worst case amortized Operation union O(n) O(log n) O(1) decrease-key O(log n) O(log n) O(1) 112

113 Questions? 113