Sorting. Data Structures LECTURE 4. Comparison-based sorting. Sorting algorithms. Quick-Sort. Example (1) Pivot

Transcription

1 Data Structures, Sprg 004. Joskowcz Data Structures ECUE 4 Comparso-based sortg Why sortg? Formal aalyss of Quck-Sort Comparso sortg: lower boud Summary of comparso-sortg algorthms Sortg Defto Iput: A seuece of umbers A = (a, a,, a ) Output: A permutato (reorderg) (a,, a ) such that a a Why sortg? Fudametal problem Computer Scece May algorthms use t as a key subroute Wde varety wth a rch set of techues Kow lower bouds, asymptotcally optmal May programmg ad mplemetato ssues come up! Data Structures, Sprg 004. Joskowcz Sortg algorthms wo types of sortg algorthms:. Comparso sortg: the basc operato s the comparso betwee two elemets: a a j Merge-Sort, Iserto-Sort, Bubble-Sort Quck-Sort: aalyss wth recurrece euatos ower bouds for comparso sortg: () = ( lg ) ad S() = () Heap Sort wth prorty ueues (later, after trees). No comparso-based: does use comparsos! eures addtoal assumptos Sortg lear tme: () = () ad S() = () Data Structures, Sprg 004. Joskowcz Quck-Sort Uses a Dvde-ad-Couer strategy: Splt A[eft..ght] to A[eft..Mddle ] ad A[Mddle+..ght] such that the elemets of A[eft..Mddle ] are smaller or eual tha those A[Mddle+..ght] Sort each part recursvely Quck-Sort(A, eft, ght). f eft < ght the do. Mddle Partto(A, eft, ght) 3. Quck-Sort(A, eft, Mddle ) 4. Quck-Sort(A, Mddle +, ght) Data Structures, Sprg 004. Joskowcz Partto earrages the array ad returs the parttog dex he partto s the leftmost elemet larger tha the last Partto(A, eft, ght). Pvot A[ght]. eft 3. for j eft to ght 4. do f (A[j] Pvot) 5. the + 6. Exchage(A[], A[j]) 7. Exchage (A[+], A[ght]) 8. retur + Data Structures, Sprg 004. Joskowcz j Example () Pvot j j j st terato Data Structures, Sprg 004. Joskowcz swapped wth tself

2 Data Structures, Sprg 004. Joskowcz Example () Pvot j j j ad 8 swapped 3 ad 7 swapped eft lst Example (3) eft lst A[] Pvot Pvot ght lst Pvot j ght lst A[] > Pvot Data Structures, Sprg 004. Joskowcz Urestrcted lst d terato geeral patter Quck-Sort complexty he complexty of Quck-Sort depeds o whether the parttog s balaced or ubalaced, whch depeds o whch elemets are used for parttog. Ubalaced partto: there s o partto, so the sub-problems are of sze ad 0.. Perfect partto: the partto s always the mddle, so the sub-problems are both of sze /. 3. Balaced partto: the partto s somewhere the mddle, so the sub-problems are of sze k ad k. et us study each case separately! Data Structures, Sprg 004. Joskowcz Ubalaced partto he recurrece euato s: () = ( ) + (0) + k ( k ) k () k Data Structures, Sprg 004. Joskowcz Data Structures, Sprg 004. Joskowcz Perfect partto he recurrece euato s: () (/) + (/) + () / lg Geeral case he recurrece euato s: ( ) max ( ) 0 Average case s somewhere betwee ubalaced ad perfect partto: whch oe domates? lg ( ) Data Structures, Sprg 004. Joskowcz

3 Example: 9-to- proportoal splt Suppose that the parttog algorthm always produces a 9-to- proportoal splt. he complexty s: () = (/0) + (9/0) + () At every level, the boudary codto s reached at depth log 0 wth cost (). he recurso termates at depth log 0/9 herefore, the complexty s () = O( lg ) I fact, ths s true for ay proportoal splt! Clam: Proof: Worst-case aalyss: proof () max ( ) c O Base of ducto:rue for =. Iducto step: Assume for <, ad prove for. ' max ( ) ' ' ' c c( ' ) d' c c( ') c' d' Data Structures, Sprg 004. Joskowcz Data Structures, Sprg 004. Joskowcz Worst-case aalyss: proof () o prove the clam, we eed to show that ths s smaller tha c('), or euvaletly that: d ' c ( ' ) Sce ( -) s always greater tha /, as ca be easly verfed by checkg the two cases: or we ca pck c such that the eualty holds. Data Structures, Sprg 004. Joskowcz Average case complexty We must frst defe what s a average case he behavor s determed by the relatve orderg of the elemets, ot by the elemets themselves. hus, we are terested the average of all permutatos, where each permutato s eually lkely to appear (uformly radom put). he average complexty s the umber of steps averaged over a uformly radom put. he complexty s determed by the umber of bad splts ad the umber of good splts. Data Structures, Sprg 004. Joskowcz Bad splts ad good splts -- tuto 0 ( )/ ( )/ Alterate bad splt Data Structures, Sprg 004. Joskowcz () ( )/ Good splt I both cases, the complexty s (). hus the bad splt was absorbed by a good oe! () ( )/ adomzato ad average complexty Oe way of studyg the average case aalyss s to aalyze the performace of a radomzed verso of the algorthm. I the radomzed verso, choces are made wth a uform probablty, ad ths mmcks put geeralty essetally, we reduce the chaces of httg the worst put! adomzato esures that the performace s good wthout makg assumptos o the put adomess s oe of the most mportat cocepts ad tools moder Computer Scece! Data Structures, Sprg 004. Joskowcz

4 adomzed Quck-Sort adomzed Complexty: he umber of steps, (for the WOS put!) averaged over the radom choces of the algorthm. For Quck-Sort, the pvot determes the umber of good ad bad splts We chose the leftmost elemet to select a pvot. What f we choose stead ay elemet radomly? I Partto, use Pvot A[adom(eft,ght)] stead of Pvot A[eft] Note that the algorthm remas correct! Data Structures, Sprg 004. Joskowcz adomzed complexty adomzed-case recurrece: he pvot s eually lkely to be ay place, ad sce there are places, each case occurs / of the puts. We get: ( ) hs s ecurrece wth Full Hstory, sce t depeds o all prevous szes of the problem. It ca be prove, usg methods whch we wll ot get to ths tme, that the soluto for ths recurrece satsfes: clg O( lg ) Data Structures, Sprg 004. Joskowcz Sortg wth comparsos he basc operato of all the sortg algorthms we have see so far s the comparso betwee two elemets: a a j he sorted order they determe s based oly o comparsos betwee the put elemets! We would lke to prove that ay comparso sortg algorthm must make ( lg ) comparsos the worst case to sort elemets (lower boud). Sortg wthout comparsos takes () the worst case, but we must make assumptos about the put. Comparso sortg lower boud We wat to prove a lower boud ( ) o the worst-case complexty sortg for ANY sortg algorthm that uses comparsos. We wll use the decso tree model to evaluate the umber of comparsos that are eeded the worst case. Every algorthm has ts ow decso tree, depedg o how t does the comparsos betwee elemets. he legth of the logest path from the root to the leaves ths tree wll determe the maxmum umber of comparsos that the algorthm must perform. Data Structures, Sprg 004. Joskowcz Data Structures, Sprg 004. Joskowcz Decso trees A decso tree s a full bary tree that represets the comparsos betwee elemets that are performed by a partcular algorthm. Decso tree for 3 elemets he tree has teral odes, leaves, ad braches: Iteral ode: two dces :j for, j eaf: a permutato of the put (), () Braches: result of a comparso a a j (left) or a > a j (rght) Data Structures, Sprg 004. Joskowcz Data Structures, Sprg 004. Joskowcz

5 Paths decso trees he executo of sortg algorthm o put I correspods to tracg a path from the root to a leaf Each teral ode s assocated wth a yes/o uesto, regardg the put, ad the two edges that are comg out of t are assocated wth oe of the two possble aswers to the uesto. he leaves are assocated wth oe possble outcome of the tree, ad o edge s comg out of them. At the leaf, the permutato s the oe that sorts the elemets! Data Structures, Sprg 004. Joskowcz Decso tree for 3 elemets ogest path: 3 (7,9,6) 7 9 (7,9,6) 9 > 6 (7,9,6) 7 > 6 (A)=(6,7,9) Data Structures, Sprg 004. Joskowcz Decso tree computato he computato for a put starts at the root, ad progresses dow the tree from oe ode to the ext accordg to the aswers to the uestos at the odes. he computato eds whe we get to a leaf. ANY correct algorthm MUS be able to produce each permutato of the put. here are at most! permutatos ad they must all appear the leafs of the tree. Worst case complexty he worst-case umber of comparsos s the legth of the logest root-to-leaf path the decso tree. he lower boud o the legth of the logest path for a gve algorthm gves a lower boud o the worstcase umber of comparsos the algorthm reures. hus, fdg a lower boud o the legth of the logest path for a decso tree based o comparsos provded a lower boud o the worst case complexty of comparso based sortg algorthms! Data Structures, Sprg 004. Joskowcz Data Structures, Sprg 004. Joskowcz Comparso-based sortg algorthms Ay comparso-based sortg algorthm ca be descrbed by a decso tree. he umber of leaves the tree of ay comparso based sortg algorthm must be at least!, sce the algorthm must gve a correct aswer to every possble put, ad there are! possble aswers. Why at least? Because there mght be more tha oe leaf wth the same aswer, correspodg to dfferet ways the algorthm treats dfferet puts. Data Structures, Sprg 004. Joskowcz egth of the logest path ()! dfferet possble aswers. Cosder all trees wth! leaves. I each oe, cosder the logest path. et d be the depth (heght) of the tree. he mmum legth of such logest path must be such that! d herefore, log (!) log ( d ) = d Quck check: (/) (/)! (/) log (/) log (!) log log (!) = ( log ) Data Structures, Sprg 004. Joskowcz

6 Clam: Proof: egth of the logest path () log(!) log(!) log( O the other had: ( log( )) for ) log( ) ( log( )). log( log( ) log(!) log( ) log( ). hs s the lower boud o the umber of comparsos ay comparso-based sortg algorthm. Data Structures, Sprg 004. Joskowcz ) Complexty of comparso-sortg algorthms Bubble-Sort Iserto-Sort Merge-Sort Quck-Sort Space O() O() O( lg ) O() Worst case O( ) O( ) O( lg ) O( ) Best case O() O() O( lg ) O( lg ) Average case O( ) O( ) O( lg ) O( lg ) adom. case O( ) O( ) O( lg ) ower bouds for comparso sortg s () = ( lg ) ad S() = () for worst ad average case, determstc ad radomzed algorthms. Data Structures, Sprg 004. Joskowcz ---