Algorithm Analysis. x is a member of the set P x is not a member of the set P The null or empty set. Cardinality: the number of members

Transcription

1 Algorthm Aalyss Mathematcal Prelmares: Sets ad Relatos: A set s a collecto of dstgushable members or elemets. The members are usually draw from some larger collecto called the base type. Each member of a set s ether a prmtve elemet of the base type or s a set tself. Each value from the base type s ether the set or ot the set. For example, a set amed P mght be composed of the tegers 3, 4, ad 5. I the case, P s members are 3, 4, 5 ad the base type s teger. The table below shows the symbols commoly used to express sets ad ther relatoshps. Let s have two sets P ad Q {,4 A set composed of members ad 4 {x x s a postve A set defto usg a membershp teger codto x P x s a member of the set P x P x s ot a member of the set P The ull or empty set P Cardalty: the umber of members P Q Q P P Q P Q set P Set P s cluded set Q: set P s a subset of set Q set Q s a superset of set P Set Uo: all elemets belogg to P or Q Set Itersecto: all elemets belogg

2 P Q to P ad Q Set dfferece: all elemets of set P NOT set Q The powerset of a set S, s the set of all possble subsets for S. A sequece s a collecto of elemets wth a order, ad whch may cota duplcate-valued elemets. A sequece s sometmes called a tuple or a vector. A relato R over a set S s a set of ordered pars from S. We defe the followg propertes of relatos, where R s a bary relato over set S R s reflexve f ara for all a S R s symmetrc f wheever arb, the bra for all a,b S R s atsymmetrc f wheever arb ad bra the a=b for all a,b S R s trastve f wheever arb, ad brc, the arc for all a,b, c S R s a equvalece relato o set S f t s reflexve, symmetrc ad trastve A bary relato s called a partal order f t s reflexve, atsymmetrc ad trastve Some Importat fuctos:

3 Factoral fucto: the factoral fucto wrtte! for postve teger, s the product of the umbers betwee ad cluded. Thus 5!= The factoral fucto grows quckly as becomes larger. Permutatos: A permutato of a sequece s the members of the sequece arraged some order. If a sequece cotas dstct members, the there are! dfferet permutatos for the sequece. Boolea varables: A Boolea varable s a varable that takes oe of the two values: true or false. Floor ad celg: The floor of x( wrtte (x) takes a real value x ad returs the greatest teger x, example 3.4 returs 3, as does 3.0. The celg of x (wrtte x) takes real value x ad returs the least teger x. Example 3.4 returs 4. Modulus operator: The modulus (or mod) fucto returs the remader of a teger dvso. Sometmes wrtte as mod m or %m

4 Logarthm: A logarthm of base b for value y s the power to whch b s rased to get y. Ths s wrtte as log b y= x. Thus, f log b y = x, the b x = y ad b log b y = y I most applcatos we ll see, we use a logarthm base of two. Logarthms have the followg propertes, for ay postve values of m, ad r ad ay postve tegers a ad b:. log m= log m + log 2. log /m= log - log m 3. log r = r log 4. log a = log b / log b a Algorthm Defto: A algorthm s a step-by-step method of solvg some problem. A algorthm has a fte set of structo havg the followg characterstcs: - Precso: The steps are precsely stated. - Uqueess: The termedary results of each step of executo are uquely defed ad deped oly o the puts ad the results of the precedg steps - Fteess: The algorthm stops after a ftely may structos have bee executed. - Iput: The algorthm receves put - Output: The algorthm produces output - Geeralty : The algorthm apples to a set of puts

5 Recurso: A algorthm s recursve f t calls tself to do part of ts work. For ths to be successful, the call to tself must be o a smaller problem tha the orgal oe. A recursve algorthm has two parts: - the base case, whch hadles a smple put that ca be solved wthout usg recurso ad -the recursve part whch cotas oe or more recursve calls to the algorthm where the parameters are some sese closer to the base case. May data structures are aturally recursve, that they ca be defed as beg made up of smlar parts. May tree structures are ofte represeted recursvely. May searchg ad sortg algorthms are based o a strategy of dvde ad coquer. That s the soluto s foud by breakg dow the problem to smaller (smlar) subproblems, solvg the subproblems the combg the subproblem soluto to form the soluto to the orgal problem. Ths process s mplemeted va recurso. Summatos ad recurreces: Most programs cota loop costructs. Whe aalyzg tme cost for programs wth loops, we eed to add up the cost for each tme the loop s executed. Ths s a example of summato. The followg s a lst of useful summatos, alog wth ther closed-form solutos:

6 (6) (5) (4) 0 (3) log (2) () 2 ) ( 0 0 log a for a a a a for a a Estmatg Estmatg ca be formalzed by the followg three-step process:. Determe the major parameters that affect the problem. 2. Derve a equato that relates the parameters to the problem. 3. Select values for the parameters ad apply the equato to yeld a estmated soluto.

7 Itroducto to Algorthm Aalyss: Algorthm aalyss measures the effcecy of a algorthm or ts mplemetato as a program, as the put sze becomes large. Typcally, we wll aalyze the tme requred for a algorthm ad the space requred for a data structure, Of prmary cosderato whe estmatg a algorthm performace s the umber of basc operatos requred by the algorthm to process a put of a certa sze. Sze ofte refers to the umber of puts beg processed. A basc operato must have the property that ts tme to complete does ot deped o the partcular value of ts operads. (addg, comparg two tegers versus summg a array of tegers) Cosder a smple algorthm to solve the problem of fdg the largest value a array of tegers. The algorthm looks at each teger tur, savg the posto of the largest value see so far. Ths algorthm s called the largest-value sequetal search: t largest(t array[], t ) { t currlarge =0; for (t =; <; ++) f (array[currlarge] <array[]) currlarge=; retur currlarge; ms per operato

8 =00, tme = = 000, tme= 2(999) +5= 998+5= 2003ms Here the sze of the problem s ; the umber of tegers stored array. The basc operato s to compare a teger s value to that of the largest see so far. It s assumed that t takes a fxed amout of tme to do oe such comparso, regardless of the value of the two tegers or ther posto the array. Sce the most mportat factor affectg the rug tme s the sze of the put, for a gve put sze we say that the tme T to ru the algorthm s a fucto of or T(). Let us call c the tme requred to compare two tegers. We are ot cocered wth the tme requred to cremet varable, or the tme of actual assgmets. Therefore, a good approxmato of the total tme requred to ru largest s c, sce we must make comparsos, wth each oe costg c tme. We say the fucto largest rug tme s expressed by the equato: T() = c The equato descrbes the growth rate for the rug tme of the largest value sequetal search algorthm The growth rate for a algorthm s the rate at whch the cost of the algorthm grows as the sze of the put grows. A growth rate of c s referred to as lear growth rate or rug tme. A algorthm whose rug-tme equato has a hghest order term cotag a factor of 2 s sad to be quadratc.

9 If the term of the equato s 2, we say that the algorthm has a expoetal growth rate. Best, Worst ad Average Cases: Best-Case Tme: The mmum tme eeded to execute the algorthm for a seres of put all of sze. Worst-Case Tme: The maxmum tme eeded to execute the algorthm for a seres of put all of sze. Average-Case Tme: The average tme eeded to execute the algorthm for a seres of put all of sze. Ofte, we are more terested how the best-case or the worst-case grows as the tme grows. For example, suppose that the worst case tme of a algorthm s: T() = , for put of sze. For large, the term 60 2 s approxmately equal to t(). I ths sese, t() grows lke T() = t()= , ,50 60, 005,00 6, 000,050, , ,000,000 6,000,000,000

10 Whe, we descrbe how the best case tme ad the worst case tme grows as the sze of the put creases, we oly look for the domat term ad gore the others, other words, we ca say that t() grows lke 2 as creases. We say ths case that t() s of order 2 ad we wrte: t() = ( 2 ) (read as t() s theta of 2 ) Let f ad g be fuctos wth doma {, 2, 3,. Bg Oh Notato: We wrte: f() = O(g()) Ad we say that f() s of order at most g() f there exsts a postve costat C such that f() C g() for all but a ftely may postve tegers.(worst case) Omega Notato: We wrte: f() = (g()) Ad we say that f() s of order at least g() f there exsts a postve costat C2 such that f() C2 g() for all but a ftely may postve tegers. (best case) Theta Notato: We wrte:

11 f() = (g()) Ad we say that f() s of order g() f f() = O(g()) ad f() = (g()). Fdg the Theta fucto of a fucto: I order to fd the Theta fucto of a fucto f(), we eed to fd two costats C ad C2 so that: f() C g() ad f() C2 g() wth g() = 2 Example: Fdg C = 66 2 for So we ca take C = = O ( 2 ) Fdg C So we ca take C2 = = ( 2 )

12 Sce = O ( 2 ) ad = ( 2 ) we say that = ( 2 ) Ths method ca be used to show that a polyomal of degree k wth o egatve coeffcet s ( k ) Order of growth of some commo fuctos: O() < O(log ) < O() < O( * log ) < O( 2 ) < O( 3 ) < O(2 ) Executo tme of Programmg Costructs We wll study the aalyss of the rug tmes usg several algorthms. Costat Tme: - We wll assume that operatos o prmtve data type values (such as addto, subtractos) ca be performed wth a costat amout of tme. - Assgmet of a prmtve value to a varable also requres oly a costat tme. - A fxed sequece of costat tme operatos performed o a structure stll requres oly costat tme. - The maxmum tme t takes to perform a codtoal f statemet s the sum of the tmes t takes to perform a test ad the maxmum tmes requred by ether alteratve. If all three ca be performed costat tme, the the tme to execute the f statemet ca stll be bouded by some costat value.

13 - Eve legthy procedures, f they do ot clude loops or procedure calls, the the total executo tme must rema costat. Smple Loops: A loop that performs a costat umber of teratos s stll cosdered to be executg costat tme. Example: for (t = ; <=00; ++) array[]=; More commoly, the umber of teratos a loop wll perform s determed by put values. To descrbe the rug tme of a loop we eed to characterze how may teratos the loop wll perform ad the multply ths value by the rug tme of the body of the loop. The smplest case occurs whe the lmts of the loop are fxed by the put value, ad the body of the loop uses costat tme. Example of a program that returs the smallest value the vector:

14 double mmum (double values [], usged t ) { assert (>); double mvalue= values[0]; for (usged t =; <; ++) { f (values[] < mvalue) mvalue= values[]; retur mvalue; Here the loop wll clearly execute tmes. Because at each terato we are performg at most oe comparso ad oe assgmet, the executo tme for the body of the loop s costat. The total rug tme for the procedure s therefore O(). Case 2: bool sprme (usged t ) { for (usged t = 2; * <= ; ++) { f ( % = = 0) retur false; retur true; I ths case, the loop termates whe the value of the loop varable exceeds the square root of the orgal value. Thus, the umber of teratos s approxmately.

15 Because each terato of the loop s performed a costat amout of tme, the etre algorthm s sad to be O( ). The algorthmc executo tme represets the worst case executo tme. Nested Loops: The executo tme for a loop statemet s the umber of teratos of the loop multpled by the executo tme of the body of the loop. Case : The smplest case occurs whe the lmts of the loops are depedet of oe aother. I ths stuato, the executo tme s the product of the values represetg the umber of tmes each loop wll terate multpled by the executo tme of the body. Example: A classc algorthm for multplyg two by matrces, to produce a ew by matrx product. vod matprod( double a[][], double b[][],double c[][]) { for (usged t =0; <; ++) { for ( usged t j =0; j < ; j ++) { c[][j]= 0.0; for (usged t k =0; k < ; k++){ c[][j] +=a[][k]*b[k][j];

16 The umber of teratos each loop s. The body of the ermost loop, whch performs oly a multplcato, a addto, ad a assgmet, s costat tme. Therefore the total rug tme s O( 3 ). Case 2: A slghtly more complex aalyss s requred whe the lmts of teratos for the er loop s lked to a elemet the outer loop. A example of ths s bubblesort: double v[0]={5, 4, 7, 6, 2, 9,, 89, 35, vod bubblesort( double v [], usged t ) { for (usged t = -; > 0; --){ for (usged t j =0; j <; j++) { f ( v[j] > v[j+]) { double temp = v[j]; v[j]= v[j+]; v[j+] = temp; for (k=0; k< ; k++) cout << v[k]; To determe the rug tme of ths procedure, we ca smulate o a few values. O the frst terato of the outermost loop, the er loop wll execute - tmes. O the secod terato of the outermost loop, the ermost loop wll execute -2 tme, ad so o.

17 The umber of teratos s therefore: (-) + (-2)+.+ = ( ) 2 Cosequetly, ths algorthm has a rug tme of O( 2 ) Whle loops: The aalyss of whle loops s smlar to that of for loops. The key s to determe the umber of teratos the loop wll perform. Case : vod sertosort (double v [], usged t ) { for (usged t = ; < ; ++) { double elemet = v[]; t j = -; whle (j >=0 && elemet < v[j]) { v[j+] = v[j]; j= j -; v[j+]= elemet; Ulke the bubble sort algorthm, the serto sort algorthm places the lower porto, of the array to sequece frst. Each ew value s serted to place, sldg elemets over utl the proper locato for each ew value s establshed.

18 The outer loop executes steps. The er loop may termate early, but the worst case must shft over elemets all the way to the bottom. We see, therefore, that the worst case the umber of teratos of the er loop follows the patter: (-). As we have see, the sum of ths seres s the algorthm s O( 2 ). ( ) 2 ad thus Aother classc example of a otrval whle loop occurs the bary search algorthm. Here we assume that the put vector s a already ordered collecto of values. The task s to determe f a partcular value occurs the lst ad, f ot, the posto mmedately pror to the locato where the elemets would be placed: usged t barysearch(double v[], usged t, double value) { usged t low =0; usged t hgh =; whle (low < hgh) { usged md= (low +hgh)/2; f (v[md] < value) low = md +; else hgh = md; retur low;

19 The program works by repeatedly dvdg half the rage of values beg searched. Because there were orgally values, we kow the umber of tmes the collecto ca be subdvded s o larger tha roughly log (see ote). Ths s suffcet to tell us that the etre procedure s O(log ) Note: The log(base 2) of a postve value x s approxmately equal to the umber of tmes that x ca be dvded by 2. logarthms CS are, almost varably, used wth a base value of 2. Fucto call: Whe fucto or procedure calls occur, the rug tme of the call s take to be the rug tme of the operatos assocated wth the procedure. Suppose we wat to prt the values of all prme umbers less tha. We could use a algorthm such as the followg: vod prtprmes (usged t ) { for (usged t = 2; <=; ++) { f (sprme()) cout << << s prme \ ; else cout << << s ot a prme\ ;

20 We kow the executo tme of the sprme route s O( ). Because we are makg roughly calls, the total rug tme of the procedure s o greater tha O( ) Executo tmes for a task usg = 0 5 assumg 0 6 operatos ca be performed per secod. fucto log log Rug tme More tha a cetury 3.7 years 2.8 hours 3.6 secods.2 secods 0. secods 3.2 x 0-4 secods.2 x 0-5 secods