Suffix Trees. Outline. Introduction Suffix Trees (ST) Building STs in linear time: Ukkonen s algorithm Applications of ST.

Transcription

1 Suffi Trees Outline Introduction Suffi Trees (ST) Building STs in liner time: Ukkonen s lgorithm Applictions of ST 2 Introduction 3

2 Sustrings String is ny sequence of chrcters. Sustring of string S is string composed of chrcters i through j, i j of S. S = cter;te is sustring. cr is not sustring. Empty string is sustring of S. 4 Susequences Susequence of string S is string composed of chrcters i < i 2 < < i k of S. S = cter; te is susequence. cr is susequence. Empty string is susequence of S. 5 String/Pttern Mtching - I You re given source string S. Suppose we hve to nswer queries of the form: is the string p i sustring of S? Knuth-Morris-Prtt (KMP) string mtching. O( S + p i ) time per query. O(n S + Σ i p i ) time for n queries. Suffi tree solution. O( S + Σ i p i ) time for n queries. 6 2

3 String/Pttern Mtching - II KMP preprocesses the query string p i, wheres the suffi tree method preprocesses the source string (tet) S. The suffi tree for the tet is uilt in O(m) time during pre-processing stge; therefter, whenever string of length O(n) is input, the lgorithm serches it in O(n) time using tht suffi tree. 7 String Mtching: Prefies & Suffies Sustrings of S eginning t the first position of S re clled prefies of S, nd sustrings tht end t its lst position re clled suffies of S. S=AACTAG Prefies: AACTAG,AACTA,AACT,AAC,AA,A Suffies: AACTAG,ACTAG,CTAG,TAG,AG,G p i is sustring of S iff p i is prefi of some suffi of S. Suffi Trees 9 3

4 Definition: Suffi Tree (ST) T for S of length m. A rooted tree with m leves numered from to m. 2. Ech internl node, ecluding the root, of T hs t lest 2 children. 3. Ech edge of T is leled with nonempty sustring of S. 4. No two edges out of node cn hve edge-lels strting with the sme chrcter. 5. For ny lef i, the conctention of the edge-lels on the pth from the root to lef i ectly spells out the suffi of S, nmely S[i,m], tht strts t position i. 0 Emple: Suffi Tree for S=c Eistence of suffi tree S If one suffi S j of S mtches prefi of nother suffi S i of S, then the pth for S j would not end t lef. S = S = nd S 4 = How to void this prolem? Assume tht the lst chrcter of S ppers nowhere else in S. Add new chrcter not in the lphet to the end of S. 2 4

5 Emple: Suffi Tree for S=c 3 Building STs in liner time: Ukkonen s lgorithm 4 Building STs in liner time Weiner s lgorithm [FOCS, 973] The lgorithm of 973 clled y Knuth First lgorithm of liner time, ut much spce McGreight s lgorithm [JACM, 976] Liner time nd qudrtic spce More redle Ukkonen s lgorithm [Algorithmic, 995] Liner time lgorithm nd less spce This is wht we will focus on 5 5

6 6 Implicit Suffi Trees Ukkonen s lgorithm constructs sequence of implicit STs, the lst of which is converted to true ST of the given string. An implicit suffi tree for string S is tree otined from the suffi tree for S y removing from ll edge lels removing ny edges tht now hve no lel removing ny node tht does not still hve t lest two children An implicit suffi tree for prefi S[,i] of S is similrly defined sed on the suffi tree for S[,i]. I i will denote the implicit suffi tree for S[,i]. Ech suffi is in the tree, ut my not end t lef. Emple: Construction of the Implicit ST Implicit tree for from tree for {,,,,, } Construction of the Implicit ST: Remove Remove {,,,,, }

7 Construction of the Implicit ST: After the Removl of Remove {,,,, } Construction of the Implicit ST: Remove unleled edges Remove unleled edges {,,,, } Construction of the Implicit ST: After the Removl of Unleled Edges Remove unleled edges {,,,, }

8 Construction of the Implicit ST: Remove interior nodes Remove internl nodes with only one child {,,,, } Construction of the Implicit ST: Finl implicit tree Remove internl nodes with only one child {,,,, } Ukkonen s Algorithm (UA) I i is the implicit suffi tree of the string S[, i] Construct I /* Construct I i+ from I i */ for i = to m- do /* phse i+ */ for j = to i+ do /* etension j */ Find the end of the pth P from the root whose lel is S[j, i] in I i nd etend P with S[i+] y suffi etension rules; Convert I m into suffi tree S 24 8

9 Emple S = cd i+ = i+ = 2 etend to i+ = 3 etend to etend to 25 Etension Rules Gol: etend ech S[j,i] into S[j,i+] Rule : S[j,i] ends t lef Add chrcter S(i+) to the end of the lel on tht lef edge Rule 2: S[j,i] doesn t end t lef, nd the following chrcter is not S(i+) Split new lef edge for chrcter S(i+) My need to crete n internl node if S[j,i] ends in the middle of n edge Rule 3: S[j,i+] is lredy in the tree No updte 26 Emple: Etension Rules Implicit tree for from tree for Rule : 2: 3: t dd lredy lef lef in node edge tree (nd n interior node) 9

10 UA for c () S[,3]= E S(j,i) S(i+) UA for c (2) 29 UA for c (3) 30 0

11 UA for c (4) 3 Oservtions Once S[j,i] is locted in the tree, etension rules tke only constnt time Nively we could find the end of ny suffi S[j,i] in O(S[j,i]) time y wlking from the root of the current tree. By tht pproch, I m could e creted in O(m 3 ) time. Mking Ukkonen s lgorithm O(m) Suffi links Skip nd count trick Edge-lel compression A stopper Once lef, lwys lef 32 Suffi Links Consider the two strings nd Suppose some internl node v of the tree is leled with nd nother node s(v) in the tree is leled with The edge (v,s(v)) is clled suffi link Do ll internl nodes (the root is not considered n internl node) hve suffi links? 33

12 Emple: suffi links S = ACACACAC AC C AC AC AC AC AC AC 34 Suffi Link Lemm If new internl node v with pth-lel is dded to the current tree in etension j of some phse i+, then the pth leled lredy ends t n internl node of the tree or the internl node leled will e creted in the etension of j+ in the sme phse i+ string is empty nd s(v) is the root Proof of Suffi Link Lemm A new internl node is creted only y the etension rule 2 This mens tht there re two distinct suffies of S[,i+] tht strt with S(i+) nd c where c is not S(i+) This mens tht there re two distinct suffies of S[,i+] tht strt with S(i+) nd c where c is not S(i+) Thus, if is not empty, will lel n internl node once etension j+ is processed which is the etension of 2

13 Corollry of Suffi Link Lemm Every internl node of n implicit suffi tree hs suffi link from it. 37 How to use suffi links - S[,i] must end t lef since it is the longest string in the implicit tree I i Keep pointer to this lef in ll cses nd etend ccording to rule Locting S[j+,i] from S[j,i] which is t node w If w is n internl node, set v to w Otherwise, set v = prent(w) If v is the root, you must trverse from the root to find S[j+,i] If not, go to s(v) nd egin serch for the remining portion of S[j,i] from there 38 How to use suffi links

14 Skip nd Count Trick () Prolem: Moving down from s(v), directly implemented, tkes time proportionl to the numer of chrcters compred Solution: To mke running time proportionl to the numer of nodes in the pth serched, insted of the numer of chrcters 40 Skip nd Count Trick (2) After 4 nodes down-skips, the end of S[j, i] is found. 4 Skip nd Count Trick (3) Node-depth of v, denoted (ND(v)), is the numer of nodes on the pth from the root to the node v Lemm: For ny suffi link (v, s(v)) trversed in Ukkonen s lgorithm, t tht moment, ND(v) ND(s(v))+ 42 4

15 Skip nd Count Trick (4) At the moment of trversing (v,s(v)): ND(v) ND(s(v)) Skip nd Count Trick (5) The current node-depth of the lgorithm is the node depth of the node most recently visited y the lgorithm Lemm: Using the skip nd count trick, ny phse of Ukkonen s lgorithm tkes O(m) time. Up-wlk: decreses the current node-depth y Suffi link trversl: sme s up-wlk Totlly, the current node-depth is decresed y 2m. No node hs depth >m. The totl possile increment to the current node-depth is 3m. 45 Edge Lel Representtion Potentil Prolem Size of edge lels my require Ω(m 2 ) spce Thus, the time for the lgorithm is t lest s lrge s the size of its output Emple S = cdefghijklmnopqrstuvwyz Totl length is Σ j<m+ j = m(m+)/2 Similr prolem cn hppen when the length of the string is ritrrily lrger thn the lphet size Solution Lel edges with pir of indices indicting eginning nd end of positions of the sustring in S 5

16 Modified Etension Rules Rule 2: new lef edge (phse i+) crete edge (i+, i+) split edge (p, q) => (p, w) nd (w +, q) Rule : lef edge etension lel hd to e (p,i) efore etension given rule 2 ove nd n induction rgument: (p, q) => (p, q + ) Rule 3 Do nothing 46 Full edge lel representtion String S = Edge-lel Compression String S = 3 (3,6) (,2) [or (4,5)?] (2,2) (6,6) 6 (6,6) 5 (6,6) (3,6) (3,6)

17 A Stopper In ny phse, if suffi etension rule 3 pplies in etension j, it will lso pply in ll etensions k, where k>j, until the end of the phse. The etensions in phse i+ tht re done fter the first eecution of rule 3 re sid to e done implicitly. This is in contrst to ny etension j where the end of S[j, i] is eplicitly found. An etension of tht kind is clled nd eplicit etension. Hence, we cn end ny phse i+ when the first etension rule 3 pplies. 49 Once lef, lwys lef () If t some point in UA lef is creted nd leled j (for the suffi strting t position j of S), then tht lef will remin lef in ll successive trees creted during the lgorithm. In ny phse i, there is n initil sequence of consecutive etensions (strting with etension ) in which only rule or 2 pplies, where let j i e the lst etension in this sequence. Note tht j i j i+. 50 Once lef, lwys lef (2) Implicit etensions: for etensions to j i, write [, e] on the lef edge, where e is symol denoting the current end nd is set to i + once t the eginning. In lter phses, we will not need to eplicitly etend this lef ut rther cn implicitly etend it y incrementing e once in its glol loction. Eplicit etensions: from etension j i+ till first rule 3 etension is found (or until etension i+ is done) 5 7

18 Once lef, lwys lef (3): Single phse lgorithm j* is the lst eplicit etension computed in phse i+ Phse i+ Increment e to i+ (implicitly etending ll eisting leves) Eplicitly compute successive etensions strting t j i+ nd continuing until reching the first etension j* where rule 3 pplies or no more etensions needed Set j i+ to j* -, to prepre to the net phse Oservtion Phse i nd i+ shre t most eplicit etension 52 Emple: S= - () I (,e) e =, J = I 2 (,e) (2,e) 2 e = 2, S[,2] : skip S[2,2] : rule 2, crete(2, e) j 2 = 2 I 3 (,e) (2,e) 2 e = 3, S[,3].. S[2,3] : skip S[3,3] : rule 3 j 3 = 2 53 Emple: S= - (2) I 4 (,e) (2,e) 2 e = 4, S[,4].. S[2,4] : skip S[3,4] : rule 3 S[4,4] : uto skip j 4 = 2 I 5 (,2) (3,e) (5,e) 3 (2,2) (5,e) 5 (3,e) (5,e) 2 4 e = 5, S[,5].. S[2,5] : skip S[3,5] : rule 2, split (,e) (, 2) nd (3,e), crete (5,e) S[4,5] : rule 2, split (2,e) (2,2) nd (3,e), crete (5,e) S[5,5] : rule 2, crete (5,e) j 5 =

19 Emple: S= - (3) I 6 (,2) (3,e) (5,e) 3 (2,2) (5,e) 5 (3,e) (5,e) 2 4 e = 6, S[,6].. S[5,6] : skip S[6,6] : rule 3 j 6 = 5 I 7 (,2) (3,e) (5,e) 3 55 (7,e) 7 (2,2) (5,5) (7,e) 6 (3,e) (5,e) (6,e) e = 7, S[,7].. S[5,7] : skip S[6,7] : rule 2, split (5,e) (5,5) nd (6,e), crete (6,e) S[7,7] : rule 2, crete (7,e) j 7 = 7 Compleity of UA Since ll the implicit etensions in ny phse is constnt, their totl cost is O(m). Totlly, only 2m eplicit etensions re eecuted. The m numer of down-wlking skips is O(m). Time-compleity of Ukkonen s lgorithm: O(m) 56 Finishing up Convert finl implicit suffi tree to true suffi tree Add using just nother phse of eecution Now ll suffies will e leves Replce e in every lef edge with m Just requires trversl of tree which is O(m) time 57 9

20 58 Implementtion Issues () When the size of the lphet grows: For lrge trees suffi links llow n lgorithm to move quickly from one prt of the tree to distnt prt of the tree. This is gret for worst-cse time ounds, ut its horrile if the tree isn't entirely in memory. Thus, implementing ST to reduce prcticl spce use cn e serious concern. The min design issues re how to represent nd serch the rnches out of the nodes of the tree. A prcticl design must lnce etween constrints of spce nd need for speed 59 Implementtion Issues (2) There re four sic choices to represent rnches An rry of size Θ( Σ ) t ech non-lef node v A linked list t node v of chrcters tht pper t the eginning of the edge-lels out of v. If its kept in sorted order it reduces the verge time to serch for given chrcter In the worst cse it, dds time Σ to every node opertion. If the numer of children of v is lrge, then little spce is sved over the rry while noticely degrding performnce A lnced tree implements the list t node v Additions nd serches tke O(logk) time nd O(k) spce, where k is the numer of children of v. This lterntive mkes sense only when k is firly lrge. A hshing scheme. The chllenge is to find scheme lncing spce with speed. For lrge trees nd lphets hshing is very ttrctive t lest for some of the nodes Implementtion Issues (3) When m nd Σ re lrge enough, the est design is proly miture of the ove choices. Nodes ner the root of the tree tend to hve the most children, so rrys re sensile choice t those nodes. For nodes in the middle of suffi tree, hshing or lnced trees my e the est choice. Sometimes the lphet size is eplicitly presented in the time nd spce ounds Construction time is O(m log Σ ),using Θ(m Σ ) spce. m is the size of the string

21 Applictions of Suffi Trees 6 Applictions of Suffi Trees Ect string mtching Sustring prolem for dtse Longest common sustring Suffi rrys 62 Ect String Mtching () Ect mtching prolem: given pttern P of length n nd tet T of length m, find ll occurrences of P in T in O(n+m) time. Overview of the ST pproch: Built ST for tet T in O(m) time Mtch the chrcters of P long the unique pth in ST until either P is ehusted or no more mtches re possile In the ltter cse, P doesn t pper nywhere in T In the former cse, every lef in the sutree elow the point of the lst mtch is numered with strting loction of P in T, nd every strting loction of P in T numers such lef ST pproch spends O(m) preprocessing time nd then O(n+k) serch time, where k is the numer of occurrences of P in T 63 2

22 Ect String Mtching (2) When serch termintes t n element node, P ppers ectly once in the source string T. When the serch for P termintes t rnch node, ech element node in the sutree rooted t this rnch node gives different occurrence of P. c c c c c c Notes: P = ; T = c Pttern P mtches pth down to the point shown y n rrow, nd s required, the leves elow tht point re numered nd Sustring Prolem for Dtse Input: dtse D ( set of strings) nd string S Output: find ll the strings in D contining S s sustring Usge: identity of the person Ect string mtching methods: cnnot work Suffi tree: D is stored in O(m) spce, where m is the totl length of ll the strings in D. Suffi tree is uilt in O(m) time Ech lookup of S (the length of S is n) costs O(n) time 65 Longest common sustring () Input: two strings S nd S 2 Output: find the longest sustring S common to S nd S 2 Emple: S =common-sustring S 2 =common-susequence Then, S=common-sus 66 22

23 Longest common sustring (2) Build suffi tree for S nd S 2 Ech lef represents either suffi from one of S nd S 2, or suffi from oth S nd S 2 Mrk ech internl node v with (2) if there is lef in the sutree of v representing suffi from S (S 2 ) The pth-lel of ny internl node mrked oth nd 2 is sustring common to oth S nd S 2, nd the longest such string is the longest common sustring. 67 Longest common sustring (3) S = c, S 2 =, S = 68 Suffi Arrys () A suffi rry of n m-chrcter string S, is n rry of integers in the rnge to m, specifying the leicogrphic order of the m suffies of S. Emple: S = c 69 23

24 Suffi Arrys (2) A suffi rry of S cn e otined from the suffi tree T of S y performing leicl depth-first serch of T 70 Suffi Arrys: Ect string mtching Given two strings T nd P, where T = m nd P = n, find ll the occurrences of P in T? Using the inry serch on the suffi rry of T, ll the occurrences of P in T cn e found in O(n logm) time. Emple: let T = c nd P = c 7 References Dn Gusfield: Algorithms on Strings, Trees, nd Sequences. University of Cliforni, Dvis.Cmridge University Press,997 El Hunt et l.: A dtse inde to lrge iologicl sequences. Slides t VLDB, 200 R.C.T. Lee nd Chin Lung Lu.: Suffi Trees. Slides t CS 533 Algorithms for Moleculr Biology 72 24