Latent Semantic Indexing

Transcription

1 Laen Semanic Indexing Vecor Space Rerieval migh lead o poor rerieval Unrelaed documens migh be included in he answer se Relevan documens ha do no conain a leas one index erm are no rerieved Reasoning rerieval based on index erms is vague and noisy The user informaion need is more relaed o conceps and ideas han o index erms Key Idea: map documens and queries ino a lower dimensional space composed of higher level conceps which are fewer in number han he index erms Dimensionaliy reducion: Rerieval (and clusering) in a reduced concep space migh be superior o rerieval in he high-dimensional space of index erms 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 36 Despie is success he vecor model suffers some problems. Unrelaed documens may be rerieved simply because erms occur accidenally in i, and on he oher hand relaed documens may be missed because no erm in he documen occurs in he query (consider synonyms, here exiss a sudy ha differen people use he same keywords for expressing he same conceps only 20% of he ime). Thus i would be an ineresing idea o see wheher he rerieval could be based on conceps raher han on erms, by mapping firs erms o a "concep space" (and queries as well) and hen esablish he ranking wih respec o similariy wihin he concep space. This idea is explored in he following.

2 Using Conceps for Rerieval 1 d1 1 d1 d2 c1 d2 2 2 d3 c2 d3 3 3 d4 d4 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 37 This illusraes he approach: raher han direcly relaing documens and erms as in vecor rerieval, here exiss a middle layer ino which boh queries and documens map. The space of conceps can be of smaller dimension. For example, we could deermine ha he query 3 reurns d2, d3, d4 in he answer se based on he observaion ha hey relae o concep c2, wihou requiring ha he documen conains erm d3. The quesion is, of how o obain such a concep space. One possible way would be o find canonical represenaions of naural language, bu his is a difficul ask o achieve. Much simpler, we could ry o use mahemaical properies of he erm-documen marix, i.e. deermine he conceps by marix compuaion.

3 Basic Definiions Problem: how o idenify and compue conceps? Consider he erm-documen marix Le M ij be a erm-documen marix wih rows (erms) and N columns (documens) To each elemen of his marix is assigned a weigh w ij associaed wih k i and d j The weigh w ij can be based on a f-idf weighing scheme 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 38

4 Compuing he Ranking Using M M q M q query doc1 query doc2 doc1 doc2... N doc3 doc4 query doc5 query doc6 doc6 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 39 This figure illusraes how he erm-documen marix M can be used o compue he ranking of he documens wih respec o a query q. (The columns in M and q would have o be normalized o 1)

5 Singular Value Decomposiion Key Idea: exrac he essenial feaures of M and approximae i by he mos imporan ones Singular Value Decomposiion (SVD) M = K S D K and D are marices wih orhonormal columns S is an r x r diagonal marix of he singular values sored in decreasing order where r = min(, N), i.e. he rank of M Such a decomposiion always exiss and is unique (up o sign) Consrucion of SVD K is he marix of eigenvecors derived from M M D is he marix of eigenvecors derived from M M Algorihms for consrucing he SVD of a m n marix have complexiy O(n 3 ) if m n 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 40 For exracing "concepual feaures" a mahemaical consrucion from linear algebra is used, he singular value decomposiion (SVD). I decomposes a marix ino he produc of hree marices. The middle marix is a diagonal marix, where he elemens of his marix are singular values. This decomposiion can always be consruced in O(n^3). Noe ha he complexiy is considerable, which makes he approach compuaionally expensive. There exis however also approximaion echniques o compue S more efficienly.

6 Illusraion of Singular Value Decomposiion N r r r M = K S D N x N x r r x r r x N Assuming N Â 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 41

7 Inerpreaion of SVD Firs inerpreaion: we can wrie M r = i = 1 skd i i i The s i are ordered in decreasing size, hus by aking only he larges ones we obain a good "approximaion" of M Second inerpreaion: he singular values s i are he lenghs of he semiaxes of he hyperellipsoid E defined by { 1} E = Mx x = 2 Each value s i corresponds o a dimension of a "concep space" Third inerpreaion: he SVD is a leas square approximaion 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 42 The singular value decomposiion is an exremely useful consrucion o reveal properies of a marix. This is illusraed by he following facs: We can wrie he marix as he sum of componens weighed by he singular values, hus we can obain approximaions of he marix by only considering he larger singular values. The singular values have also a geomerical inerpreaion, as hey ell us how a uni ball ( x =1) is disored by he muliplicaion wih he marix M. Thus we can view he axes of he hyperellipsoid E as he dimensions of he concep space. The SVD afer eliminaing less imporan dimensions (smaller singular values) can be inerpreed as a leas square approximaion o he original marix.

8 Laen Semanic Indexing In he marix S, selec only he s larges singular values Keep he corresponding columns in K and D The resulan marix is called M s and is given by M s = K s S s D s where s, s < r, is he dimensionaliy of he concep space The parameer s should be large enough o allow fiing he characerisics of he daa small enough o filer ou he non-relevan represenaional deails 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 43 Using he singular value decomposiion, we can now derive an "approximaion" of M by aking only he s larges singular values in marix S. The choice of s deermines on how many of he "imporan conceps" he ranking will be based on. The assumpion is ha conceps wih small singular value in S are raher o be considered as "noise" and hus can be negleced.

9 Illusraion of Laen Semanic Indexing N s N M s = K s S s D s Documen vecors Term vecors x N x s s x s s x N 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 44 This illusraes of how he sizes of he involved marices reduce, when only he firs s singular values are kep for he compuaion of he ranking. The columns in marix K_s correspond o erm vecors, whereas he columns in marix D_s^ correspond o documen vecors. By using he cosine similariy measure beween columns he similariy of he documens can be evaluaed.

10 Answering Queries Documens can be compared by compuing cosine similariy in he "documen space", i.e. comparing heir rows d i and d j in marix D s A query q is reaer like one furher documen i is added like an addiional column o marix M he same ransformaion is applied as for mapping M o D Mapping of M o D M = K S D S -1 K M = D (since K K = 1) D = M K S -1 Apply same ransformaion o q: q* = q K s S s -1 Then compare ransformed vecor by using he sandard cosine measure sim( q*, d ) = i q* ( D ) s q*( D ) s i i 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 45 (D s ) i denoes he i-h column of marix D s The mapping of query vecors can be mahemaically explained as follows: i corresponds o adding a new column (like a new documen) o marix M. We can use he fac ha K s K s =1 Since M s = K s S s D s we obain S s 1 K s M s = D s or D s = M s K s S s 1 Thus adding columns o D s requires he ransformaion ha is applied o obain q*.

11 Example (SVD, s=2) K s S s D s 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 46 SVD for Term-Documen Marix from he running example.

12 Mapping of Query Vecor ino Documen Space = (query "applicaion heory") 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 47

13 Ranked Resul s=2 s=4 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 48 This is he ranking produced for he query for differen values of s.

14 Plo of Terms and Documens in 2-d Space 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 49 Since he concep space has wo dimensions we can plo boh he documens and he erms in he 2 dimensional space. I is ineresing o observe of how semanically "close" erms and documens cluser in he same regions. This illusraes very well he power of laen semanic indexing in revealing he "essenial" semanics in documen collecions.

15 Discussion of Laen Semanic Indexing Laen semanic indexing provides an ineresing concepualizaion of he IR problem Advanages I allows reducing he complexiy of he underline represenaional framework For insance, wih he purpose of inerfacing wih he user Disadvanages Compuaionally expensive Assumes normal disribuion of erms (leas squares), whereas erm frequencies are a coun 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 50

16 4. Classificaion Daa: uples wih muliple caegorical and quaniaive aribues and a leas one caegorical aribue (he class label aribue) Classificaion Predics caegorical class labels Classifies daa (consrucs a model) based on a raining se and he values (class labels) in a class label aribue Uses he model in classifying new daa Predicion/Regression models coninuous-valued funcions, i.e., predics unknown or missing values Typical Applicaions credi approval, arge markeing, medical diagnosis, reamen effeciveness analysis 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 51 Classificaion creaes a GLOBAL model, ha is used for PREDICTING he class label of unknown daa. The prediced class label is a CATEGORICAL aribue. Classificaion is clearly useful in many decision problems, where for a given daa iem a decision is o be made (which depends on he class o which he daa iem belongs).

17 Classificaion Process Model: describing a se of predeermined classes Each uple/sample is assumed o belong o a predefined class based on is aribue values The class is deermined by he class label aribue The se of uples used for model consrucion: raining se The model is represened as classificaion rules, decision rees, or mahemaical formulae Model usage: for classifying fuure or unknown daa Esimae accuracy of he model using a es se Tes se is independen of raining se, oherwise over-fiing will occur The known label of he es se sample is compared wih he classified resul from he model Accuracy rae is he percenage of es se samples ha are correcly classified by he model 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 52 In order o build a global model for classificiaion a raining se is needed from which he model can be derived. There exis many possible models for classificaion, which can be expressed as rules, decision rees or mahemaical formulae. Once he model is buil, unknown daa can be classified. In order o es he qualiy of he model is accuracy can be esed by using a es se. If a cerain se of daa is available for building a classifier, normally one splis his se ino a larger se, which is he raining se, and a smaller se which is he es se.

18 Classificaion: Training Training Se Classificaion Algorihms NAME RANK YEARS TENURED Mike Assisan Prof 3 no Mary Assisan Prof 7 yes Bill Professor 2 yes Jim Associae Prof 7 yes Dave Assisan Prof 6 no Anne Associae Prof 3 no Classifier (Model) IF rank = professor OR years > 6 THEN enured = yes 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 53 In classificaion he classes are known and given by so-called class label aribues. For he given daa collecion TENURED would be he class label aribue. The goal of classificaion is o deermine rules on he oher aribues ha allow o predic he class label aribue, as he one shown righ on he boom.

19 Classificaion: Model Usage Classifier Tes Se Unseen Daa (Jeff, Professor, 4) NAME RANK YEARS TENURED Tom Assisan Prof 2 no Merlisa Associae Prof 7 no George Professor 5 yes Joseph Assisan Prof 7 yes Tenured? YES 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 54 In order o deermine he qualiy of he rules derived from he raining se, he es se is used. We see ha he classifier ha has been found is correc in 75% of he cases. If rules are of sufficien qualiy hey are used in order o classify daa ha has no been seen before. Since he reliabiliy of he rule has been evaluaed as 75% by esing i agains he es se and assuming ha he es se is a represenaive sample of all daa, hen he reliabiliy of he rule applied o unseen daa should be he same.

20 Crieria for Classificaion Mehods Predicive accuracy Speed and scalabiliy ime o consruc he model ime o use he model efficiency in disk-residen daabases Robusness handling noise and missing values Inerpreabiliy undersanding and insigh provded by he model Goodness of rules decision ree size compacness of classificaion rules 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 55

21 Classificaion by Decision Tree Inducion Decision ree A flow-char-like ree srucure Inernal node denoes a es on an aribue Branch represens an oucome of he es Leaf nodes represen class labels or class disribuion Decision ree generaion consiss of wo phases Tree consrucion A sar, all he raining samples are a he roo Pariion samples recursively based on seleced aribues Tree pruning Idenify and remove branches ha reflec noise or ouliers Use of decision ree: Classifying an unknown sample Tes he aribue values of he sample agains he decision ree 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 56 In he following we will inroduce a mehod o consruc a specific kind of classificaion models, namely decision rees. A decision ree splis a each node he daa se ino smaller pariions, based on a es predicae ha is applied o one of he aribues in he uples. Each leaf of he decision ree is hen associaed wih one specific class label. Generally a decision ree is firs consruced in a op-down manner by recursively spliing he raining se using condiions on he aribues. How hese condiions are found is one of he key issues of decision ree inducion. Afer he ree consrucion i usually is he case ha a he leaf level he granulariy is oo fine, i.e. many leaves represen some kind of excepional daa. Thus in a second phase such leaves are idenified and eliminaed. Using he decision ree classifier is sraighforward: he aribue values of an unknown sample are esed agains he condiions in he ree nodes, and he class is derived from he class of he leaf node a which he sample arrives.

22 Classificaion by Decision Tree Inducion income suden credi_raing buys_compuer high no fair no high no excellen no high no fair yes medium no fair yes low yes fair yes low yes excellen no low yes excellen yes medium no fair no low yes fair yes medium yes fair yes medium yes excellen yes medium no excellen yes high yes fair yes medium no excellen no age? <=30 overcas >40 suden? credi raing? no yes fair excellen no yes yes no yes buys_compuer? 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 57 A sandard approach o represen he classificaion rules is by a decision ree. In a decision ree a each level one of he exising aribues is used o pariion he daa se based on he aribue value. A he leaf level of he classificaion ree hen he values of he class label aribue are found. Thus, for a given daa iem wih unknown class label aribue, by raversing he ree from he roo o he leaf is class can be deermined. Noe ha in differen branches of he ree, differen aribues may be used for classificaion. The key problem of finding classificaion rules is hus o deermine he aribues ha are used o pariion he daa se a each level of he decision ree.

23 Algorihm for Decision Tree Consrucion Basic algorihm for caegorical aribues (greedy) Tree is consruced in a op-down recursive divide-and-conquer manner A sar, all he raining samples are a he roo Examples are pariioned recursively based on es aribues Tes aribues are seleced on he basis of a heurisic or saisical measure (e.g., informaion gain) Condiions for sopping pariioning All samples for a given node belong o he same class There are no remaining aribues for furher pariioning majoriy voing is employed for classifying he leaf There are no samples lef Aribue Selecion Measure Informaion Gain (ID3/C4.5) 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 58 The basic algorihm for decision ree inducion proceeds in a greedy manner. Firs all samples are a he roo. Among he aribues one is chosen o pariion he se. The crierion ha is applied o selec he aribue is based on measuring he informaion gain ha can be achieved, or how much uncerainy on he classificaion of he samples is removed by he pariioning. Three condiions can occur such ha no furher splis can be performed: all samples are in he same class, herefore furher spliing makes no sense, no aribues are lef which can be used o spli. Sill samples from differen classes can be in he leaf, hen majoriy voing is applied. no samples are lef.

24 Which Aribue o Spli? Maximize Informaion Gain Class P: buys_compuer = yes Class N: buys_compuer = no I(p, n) = I(9, 5) =0.940 The amoun of informaion, needed o decide if an arbirary example in S belongs o P or N I( p, n) p p n log 2 log p+ n p+ n p+ n = 2 n p+ n age pi ni I(pi, ni) <= > E ( age ) 5 4 = I (2,3) + I (4,0 ) I (3,2) = Gain( age) = I( pn, ) Eage ( ) = Gain( income) = Gain( suden) = Gain( credi _ raing) = Aribue A pariions S ino {S 1, S 2,, S v } If S i conains p i examples of P and n i examples of N, he expeced informaion needed o classify objecs in all subrees S i is E ( A) 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 59 p + ni n = ν i i = 1 p + I ( p, n ) The encoding informaion ha would be gained by branching on A Gain( A ) = I ( p, n ) E ( A ) i i Here we summarize he basic idea of how spli aribues are found during he consrucion of a decision ree. I is based on an informaion-heoreic argumen. Assuming ha we have a binary caegory, i.e. wo classes P and N ino which a daa collecion S needs o be classified, we can compue he amoun of informaion required o deermine he class, by I(p, n), he sandard enropy measure, where p and n denoe he cardinaliies of P and N. Given an aribue A ha can be used for pariioning furher he daa collecion in he decision ree, we can calculae he amoun of informaion needed o classify he daa afer he spli according o aribue A has been performed. This value is obained by calculaing I(p, n) for each of he pariions and weighing hese values by he probabiliy ha a daa iem belongs o he respecive pariion. The informaion gained by a spli hen can be deermined as he difference of he amoun of informaion needed for correc classificaion before and afer he spli. Thus we calculae he reducion in uncerainy ha is obained by spliing according o aribue A and selec among all possible aribues he one ha leads o he highes reducion. On he lef hand side we illusrae hese calculaions for our example.

25 Pruning Classificaion reflecs "noise" in he daa Remove subrees ha are overclassifying Apply Principle of Minimum Descripion Lengh (MDL) Find ree ha encodes he raining se wih minimal cos Toal encoding cos: cos(m, D) Cos of encoding daa D given a model M: cos(d M) Cos of encoding model M: cos(m) cos(m, D) = cos(d M) + cos(m) Measuring cos For daa: coun misclassificaions For model: assume an appropriae encoding of he ree 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 60 I is imporan o recognize ha for a es daase a classifier may overspecialize and capure noise in he daa raher han general properies. One possibliy o limi overspecializaion would be o sop he pariioning of ree nodes when some crieria is me (e.g. number of samples assigned o he leaf node). However, in general i is difficul o find a suiable crierion. Anoher alernaive is o firs buil he fully grown classificaion ree, and hen in a second phase prune hose subrees ha do no conribue o an efficien classificaion scheme. Efficiency can be measured in ha case as follows: if he effor in order o specify a class (he implici descripion of he class exension) exceeds he effor o enumerae all class members (he explici descripion of he class exension), hen he subree is overclassifying and non-opimal. This is called he priniciple of minimum descripion lengh. To measure he descripion cos a suiable merics for he encoding cos, boh for rees and daa ses is required. For rees his can be done by suiably couning he various srucural elemens needed o encode he ree (nodes, es predicaes), whereas for explici classificaion, i is sufficien o coun he number of misclassificaions ha occur in a ree node.

26 Exracing Classificaion Rules from Trees Represen he knowledge in he form of IF-THEN rules One rule is creaed for each pah from he roo o a leaf Each aribue-value pair along a pah forms a conjuncion The leaf node holds he class predicion Rules are easier for humans o undersand Example IF age = <=30 AND suden = no IF age = <=30 AND suden = yes IF age = IF age = >40 AND credi_raing = excellen IF age = >40 AND credi_raing = fair THEN buys_compuer = no THEN buys_compuer = yes THEN buys_compuer = yes THEN buys_compuer = yes THEN buys_compuer = no 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 61 A decision ree can also be seen as an implici descripion of classificaion rules. Classificaion rules represen he classificaion knowledge as IF- THEN rules and are easier o undersand for human users. They can be easily exraced from he classificaion ree as described.

27 Decision Tree Consrucion wih Coninuous Aribues Binary decision rees For coninuous aribues A a spli is defined by val(a) < X For caegorical aribues A a spli is defined by a subse X domain(a) coninuous caegorical class Deermining coninuous aribue splis Soring he daa according o aribue value Deermine he value of X which maximizes informaion gain by scanning hrough he daa iems 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 62 If coninuous aribues occur he decision ree can be consruced as a binary decision ree, by finding an aribue value ha splis he samples ino 2 pariions. Consequenly also caegorical aribues are reaed ha way. In order o deermine suiable spli poins for coninuous aribues he samples need firs o be sored. On he lef we see a sample daabase wih he class label aribue Risk and a coninuous aribue Age and a caegorical aribue Car Type used for classificaion.

28 Example I(p, n) = I(4, 2) =0.918 E(A) = 0 + ½ I(1, 2) =0.459 Gain = I(p, n) E(A) = spliing o {spors} and {family, ruck} E(A) = 0 + 2/3 I(2, 2) =0.666 Gain = I(p, n) E(A) = /4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 63 This example illusraes he principle of how spliing is performed boh for coninuous and caegorical aribues. For reasons we will discuss laer we consruc separae aribue liss for each aribue, ha is used for classificaion. The aribue lis conains he aribue for which i is consruced (i.e. Age and Car Type), he class label aribue and he ransacion idenifier id. The aribue lis is sored for coninuous aribues. Now le us see of how a spli poin is found for he coninuous aribue Age. The disribuion of he class aribue for he whole daa se (4 High and 2 Low) is sored in variables C_above and a poiner is posiioned on op of he aribue lis. Then he poiner is moved downwards. Whenever he Age value changes he values C_below and C_above are updaed (such ha hey always keep he disribuion of H and L values above and below he poiner). Also, when he Age value changes he informaion gain is compued if he spli were performed a ha poin (in he same way as done for caegorical aribues before). Afer passing hrough he aribue lis he opimal spli value for he Age aribue is known. For he caegorical aribue we have o esablish a saisics of he disribuion of he classes for each of he possible aribue values and sore i in a marix. Then we check he informaion gain ha can be obained for each of he possible subses of aribue values and hus deermine he opimal "spli" for he caegorical aribue. Finally, he aribue is chosen ha resuls in he bes (binary) spli.

29 Scalabiliy Naive implemenaion A each sep he daa se is spli and associaed wih is ree node Problem wih naive implemenaion For evaluaing which aribue o spli daa needs o be sored according o hese aribues Becomes dominaing cos Idea: Presoring of daa and mainaining order hroughou ree consrucion Requires separae sored aribue ables for each aribue Aribue seleced for spli: spliing aribue able sraighforward Build Hash Table associaing TIDs of seleced daa iems wih pariions Selec daa from oher aribue ables by scanning and probing he hash able 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 64 If we would associae he complee able of samples wih he nodes of he classificaion ree during is inducion, we would have o re-sor he able consanly, when searching for spli poins for differen coninuous aribues. This would become for large daabases he dominaing cos in he algorihm. For ha reason he aribue values are sored in separae ables (as we have already seen in he example before) which are sored once a he beginning. Afer a spli he order in he aribue ables can be mainained: For he aribue able on which he spli occurs he able needs jus o be cu ino wo pieces. For he oher ables a scan is performed, afer building a hash able of he TIDs is consruced for associaing he TIDs wih heir pariion. During he scan he hash able is probed in order o redirec he uples o heir proper pariion. The order of he aribue able is mainained during ha process. Remark: his is a similar idea as was employed in he consrucion of invered files!

30 Example hash able 0 L L probe 1 2 L R 3 R R 4 5 R L 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 65 This illusraion demonsraes a spli of aribue ables.

31 Summary Wha is he difference beween clusering and classificaion? Wha is he difference beween model consrucion, model es and model usage in classificaion? Which crierion is used o selec an opimal aribue for pariioning a node in he decision ree? How are clusers characerized? When is he k-means algorihm erminaing? 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 66

32 References Texbook Jiawei Han, Daa Mining: conceps and echniques, Morgan Kaufman, 2000, ISBN Some relevan research lieraue R. Agrawal, T. Imielinski, and A. Swami. Mining associaion rules beween ses of iems in large daabases. SIGMOD'93, , Washingon, D.C. 2003/4, Karl Aberer, EPFL-SSC, Laboraoire de sysèmes d'informaions réparis Daa Mining - 67