Induction of Quadratic Decision Trees using Genetic Algorithms and k-d Trees

Transcription

1 Inducton of Quadratc ecson Trees usng Genetc Algorthms and k- Trees SAI-CHEONG NG, KWONG-SAK LEUNG epartment of Computer Scence and Engneerng The Chnese Unversty of Hong Kong Shatn, New Terrtores HONG KONG SPECIAL AMINISTRATIVE REGION Abstract: - Genetc Algorthm-based Quadratc ecson Tree ( QT) has been appled successfully n varous classfcaton problems wth non-lnear class boundares. However, the executon tme of QT s qute long. In ths paper, a new verson of QT, called Genetc Algorthm-based Quadratc ecson Tree wth k- Tree ( QT wth k- Tree), s proposed. In the proposed algorthm, a k- tree s constructed at each non-leaf node of a decson tree to evaluate the mpurty reducton due to a quadratc hypersurface. Experments show that the proposed algorthm s faster than QT on datasets wth lower dmensonalty or the number of tranng samples s suffcently large, wthout sacrfcng the qualty of a quadratc decson tree. Although the Gn ndex s used to measure the mpurty of a set of samples, varous mpurty measures ncludng entropy and the Twong value can also be used n the proposed algorthm. Key-Words: - ecson Trees, Genetc Algorthms, k- Trees, Quadratc Hypersurface, Impurty, ata Mnng Introducton ecson trees are popular supervsed classfcaton algorthms. Snce the last century, decson trees have been appled successfully n varous felds, ncludng character recognton, remote sensng, medcal dagnoss and so on []. In addton, a decson tree can be transformed to a set of rules, whch can be more easly understood by doman experts []. A decson tree s a tree structure whch classfes an nput sample nto one of ts possble classes. Each non-leaf node contans a decson crteron to select whch chld node should be chosen next. An nput sample arrvng at a leaf node s classfed accordng to the class label assocated wth the node. To learn a decson tree, a set of samples s parttoned nto two dsjont subsets, called tranng set and testng set. The tranng set s employed to construct a decson tree. A constructed decson tree should mnmze the number of msclassfcatons on the testng set, nstead of the tranng set. To start the constructon of a decson tree, a root node s created frst. The decson crteron assocated wth the root node s determned n order to partton all the tranng samples nto two dsjont subsets. The optmalty of the decson crteron can be measured usng varous mpurty measures, ncludng the Gn ndex [3], the Twong value [3] and so on. After determnng the optmal decson crteron, two chld nodes are created recursvely. At each chld node, the assocated optmal decson crteron s determned usng the tranng samples arrvng at t untl a stoppng crteron s satsfed. Genetc Algorthms (GAs) are optmzaton algorthms nspred by the prncples of evoluton and genetcs to fnd solutons to problems [4]. A populaton of chromosomes are ntalzed, representng canddate solutons to a problem. Each chromosome can be coded as a fnte bnary strng or a vector of real numbers. Each chromosome has ts assocated ftness value, reflectng the optmalty of the correspondng soluton to a problem. Better chromosomes are more lkely to be selected for reproducton. Crossover and mutaton operators are appled to the chromosomes n the populaton. In crossover, a par of chromosomes are selected and combned to generate two offsprng chromosomes. In mutaton, one or more elements of a chromosome are modfed to ncrease the dversty of the populaton. A k- tree s a bnary search tree whch recursvely dvdes a d-dmensonal space usng an axs-parallel hyperplane [5]. A k- tree has been appled to speed up nearest neghbor queres [6]. In ths study, a k- tree s employed to evaluate the mpurty reducton due to a quadratc hypersurface. The rest of the paper s organzed as follows. In the next secton, we provde the related work of decson trees, the role of GAs, and the motvaton n ths study. In secton 3, we descrbe how to construct a quadratc decson tree usng GAs. In secton 4, the algorthm to fnd the optmal quadratc hypersurface at each non-leaf node of a decson tree s dscussed.

2 In secton 5, the algorthm to evaluate the mpurty reducton due to a quadratc hypersurface s descrbed. In secton 6, the performance of the proposed algorthm s evaluated and compared wth varous decson tree algorthms. A concluson of the paper s gven n the last secton. Background. ecson Tree Algorthms A unvarate decson tree consders one attrbute at each non-leaf node. For a numerc attrbute, the decson crteron s of the form: x > γ () where x s an nput attrbute and γ s a real constant. If a sample x = (x, x, x d ) T wth d nput attrbutes satsfes (), the rght chld node wll be chosen next. Otherwse, the left chld node wll be chosen. Common unvarate decson tree algorthms nclude classfcaton and regresson trees (CART) [3], C4.5 [7], C5.0 [8] and so on. When the nput samples are correlated, a large unvarate decson tree wth poor generalzaton may be constructed. A multvarate decson tree consders more than one nput attrbutes at each non-leaf node. In a lnear decson tree, the decson crteron s of the form: ω x + ω x + + ω d x d > γ () where ω, ω, ω d are the coeffcents of x, x, x d respectvely and γ s a real constant. The decson crteron s equvalent to a hyperplane wth arbtrary orentaton n the attrbute space. Common lnear decson tree algorthms nclude classfcaton and regresson trees wth lnear combnatons (CART-LC) [3], OC [], lnear machne dscrmnant tree (LMT) [9], lnear dscrmnant tree by Yıldız et al. [0] and so on. In non-lnear decson trees (NT) proposed by Ittner and Schlosser [], new attrbutes are constructed from the par-wse products and the squares of the orgnal attrbutes n each nput sample. A new sample s created by these constructed attrbutes and the orgnal attrbutes. For example, a new sample y = (3, 5, 3x5, 3, 5) T s created from an nput sample x = (3, 5) T. At each non-leaf node, OC algorthm s employed to determne the optmal hyperplane n the new attrbute space. A hyperplane n the new attrbute space s equvalent to a quadratc hypersurface n the orgnal attrbute space.. Applcaton of Genetc Algorthms to Constructon of ecson Trees In bnary tree-genetc algorthm (BTGA) [] and OC-GA [3], GAs are appled to fnd the optmal hyperplane at each non-leaf node of a decson tree because the problem of fndng the optmal hyperplane s NP-complete [4]. In genetc algorthm-based quadratc decson tree ( QT) [5], the decson crteron at each non-leaf node of a decson tree s of the form: x T Ax + b T x > γ (3) where A = (a j,k ) s a symmetrc matrx of order d, b = (b, b, b d ) T s a d-dmensonal column vector, γ s a real constant. The decson crteron s equvalent to a quadratc hypersurface n the attrbute space. GAs are employed to fnd the optmal quadratc hypersurface to partton samples nto two dsjont subsets..3 Motvaton Although QT provdes a better approxmaton to non-lnear class boundares, the executon tme of QT s qute long. A novel decson tree, called Genetc Algorthm-based Quadratc ecson Tree wth k- Tree ( QT wth k- Tree), s proposed. A k- tree s constructed before searchng for the optmal quadratc hypersurface at each non-termnal node of a decson tree. To evaluate the mpurty reducton after parttonng a set S of tranng samples nto two dsjont subsets, only some of the tranng samples n S are requred to be consdered n the proposed algorthm. 3 Constructon of Quadratc ecson Trees To construct a quadratc decson tree, the procedure createtnode() s nvoked recursvely. To create the root node of a quadratc decson tree, the procedure createtnode() accepts the tranng set as the parameter. Fg. shows the algorthm of createtnode(), whch s modfed from [, 5]. 4 Evolvng Quadratc Hypersurface usng Genetc Algorthms At each non-leaf node of a decson tree, GAs are appled to evolve the optmal quadratc hypersurface for parttonng tranng samples arrvng at t nto

3 two dsjont subsets. In the followng sectons, we defne as the number of terms requred to represent a quadratc hypersurface. From (3), the relaton between and d s gven by: = (d+)(d+)/. (4) 4. Populaton Intalzaton Gven a populaton of L chromosomes P = {θ, θ, θ L }. Each chromosome θ, =,, L, represents the followng quadratc hypersurface: x T A x + b T x > γ, =,, L, (5) where A = (a,j,k ), =,, L, are symmetrc matrces of order d, b = (b,, b,, b,d ) T, =, L, are d-dmensonal column vectors and γ, γ, γ L are real constants. The chromosome θ = (w,, w,, w, ) s encoded such that: a,j,j f j =, d a,,j-d+ f j = d+, d+, d- a,,j-(d-)+ f j = d, d+, 3d-3 w,j = a,d-,d f j = d(d+)/ b, j-d(d+)/ f j = d(d+)/+, - γ f j =. Each value of w,, w,, w,-, =,, L, s ntalzed usng a unform random number wthn the range [-, ]. The values of w,, w,..., w L, are ntalzed so that each quadratc hypersurface PROCEURE createtnode INPUT: A set of tranng samples S OUTPUT: A new node N of a decson tree. If S < n 0 or the mpurty of S s less than a threshold g 0, N s a termnal node and go to step 7.. Construct a k- tree usng the set S 3. Fnd the optmal quadratc hypersurface, specfed by A = (a j,k ), b = (b, b,, b d ) T and constant term γ, usng GAs such that the mpurty reducton s maxmzed. 4. If the mpurty reducton s less than g 0, N s a termnal node and go to step efne S L = {x S x T Ax + b T x γ} and S R = S \ S L 6. Invoke createtnode(s L ) and createtnode(s R ), and then skp step etermne the class label assocated wth the node N. Fg. Algorthm to create a node of the proposed decson tree contans a pont on the lne segment jonng a par of tranng samples wth dstnct classes. Moreover, the values of w,, w,, w,, =,,, L, are normalzed such that the followng condton s satsfed: j = w (6), j = Each chromosome represents a canddate quadratc hypersurface n the attrbute space. It s hoped that the qualty of each chromosome s mproved by reproducton, crossover and mutaton. 4. Ftness Evaluaton The ftness value of a chromosome equals the mpurty reducton after parttonng a set of tranng samples nto two dsjont subsets. The algorthm to measure the mpurty reducton due to a quadratc hypersurface s descrbed n secton Selecton After the ftness value of each chromosome s evaluated, L chromosomes are selected and replcated to the matng pool usng the roulette wheel selecton method. Each chromosome θ s selected wth a probablty p, whch s proportonal to ts ftness value. Chromosomes wth hgher ftness values are more lkely to be replcated to the matng pool. 4.4 Crossover A par of chromosomes are selected (wthout replacement) frst. Crossover s appled to each par of selected chromosomes wth a fxed probablty p c. Gven two parent chromosomes u = (u, u, u ) and v = (v, v, v ), f u v < 0, the values of v, v, v wll be negated so that u v > 0. Two random numbers t, t [-α, α+] are generated, where α s a user-defned real constant. Two offsprng o and o are generated such that: o = (t u +( t )v, t u +( t )v, t u + ( t )v ) o = (t v +( t )u, t v +( t )u. t v + ( t )u ) We set α = 0.5 for all the experments reported n secton Mutaton After crossover operator s appled to these chromosomes, each offsprng chromosome

4 undergoes mutaton. Non-unform mutaton [6] s employed n ths paper. Each element of a chromosome s mutated wth a mutaton probablty p m. Suppose mutaton operator s appled to a chromosome θ = (w, w, w ). When the value of w, =,, -, s selected to mutate, t s set to zero wth probablty p m and modfed by non-unform mutaton wth probablty - p m f w 0. If w = 0, the value of w s set to zero wth probablty - p m and modfed by non-unform mutaton wth probablty p m. When the value of w s selected to mutate, ts value s modfed by non-unform mutaton. 4.6 Evolvng the Optmal Quadratc Hypersurface usng Genetc Algorthms When a new node of a quadratc decson tree s created, GAs are appled to evolve the optmal quadratc hypersurface provded that the number of tranng samples arrvng at t s greater than a threshold n 0. An eltst strategy s employed to ensure that the best chromosome n the current generaton s preserved n the next generaton. The followng outlnes the steps to fnd the optmal quadratc hypersurface usng GAs:. Intalze a populaton of chromosomes P = {θ, θ, θ L }.. Evaluate the ftness value of each chromosome n P. 3. enote θ best the best chromosome n the populaton P, T the number of generatons, τ the current generaton number. Set τ = o the followng whle τ < T Select L chromosomes θ (), θ (), θ (L) P usng the roulette wheel selecton method. enote the matng pool M = {θ (), θ (), θ (L) }. Set M = Crossover(M). Set M = Mutaton(M). Calculate the ftness value of each chromosome n M. enote θ worst the worst chromosome n M. Set P = M \ {θ worst } + {θ best }. Set θ best as the best chromosome n P. Set τ = τ The best chromosome θ best s chosen to dvde the set of tranng samples arrvng at node N h of nto two dsjont subsets, provded that the ftness value of θ best s greater than a threshold g 0. 5 Evaluatng the Impurty Reducton due to a Quadratc Hypersurface wth a k- Tree Suppose S h s the set of tranng samples arrvng at node N h of a decson tree. In order to evaluate the mpurty reducton after parttonng S h nto two dsjont subsets usng the decson crteron n (3), a k- tree s constructed so that t s requred to test whether the condton (3) s satsfed or not for some of the tranng samples n S h only. In QT, all the tranng samples n S h are requred to be tested whether the condton (3) s satsfed or not. In ths secton, we denote Q as a node of a k- tree and S Q s the set of tranng samples arrvng at node Q. Let T Q,L be the set of tranng samples arrvng at the left chld node of Q and T Q,R be the set of tranng samples arrvng at the rght chld node of Q. efne y Q = (y Q,, y Q,, y Q,d ) and z Q = (z Q,, z Q,, z Q,d ), where y Q, and z Q,, =,, d, are the mnmum and maxmum values of the th nput attrbute n S Q respectvely. 5. Constructon of a k- Tree To construct a k- tree, the procedure createktnode() s nvoked recursvely. To create the root node of a k- tree, the procedure createktnode(s h, 0) s nvoked. Fg. shows the algorthm of createktnode(). PROCEURE createktnode(s Q, k) INPUTS: - A set of tranng samples S Q. - An nteger k {0,, d-}, where the (k+) th nput attrbute s used to partton S Q. OUTPUT: A new node Q of a k- tree. If S Q < d+, go to step 7.. Set y Q and z Q usng the set S Q. 3. Set T Q,L = T Q,R = φ. 4. For each x = (x,, x,, x,d ) S Q If x,k+ < β, set T Q,L = T Q,L + {x }, where β = x / S x S Q, k+ Q Otherwse, set T Q,R = T Q,R + {x }. 5. Invoke createktnode(t Q,L, (k+) mod d) and createktnode(t Q,R, (k+) mod d) 6. Return the node Q and skp step Label Q as a leaf node and create an array of ponters to the tranng samples n S Q. Fg. Algorthm to create a node of a k- tree

5 5. Evaluatng the Impurty Reducton ue to a Quadratc Hypersurface Gven a chromosome θ = (w, w, w ) represents the quadratc hypersurface n (3), the mpurty reducton s calculated by nvokng the procedure evaluateftness() recursvely. The root node of the constructed k- tree s consdered frst. At each non-leaf node of a k- tree, ts chld nodes wll be consdered f necessary. Suppose the node Q s beng consdered, f ether (7) or (8) s satsfed, = = where λ = and ρ = w [ λ ( + sgn( w )) + ρ ( sgn( w ))] > w (7) w[ λ ( sgn( w )) + ρ ( + sgn( w ))] w (8) y Q, f =, d y Q, y Q,-d+ f = d+, d+, d- y Q, y Q,-(d-)+ f = d, d+, 3d-3 y Q,d- y Q,d f = d(d+)/ f = d(d+)/+, -, y Q,-d(d+)/ z Q, f =, d z Q, z Q,-d+ f = d+, d+, d- z Q, z Q,-(d-)+ f = d, d+, 3d-3 z Q,d- z Q,d f = d(d+)/ f = d(d+)/+, -, z Q,-d(d+)/ t s not requred to test whether the condton (3) s satsfed or not for each sample n S Q. If nether (7) nor (8) holds, the chld nodes of Q wll be consdered f Q s a non-leaf node of a k- tree. If Q s a leaf node of a k- tree, all the tranng samples n S Q wll be consdered. Fg. 3 shows the algorthm of procedure evaluateftness(). 6 Performance Evaluaton To evaluate the performance of the proposed algorthm, several artfcal and publc-doman datasets (from the UCI machne learnng repostory [7]) are chosen. Several decson tree algorthms, ncludng C4.5, C5.0, OC, LMT, NT, BTGA, OC-GA, OC-ES (a hybrd algorthm of decson trees and evoluton strateges [3] and QT, are used for performance comparson. All the experments are executed on a dual Intel Xeon machne (.GHz). The frst dataset s an artfcal data set, called AS. AS s a three-class problem and there are PROCEURE evaluateftness(q, S Q, θ, r, l) INPUTS: - A node Q of a k- tree - A set S Q of tranng samples arrvng at node Q - h Q = (h Q,, h Q,, h Q,C ), where C s the number of classes and h Q, s the number of tranng samples of class arrvng at node Q - θ = (w, w,w ), the chromosome representng the quadratc hypersurface n (3) INPUT/OUTPUT: r = (r, r r C ), where r s the number of samples of class such that (3) s satsfed. l = (l, l, l C ) where l s the number of samples of class such that (3) s volated. OUTPUT: f, the ftness value of θ ALGORITHM: - If Q s the root node of a k- tree, set r = l = 0. - If S Q > and (7) s satsfed, set r = r + h Q Else If S Q > and (8) s satsfed, set l = l + h Q Else If Q s a non-leaf node of a k- tree, - Invoke evaluateftness(q, T Q,L, θ, r, l) - Invoke evaluateftness(q, T Q,R, θ, r, l) Else - For each tranng sample x S Q Let K be the class label of the sample x If (3) s satsfed for the sample x, set r K = r K + Else set l K = l K + - End For End If - If Q s the root node of a k- tree, set C h C Q, r g' = ( ), g = S ( ), T = Q = Q, R C l T Q, R TQ, L g = ( ), f = g' g g = TQ, L SQ SQ Else set f = 0 Fg. 3 Algorthm of the procedure evaluateftness() 000 samples n AS. Each sample s a two-dmensonal vector (x, x ), where x, x [0, 000]. If a sample satsfes (9), t wll be labelled as class. If a sample volates (9) but satsfes (0), t wll be labelled as class. If nether (9) nor (0) s satsfed, t wll be labelled as class 3. (x -550) /500+(x -500) /44400> (9) x >600sn(πx /000) (0)

6 The second dataset s also an artfcal data set, called AS. There are 000 samples n AS. Each sample s a two-dmensonal vector (x, x ), where x, x [0, 000]. If a sample satsfes (), t wll be labelled as class. If a sample volates () but satsfes (), t wll be labelled as class. If nether () nor () s satsfed, t wll be labelled as class 3. (x -500) /0500+(x -500) /500> () (x -500) /90000+(x -50) /6500> () The ECOLI and BALANCE datasets are publc-doman datasets from the UCI machne learnng repostory [7]. For the ECOLI dataset, there are 336 samples and 8 possble classes. Each sample has 7 numerc attrbutes. For the BALANCE dataset, there are 65 samples. Each sample has 4 numerc attrbutes and a class label. The BALANCE dataset s a three-class problem. Tables and respectvely show the average and standard devaton of valdaton accuracy (%) and executon tme (n seconds) on each dataset when 0-fold cross-valdaton s appled wth 0 runs. The populaton sze for OC-GA, BTGA, QT and the proposed algorthm s 00. Tables 3, 4 and 5 show the other parameters for the decson tree algorthms nvolvng GAs and ES on each dataset so as to maxmze ther valdaton accuraces. Snce the executon tme of one generaton s dfferent for each of these algorthms even the tranng set s dentcal, the number of generatons for OC-GA, OC-ES and BTGA are adjusted so that the executon tme for each of these algorthms s comparable or larger than that of QT on each dataset. Preprunng s employed n BTGA, QT and the proposed algorthm. Table 5 shows the optmal values of n 0 (mnmum number of tranng samples at each node) and g 0 (mnmum mpurty reducton) for these algorthms on each dataset. AS AS ECOLI BALANCE C ± ± ± ± 0.7 C ± ± ± ± 0.8 OC 95.3 ± ± ±.8 9. ± 0.6 LMT 94.8 ± ± ± ±. NT 95.5 ± ± ± ±. OC-GA 94.6 ± ± ±. 94. ± 0.7 OC-ES 95.7 ± ± ± ±. BTGA 96.5 ± ± ± ± ± ± ± ± 0.5 QT 99. ± ± ± ± 0.5 Algorthm Table : Comparson of Valdaton Accuracy (%) of Varous ecson Tree Algorthms The valdaton accuraces of varous decson tree algorthms are compared usng one-sded t-test. From Table, QT and the proposed algorthm outperform the others on AS, AS, ECOLI and BALANCE n terms of valdaton accuracy at 95% sgnfcance level. The qualty of a quadratc decson tree constructed by the proposed algorthm s dentcal to that constructed by QT. AS AS ECOLI BALANCE C4.5 < < < < C5.0 < < < < OC 3.± ± ± ± 0.6 LMT 7.3 ± ± ± ± 0.5 NT 4.±. 0 ± 5 4 ± ±.6 OC-GA 65 ± 5 66 ± 6 80 ± ± 8 OC-ES 57 ± 6 5 ± 3 070±07 4 ± BTGA 467 ± 6 77 ± 80 54± ± 68 QT 9 ± 3 ± 894 ± ± 7 Algorthm 53 ± 94 ± 94 ± ± 7 Table : Comparson of Executon Tme (n seconds) of Varous ecson Tree Algorthms AS AS ECOLI BALANCE OC-GA OC-ES BTGA QT 000 Algorthm 000 Table 3: Number of Generatons for OC-GA, OC-ES, BTGA, QT and the Algorthm AS AS ECOLI BALANCE p c p m p c p m p c p m p c p m OC-GA BTGA QT Algorthm Table 4: Crossover Probablty (p c ) and Mutaton Probablty (p m ) of OC-GA, BTGA, QT and the Algorthm AS AS ECOLI BALANCE n 0 g 0 n 0 g 0 n 0 g 0 n 0 g 0 BTGA QT Algorthm Table 5: Mnmum Number of Samples (n 0 ) and Mnmum Impurty Reducton (g 0 ) of BTGA, QT and the Algorthm

7 From Table, the proposed algorthm and QT run slower than other decson tree algorthms. The executon tme of the proposed algorthm s compared wth that of QT. The parameters for QT and the proposed algorthm are dentcal n order to have a far comparson of ther executon tmes. From Table, the proposed algorthm s faster than QT on AS, AS and BALANCE. However, the executon tme of the proposed algorthm s longer than that of QT on ECOLI. It s possble that all the tranng samples n S Q le on the same sde of the quadratc hypersurface n (3) even both of the condtons (7) and (8) are volated. Ths stuaton s more lkely to occur when the number of nput attrbutes of a sample ncreases. (Recall that there are 7 nput attrbutes on ECOLI) Although quadratc programmng can be used to reduce the chance of occurrence of ths stuaton, t s computatonally ntensve to use quadratc programmng nstead of the condtons (7) and (8) to determne whether the chld nodes of Q are consdered or t s necessary to evaluate the sgn of all the samples n S Q. 7 Concluson QT and the proposed algorthm provde a better approxmaton to non-lnear class boundares. The qualty of a quadratc decson tree constructed by the proposed algorthm s dentcal to that constructed by QT provded that they use the dentcal set of parameters. Although the executon tme of the proposed algorthm s longer than that of QT on ECOLI, the proposed algorthm provdes an alternatve algorthm to calculate the mpurty of a set of tranng samples. Moreover, varous mpurty measures, ncludng entropy, the Gn ndex and the Twong value, may be used n the proposed algorthm. In the future, the sze of a k- tree can be controlled dynamcally n order to mnmze the tme requred to evaluate the mpurty reducton due to a quadratc hypersurface. References: [] S.R. Safavan,. Landgrebe, A Survey of ecson Tree Classfer Methodology, IEEE Trans. on Systems, Man, & Cybernetcs, Vol., No. 3, May 99, pp [] S.K. Murthy, S. Kasf, S. Salzberg, A System for Inducton of Oblque ecson Trees, Journal of Artfcal Intellgence Research, Vol., 994, pp. -3. [3] L. Breman, J. Fredman, R. Olshen, C. Stone, Classfcaton and Regresson Trees, Wadsworth Internatonal Group, 984. [4].E. Goldberg, Genetc Algorthms n Search, Optmzaton and Machne Learnng, Addson Wesley, 989. [5] H. Samet, The esgn and Analyss of Spatal ata Structures, Addson Wesley, 990. [6] N. Yanlos, ata Structures and Algorthms for Nearest Neghbor Search n General Metrc Spaces, SOA: ACM-SIAM Symposum on screte Algorthms, 993, pp [7] J.R. Qunlan, C4.5: Programs for Machne Learnng, Morgan Kaufmann, 993. [8] J.R. Qunlan, See5/C5.0, Rulequest Research, 999. [9] C.E. Brodley, P.E. Utgoff, Multvarate ecson Trees, Machne Learnng, Vol. 9, 995, pp [0] O.T. Yıldız, E. Alpaydın, Lnear scrmnant Trees, Proceedngs of the 7th Internatonal Conference on Machne Learnng, pp [] A. Ittner, M. Schlosser, Non-lnear ecson trees NT, Proceedngs of the 3th Internatonal Conference on Machne Learnng, pp [] B.B. Cha, T. Huang, X. Zhuang, Y. Zhao, J.Sklansky, Pecewse Lnear Classfers usng Bnary Tree Structure and Genetc Algorthm, Pattern Recognton, Vol. 9, No., November 996, pp [3] E. Cantú-Paz, C. Kamath, Inducng Oblque ecson Trees wth Evolutonary Algorthms, IEEE Trans. on Evolutonary Computaton, Vol. 7, No., February 003, pp [4]. Heath, S. Kasf, S. Salzberg, Inducton of Oblque ecson Trees, Proceedngs of the 3th Internatonal Jont Conference on Artfcal Intellgence, pp [5] S. C. Ng, K. S. Leung, Inducton of Quadratc ecson Trees usng Genetc Algorthms, 003 Intellgent Automaton Conference, ecember 003, pp [6] Z. Mchalewcz, Genetc Algorthms + ata Structures = Evoluton Programs, Sprnger Verlag, 99. [7] C.L. Blake, C.J. Merz, UCI Repostory of Machne Learnng atabases [ y.html], Irvne, CA: Unversty of Calforna, epartment of Informaton and Computer Scence, 998.