Measuring Search Trees

Transcription

1 Mesuring Serch Trees Christin Bessiere 1, Bruno Znuttini 2, nd Cèsr Fernández 3 1 LIRMM-CNRS, Montpellier, Frnce 2 GREYC, Cen, Frnce 3 Univ. de Lleid, Lleid, Spin Astrct. The SAT nd CSP communities mke gret use of serch effort comprisons to ssess the vlidity of n lgorithm or heuristic. There exist different wys of counting the size of serch tree developed y procedure. However, they re seldomly defined, nd their similrities/differences re not cler. This note ims t clrifying this issue. We formlly define (some of) the existing mesures nd chrcterize the properties tht we cn expect from them. We show some weknesses of the existing mesures with respect to those properties. We propose the numer of wrong decisions s new wy of mesuring the serch effort tht is more ccurte to effectively represent the effort n lgorithm hs devoted to the serch. We show tht this new wy of mesuring the serch spce ehves well with respect to the desired properties. 1 Introduction In the stisfiility nd constrint communities, we often need to compre the reltive performnce of different lgorithms. There re different mesures tht cn e used, nd tht hve dvntges nd/or drwcks. In the constrint community, the numer of times constrint hs een checked s stisfied or not (constrint check) hs een used for long time. It is considered s the sic opertion ll lgorithms do. It hs lso some dvntges such s eing proportionl to the quntity of informtion retrieved from the constrint network during the solving process. But it hs shortcomings which re evident s soon s we use complex lgorithms tht perform other types of computtion thn just checking constrints. (AC-4 ws n ovious exmple of n lgorithm storing once nd for ll the constrint checks, nd then working only with its dt structure.) Another mesure is the cpu time. It hs the dvntge of tking into ccount ll the prts of the effort devoted to the solving process, ut is ffected y the qulity of the implementtion. This issue is prticulrly ccurte when we wnt to compre the performnce of n lgorithm implemented on site with stndrd one ville on the we or in mrket product. The two versions re not necessrily implemented with the sme skills. The lst mesure, which is widely used, oth in CSP nd SAT solvers (t lest in those using cktrck serch), is the size of the serch tree explored during the solving process. This size is often evluted s the numer of cktrcks.

2 But, if we check wht is clled numer of cktrcks y different people, inside different solvers, or in different ppers, it ppers tht this concept is rther fuzzy, nd different wys of mesuring the size of the serch tree cn e reported under this nme. The gol of this note is to clrify our picture on those mesures. We present them nd provide forml definition. We propose new mesure, the numer of wrong decisions, tht is more consistent with wht is effectively done during the solving process of n NP-complete prolem. We chrcterize properties tht we cn expect from serch mesure. We show tht ll mesures ut wrong decisions fil on one of the properties. 2 Serch Trees We give some definitions tht will permit to hndle more formlly the notion of serch tree. These definitions re sed on stndrd CSP understnding, ut ll the definitions hold for SAT, or ny similr type of serch. A serch tree T is composed of nodes, rcs, nd n ordering < c on the outgoing rcs of ech node. A node u represents n ordered prtil instntition I(u) = x i1 = v i1,..., x ik = v ik. A serch tree is rooted t the prticulr node u 0 with I(u 0 ) =. There is n rc from node u to node u 1 if I(u 1 ) = I(u) (x = v), x nd v eing vrile nd one of its vlues. ( denotes conctention.) u 1 is clled child of u, nd u the prent of u 1. For every node u, T(u) denotes the sutree of T rooted t u. The set ch(u, T) of the children of u in T is totlly ordered y < c. The serch tree TP A of cktrck serch lgorithm A solving prticulr prolem P is the serch tree such tht there is one-to-one mpping etween nodes in the tree nd instntitions visited y A until it reched solution or proved inconsistency of P. Given two nodes u 1 nd u 2 children of node u TP A, u 1 < c u 2 iff I(u 1 ) hs een visited y A efore I(u 2 ). The complete serch tree TP A of A on P is the serch tree developed y A if A ws not stopped fter solution is found, ut continued until the end of the enumertion. 3 Serch Cost We define severl wys of counting the serch cost of n lgorithm A on prticulr prolem P. All ut the lst one hve lredy extensively een used in ppers compring serch lgorithms: 1. numer of nodes, 2. numer of cktrcks, 3. numer of decisions, 4. numer of wrong decisions. We now define these four concepts, nd we will illustrte their differences on n exmple (Fig. 2). For ll of them we lso discuss where counter should e plced in n lgorithm for computing the mesure. For tht purpose, we give

3 function BT_serch(int level) /* I = level */ 1. if preprocessing(level) then 2. if termintion condition then return 1 3. select vrile xi 4. for ech vlue vi in dom(xi) do 5. ssign vi to xi 6. if BT_serch(level+1) then return 1 7. undo(level) 8. return 0 Fig. 1. Bcktrck scheme the generl scheme of cktrck-like lgorithm on Fig. 1. The procedure is initilly clled with level=0. The function preprocessing() cn contin ny simplifiction lgorithm. It usully prunes the domins with respect to given locl consistency: unit propgtion in DPLL for CNF formuls, ound consistency in constrint solvers involving numericl constrints, or some form of rc consistency in CSP serch lgorithm such s forwrd checking [HE80] or MAC [SF94]. In the sic cktrck lgorithm [GB65], it just checks whether the new ssignment together with the current one stisfies ll the relevnt constrints. In ll cses, we ssume tht preprocessing() fils if the domin of vrile is empty. Finlly, we ssume tht the recursive cll on Line 6 of Fig. 1 my modify the domin of x i. Think indeed of ckjump lgorithm for instnce, where the preprocessing on the ith level could empty the domin of the i-1th vrile ecuse the culprit ssignment is tht to the i-2th one. Numer of nodes. The simplest technique is to count the numer of nodes visited y procedure. The definition is ovious. Definition 1 (#nodes). The numer of nodes of the resolution of prolem P y n lgorithm A is simply the numer of nodes in T A P. Counting the numer of nodes visited y n lgorithm on given prolem is ovious s well: just increment the counter y 1 ech time the functionbt serch is entered. This mens dding the following line to Fig. 1: 0. #nodes++ A wekness of this mesure is tht it doesn t discriminte etween the effort devoted to trversing the tree top-down nd left-right. In other words, visiting n filing nodes, or going deterministiclly from the root to lef without ny wrong choice will dd the sme n nodes to the count (n is the numer of vriles), while in the second cse not single mistke ws done.

4 Numer of cktrcks. This is the est known mesure. It counts the numer of times the procedure goes ck from vrile x ik to its predecessor x ik 1 fter hving proved tht none of the extensions of I(x ik ) cn e extended to solution. In terms of the serch tree, it counts the numer of times the serch goes up in the tree from node u to its predecessor fter hving exhusted the lst child of u. More formlly: Definition 2 (#cktrcks). The numer of cktrcks of the resolution of prolem P y n lgorithm A is equl to {u T A P ch(u, T A P ) & P I(u) }. The condition for node u to e counted s cktrck node is tht it hs some children which hve een visited (first component of the condition), nd tht none of them were successful (second component). Counting the cktrcks of P on A is quite strightforwrd: increment the counter y 1 ech time the function BT serch fils fter hving tried to extend the current instntition. This mens dding the following line on Fig. 1: 6is. #cktrcks++ A mjor drwck of this wy of counting the serch effort is tht it doesn t tke into ccount the numer of vlues tried for vrile. In other words, given node u cn hve 2 or 100 children, we will count them oth s one cktrck, if the lst child fils. Numer of decisions. This mesure is used in severl modern SAT solvers ville on the we [MMZ + 01,EY02]. It counts the numer of times choice hd to e mde during the serch. This prevents us from counting instntitions tht were ovious, s when domin is singleton. (As opposed to #nodes.) Definition 3 (#decisions). The numer of decisions of the resolution of prolem P y n lgorithm A is mx(0, ch(u, TP A ) 1) + {u TP A ch(u, TP A ) ch(u, TP A)} u T A P The formul identifies the numer of decisions tken in ech node u to its numer of children minus 1 (first component), nd dds one for ech node such tht the lst explored child ws not the lst possile one it ws itself decision (second component). In order to count the numer of decisions of n lgorithm on prolem, we simply count how mny times vlue is selected while it ws not the only lterntive for x i : 4is. if vi is not the only remining vlue for xi then #decisions++ Wheres this mesure hs dvntges over #nodes, it still hs the sme drwck s #nodes, nmely, counting choices tht will e successful. In other words, we cn otin score of n for cktrck-free serch in which ll the choices were directly the good ones.

5 2+0 dec 2 wd x2 x1 1 t 2+0 dec 2 wd c c 1+1 dec 1wd 13 nodes 2 cktrcks (t) 9 decisions (dec) 7 wrong decisions (wd) 1 t 2+0 dec 2 wd 0+1 dec x3 c solution Fig.2. A serch tree with three vriles nd three vlues Numer of wrong decisions. This mesure tries to overcome ll the weknesses pointed out so fr. As opposed to #nodes or #decisions, it reports 0 if serch ws successful on ll its choices. But s opposed to #cktrcks, it tkes into ccount the numer of children given node hs. It counts the numer of times choice hd to e mde during the serch, except if this choice finlly leds to solution. Definition 4 (#wrong decisions). The numer of wrong decisions of the resolution of prolem P y n lgorithm A is mx(0, ch(u, TP A ) 1) u T A P This definition is expressed in terms of the serch the lgorithm A performs on the prolem P, ut we cn lso give n lterntive one. Property 1 For ny lgorithm A nd prolem P, the numer of leves in T A P equls #wrong-decisions(a,p)+1. Proof. This is esily seen y induction on TP A. Indeed, if T P A hs only one node, nd thus one lef the numer of wrong decisions is 0. Now if the root of TP A hs k children, write #leves i for the numer of leves in its ith child nd #wd i for the numer of wrong decisions in the suprolem defined y the ith child. Then the numer of leves in TP A is k i=1 #leves i, i.e., k i=1 (#wd i + 1) y the induction hypothesis. This is k+ k i=1 #wd i = 1+((k 1)+ k i=1 #wd i), which is exctly the numer of wrong decisions of A on P plus 1. The intuition ehind this equivlence is tht the numer of wrong decisions in serch counts exctly the prtil ssignments tht re explored while they cnnot e extended into solution, i.e., the leves of the serch tree, except for the lst one tht is either solution or the lst element of proof tht the prolem is inconsistent.

6 As for counting the wrong decisions of n lgorithm A on prolem P, increment counter y 1 ech time A undoes the modifictions coming from the selection of vlue v i if this ws not the lst possiility for x i : 6is. if there re still some vlues in dom(xi) then #wrong-decisions++ The four mesures ove re illustrted on serch tree on Fig Properties of mesures We now formlly define two properties tht we should expect from mesure of the serch cost, nd evlute those mesures defined in Section 3 with respect to them. These two properties re the following: 1. strict monotonicity, 2. stility under inriztion of the serch tree. 4.1 Strict monotonicity The first property tht we cn expect from mesure is very nturl: we wnt tht the mesure of serch tree TP A A is more thn tht of serch tree T tht is included in TP A A P. Inclusion here is intended in the following sense: TP is sid to e included in TP A A if every node in TP is in T P A nd there is t lest one node u tht hs k 1 children in TP A nd more thn k in T P A. The ltter condition llows to consider tht two serch trees re equivlent if one is otined from the other only y extending some of its rnches; indeed, the serch spce explored in this cse is essentilly the sme. Definition 5 (strict monotonicity). A mesure of serch cost µ is sid to e strictly monotonic if for ny two serch trees TP A, T P A such tht T P A is included in T A P, µ(t A P ) < µ(t A P ) holds. Property 2 Mesures #nodes nd #wrong decisions re strictly monotonic, while #cktrcks nd #decisions re not. Proof. As for #cktrcks nd #decisions, counter-exmple is given on Fig. 3. The tree on the left is indeed included in tht on the right, ut it is esily seen tht #decisions=3 nd #cktrcks=1 for oth (ssuming the prolem is consistent). As for #nodes, it is oviously strictly monotonic, nd finlly #wrong decisions is s well ecuse if TP A more lef thn TP A. is included in T A P then T A P hs t lest one

7 x1 x2 solution solution Fig.3. Two serch trees with two vriles nd two vlues 4.2 Binry trees More nd more CSP techniques, such s the well-known MAC procedure [SF94], use refuttion t ech node of the tree. Thus, these procedures develop inry tree. The left rnch is choice of vlue, nd the right rnch is the refuttion of this vlue, except if there remins single vlue in the domin in which cse the right rnch instntites tht remining vlue. (See [Mit03,KB03] for comprison of regulr nd inry CSP tree serch.) We give generl scheme for inry tree serch on Fig. 4. function Binry_BT_serch(int level) /* I = level */ 1. if preprocessing(level) then 2. if termintion condition then return 1 3. select vrile xi with dom(xi) >1 4. select vlue vi in dom(xi) nd ssign it to xi 5. if Binry_BT_serch(level+1) then return 1 6. dom(xi) := dom(xi)-{vi} 7. if Binry_BT_serch(level+1) return 1 8. undo(level) 9. return 0 Fig.4. Binry cktrck scheme Binry tree serch is more generl thn non-inry since it is possile to chnge not only the vlue, ut lso the vrile fter ech filure (lines 3 nd 4 on Fig. 4). In ddition, ny non-inry serch tree TP A hs inrized counterprt, noted BP A, which corresponds to the inry serch in which the selection of vriles nd vlues re done in exctly the sme ordering s in the non-inry one, nd function preprocessing() in line 1 does nothing when its cll does not follow n instntition (in line 7). The inrized counterprt of the serch tree of Fig. 2 is presented on Fig. 5.

8 x2= x1= x1= x1=c 17 nodes 4 t 9 decisions 7 wrong decisions x2= x2=c x2= x3= x2= x3= x3=c x3= solution Fig.5. The inrized counterprt of the serch tree in Fig. 2. The definitions given in Section 3 still hold for inry serch trees, with counters dded to the scheme on Fig. 4 in the following wy: 1. for numer of nodes, 0. #nodes++ 2. for numer of cktrcks, 7is. #cktrcks++ 3. for numer of decisions, 3is. #decisions++ 4. for numer of wrong decisions. 5is. #wrong-decisions++ It is nturl to expect from mesure of the serch cost tht it llows us to compre non-inry cktrck lgorithms nd inrized ones. In prticulr, we expect from mesure tht the cost of non-inry serch tree is the sme s tht of its inrized counterprt, since the serch spce explored is essentilly the sme. The following definition formlizes the stility of mesure: Definition 6 (stility under inriztion). A mesure of serch cost µ is sid to e stle under inriztion if for ny two serch trees TP A, BA P such tht BP A is the inriztion of T P A, µ(t P A) = µ(ba P ) holds. Property 3 Mesures #decisions nd #wrong decisions re stle under inriztion, while #nodes nd #cktrcks re not.

9 u 1 u 1 u 2 u T. u. k T. 1 1 T 2 T 2 u k 1 T 1 T 2 T k T k T k 1 T k Generl tree Binriztion when d >k Binriztion when d =k Fig.6. The two possile inriztions of serch tree Proof. As for #nodes nd #cktrcks, the tree on Fig. 2 nd its inriztion on Fig. 5 give counter-exmple for oth. As for #wrong decisions, it is esily seen tht the numer of leves in the inrized tree is the sme s in the originl one, nd thus Proposition 1 concludes. Thus we re left with #decisions; we show the result y induction on TP A. Denote y BP A the inriztion of T P A, nd first remrk tht if T P A hs only one node then BP A = T P A. Now ssume the root of T P A hs k 1 children, nd write #dec i for the numer of decisions in the suprolem defined y its ith child. By definition of inrized serches BP A cn tke one of two forms, depending on whether there re more thn k vlues in the domin of the first vrile. These two forms re depicted on Figure 6: tht on the middle corresponds to the cse where there re more thn k vlues, nd tht on the right to tht where there re exctly k of them. As for the generl tree, y definition the numer of decisions of A on P is k+ k i=1 #dec i in the first cse nd k 1+ k i=1 #dec i in the second cse. Now in the first cse, ech node u i, i < k in BP A dds 1 to the numer of decisions since it hs two children nd ch(u i, BP A) = ch(u i, BP A), nd node u k dds 1 s well since it hs only one child ut ch(u k, BP A) ch(u k, BP A ) ; thus the numer of decisions in the inrized serch is k + k i=1 #dec i y the induction hypothesis, tht is, the numer of decisions in the generl serch. Finlly, in the second cse ech node u i, 1 i k 1 dds 1 to the numer of decisions since it hs two children nd ch(u i, BP A) = ch(u i, BP A ), thus the totl numer of decisions is k 1 + k i=1 #dec i, which gin equls the numer of decisions in the generl serch. 5 Summry nd Conclusion We hve contriuted to clrifiction of the different wys of mesuring the size of serch tree developed y cktrck procedure. We hve formlly defined the existing mesures nd we hve chrcterized some properties tht cn e

10 expected from such mesures. We hve shown some weknesses of the existing mesures with respect to those properties. We hve proposed the numer of wrong decisions s new wy of mesuring the serch effort. This new mesure ehves well with respect to the desired properties. References [EY02] E.Golderg nd Y.Novikov. Berkmin: fst nd roust st-solver. In Proceedings DATE 02, pges , [GB65] S.W. Golom nd L.D. Bumert. Bcktrck progrmming. Journl of the ACM, 12(4): , Octoer [HE80] R.M. Hrlick nd G.L. Elliott. Incresing tree sech efficiency for constrint stisfction prolems. Artificil Intelligence, 14: , [KB03] G. Ktsirelos nd F. Bcchus. Unrestricted nogood recording in csp serch. In Proceedings CP 03, pges , Kinsle, Irelnd, [Mit03] D.G. Mitchell. Resolution nd constrint stisfction. In Proceedings CP 03, pges , Kinsle, Irelnd, [MMZ + 01] M. Moskewicz, C. Mdign, Y. Zho, L. Zhng, nd S. Mlik. Chff: Engineering n efficient st solver. In Proc. Intl. Design Automtion Conference (DAC-01), pges , Ls Vegs NV, [SF94] D. Sin nd E.C. Freuder. Contrdicting conventionl wisdom in constrint stisfction. In Proceedings PPCP 94, Settle WA, 1994.