Journal of Statistical Software

Transcription

1 JSS Joural of Statistical Software Jue 2007, Volume 19, Issue 6. Ratioal Arithmetic Mathematica Fuctios to Evaluate the Oe-sided Oe-sample K-S Cumulative Samplig Distributio J. Radall Brow Ket State Uiversity Milto E. Harvey Ket State Uiversity Abstract Oe of the most widely used goodess-of-fit tests is the Kolmogorov-Smirov (K- S) family of tests which have bee implemeted by may computer statistical software packages. To calculate a p value (evaluate the cumulative samplig distributio), these packages use various methods icludig recursio formulae, limitig distributios, ad approximatios of ukow accuracy developed over thirty years ago. Based o a extesive literature search for the oe-sided oe-sample K-S test, this paper idetifies two direct formulae ad five recursio formulae that ca be used to calculate a p value ad the develops two additioal direct formulae ad four iterative versios of the direct formulae for a total of thirtee formulae. To esure accurate calculatio by avoidig catastrophic cacelatio ad elimiatig roudig error, each formula is implemeted i ratioal arithmetic. Liear search is used to calculate the iverse of the cumulative samplig distributio (fid the cofidece iterval badwidth). Extesive tables of badwidths are preseted for sample sizes up to 2, 000. The results cofirm the hypothesis that as the umber of digits i the umerator ad deomiator itegers of the ratioal umber test statistic icreases, the computatio time also icreases. I comparig the computatioal times of the thirtee formulae, the direct formulae are slightly faster tha their iterative versios ad much faster tha all the recursio formulae. Computatioal times for the fastest formula are give for sample sizes up to fifty thousad. Keywords: K-S samplig distributios, K-S oe-sided oe-sample probabilities, K-S cofidece bads, ratioal arithmetic. 1. Itroductio The Kolmogorov-Smirov (K-S) family of tests is oe of the most widely used goodess-offit tests ad is icluded i may oparametric statistics texts (see recet texts Gibbos

2 2 Evaluatig the Oe-sided Oe-sample K-S Test Samplig Distributio ad Chakraborti (2003), Spret ad Smeeto (2001), Coover (1999), Daiel (1990)). These iclude the oe-sided oe-sample, two-sided oe-sample, oe-sided two-sample, ad the twosided two-sample tests. The K-S family of tests also iclude restricted rage tests (comparig distributios over a portio of their rage) ad ratio tests (the ratio of oe distributio to aother). For sample size, the most commo K-S test is the two-sided oe-sample test which uses the maximum absolute distace D betwee the hypothesized cotiuous cumulative distributio F (x) ad the empirical cumulative distributio F (x), D = sup <x< F (x) F (x), as the radom variable. I a hypothesis testig applicatio, computig the test statistic d is relatively easier tha evaluatig the cumulative samplig distributio to determie the p value, P [D d]. The cumulative samplig distributio is a piecewise polyomial that is differet for each sample size ad whose complexity rapidly grows with icreasig so that it has ot eve bee geerated let aloe used for > 31 (see Rube ad Gambio (1982) ad Drew, Gle, ad Leemis (2000)). Cosequetly, the limitig distributio, various recursio formulae, ad various approximatios have bee used to evaluate the cumulative samplig distributio. I additio, may computer statistical software packages such as SPSS, STATISTICA, R, Numerical Recipes, ad IMSL iclude K-S tests. Although a recursio formula will theoretically determie the p value P [D d] for a particular value d of the test statistic, the complexity of the formula is such that roudoff error ad catastrophic cacelatio ca greatly reduce the accuracy of the calculatios. Sice most procedures used today were developed o pre-1978 computers where oly machie precisio was available, the accuracy of their results is ot kow exactly. Cosequetly, recursio formulae have oly bee used to geerate tables for sample sizes of 40 ad various approximatios of ukow accuracy have bee used for > 40. Usig recursio formula ad ratioal arithmetic, Brow ad Harvey (2005) were able to compute p values for sample sizes up to two thousad, = 2, 000. I additio to the two-sided oe-sample case (absolute differece betwee hypothesized ad empirical), the oe-sided oe-sample (differece betwee hypothesized ad empirical) cumulative samplig distributio is a complex series that ca also be evaluated by recursio formulae. Ideed, may computer statistical software packages that implemet both the two-sided ad oe-sided oe-sample K-S test use differet methods to calculate the p values. Table 1 summarizes the strategies used by some of these commercial statistical packages to calculate two-sided ad oe-sided oe-sample p values. Although these packages compute the K-S test statistic i the same way, there is cosiderable differece i the way they evaluate the cumulative samplig distributio (calculate p values). The Numerical Recipes statistical subrouties i Press, Teukolsky, Vetterlig, ad Flaery (1992) use a approximatio by Stephes (1970) to geerate p values for the two-sided oe-sample K-S cumulative samplig distributio. SPSS 15.0 (SPSS Ic. 2006) does ot state how the p value for the two-sided oe-sample K-S test is calculated. However, i 2002, the maual for SPSS 11.0 stated that the statistical software package used a modificatio of the limitig distributio derived by Feller (1948) ad used by Smirov (1948). Assumig SPSS has ot chaged how the p value is calculated, the 2002 method is listed i Table 1. The STATISTICA (StatSoft, Ic. 2006) software package uses the critical values tabulated by Massey (1950) ad Massey (1951) to geerate their p values. IMSL (Visual Numerics 2006) computes both the oe-sided ad twosided K-S test statistics ad the gives p values for each test. Specifically, for the oe-sided K-S test ad sample sizes 80, IMSL uses a recursio formula i Coover (1972) to compute the exact p values for the oe-sided samplig distributio, but for large sample sizes

3 Joural of Statistical Software 3 Statistical Type of Used To Software Oe-sample Calculate Package K-S Test p values IMSL Oe-sided For 80, recursio formula by Coover (1972). Two-sided For > 80, limitig distributio derived by Feller (1948). Double the correspodig oe-sided p value. Numerical Two-sided Approximatio by Stephes (1970). Recipes R Oe-sided For 100, direct formula by Smirov (1944). For > 100, limitig distributio derived by Feller (1948). Two-sided For 100, program by Marsaglia et al. (2003). For > 100, limitig distributio derived by Kolmogorov (1933). SPSS Two-sided Modificatio of limitig distributio derived by Feller (1948). STATISTICA Two-sided Critical values tabulated by Massey (1950) ad Massey (1951). Table 1: Statistical software packages ad the oe-sample K-S test. > 80, it uses the oe-sided limitig distributio. IMSL the doubles the oe-sided p values to get the correspodig two-sided p values. Like IMSL, the R statistical software package computes both the oe-sided ad two-sided K-S test statistics but uses differet methods to compute the p values. For large sample sizes > 100, R Developmet Core Team (2006) states that the asymptotic distributios are used (presumably the formula by Feller (1948) for the oe-sided K-S test ad the formula by Kolmogorov (1933) for the two-sided case). For small sample sizes 100, the oe-sided K-S test uses the direct formula by Smirov (1944) to calculate the p value ad the two-sided K-S test uses the matrix formula of Durbi (1973) as implemeted by Marsaglia, Tsag, ad Wag (2003). I 2002, the R statistical software package istead computed the oe-sided p values usig the techiques i Coover (1972) ad doubled the oe-sided p value to get the two-sided p value. It is ot clear i 2002 whether R also used the oe-sided limitig distributio for large sample sizes. I additio to the oe-sample K-S cumulative samplig distributios, there are samplig distributios for may other K-S type statistics icludig two-sample (differece betwee two empirical distributios), restricted rage (two distributios compared over a portio of their rage), ad ratios (the ratio of oe distributio to aother). For the two-sample K-S case with sample sizes m ad, the oe-sided cumulative samplig distributio is a simple formula for m = ad a complex series for m. The two-sided two-sample K-S cumulative samplig distributio is a complex series for m = while a recursio formula is eeded for m. There are may K-S oe-sided oe-sample restricted rage ad ratio cumulative samplig distributios whose formulae are kow but are very complex expressios. Give the advaces i computig power ad computatioal software i the past thirty years, it is time to recosider the etire area ad if possible, devise techiques to accurately ad

4 4 Evaluatig the Oe-sided Oe-sample K-S Test Samplig Distributio quickly evaluate K-S cumulative samplig distributios. Sice the etire K-S cumulative samplig distributio area is so large, the questio is where we should begi a comprehesive evaluatio of the alterate formulae for the various K-S cumulative samplig distributios. Because of the large umber ad complexity of the formulae, the oe-sided oe-sample K-S restricted rage ad ratio areas are ot good startig poits. Similarly, sice two-sample sizes require more computatioal work tha oe-sample size, the two-sample area should be doe after the oe-sample area. The oe-sample area cotais two tests, the two-sided oesample K-S test ad the oe-sided oe-sample K-S test. Sice Brow ad Harvey (2005) have already ivestigated the two-sided oe-sample case, this paper will ivestigate the oe-sided oe-sample case. This paper reviews the oe-sided oe-sample K-S cumulative samplig distributio formulae, devises ratioal arithmetic implemetatios of each formula, verifies the validity of each implemetatio by determiig if each implemetatio gets exactly the same p value over a broad rage of examples, develops a efficiet method to calculate the badwidth (the iverse of the cumulative samplig distributio), ad fially compares the computatioal times eeded for each implemetatio to determie the fastest formula. 2. Oe-sided oe-sample K-S samplig distributio formulae There are two oe-sided oe-sample radom variables: the oe-sided upper radom variable D + = sup <x< {F (x) F (x)} ad the oe-sided lower radom variable D = sup <x< {F (x) F (x)}. Sice by symmetry D + ad D have the same cumulative samplig distributio, D + is used to represet both cases. Based o a extesive literature search for the oe-sided oe-sample K-S test, this paper idetifies two direct formulae ad five recursio formulae that ca be used to calculate a p value, P [D + d + ], ad the develops two additioal direct formulae ad four iterative versios of the direct formulae for a total of thirtee formulae. Table 2 cotais a summary of the thirtee formulae which are developed i this sectio Direct formulae A closed form expressio of the oe-sided oe-sample K-S cumulative samplig distributio was developed by Smirov (1944) ad verified by may scholars icludig Feller (1948) ad Birbaum ad Tigey (1951). For 0 < d + 1 ad sample size, Smirov s formula deoted by SmirovD i this paper is show i the first row of Table 3 where (1 d + ) is the greatest iteger less tha or equal to (1 d + ). Dwass (1959) derived a differet formula deoted by DwassD that is also show i Table 3. Both the SmirovD ad DwassD formulae are also derived i Durbi (1973). A secod form of the Smirov distributio deoted by SmirovAltD ad a secod form of the Dwass distributio deoted by DwassAltD are derived by factorig 1/ 1 out of their respective formulae. The alterate forms, SmirovAltD ad DwassAltD, show i Table 3 may be faster tha the origial formulatios because the terms iside the summatio are simpler ratioal umbers. I most applicatios, the test statistic d + is less tha 0.5 ad usually much less tha 0.5 which meas the umber of terms (1 d + ) + 1 i the SmirovD ad SmirovAltD formulae ca be close to the sample size. I compariso, the umber of terms d + 1 i the DwassD ad DwassAltD formulae is much less tha the umber i the SmirovD ad SmirovAltD

5 Joural of Statistical Software 5 Type Formula Referece Formula Name Smirov (1944) Direct SmirovD Direct SmirovAltD Iterative SmirovI Iterative SmirovAltI Dwass (1959) Direct DwassD Direct DwassAltD Iterative DwassI Iterative DwassAltI Daiels (1945) Recursio Daiels Noe ad Vadewiele (1968) Recursio Noe Steck (1969) Recursio Steck Coover (1972) Recursio Coover Kotelikov ad Chmaladze (1983) Recursio Bolshev Table 2: Thirtee formulae to calculate a K-S oe-sided oe-sample p value formulae ad should therefore take less computatio time. O the other had, all the terms i the SmirovD ad SmirovAltD formulae are positive while the terms i the DwassD ad DwassAltD formulae alterate sigs. This meas that the Dwass formulae are much more susceptible to error tha the Smirov formulae. Note that implemetig the formulae i ratioal arithmetic removes the issue of computatioal error as all computatios are exact. This will be discussed i detail i Sectio Iterative formulae Each of the four formulae i Table 3 ca be trasformed ito a iterative formula which might be faster tha the origial formula. The Smirov formula will be used to illustrate the process. Let γ j be the value of the jth term i the series ad let x j be the iterative factor that coverts γ j 1 to γ j so that γ j = x j γ j 1. The followig derives x j for the SmirovD ad SmirovAltD formula. Note that the x j must be the same for the SmirovD ad SmiroAltD formulae sice the SmirovAltD formula is simply the SmirovD formula with 1/ 1 factored out. ( ) ( γ j = d + 1 d + j ) j ( d + + j ) j 1 j γ j 1 = ( ) ( d + 1 d + j 1 ) j+1 ( d + + j 1 j 1 x j = ( ) ( γ d + 1 d + j j j = ( ) γ ( j 1 d + 1 d + j 1 ) j+1 ( j 1 ) j ( d + + j ) j 2 ) j 1 d + + j 1 ) j 2

6 6 Evaluatig the Oe-sided Oe-sample K-S Test Samplig Distributio Type Formula to Compute P [D + d + ] for 0 < d + 1 (1 d+ ) SmirovD d + j=0 ( ) ( 1 j ) j ( ) j j 1 j d+ + d+ DwassD SmirovAltD d+ 1 d + d + 1 j=0 (1 d + ) j=0 ( ) ( 1 j ) j 1 ( ) j j j + d+ d+ ( ) ( d + j ) j ( d + + j ) j 1 j DwassAltD 1 d+ 1 d + j=0 ( ) ( j + d + ) j 1 ( j d + ) j j (1 d + ) is the greatest iteger less tha or equal to (1 d + ) Table 3: K-S oe-sided oe-sample direct formulae. = ( j + 1) (d+ + j) j ( d + j) [ 1 ] 1 j+1 [ d j + 1 ] 1 j 2 d + + j 1 For the DwassD ad DwassAltD formulae, let y j be the iterative factor that coverts γ j 1 to γ j so that γ j = y j γ j 1. The followig derives y j for the DwassD ad DwassAltD formulae. γ j = ( ) ( d + 1 j ) j 1 ( ) j j j + d+ d+ γ j 1 = ( ) ( d + 1 j 1 ) j ( ) j 1 j 1 j 1 + d+ d+ x j = ( ) ( γ d + 1 j ) j 1 ( ) j j j j + d+ d+ = ( ) γ ( j 1 d + 1 j 1 j 1 + d+ = ( j + 1) (j [ d+ ) j ( j d + ) 1 1 j d + ) j ( ) j 1 j 1 d+ ] j 1 [ 1 + ] 1 j 1 j 1 d + Usig the same process as above, the iterative formula for SmirovD deoted by SmirovI ca be derived. Similarly, the iterative formulae for SmirovAltD, DwassD, ad DwassAltD ca be derived ad are deoted by SmirovAltI, DwassI, ad DwassAltI respectively. The results are show i Table 4.

7 Joural of Statistical Software 7 Type Smirov x j = ( j + 1) (d+ + j) j ( d + j) [ 1 Dwass y j = ( j + 1)(j [ d+ ) j( j d + ) 1 Iterative Formula ] 1 j+1 [ d j + 1 ] 1 j 1 [ j d ] 1 j 2 d + + j 1 ] 1 j 1 j 1 d + Name Iitial Value Iteratio P [ D + d +] (1 d+ ) SmirovI γ 0 = (1 d + ) γ j = x j γ j 1 (1 d+ ) SmirovAltI γ 0 = (1 d + ) γ j = x j γ j 1 d + DwassI γ 0 = d + (1 + d + ) 1 γ j = y j γ j 1 1 DwassAltI γ 0 = d + ( + d + ) 1 γ j = y j γ j 1 1 (1 d + ) is the greatest iteger less tha or equal to (1 d + ) j=0 j=0 γ j γ j γ j j=0 d + γ j j=0 / 1 Table 4: K-S oe-sided oe-sample iterative formulae. I additio to the four direct formulae ad four iterative formulae, five recursio formulae to compute the oe-sided oe-sample K-S p value have bee derived ad are preseted i chroological order i the ext five subsectios Daiels recursio formula Daiels (1945) derived a differece equatio that was later restated by Noe ad Vadewiele (1968). The form of Daiels recursio formula (referred to heceforth as Daiels) show below is derived by solvig the differece equatio for Q i (1). The recursio formulae use the test statistic d + = t/ or t = d +. P Q 0 (1) = 1 i 1 Q i (1) = ( D + t ) k=0 = 1 Q (1) ( ) [ ( ) i i t i k Q k (1) max k, 0 1] for i = 1, 2,,

8 8 Evaluatig the Oe-sided Oe-sample K-S Test Samplig Distributio 2.4. Noe ad Vadewiele recursio formula Sice the Daiels recursio formula has both positive ad egative terms, Noe ad Vadewiele (1968) derived a alterate recursio formula that has oly o-egative terms. Noe (1972) later added a correctio to this recursio formula. The particular form of the recursio formula (referred to heceforth as Noe) listed below cotaiig Noe s correctio is take from Shorack ad Weller (1986), page 363, formulas (24) through (28). P Q 0 (0) = 1 Q m (m) = 0 for 1 m + 1 ( ) i [ ( ) i m t Q i (m) = Q k (m 1) max k, 0 max ( D + t ) k=0 = 1 Q ( + 1) ( m t 1 for 0 i m 1, 1 m + 1 )] i k, Steck recursio formula Steck (1969) derived the recursio formula (referred to heceforth as Steck) show below that was later listed i Shorack ad Weller (1986). P ( D + t ) b j = ( ) j 1 + t mi, 1 P 0 = 1 P 1 = b 1 P i = b i i i 2 m=0 = 1 P for j = 1, 2,, ( ) i [b i b m+1 ] i m P m for i = 2, 3,, m 2.6. Coover recursio formula Coover (1972) derived a recursio formula (referred to heceforth as Coover) that simplifies to the followig for a hypothesized cotiuous cumulative distributio F (x). P ( D + t ) e 0 = 1 k 1 e k = 1 = t j=0 j=0 ( j ( ) ( k 1 j j t ) k j e j for k = 1, 2,, t ) ( 1 j t ) j e j Although ot stated explicitly, this appears to be the recursio formula used by the IMSL ad R statistical software packages.

9 Joural of Statistical Software Bolshev recursio formula Kotelikov ad Chmaladze (1983) used the recursio formula (referred to heceforth as Bolshev) show below that was later called the Bolshev recursio i Shorack ad Weller (1986). P ( D + t ) b j = ( j 1 + t mi P 0 = 1 P i = i 1 m=1 = 1 P ), 1 for j = 1, 2,, ( ) i [1 b i m+1 ] m P i m for i = 1, 2,, m 3. Computatioal ad research issues The thirtee formulae preseted i the last sectio are complex. Cosequetly, implemetig them raises certai computatioal issues that eed to be studied. The followig are the three major computatioal questios that eed to be resolved before the thirtee formulae are implemeted. 1. What type of computatioal arithmetic should be used? 2. What arithmetic form should be used to iput the test statistic d + to the thirtee formulae? 3. What is the most efficiet way to calculate the biomial coefficiets? These questios will be cosidered ad aswered i the order show above. After these implemetatio questios have bee resolved, the followig four research questios will be aswered. 1. What is the best way to calculate the badwidth (the iverse of the cumulative samplig distributio)? The aswer to this questio will be used to compute ad preset detailed badwidth tables. 2. What is the relatioship betwee computatio time ad sample size? 3. What is the fastest formulae? 4. What is the relatioship betwee the accuracy of the test statistic d + ad the computatio time? 3.1. Computatioal optios Usig curret computatioal software, the formulae ca be implemeted usig either ratioal arithmetic, arbitrary precisio arithmetic, or machie precisio arithmetic. Ratioal arithmetic stores every umber as a ratio of two itegers (a ratioal umber) where each iteger

10 10 Evaluatig the Oe-sided Oe-sample K-S Test Samplig Distributio ca have as may decimal digits as eeded to express the umber exactly. Although the speed of ratioal arithmetic declies as the umber of digits i the umerator/deomiator itegers icrease, it has the advatage of o error as log as o irratioal umbers are used. Coversely, machie precisio arithmetic specifies the umber of decimal digits (usually less tha twety ad determied by the computer hardware) to use i computatios so it is subject to roudoff error ad catastrophic cacelatio. Catastrophic cacelatio occurs whe oe umber is subtracted from aother umber of about the same value. For example, if is subtracted from both with ie decimal digits of precisio, the the result is with two decimal digits of precisio. Although machie precisio is fast, it is possible to sigificatly degrade the accuracy ad eve worse, ot be aware that the accuracy has bee reduced. Arbitrary precisio arithmetic is like machie precisio except that the umber of decimal digits of precisio is ot depedet o the computer hardware ad the user ca specify the umber of decimal digits of precisio. Although arbitrary precisio is slower tha machie precisio, it is faster tha ratioal arithmetic. I additio, the software system Mathematica, Wolfram (2003), keeps track of the resultig precisio rp so that for the example above, Mathematica would also kow that the result had a precisio of rp = 2. The trick i usig arbitrary precisio arithmetic is specifyig the precisio to be used i iteral calculatios (iteral precisio ip) so that the fial aswer has a specified desired precisio dp or greater. I other words, the user must specify both ip ad dp so that the fial aswer has rp dp. Sice all the K-S formulae ca be modified so that o irratioal umbers are used, this paper will use ratioal arithmetic implemetatios as they produce exact p values with o error. Future research will develop machie precisio ad arbitrary precisio implemetatios whose accuracy will the be verified by the ratioal arithmetic implemetatios i this paper. I terms of accuracy, ratioal arithmetic gives the exact probability (o error) as log as the test statistic d ca be expressed exactly as a ratioal umber; it caot be a irratioal umber like π/100. The oly way d + ca be a irratioal umber is if the hypothesized distributio F (x) for some x is a irratioal umber because by defiitio the empirical distributio F (x) is a ratioal umber (i/ for i = 0, 1, 2,..., ). I such cases d ca be approximated arbitrarily closely by ratioal umbers above ad below d +. These are the used to calculate the p value P [D + d + ] to ay desired accuracy. Thus, ratioal arithmetic either provides the exact p value if d + is a ratioal umber or ca get as close as the user desires if d + is a irratioal umber Test statistic complexity Usig ratioal arithmetic, the sample size ad the test statistic d + ca produce a p value with may digits i the umerator ad deomiator itegers. For example, whe = 200 ad d + = 13/200; d + has two digits i the umerator ad three digits i the deomiator ([2/3] umerator/deomiator digits) while the correspodig p value P [D + d + ] has [456/459] umerator/deomiator digits. To show how the umber of umerator/deomiator digits ca grow, cosider aother example with = 2, 000 ad d + = 83/2000 where d + has [2/4] umerator/deomiator digits while the correspodig p value P [D + d + ] has [6599/6602] umerator/deomiator digits. The large umber of umerator/deomiator digits for the p values P [D + d + ] suggest that computatioal time might vary with the umber of umerator/deomiator digits i test statistic d +. This hypothesis is tested i Sectio 8.

11 Joural of Statistical Software 11 A ratioal umber implemetatio of each of the thirtee formula has two iputs, the sample size that is by defiitio a ratioal umber (a iteger) ad the test statistic d + that is a umber betwee zero ad oe, 0 d + 1. If d + is either zero, oe, or a irratioal umber, the d + ca be expressed as either a ratioal umber or a arbitrary precisio umber. For example, with = 100 ad d + = , d + ca be used i the program as the ratioal umber d + = / or as the arbitrary precisio umber d + = while all the rest of the computatios i the Mathematica program are i ratioal arithmetic. Sice Mathematica treats ratioal umbers ad arbitrary precisio umbers differetly, the same Mathematica program that implemets the SmirovD formula i ratioal arithmetic will yield differet probabilities depedig o whether d + is used as ratioal umber or a arbitrary precisio umber i the program. Whe used as a ratioal umber i the Mathematica program, d + = / produces a ratioal umber probability with 597 digits i the umerator ad 600 digits i the deomiator ([597/600] umerator/deomiator digits) that whe [ coverted to a arbitrary precisio umber with 20 decimal digits of accuracy yields P D d+] = However, whe used as a arbitrary precisio [ umber i the Mathematica program, d + = yields a differet p value, P D d+] = / = , tha that produced by the ratioal umber d +. Sice the correct probability is the oe produced by the ratioal umber iput, the iput d + is always coverted to a ratioal umber before it is used i ay Mathematica program Calculatig a series of biomial coefficiets Sice all thirtee formulae use the biomial coefficiet i almost every term, a importat ( ) k cosideratio is how to calculate them. Although the otatio varies i each formulae, let j represet the biomial coefficiet. The four iterative formulae iclude the biomial coefficiet i the iterative formulae for x j ad y j so that the biomial coefficiet is automatically icluded ad eed ot be calculated separately. However, every oe of the direct formulae ad the recursio formulae use the biomial coefficiet ( ) i each term of the summatios. I each of k these summatios, the biomial coefficiet for each succeedig term chages by addig j oe to j so that each formula uses a series of biomial coefficiets. The efficiecy of the method for calculatig the series of biomial coefficiets as ratioal umbers will effect the speed of the implemetatio for each formulae. The two followig competig methods ca be used to compute every biomial coefficiet i the series as a ratioal umber. Ratioal Number Biomial Fuctio (RNBF) Use the Mathematica Biomial fuctio to geerate each coefficiet as a ratioal umber. Ratioal Number ( ) Iterative ( Calculatio ) (RNIC) Use ratioal arithmetic to iteratively k k compute from by multiplyig it by k j + 1. j j 1 j Brow ad Harvey (2006) compared the RNBF method versus the RNIC method ad for a small umber of terms t foud little differece i the computatio times. However, for a large umber of terms, the RNIC method is faster tha the RNBF method ad the differece

12 12 Evaluatig the Oe-sided Oe-sample K-S Test Samplig Distributio Mathematica Listed Formula Type Fuctio I Name Formula Name Sectio SmirovD Direct SmirovDKS1SidedRTRatioal 1 DwassD Direct DwassDKS1SidedRTRatioal 2 SmirovAltD Direct SmirovAltDKS1SidedRTRatioal 3 DwassAltD Direct DwassAltDKS1SidedRTRatioal 4 SmirovI Iterative SmirovIKS1SidedRTRatioal 5 DwassI Iterative DwassIKS1SidedRTRatioal 6 SmirovAltI Iterative SmirovAltIKS1SidedRTRatioal 7 DwassAltI Iterative DwassAltIKS1SidedRTRatioal 8 Daiels Recursio DaielsKS1SidedRTProbRatioal 9 Noe Recursio NoeKS1SidedRTProbRatioal 10 Steck Recursio SteckKS1SidedRTProbRatioal 11 Coover Recursio CooverKS1SidedRTProbRatioal 12 Bolshev Recursio BolshevKS1SidedRTProbRatioal 13 Table 5: Mathematica fuctio ame for the thirtee formulae listed i file KS1SidedOeSampleRatioal.b i time icreases as the sample size icreases. This implies that as the umber of terms grow, a RNBF implemetatio will evetually exceed the time eeded by the correspodig RNIC implemetatio. I additio, RNBF uses the Mathematica Biomial fuctio so ay code usig the RNBF method must be implemeted i Mathematica while the RNIC method ca be implemeted usig ay ratioal arithmetic software. Thus, RNIC is more portable tha RNBF ad is aother reaso for adoptig RNIC. As a result, this paper will use the RNIC method exclusively to calculate the biomial coefficiets for the thirtee formulae. 4. Implemetatios of the thirtee formulae All the Mathematica code geerated for this paper is cotaied i oe Mathematica file amed KS1SidedOeSampleRatioal.b which is divided ito 23 sectios. Each sectio cotais oe program ad sample output. Table 5 cotais a list of all the formulae, their type, the Mathematica fuctio ame implemetig them, ad the sectio umber i file KS1SidedOeSampleRatioal.b that cotais the Mathematica code ad sample output. 5. Calculatig the oe-sided badwidth d + (, α, ρ) I additio to calculatig the p value for hypothesis testig, the oe-sided oe-sample K- S cumulative samplig distributio ca be used to costruct a oe-sided cofidece bad aroud the empirical distributio F (x). The badwidth of a oe-sided cofidece bad with cofidece coefficiet 1 α ad sample size is the value of the test statistic d + that satisfies

13 Joural of Statistical Software 13 P (D + d + ) = α. Determiig a badwidth d + for a particular sample size ad cofidece coefficiet 1 α meas evaluatig the iverse of the cumulative samplig distributio which ca oly be doe by search techiques such as biary search. Ulike the p value, a badwidth d + caot i practice be determied exactly because the search techique may ot coverge to the exact value. For example, biary search with startig values of 0 ad 1 would ever fid d + = 1/3 ad would iterate forever. Thus, search techiques are desiged to stop whe a specified accuracy is reached. Let d + (, α, ρ) represet the badwidth rouded to ρ sigificat digits for sample size ad cofidece coefficiet 1 α. Note that badwidth d + (, α, ρ) is also the hypothesis testig critical value for a α level of sigificace. Fidig the badwidth d + (, α, ρ) is a three step process: (1) use a approximatio to fid a iitial value close to d + (, α, ρ), (2) use the iitial value to fid upper ad lower bouds o d + (, α, ρ), ad (3) use a search procedure to determie d + (, α, ρ) betwee the lower ad upper bouds. The first step will use the approximatio of Maag ad Dicaire (1971) to fid the iitial value ( [6d + + 1] 2 ) by solvig α exp for d + yieldig d + l(α) Sice the iitial 6 value foud by the approximatio i the first step should be fairly close to the actual value, the secod step gradually icreases the distace away from the iitial value util a lower ad upper boud o the actual value is foud. Although there are may search techiques that ca be used i the third step to determie the badwidth, this paper will cosider the two most commo techiques: biary search ad liear search. Note that biary search takes the midpoit betwee the upper ad lower bouds as the ext value to test while liear search uses liear iterpolatio to fid the ext value. Prelimiary computatioal experiece showed that liear search was always faster tha biary search so the followig liear search algorithm is used to fid the badwidth d + (, α, ρ). Liear search algorithm for calculatig the badwidth d + (, α, ρ) Step 1 (Fid Iitial Value): Calculate d + = l(α) to ρ digits of precisio. If d + > 1, set d + = 1. If d + < 0, set d + = 0. Go to Step 2. Step 2 (Determie If Iitial Value Is Lower Or Upper Boud): Calculate p = P [D + d + ]. If p > α, the d + is a lower boud, set d + L = d+ set p L = p, ad go to Step 3. Otherwise, d + is a upper boud, set d + U = d+, ad go to Step 6. Step 3 (Determie a Upper Boud): Covert d + L to a umerator iteger dumerator+ L ad a deomiator iteger ddeomiator L + where ddeomiator+ L is a power of te ad d + L = dumerator+ L /ddeomiator+ L. If the umber of digits of precisio does ot exceed four, ρ 4, set icremet ic = 1. Otherwise ρ > 4 ad set the icremet ic = 10 ρ 5. Go to Step 4. Step 4 (Costruct ad Test a Possible Upper Boud): Set dtry = dumerator L + + ic, calculate p = P [D + dtry/ddeomiator L + ]. If p > α, the a ew lower boud has bee foud ad go to Step 5. Otherwise, the iitial upper boud has bee foud, set dumerator U + = dtry, set dumerator+ U = dumerator+ L, set p U = p, ad go to Step 9.

14 14 Evaluatig the Oe-sided Oe-sample K-S Test Samplig Distributio Step 5 (New Lower Boud Foud): Set dumerator L + = dtry, set p L = p, set ic = ic 10, ad go to Step 4. Step 6 (Determie a Lower Boud): Covert d + U to a umerator iteger dumerator+ U ad a deomiator iteger ddeomiator U + where ddeomiator+ U is a power of te ad d + U = dumerator U + /ddeomiator+ U. If the umber of digits of precisio does ot exceed four, ρ 4, set icremet ic = 1. Otherwise ρ > 4 ad set the icremet ic = 10 ρ 5. Go to Step 7. Step 7 (Costruct ad Test a Possible Lower Boud): Set dtry = dumerator U + ic, calculate p = P [D + dtry/ddeomiator U + ]. If p < α, the a ew upper boud has bee foud ad go to Step 8. Otherwise, the iitial lower boud has bee foud, set dumerator L + = dtry, set dumerator+ L = dumerator+ U, set p L = p, ad go to Step 9. Step 8 (New Upper Boud Foud): Set dumerator U + = dtry, set p U = p, set ic = ic 10, ad go to Step 7. Step 9 (Liear Search Iteratio): If dumerator U + dumerator+ L 1, go to Step 12. Set dtry = dumerator L + + (dumerator+ U dumerator+ L ) (p L α)/(p L p U ). If dtry dumerator U +, set dtry = dumerator+ U 1. If dtry dumerator+ L, set dtry = dumerator L Calculate p = P [D+ dtry/ddeomiator L + ]. If p > α, the a ew lower boud has bee foud ad go to Step 10. Otherwise, a ew upper boud has bee foud ad go to Step 11. Step 10 (Liear Search New Lower Boud): Set dumerator + L = dtry, p L = p, ad go to Step 9. Step 11 (Liear Search New Upper Boud): Set dumerator + U = dtry, p U = p, ad go to Step 9. Step 12 (Determie Whether to Use Lower or Upper Boud): Calculate p = P [D + (dumerator L )/(ddeomiator+ L 10)]. If p < α, the use the lower boud by settig d + = dumerator L + /ddeomiator+ L ad go to Step 13. Otherwise, use the upper boud by settig d + = dumerator U + /ddeomiator+ U ad go to Step 13. Step 13 (Badwidth Foud): Termiate the algorithm with the badwidth d +. The liear search algorithm is implemeted usig the direct formulae: SmirovD, SmirovAltD, DwassD, ad DwassAltD. The sectio umber i file KS1SidedOeSampleRatioal.b cotaiig the Mathematica fuctio that implemets the liear search algorithm for each direct formula is listed i Table 6. Tables 7 ad 8 cotai computatioal experiece of the liear search algorithms for each direct formula to fid the badwidths. Sice the DwassAltD liear search algorithm is faster tha the other three direct formula implemetatios, it will be used i the remaider of the paper to fid badwidths.

15 Joural of Statistical Software 15 Direct I Mathematica File KS1SidedOeSampleRatioal.b Formula Mathematica Fuctio Name Sectio Number DwassD KS1SidedBadwidthByLiearSearchDwassD 15 DwassAltD KS1SidedBadwidthByLiearSearchDwassAltD 14 SmirovD KS1SidedBadwidthByLiearSearchSmirovD 17 SmirovAltD KS1SidedBadwidthByLiearSearchSmirovAltD 16 Table 6: Direct formula implemetatios of the liear search algorithm to fid badwidths The Mathematica fuctio KS1SidedOeSampleBadwidthsToFile cotaied i Sectio 18 of the KS1SidedOeSampleRatioal.b file fids badwidths usig liear search with DwassAltD ad writes these badwidths to a comma delimited file for iput ito Excel ad a text file that ca be used as the iput ito timig programs. The text file cotais badwidths where every digit i a badwidth is output separately so the badwidth ca be recostructed to ay desired accuracy. These text files will be used as iput files to produce the computatioal experiece i Sectios 6, 8, ad 9. Tables 9, 10, ad 11 cotai the badwidths to six digits of precisio (ρ = 6) for α = 0.2, 0.1, 0.05, 0.02, 0.01, ad represetative sample sizes from = 2 through = 2, Computatioal experiece comparig all thirtee formulae This sectio compares the computer time eeded to calculate the same p value by all thirtee formulae with the objective of determiig the fastest formula. Usig the program i Sectio 5, badwidths are geerated for sample sizes = 1000, 2500, 5000 ad cofidece coefficiets α = 0.001, 0.01, 0.1, 0.25, 0.5, 0.9. The resultig badwidths with ρ = 20 digits of precisio are put ito data file KS1SidedOeSampleBadwidthsN1000to5000.dat ad Excel file KS1SidedOeSampleBadwidthsN1000to5000.csv. The six cofidece coefficiets for α = 0.001, 0.01, 0.1, 0.25, 0.5, 0.9 were chose so that the etire rage of α s from 0 to 1 had some represetatio but the rage of greatest p value iterest from to 0.1 has the most represetatio. To illustrate the results, Table 12 cotais the ρ = 20 badwidths for both α = ad α = 0.9. I additio, Table 12 cotais all the ρ = 3 badwidths i both decimal ad ratioal form Direct formulae computatioal experiece I comparig the computatioal times across various sample sizes, the questio is what values of the test statistic d + should be used for compariso. The two alteratives are comparig the computatioal times for a fixed value of the test statistic d + or comparig the computatioal times eeded to produce a specified p value α. The difficulty with comparig the computatio times for a fixed value of the test statistic d + is that the p value will vary with. For example, P [D + 1, /5000] while P [D+ 5, /5000] A more useful

16 16 Evaluatig the Oe-sided Oe-sample K-S Test Samplig Distributio Time I Secods (Time) ad Number Iteratios (Iters) Sample p value Direct ρ = 3 ρ = 6 ρ = 9 ρ = 12 Size α Formula Time Iters Time Iters Time Iters Time Iters SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD SmirovD SmirovAltD Note: All timigs o a Petium IV ruig at 2.4 GHz. Table 7: SmirovD ad SmirovAltD liear search algorithms calculatig badwidth d + (, α, ρ)

17 Joural of Statistical Software 17 Time I Secods (Time) ad Number Iteratios (Iters) Sample p value Direct ρ = 3 ρ = 6 ρ = 9 ρ = 12 Size α Formula Time Iters Time Iters Time Iters Time Iters 1, DwassD DwassAltD DwassD DwassAltD DwassD DwassAltD DwassD DwassAltD DwassD DwassAltD DwassD DwassAlt , DwassD DwassAltD DwassD DwassAltD DwassD DwassAltD DwassD DwassAltD DwassD DwassAltD DwassD DwassAltD , DwassD DwassAltD DwassD DwassAltD DwassD DwassAltD DwassD DwassAltD DwassD DwassAltD DwassD DwassAltD Note: All timigs o a Petium IV ruig at 2.4 GHz. Table 8: DwassD ad DwassAltD liear search algorithms calculatig badwidth d + (, α, ρ)

18 18 Evaluatig the Oe-sided Oe-sample K-S Test Samplig Distributio Sample Badwidth d + (, α, ρ = 6) Size α = 0.2 α = 0.1 α = 0.05 α = 0.02 α = 0.01 α = Table 9: Badwidth d + (, α, ρ = 6) to six digits of precisio for = 2 to = 49

19 Joural of Statistical Software 19 Sample Badwidth d + (, α, ρ = 6) Size α = 0.2 α = 0.1 α = 0.05 α = 0.02 α = 0.01 α = Table 10: Badwidth d + (, α, ρ = 6) to six digits of precisio for = 50 to = 900

20 20 Evaluatig the Oe-sided Oe-sample K-S Test Samplig Distributio Sample Badwidth d + (, α, ρ = 6) Size α = 0.2 α = 0.1 α = 0.05 α = 0.02 α = 0.01 α = Table 11: Badwidth d + (, α, ρ = 6) to six digits of precisio for = 920 to = 2, 000 approach is to compare the computatio times eeded to produce the same p value across various sample sizes. I order to do this, we eed to calculate the value of the test statistic that yields a specified p value α for a sample size. I other words, we will use the badwidth d + (, α, ρ) where P [D + d + (, α, ρ)] α (see Sectio 5). For ρ = 3, the Mathematica fuctio TimigKS1SidedOeSampleRatioalDirectFormulae cotaied i Sectio 19 of the KS1SidedOeSampleRatioal.b file iputs the test statistics listed Table 12 ad produces the timigs i Table 13. The fastest direct formula i Table 13 was DwassAltD followed i order by DwassD, SmirovAltD, ad SmirovD. Sice the umber of terms i the SmirovD ad SmirovAltD formulae for the same sample size icrease with icreasig α, we would expect that the time would also icrease with α. This is the patter followed by the times i Table 13 with two exceptios which will be dealt with i Sectio 8: the time decreases for = 1, 000 goig from α = 0.25 to α = 0.5 ad the time also decreases for = 2, 500 goig from α = 0.01 to α = Iterative formulae computatioal experiece For ρ = 3, the Mathematica fuctio TimigKS1SidedDirectVersusIterFormulae cotaied i Sectio 20 of the KS1SidedOeSampleRatioal.b file iputs the test statistics listed

21 Joural of Statistical Software 21 Sample Badwidth d + (, α, ρ = 20) Size α = α = 0.9 1, , , Sample Badwidth d + (, α, ρ = 3) Size α = α = 0.01 α = 0.1 α = 0.25 α = 0.5 α = 0.9 1, / / / / / / , / / / / / / , / / / / / /10000 Table 12: Badwidth d + (, α, ρ) to produce P [D + d + (, α, ρ)] α Sample Time i Secods to calculate P [D + d + (, α, 3)] Size Formula α = α = 0.01 α = 0.1 α = 0.25 α = 0.5 α = 0.9 1, 000 SmirovD SmirovAltD DwassD DwassAltD , 500 SmirovD SmirovAltD DwassD DwassAltD , 000 SmirovD SmirovAltD DwassD DwassAltD Note: All timigs o a Petium IV ruig at 2.4 GHz. Table 13: Time i secods for direct formulae to calculate P [D + d + (, α, 3)] usig ratioal arithmetic

22 22 Evaluatig the Oe-sided Oe-sample K-S Test Samplig Distributio Sample Test Time i Secods to Compute P [D + d + (, α, 3)] for Formula Size p value Statistic Smirov SmirovAlt Dwass DwassAlt α d + (, α, 3) Direct Iter Direct Iter Direct Iter Direct Iter 1, , , Note: All timigs o a Petium IV ruig at 2.4 GHz. Table 14: Comparig time i secods for direct ad iterative formulae usig ratioal arithmetic Table 12 ad produces the timigs show i Table 14 for all direct ad iterative formulae. I this timig program, the p values produced by the direct ad iterative formulae for the same sample size ad test statistic d + are compared ad a error message is writte if they are ot all exactly equal. For all the computatioal experiece performed, o error message was ever geerated which is a idicatio that the Mathematica implemetatios of the direct ad iterative formulae are correct. The results clearly show that the direct formulae are faster tha their correspodig iterative formulae. Sice the DwassAltD direct formula is the fastest of all the direct ad iterative formulae, it will be used as a compariso i the computatioal experiece of the recursio formulae.