Confidence Intervals for a Proportion Using Inverse Sampling when the Data is Subject to False-positive Misclassification

Transcription

1 Journal of Data Science 13(015), Confidence Intervals for a Proortion Using Inverse Samling when the Data is Subject to False-ositive Misclassification Kent Riggs 1 1 Deartment of Mathematics and Statistics, Stehen F. Austin State University Abstract: Of interest in this aer is the develoment of a model that uses inverse samling of binary data that is subject to false-ositive misclassification in an effort to estimate a roortion. From this model, both the roortion of success and falseositive misclassification rate may be estimated. Also, three first-order likelihoodbased confidence intervals for the roortion of success are mathematically derived and studied via a Monte Carlo simulation. The simulation results indicate that the score and likelihood ratio intervals are generally referable over the Wald interval. Lastly, the model is alied to a medical data set. Key words: Misclassification; Double samling; Inverse samling; Interval estimation; Likelihood methods 1. Introduction Statistical inference of a roortion using binary data that is subject to misclassification has been a art of the statistical literature for some time. Bross (1954) demonstrated that traditional estimation of a roortion in the resence of misclassification roduces biased estimators. Tenennbein (1970) accounts for the misclassification with a double samling scheme. Lie et al. (1994) and Moors et al. (000) considered the case where only false-negative counts are obtained. Boese et al. (006) develoed several interval estimators in the case when only false-ositive counts are obtained. All of the above authors used a fixed samling scheme(s) in order to estimate the roortion of interest. While this aroach often works well, it has been demonstrated by Tian et al. (009) that fixed samles can sometimes be too small to effectively estimate the roortion of success when it is small. As a remedy to this roblem, inverse samling and the negative binomial distribution can be emloyed. In this aer, I wish to develo a statistical model that uses a two-stage samling scheme to estimate a roortion of success where the first stage is generated under inverse samling and is subject to false-ositives, while the second stage is generated under fixed samling. The aer is organized as follows. In Section, the statistical model is develoed, and in Section 3 maximum likelihood estimators (MLE) are resented for the roortion of success and falseositive rate. Interval estimators are also derived by inverting the Wald, score, and likelihood

2 64 Confidence Intervals for a Proortion Using Inverse Samling when the Data is Subject to Falseositive Misclassification ratio statistics in Section 3. In Section 4, the coverage roerties and average widths of the three interval estimators are comared using a Monte Carlo simulation. Finally, in Section 5, the three confidence intervals are alied to a real-world data examle.. The False-Positive Model Using Inverse Samling The first stage in the two-stage samling scheme involves the use of a fallible classifier that is rone to roducing false-ositives under inverse samling. The second stage involves using an infallible classifier under fixed samling. In conjunction with the fallible classifier, one can obtain the actual number of successes, actual number of failures, and actual number of false ositives for this fixed samle in the second stage. The two stages are considered indeendent. The two-stage samling scheme is very useful to aroriately estimate the main roortion of interest as well any false-ositive misclassification arameter. Aroriate estimation of these arameters gives the researcher better insight into the attribute of interest and the rate at which a fallible classifier may roduce a false-ositive. Let Y be the number of failures labeled by the fallible device in stage 1 until the k th success is observed. Hence, Y NegBin ( k, ), where is the robability the fallible device labels an observation as a success. Once stage 1 is comleted, the fixed samling for stage is imlemented, where both the fallible and infallible devices make classifications on observations. Let be the number of observations labeled failure by both the fallible and infallible devices, n 00 n 10 be the number of observations labeled success by the fallible device but n 11 be the number of observations labeled success labeled failure by the infallible device, and by both the fallible and infallible devices. Thus, n n00 n10 n11, and ( n, n, n ) trinomial n,(1 )(1 ), (1 ),, where is the robability of the fallible device yielding a false-ositive, and interested robability of success. Therefore, it can be shown that (1 ). n is the Table 1 rovides an examle of the two-stage samling lan. Observations for both stages are gathered on different ortions of the oulation. 0 denotes a failure, 1 denotes a success, and 1* denotes a false-ositive by the fallible classifier. For stage 1, k 5 and y 3, while for stage, n 00 1, n10, and n11 4.

3 Kent Riggs 65 Table 1 Two-stage Samling Scheme Examle Poulation Stage 1 Fallible * Stage Fallible 1 1 1* * Infallible Table gives insight into the resulting robabilities from the different stage and classifying device combinations. Table Probabilities for Each Stage/Classifier Scenario Stage Infallible Device Fallible Device First N/A Second 0 (1 )(1 ) (1 ) 1 N/A 3. Estimation Using samles from both stages results in the following likelihood function: L(, ) trinomial n,(1 )(1 ), (1 ), NegBin ( k, ) n! y k 1 n 1 00! n10! n11! k n00 n01 n11 k y (1 )(1 ) (1 ) (1 )

4 66 Confidence Intervals for a Proortion Using Inverse Samling when the Data is Subject to Falseositive Misclassification where n! n n n y k 1 11 (1 )(1 ) (1 ) n! n! n! k k y (1 ) (1 )(1 ). From (1), we can straightforwardly find Fisher s Exected Information Matrix, which is I I I(, ) E I I, (, ) ln L (1) is the log-likelihood function. The elements of this matrix are given in the aendix. Using calculus, it can be shown that the MLE s for and are n k n n ˆ ˆ n k n n n n n k y and n n ( k y n)(1 ˆ ) Also, it can be shown that the restricted MLE of for a fixed is given by ˆ n10 k( 1) n00 n10 y ( 1)( k n n y) k ( 1) k( 1)( n ( n y)) ( n ( n y)) Next, we derive confidence intervals for ( 1)( k n n y) by inverting the aroriate Wald, score, and loglikelihood statistics. Confidence intervals for can be derived in a similar manner, but we will not ursue them in this aer. All three interval estimators are based on first order asymtotic aroximations. The Wald-based interval estimator for uses the unrestricted MLE s ˆ and ˆ. For 11 sufficiently large samles,?, (, ˆ) ( ˆ, ˆ ) 1 N I, where I ( ˆ, ) is the (1,1) element of 11 ˆ I. Hence, an aroximate 100(1 )% confidence interval is given by ˆ, () 11? z / I (, ) where z / is the 100(1- / ) ercentile for the standard normal distribution..

5 Kent Riggs 67 use of A score-based confidence interval for ˆ where. For sufficiently large samles, ( ˆ ) u ˆ and 11 I (, ˆ ) involves inverting the score statistic, which requires? 11 ( ) (, ) 1 u I is the (1,1) element of ˆ 1 aroximate 100(1 )% confidence interval is comosed of the values of? 11 1, I(, ). Therefore, an that satisfy u ( ) I (, ) ( ), (3) where ( ) 1 is the 100(1 ) ercentile of a chi-square distribution with one degree of freedom. The exression on the left-hand side of (3) is comlicated, and solutions to (3) must be found numerically. The likelihood ratio confidence interval for involves inverting the log-likelihood statistic. Note that 1 ( ˆ,? ) (, ), for sufficiently large samles. Hence, an aroximate 100(1 )% confidence interval is comosed of the values of that satisfy As in (3), the values of ˆ? 1 (, ) (, ) ( ). (4) that satisfy (4) must be determined numerically. 4. Simulation Study Now we consider coverage and width roerties of the interval estimators described in () - (4). First, we examine the coverage and width roerties when n 0.1k for various combinations of and. All simulations were erformed in SAS IML V9.3 with 10,000 iterations for each arameter and samle size configuration. Figures 1 and give lots that summarize the coverage and average width roerties of confidence intervals () - (4). The nominal confidence level is 95% and all estimated coverages have standard errors less than The maximum standard error for estimated average widths in Figure is

6 68 Confidence Intervals for a Proortion Using Inverse Samling when the Data is Subject to Falseositive Misclassification =0.50 = 0.5 = 0.05

7 Kent Riggs 69 Figure 1:Coverage Plots for the Wald, Score, and Likelihood Ratio Confidence Intervals when n = 0.1k = 0.5 = 0.05

8 630 Confidence Intervals for a Proortion Using Inverse Samling when the Data is Subject to Falseositive Misclassification = 0.50 Figure :Avg. Width Plots for the Wald, Score, and Likelihood Ratio Confidence Intervals when n = 0.1k From Figure 1, we see that the likelihood ratio interval is conservative (over-covers) or is close to nominal for all the configurations excet when 0.50 and However, for this same configuration, the likelihood interval has a coverage that is close to nominal shortly after k exceeds 100. The coverage of the score interval aears to converge to the nominal level at a comarable rate to the likelihood ratio interval, whereas the coverage of the Wald interval converges at a much slower rate. In fact, when 0.05 the Wald interval drastically undercovers, even when k is near The average width results from Figure indicate that the score interval is more narrow for smaller values of k. The disarity between the widths of the three intervals seems to go away at a quicker rate for larger values of and. It is surrising that the Wald interval is generally the widest, yet it has severe under-coverage roblems as indicated in Figure 1. It should also be noted the average widths increase as increases, which is intuitive because more uncertainty is injected into the estimation roblem. Also, not surrisingly, the average widths tend to increase as aroaches Next, we examine the coverage and width roerties when n 0.4k for various combinations of and. All simulations were erformed in SAS IML V9.3 with 10,000 iterations for each arameter and samle size configuration. Figures 3 and 4 give lots that summarize the coverage and average width roerties of confidence intervals () - (4). The nominal confidence level is 95% and all estimated coverages have aroximate standard errors less than The maximum standard error for estimated average widths in figure 4 is

9 Kent Riggs = 0.5 = 0.05

10 63 Confidence Intervals for a Proortion Using Inverse Samling when the Data is Subject to Falseositive Misclassification = 0.50 Figure 3:Coverage Plots for the Wald, Score, and Likelihood Ratio Confidence Intervals when n = 0.4k = 0.05

11 Kent Riggs 633 = 0.50 = 0.5 Figure 4:Avg. Width Plots for the Wald, Score, and Likelihood Ratio Confidence Intervals when n = 0.4k From Figure 3, we see that all three intervals generally have better coverage roerties as comared to the cases where n 0.1k. These results are not surrising, as we now have more infallible data than in Figure 1. Also, from Figure 4, the three intervals have similar average widths excet for the case where 0.05 and In this case, the score interval is narrowest for smaller values of k. In all other cases it is not surrising that the three intervals have comarable widths because of the large ortion of good data we have (i.e. n 0.4k), which translates to the common asymtotic distribution better aroximating the Wald, score, and

12 634 Confidence Intervals for a Proortion Using Inverse Samling when the Data is Subject to Falseositive Misclassification likelihood ratio statistics. The same tendencies that were observed in Figure with an increase in and are also resent in Figure 4, that is, the average widths tend to increase with an increase in and. Also, it should be stated that for the same k,, and combination, the widths from Figure 4 are considerably smaller than widths from Figure. This gain in recision is intuitive given the increase in the size of the infallible data set. 5. An Alication to a Medical Data Set We now aly the statistical model from Section and three confidence intervals from Section 3 to a medical data set from Hildesheim (1991). Boese (006) has also considered this data set. The data was generated under fixed samling, but for illustrative uroses of the model in Section, we will assume inverse samling. This data involves a large study which examines the relationshi between the heres simlex virus and cervical cancer. For us, of interest from the data is estimating the revalence,, of the heres simlex virus in women who have invasive cervical cancer. One diagnostic test for the virus is the western blot rocedure (WBP), which is fallible. A more accurate rocedure, the refined western blot rocedure (RWBP) is accurate and we will consider it as an infallible classifying device. For illustrative uroses we will only consider false ositives from the data in stage and allow false negatives to be absorbed into For the two stages, the following counts were observed: y 375, k 318, n00 13, n10 3, and n11 3. Using the regular inverse samling model that does not account for misclassification, we get the following oint estimate and 95% confidence interval estimate for : and (0.501,.581). Using the more aroriate inverse samling model that allows for false-ositive misclassification, we get the following oint estimates: ˆ and ˆ % Wald, score, and likelihood ratio confidence intervals are (0.404, 0.565), (0.397, 0.539), and (0.395, 0.546), resectively. Note that the original inverse samling model overestimates, which is not surrising due to the estimated 1.3% false-ositive rate. Also, note that the intervals resented here are slightly wider than the ones found in Boese (006). This is likely due to the fact that Boese (006) has a slightly smaller estimate (0.119) of the false-ositive rate. Now, one could use only the infallible data to estimate, but the resulting standard error is higher as comared to the estimator that uses both the fallible and infallible samles. The n infallible-only estimate is , which has a standard error of This is n considerably higher than the standard error of the estimator (0.041) that uses both the fallible and infallible data. n 11.

13 Kent Riggs Comments In this article, we derived an inverse-samling model that allows for false-ositive misclassification and three confidence intervals for the roortion of interest. All three confidence intervals are based on first-order aroximations and formed by inverting the Wald, score, and likelihood ratio statistics. Necessary for the develoment of the confidence intervals was Fisher s information matrix, unrestricted MLE s, and restricted MLE s. The Wald, score, and likelihood ratio confidence intervals were then studied via a Monte Carlo simulation study. The study indicates that the score and likelihood ratio intervals erform better than the Wald interval in terms of coverage and average width, articularly when the ratio of infallible data to fallible data is small (n =.1k). In such cases, we recommend the score or likelihood ratio confidence intervals. However, if the infallible data set is large and the ratio of infallible to fallible data is large, the Wald interval may be referred as its coverage and width roerties are comarable to the score and likelihood ratio intervals, but its comutation is easier. Finally, we alied this newly derived model and interval estimators to a real medical data set, which demonstrated the bias resent in a model that does not account for misclassification, when, in fact, misclassification is resent in the data. The author acknowledges that this statistical model and resulting estimators are similar to the ones resented in Boese (006), but with the major distinction of stage 1 in this model using inverse-samling whereas Boese (006) uses fixed-samling. The statistical model and interval estimators resented in this article should rove useful for the ractitioner who desires to estimate a articular roortion using inverse-samling but where the data is subject to false-ositive misclassification. Acknowledgements This work is suorted, in art, by a research grant from the Office of Research and Sonsored Programs at Stehen F. Austin State University. Aendix Provided are the second derivatives and negative exectations required for Fisher s information matrix: n00 n10 n11 k(1 ) y (1 ) (1 ) ( (1 ) ) (1 ),

14 636 Confidence Intervals for a Proortion Using Inverse Samling when the Data is Subject to Falseositive Misclassification n00 n10 k(1 ) y (1 ) ( (1 ) ) (1 ), k k(1 )(1 ) (1 ) ( (1 ) ), I n n n k k (1 ) (1 ) (1 ( (1 ) )) (1 ) (1 ) ( (1 ) ) ( (1 ) )(1 ), I n n k k (1 ) (1 ) (1 ) (1 ( (1 ) )) (1 ) ( (1 ) ) ( (1 ) )(1 ), I I k ( (1 ) ). References [1] Boese, D., Young, D., Stamey, J. (006). Confidence intervals for a binomial arameter [] based on binary data subject to false-ositive misclassification. Comutational Statistics and Data Analysis 50: [3] Bross, I. (1954). Misclassification in x tables. Biometrics 10: [4] Hildesheim, A., Mann, V., Brinton, L.A., Szklo, M., Reeves, W.C., Rawls, W.E. (1991). [5] Heres simlex virus tye : a ossible interaction with human aillomavirus 16/18 in the develoment of invasive cervical cancer. International Journal of Cancer 49: [6] Lie, R.T., Heuch, I., Irgens, L.M. (1994). Maximum likelihood estimation of the [7] roortion of congenital malformations using double registration systems. Biometrics 50: [8] Moors, J.J.A., van der Genugten, B.B., Strijbosch, L.W.G. (000). Reeated audit [9] controls. Statistica Neerlandica 54: [10] Tenennbien, A. (1970). A double samling scheme for estimating binomial data with [11] misclassifications. Journal of American Statistical Association 65: [1] Tian, M., Tang, M., Ng, H., and Chan, P. (009). A comarative study of confidence [13] intervals for negative binomial roortion. Journal of Statistical Comutation and Simulation 79: