Non-Inferiority Tests for the Ratio of Two Correlated Proportions

Transcription

1 Chater 161 Non-Inferiority Tests for the Ratio of Two Correlated Proortions Introduction This module comutes ower and samle size for non-inferiority tests of the ratio in which two dichotomous resonses are measured on each subject. When one is interested in showing that the true roortions are different, the data are often analyzed with McNemar s test. However, we are interested in showing non-inferiority rather than difference. For examle, suose a diagnostic rocedure is accurate, but is exensive to aly or has serious side effects. A relacement rocedure is sought which is no less accurate, but is less exensive or has fewer side effects. In this case, we are not interested in showing that the two diagnostic rocedures are different, but rather that the second is no worse than the first. Non-inferiority tests were designed for this situation. These tests are often divided into two categories: equivalence (two-sided) tests and non-inferiority (one-sided) tests. Here, the term equivalence tests means that we want to show that two diagnostic rocedures are equivalent that is, their accuracy is about the same. This requires a two-sided hyothesis test. On the other hand, non-inferiority tests are used when we want to show that a new (exerimental) rocedure is no worse than the existing (reference or gold-standard) one. This requires a one-sided hyothesis test. The rocedures discussed in this chater deal with the non-inferiority (one-sided) case. Technical Details The results of a study in which two dichotomous resonses are measured on each subject can be dislayed in a - by- table in which one resonse is shown across the columns and the other is shown down the rows. In the discussion to follow, the columns of the table reresent the standard (reference or control) resonse and the rows reresent the treatment (exerimental) resonse. The outcome robabilities can be classified into the following table. Exerimental Standard Diagnosis Diagnosis Yes No Total Yes PT No PT Total P 1 P 1 S S 161-1

2 In this table, ij Treatment, Standard. That is, the first subscrit reresents the resonse of the new, exerimental rocedure while the second subscrit reresents the resonse of the standard rocedure. Thus, roortion having a negative treatment resonse and a ositive standard resonse. 01 reresents the Sensitivity, Secificity, and Prevalence To aid in interretation, analysts have develoed a few roortions that summarize the table. Three of the most oular ratios are sensitivity, secificity, and revalence. Sensitivity Sensitivity is the roortion of subjects with a ositive standard resonse who also have a ositive exerimental resonse. In terms of roortions from a -by- table, ( ) Sensitivity 11 / / PS Secificity Secificity is the roortion of subjects with a negative standard resonse who also have a negative exerimental resonse. In terms of roortions from a -by- table, ( ) Secificity / Prevalence Prevalence is the overall roortion of individuals with the disease (or feature of interest). In terms of roortions from a -by- table, Prevalence P S Table Probabilities The outcome counts from a samle of n subjects can be classified into the following table. Exerimental Standard Diagnosis Diagnosis Yes No Total Yes No Total n n n T T n n n n n n n n S S Note that n + n is the number of matches (concordant airs) and n + n is the number of discordant airs The hyothesis of interest concerns the two marginal robabilities P T and P S. P S reresents the accuracy or success of the standard test and P T reresents the accuracy or success of the new, exerimental test. Noninferiority is defined in terms of either the difference, D PT PS, or the relative risk ratio, R P T / P S, of these two roortions. The choice between D and R will usually lead to different samle sizes to achieve the same ower. 161-

3 Non-Inferiority Hyotheses using Ratios The following is based on Nam and Blackwelder (00). We refer you to this aer for the comlete details of which we will only rovide a brief summary here. When R E < 1, the statistical hyotheses of non-inferiority are H0: PT / PS RE versus H1: P / P > R T S E Test Statistics The test statistic for an asymtotic test based on constrained maximum likelihood for large n is given by where ~ Power Formula The ower when the true value of the relative risk ratio is R E can be evaluated exactly using the multinomial distribution. When n is large, we use a normal aroximation to the multinomial distribution which leads to where 10 ( ) Z R ( 10) ( ) RE ( RE + 1) ( 1)( 1 ) E ( RE P T S) n P R ~ ~ P R P P R P R E E 4 E T S T S ~ R ~ R 01 E 10 E 00 n01 n,, n + n P, n T PS n n n c U ( ) V T n n ( ) ( ) V ( T ) z V T E T 1 α ( + ) RE n ( ) ( ) E1 T0 RA RE PS ( ) V T ( A + E ) S E 11 ( A E ) E ( + ) ( ) Φ( c ) β R R R P R R R PS n ( 10) ( ) P RE P P RE P 4RE T S T S RE ( RE + 1) ( 1)( 1 ) R R 01 E 10 E 00 A U

4 Nuisance Parameter Unfortunately, the -by- table includes four arameters 11, 10, 01, and 00, but the ower secifications above only secify two arameters: P S and D A or R A. A third arameter is defined imlicitly since the sum of the four arameters is one. One arameter, known as a nuisance arameter, remains unaccounted for. This arameter must be addressed to fully secify the roblem. This fourth arameter can be secified by secifying any one of the following: 11, 10, 01, 00, , , or the sensitivity of the exerimental resonse, / 11 PS. It may be difficult to secify a reasonable value for the nuisance arameter since its value may not be even aroximately known until after the study is conducted. Because of this, we suggest that you calculate ower or samle size for a range of values of the nuisance arameter. This will allow you to determine how sensitive the results are to its value. Procedure Otions This section describes the otions that are secific to this rocedure. These are located on the Design tab. For more information about the otions of other tabs, go to the Procedure Window chater. Design Tab The Design tab contains the arameters associated with this test such as the roortions, samle sizes, alha, and ower. Solve For Solve For This otion secifies the arameter to be solved for from the other arameters. The arameters that may be selected are Power or Samle Size. Power Calculation Power Calculation Method Select the method to be used to calculate ower. The choices are Multinomial Enumeration Power is comuted using multinomial enumeration of all ossible outcomes when N Max N for Multinomial Enumeration (otherwise, the normal aroximation is used). Multinomial enumeration of all outcomes is ossible because of the discrete nature of the data. Normal Aroximation Aroximate ower is comuted using the normal aroximation to the multinomial distribution

5 The exact calculation using the multinomial distribution becomes very time consuming for N > 500. When N > 500, the difference between the multinomial and aroximate calculations is small. For small values of N (less than 100), the Multinomial Enumeration ower may be overly otimistic because the discrete nature of the trinomial distribution results in the actual alha value being higher than its target. To be on the safe side, we recommend that you use the aroximate calculation. Max N for Multinomial Enumeration Only shown when Power Calculation Method Multinomial Enumeration Secify the maximum value of N (samle size) that uses the exact ower calculation based on the multinomial distribution. N's greater than this value will use the asymtotic aroximation. Power and Alha Power This otion secifies one or more values for ower. Power is the robability of rejecting a false null hyothesis, and is equal to one minus Beta. Beta is the robability of a tye-ii error, which occurs when a false null hyothesis is not rejected. Here, a tye-ii error occurs when you fail to conclude non-inferiority when in fact it is true. Values must be between zero and one. Historically, the value of 0.80 (Beta 0.0) was used for ower. Now, 0.90 (Beta 0.10) is also commonly used. A single value may be entered here or a range of values such as 0.8 to 0.95 by 0.05 may be entered. Alha This otion secifies one or more values for the robability of a tye-i error. A tye-i error occurs when a true null hyothesis is rejected. Here, a tye-i error occurs when you falsely conclude non-inferiority. Samle Size N (Samle Size) Enter a value for the samle size. This value must be greater than two. You may enter a range of values such as 10 to100 by 10. Effect Size Ratios Rni (Non-Inferiority Ratio) Rni is the minimum size of the relative risk ratio, P T / PS, that will still result in the conclusion of non-inferiority. Non-inferiority trials use a value that is less than one. Tyical values for this ratio are 0.8 or 0.9. Ra (Actual Ratio) Enter a value for Ra, the actual relative risk ratio P T / PS. This value is used to generate the value of P T using the formula P T P S R a. Often this value is set equal to one, but this is not necessary

6 Effect Size Standard Proortion Ps (Standard Proortion) This is the roortion of yes s (or successes), P S, when subjects received the standard treatment. This value or a good estimate is often available from revious studies. You may enter a set of values searated by blank saces. For examle, you could enter Values between, but not including, 0 and 1 are ermitted. Effect Size Nuisance Parameter Nuisance Parameter Tye Enter the tye of nuisance arameter here. Unfortunately, the -by- table cannot be comletely secified by using only the arameters Ps and Da or Ps and Ra. One other arameter must be secified. This additional arameter is called a nuisance arameter. It will be assumed to be a known quantity. Several ossible choices are available. This otion lets you secify which arameter you want to use. In all cases, the value you secify is a roortion. P11 (% Positive Matches) The roortion of subjects that are ositive on both tests. P00 (% Negative Matches) The roortion of subjects that are negative on both tests. P01 (% -Trt +Std) The roortion of subjects that are negative on the treatment, but ositive on the standard. P10 (% +Trt -Std) The roortion of subjects that are ositive on the treatment, but negative on the standard. P11+P00 (% Matches) The roortion of matches (concordant airs). P01+P10 (% Disagree) The roortion of non-matches (discordant airs). P11/Ps (Sensitivity) The sensitivity. Nuisance Parameter Value Enter the value of the nuisance arameter that you secified in the Nuisance Parameter Tye box. This value is a roortion, so it must be between 0 and

7 Examle 1 Finding Power Researchers have develoed a new treatment for migraine headaches which is less exensive than a current standard. The researchers need to show that the roortion of individuals who resond to the new treatment is not inferior to the standard treatment. The new treatment will be considered non-inferior if its success rate is no less than 95% of the success rate of the standard, which is about They want to study the ower for various samle sizes between 500 and 4000 at the 5% significance level. They ll study various values of the nuisance arameter: P11/Ps sensitivity (0.5 to 0.9). Setu This section resents the values of each of the arameters needed to run this examle. First, from the PASS Home window, load the rocedure window by exanding Proortions, then Two Correlated Proortions, then clicking on Non-Inferiority, and then clicking on. You may then make the aroriate entries as listed below, or oen Examle 1 by going to the File menu and choosing Oen Examle Temlate. Otion Value Design Tab Solve For... Power Power Calculation Method... Normal Aroximation Alha N (Samle Size) to 4000 by 500 Rni (Non-Inferiority Ratio) Ra (Actual Ratio) Ps (Standard Proortion) Nuisance Parameter Tye... P11/Ps (Sensitivity) Nuisance Parameter Value to 0.9 by 0.1 Annotated Outut Click the Calculate button to erform the calculations and generate the following outut. Numeric Results Numeric Results for a Non-Inferiority (One-Sided) Test of a Ratio Samle Non-Inf. Actual Treatment Standard Nuisance Size Ratio Ratio Proortion Proortion Parameter Power N Rni Ra Pt Ps P11/Ps Alha (reort continues) * Power was comuted using the normal aroximation method

8 Reort Definitions Power is the robability of rejecting a false null hyothesis. N is the number of subjects, the samle size. Rni is the maximum ratio between Pt and Ps that is still called 'non-inferior'. Ra is the actual ratio between Pt and Ps. That is, Ra Pt/Ps. Pt is the resonse roortion to the treatment (exerimental or new) test. Ps is the resonse roortion to the standard (reference or old) test. The Nuisance Parameter is a value that is needed, but is not a direct art of the hyothesis. Alha is the robability of rejecting a true null hyothesis. Summary Statements A samle size of 500 subjects achieves 3.55% ower at a significance level of using a one-sided non-inferiority test of correlated roortions when the standard roortion is 0.650, the maximum ratio of these roortions that still results in non-inferiority (the range of non-inferiority) is 0.950, and the actual ratio of the roortions is This reort shows the ower for the indicated scenarios. All of the columns are defined in the Reort Definitions section. Plots Section These lots show the ower versus the samle size for the various values of sensitivity. In this examle, we see that the value of the nuisance arameter has a large effect on the calculated ower

9 Examle Finding Samle Size Continuing with Examle 1, the analysts want to determine the exact samle size necessary to achieve 90% ower for all values of the nuisance arameter. Setu This section resents the values of each of the arameters needed to run this examle. First, from the PASS Home window, load the rocedure window by exanding Proortions, then Two Correlated Proortions, then clicking on Non-Inferiority, and then clicking on. You may then make the aroriate entries as listed below, or oen Examle by going to the File menu and choosing Oen Examle Temlate. Otion Value Design Tab Find... Samle Size Power Calculation Method... Normal Aroximation Power Alha Rni (Non-Inferiority Ratio) Ra (Actual Ratio) Ps (Standard Proortion) Nuisance Parameter Tye... P11/Ps (Sensitivity) Nuisance Parameter Value to 0.9 by 0.1 Outut Click the Calculate button to erform the calculations and generate the following outut. Numeric Results for a Non-Inferiority (One-Sided) Test of a Ratio Samle Non-Inf. Actual Treatment Standard Nuisance Size Ratio Ratio Proortion Proortion Parameter Power N Rni Ra Pt Ps P11/Ps Alha * Power was comuted using the normal aroximation method. These scenarios require a large samle size. In fact, the first two rows are blank because the samle size is so large it can t be determined

10 Examle 3 Validation using Nam and Blackwelder (00) Nam and Blackwelder (00) give an examle in which Ps is 0.80, P10 is 0.05, Ra is 1.00, Rni is 0.80, the significance level is 0.05, and the ower is 80%. From their Table III, the samle size is 34. Note that their calculations use the aroximate formula. Setu This section resents the values of each of the arameters needed to run this examle. First, from the PASS Home window, load the rocedure window by exanding Proortions, then Two Correlated Proortions, then clicking on Non-Inferiority, and then clicking on. You may then make the aroriate entries as listed below, or oen Examle 3 by going to the File menu and choosing Oen Examle Temlate. Otion Value Design Tab Find... Samle Size Power Calculation Method... Normal Aroximation Power Alha Rni (Non-Inferiority Ratio) Ra (Actual Ratio) Ps (Standard Proortion) Nuisance Parameter Tye... P10 (% +Trt -Std) Nuisance Parameter Value Outut Click the Calculate button to erform the calculations and generate the following outut. Numeric Results Numeric Results for a Non-Inferiority (One-Sided) Test of a Ratio Samle Non-Inf. Actual Treatment Standard Nuisance Size Ratio Ratio Proortion Proortion Parameter Power N Rni Ra Pt Ps P10 Alha * Power was comuted using the normal aroximation method. The calculated samle size of 34 matches the results of Nam and Blackwelder (00)