A SAS Program to Construct Smultaneous Confdence Intervals for Relatve Rsk Xaol Lu VA Cooperatve Studes Program, Perry Pont, MD ABSTRACT Assessng adverse effects s crtcal n any clncal tral or nterventonal study Adverse effects for testng a drug/devce often nvolve many clncal symptoms and each patent may experence multple adverse events Patents may be lost durng follow-up, or ther treatments are termnated early gven the prospectve nature of a clncal tral Consequently, commonly used SAS programs and other statstcal packages for analyss of adverse events (AE), whch form ndvdual confdence ntervals of cumulatve ncdence based relatve rsk (RR), are often not adequate and sometmes not approprate Ths paper presents a SAS program whch can be used to analyze data wth multple AE by formng smultaneous confdence ntervals of ncdence densty based RR Ths program can dynamcally estmate RR by calculatng person-tme based ncdence denstes and buld smultaneous confdence ntervals across all AE types by controllng type I error rates wth famly-wse error rate (FWER) and/or false dscovery rate (FDR) The program s desgned for analyss of multple AE n clncal trals and can be extended to any cohort study wth multple endponts INTRODUCTION Adverse effect assessment s as crucal as the effcacy measurement n the clncal nvestgaton of a medcal product va ether a randomzed clncal tral or an nterventon study Adverse effect often s comprsed of multple adverse events (AE), whch are essental for the analyss of adverse effect In common practce, AE analyses from clncal trals or nterventon studes are focused on a sngle adverse event by comparng the proportons between treatment groups usng PROC FREQ These methods are smple and easy to mplement, but they rsk losng nformaton and msnterpretng the results due to concurrence or correlaton among dfferent AE There s a need to develop methods to assess adverse effect approprately [1-3] Thus, for AE specfc nference, we propose to develop a SAS program that can analyze multple AE data by comparng ncdence denstes of multple AEs and constructng smultaneous confdence ntervals of relatve rsks (RR) Ths approach usually nvolves multple testng procedures that control for nflaton of type I error rate, such as usng famly-wse error rate (FWER) across all AE types A tradtonal approach for AE analyses has been calculatng a specfc AE cumulatve ncdence rate by countng the events and then comparng the cumulatve ncdence rate dfferences between the treatment/nterventon groups usng ether the normal approxmaton test or exact test for a two group tral/study The cumulatve ncdence rates are proportons where number of subjects s the unt for both the denomnator and numerator, mplctly assumng that all subjects are followed for the same perod of tme, whch s not always the case Some subjects may drop out or end a study earler than others Therefore, ther observaton lengths vary The 1
ncdence densty or ncdence rate s commonly used to adjust for dfferent observaton lengths by usng the unt of person-tme n prospectve cohort studes Gven the prospectve nature of clncal trals, subjects randomzed nto treatment and placebo groups are followed up to the predefned observaton perod Incdence densty calculated usng person-tme s more approprate than cumulatve ncdence rate calculated from usng ndvduals to assess adverse effects, and the smultaneous confdence ntervals of RR can be used to measure assocaton between AE and treatment usng the newly developed SAS program METHODS 1 Confdence Intervals for RR Let x be the number of AE for the treatment group and T 2 x C be the number of AE for the control group at the th type of AE, and let n T and nc be the person-tmes of the th type of AE for treatment and control groups, respectvely, where 1,2,, k The AE specfc ncdence denstes for the th C C C type are calculated as p x / n for the treatment group and p x / n for the control group, then RR can be calculated as T T T r ˆ p / p T C Confdence Ln ( rˆ ) Z1 / 2SE ( Ln ( rˆ ) nterval for RR s estmated as e and the hypothess testng for RR s based on Z Ln( rˆ) / SE( Ln(ˆ) r for Pr Z 2 Smultaneous Confdence Intervals for RR To control type I error rate, smultaneous confdence ntervals for RR are constructed Commonly used procedures nclude the Bonferron-based correcton [4] If each of k confdence ntervals has coverage probablty of 1 ( /, then the probablty that smultaneous confdence ntervals cover ther parameters s at least1 The method s straghtforward, but t s conservatve and reduces senstvty The followng are less conservatve methods ether to control the FWER or the false dscovery rate (FDR): Bonferron-Holm Step-down Method for FWER: Ths method performs more than one hypothess test smultaneously [5] Brefly, order the raw p-values, p( 1) p(2) p(, and let H( 1), H(2) be the correspondng null hypotheses Then, compare p(1 ) to / k If p ( 1) / k then reject H (1) and contnue; else, stop and fal to reject any hypothess Compare p(2) to /( k 1) If p( 2) /( k 1) then reject H (2) and contnue; else, stop and fal to reject any of hypotheses H ( 2) Contnue n ths fashon untl a stop or untl all hypotheses have been rejected Adjusted p values are p( 1) kp(1 ), p( 2) max( p(1),( k 1) p(2) ), p max( p(2),( k 2) (3) ),, p( max( p( k1), p( ) The crtcal values Z1 for the ( 3) p smultaneous confdence ntervals are based on the adjusted p s Benjamn-Hochberg Step-up Method for FDR: Ths method ams to control the FDR rather than the FWER Note that the FDR s the expectaton of false dscovery, not the probablty of rejectng at least one hypothess ncorrectly [6] Brefly, order the raw p-values, p( 1) p(2) p(, and let H( 1), H(2) be the correspondng null hypotheses Then, compare p( to k / k If p( k ) k / k, then reject all H ( 1), H(2) and stop; else, contnue
Compare p( k1) to ( k 1) / k If p( k 1) ( k 1) / k, then reject H ( 1), H (2),, H ( k 1) and stop; else, contnue Contnue n ths fashon untl a stop or untl no hypotheses are rejected Adjusted p values are p( k ) kp( k ) / k, p( k 1) mn( p( k ), kp( k1) / k 1, p( k 2) mn( p( k1), kp( k2) / k 2,, p( 1) mn( p(2), kp(1 ) ) The crtcal values Z1 are used to construct analogues of the smultaneous confdence ntervals derved from the adjusted p s Benjamn-Hochberg-Yekutel for FDR: Smlar to the Benjamn-Hochberg method, ths procedure controls the FDR under dependence assumptons [7] Under the ndependent assumpton, p k k / m, otherwse, p k k / c( m) m If the test statstcs are postvely ( ) ( ) correlated then c ( m) 1, else, f they are negatvely correlated, then c ( m) 1/ ln( m) The crtcal values Z 1 are used to construct analogues of the smultaneous confdence ntervals derved from the adjusted p s 3 Analytcal Procedures AEs are usually reported as onset date, end date, duraton, symptoms, severty, and outcome Based on the nformaton, one can count specfc events and compute person-tme to estmate the ncdence densty for subjects wthn each treatment group For subjects who have a partcular AE, the person-tmes are calculated as onset date mnus randomzaton date, whereas for those who do not have a partcular AE, the person-tmes are calculated as study end date or drop out date mnus randomzaton date AE specfc ncdence densty for each treatment group s estmated by summaton of the AE counts dvded by the person-tme for that partcular event Relatve rsk for certan AE can be estmated by the ncdence densty of the treatment group dvded by the ncdence densty of the control group The ndvdual confdence nterval of the relatve rsk for each AE s constructed as descrbed n Secton 1 The crtcal value Z1 / 2 for 95% confdence nterval s set up for 196, whereas crtcal value Z1 / 2 for 95% smultaneous confdence ntervals vary from method to method descrbed n Secton 2, dependng on the number of AEs analyzed The ndvdual 95% confdence ntervals and the smultaneous confdence ntervals based on the four methods e Bonferron, Bonferron-Holm, Benjamn- Hochberg, and Benjamn-Hochberg-Yekutel are tabulated, and the correspondng forest graphs are plotted by the SAS program EXAMPLE 1 Data Adverse event data from a phase III placebo-controlled, double-blnded, mult-center tral for lofexdne for opate wthdraw was used as an example for the above AE analyss approach The study ncluded 68 opod dependent subjects enrolled at three stes wth 35 randomzed to lofexdne and 33 to placebo In ths tral, 32 AE were coded based on clncal symptoms, and the data was stored n SAS format n a database Among these coded AE, 30 of them were observed durng the study Just for llustraton purpose, AE symptoms (types) were not revealed n the analyss All AEs were bnary varables 3
2 Results From the orgnal data, person tme was calculated for each subject Gven the short observaton tme of the tral (24 weeks), person-days were used to compute IC (event/person-days) for each group RR was derved from the rato of the ncdence densty of the treatment group over that of the placebo group If the RR s sgnfcantly larger than 1, then the AE s assocated wth the treatment A 95% confdence nterval (CI) was used to estmate true RR If the 95% CI ncludes 1, then the estmated RR s not statstcally sgnfcant Frst, AE specfc RR was estmated for each AE type or symptom, and ts correspondng 95% CI was computed to estmate whether the true RR was sgnfcantly dfferent from 1 Snce there were 30 AE types, smultaneous confdence ntervals were constructed usng the Bonferron (Bon_L, Bon_U), Bonferron-Holm (BH_L, BH_U), Benjamn-Hochberg (BHY_L, BHY_U), and Benjamn-Hochberg-Yekutel (BHY-_L, BHY-_U) methods, whch yelded wder ntervals than those wthout adjustng As shown n table one, AE types 20, 27 and 32 were sgnfcantly dfferent between the two treatment groups based on the ndvdual 95% confdence ntervals wthout adjustng for multple comparsons (Raw_L, Raw_U) However, after adjustng the multple comparsons usng the four methods, none of them were sgnfcant based on the smultaneous confdence ntervals Table One: Smultaneous Confdence Intervals from Dfferent Methods Type RR Raw_L Raw_U Bon_L Bon_U BH_L BH_U BHY_L BHY_U BHY-_L BHY-_U AE01 07941 04560 13830 03261 19337 03440 18330 04210 14977 03664 17211 AE02 06597 03490 12471 02375 18321 02437 17860 02952 14741 02544 17111 AE03 07737 04061 14742 02751 21762 02946 20318 03732 16040 03172 18875 AE04 08307 04149 16634 02728 25303 03153 21885 03988 17303 03321 20784 AE05 14660 01329 161673 00312 689311 00688 312368 01259 170769 00659 326140 AE06 07330 02122 25320 01004 53540 01337 40180 02000 26867 01439 37334 AE07 09529 04179 21731 02539 35758 04179 21731 04179 21731 03335 27225 AE08 20769 08189 52676 04667 92425 04815 89578 06270 68791 05057 85287 AE09 24922 09195 67552 05034 123379 05143 120764 06511 95391 05210 119214 AE10 17104 04423 66141 01954 149733 02290 127736 03767 77662 02670 109545 AE11 12566 06500 24291 04365 36173 04945 31929 06223 25373 05235 30165 AE13 14660 06586 32632 04062 52914 04323 49717 05742 37433 04714 45596 AE14 16755 06893 40725 04031 69644 04287 65475 05838 48080 04699 59735 AE15 73301 09383 572608 02711 1982299 02800 1918966 04259 1261490 02715 1979017 AE16 08959 05387 14899 03962 20257 04592 17480 05303 15134 04627 17346 AE17 12217 02920 51120 01230 121372 02377 62785 02889 51663 01957 76268 AE18 07330 03293 16316 02031 26457 02252 23856 03022 17777 02462 21820 AE19 06597 03922 11096 02865 15190 02906 14975 03331 13065 02960 14705 AE20 04887 02485 09610 01652 14459 01657 14409 01775 13454 01544 15470 AE21 03665 00332 40418 00078 172329 00096 139910 00228 58937 00125 107050 AE22 21990 02287 211404 00583 829613 00804 601316 01840 262850 01027 471031 AE24 02443 00254 23489 00065 92179 00072 82897 00147 40530 00086 69287 AE25 02443 00254 23489 00065 92179 00073 81568 00154 38772 00090 66681 AE26 06283 02906 13584 01824 21643 01913 20640 02484 15892 02061 19152 AE27 06173 03857 09879 02903 13124 02917 13061 03148 12105 02847 13384 AE28 08796 04859 15922 03395 22788 03957 19552 04748 16297 04052 19093 AE29 21990 02287 211404 00583 829613 00830 582414 01884 256689 01047 461683 AE30 21990 02287 211404 00583 829613 00860 562214 01927 250919 01068 452918 AE31 14660 01329 161673 00312 689311 00781 275101 01282 167590 00669 321075 AE32 03665 01569 08564 00939 14299 00939 14299 00939 14299 00796 16885 The results were also vsualzed n the followng forest plots (Fgure 1 and 2) 4
Fgure 1: 95% Confdence Intervals of RR for Each AE Fgure 2: 95% Smultaneous Confdence Intervals of RR for Each AE wth Four Methods 5
CONCLUSIONS In ths report, we propose a smple SAS program to construct smultaneous confdence ntervals usng four dfferent methods to control FWER and/or FDR for AE assessment n clncal trals The program was llustrated usng date from a randomzed, double-blnded clncal tral Although the program was desgned for AE assessment n clncal trals, t can be used for any multple endponts n both clncal trals and cohort studes TRADEMARK CITATION SAS and all other SAS Insttute Inc product or servce names are regstered trademarks or trademarks of SAS Insttute Inc n the USA and other countres ndcates USA regstraton Other brand and product names are regstered trademarks or trademarks of ther respectve companes REFERENCES 1 Chuang-Sten Ch Safety analyss n controlled clncal trals Drug Informaton Journal 1998; 32:1363S 1372S 2 O Nell RT Statstcal analyses of adverse event data from clncal trals, specal emphass on serous events Drug Informaton Journal 1987; 21:9 20 3 O Nell RT Statstcal concepts n the plannng and evaluaton of drug safety from clncal trals n drug development: ssues on nternatonal harmonsaton Statstcs n Medcne 1995; 14:1117 1127 4 Bonferron, C E "Il calcolo delle asscurazon su grupp d teste" In Stud n Onore del Professore Salvatore Ortu Carbon Rome: Italy, pp 13-60, 1935 5 Holm, S "A smple sequentally rejectve multple test procedure" Scandnavan Journal of Statstcs 1979, 6 (2): 65 70 6 Benjamn, Y and Hochberg, Y Controllng the false dscovery rate: a practcal and powerful approach to multple testng Journal of the Royal Statstcal Socety, Seres B (Methodologcal) 1995, 57 (1): 289 300 7 Benjamn,Y and Yekutel,Y False dscovery rate controllng confdence ntervals for selected parameters J Am Statst Ass, 2005, 100, 71 80 CONTACT INFORMATON Your comments and questons are valued and encouraged Contact the author at: Xaol Lu XaolLu@vagov Code samples are avalable upon request 6