Self-controlled case series analyses: small sample performance

Transcription

1 Self-controlled case seres analyses: small sample performance Patrck Musonda 1, Mouna N. Hocne 1,2, Heather J. Whtaker 1 and C. Paddy Farrngton 1 * 1 The Open Unversty, Mlton Keynes, MK7 6AA, UK 2 INSERM U780 ; Unv Pars-Sud, Vllejuf F-94807, France Abstract We derve second-order expressons for the asymptotc bas and varance of the log relatve ncdence estmator for the self-controlled case seres method n a smplfed scenaro, and study n qualtatve terms how bas and varance depend on factors such as the relatve ncdence and rato of rsk to observaton perod. Small-sample performance of the estmator n realstc scenaros s nvestgated usng smulatons. We fnd that n scenaros lkely to arse n practce, asymptotc methods are vald for numbers of cases n excess of dependng on the rato of the rsk perod to the observaton perod and on the relatve ncdence. The applcaton of Monte Carlo methods to self-controlled case seres analyses s also dscussed. Keywords: Asymptotc bas; Asymptotc varance; Bootstrap; Randomzaton test; Self-controlled case seres method; Smulaton; Small-sample performance * Correspondng author. Department of Statstcs Faculty of Mathematcs and Computng The Open Unversty Walton Hall Mlton Keynes MK7 6AA Tel. : +44 (0) Fax.: +44 (0) Emal: c.p.farrngton@open.ac.uk Ths research was supported by EPSRC (CASE0307), GlaxoSmthKlne Bologcals, and Wellcome Trust project grant

2 1 Introducton The self-controlled case seres method, or case seres method for short, s a condtonal cohort method for estmatng the strength of assocaton between the ncdence of specfed events and a tme-varyng exposure usng data only on cases. The method was orgnally developed to nvestgate assocatons between vaccnaton and acute potental adverse events [3]. Other applcatons, along wth a detaled account of the theory and ts mplementaton n standard statstcal packages are descrbed n Whtaker et al [11]. A sem-parametrc verson of the method has also been developed [7]. Whle the maxmum lkelhood estmator of the relatve ncdence s guaranteed good asymptotc propertes for both parametrc and sem-parametrc models, n practce samples are often small, especally for rare condtons. Lmted small-sample smulatons for the sem-parametrc model suggest that t performs well n samples of moderate sze [7]. However, no systematc evaluaton of the statstcal propertes of the method has been undertaken. Some comparatve evaluatons have been done, comparng the case seres method wth case-control, cohort and other case only methods [1, 4, 6]. Our am n ths paper s to nvestgate n more detal the factors that nfluence the magntude of the bas and varance of the relatve ncdence estmator, or more precsely the estmator of the log relatve ncdence. For smplcty, we confne our nvestgatons to the parametrc self-controlled case seres model and to the rsks assocated wth exogenous pont exposures [2]. The paper s organsed as follows. In secton 2 we ntroduce the case seres model. Explct expressons for the asymptotc bas, varance and mean square error n a smplfed but relevant scenaro are derved and studed n secton 3. Secton 4 descrbes a smulaton study to evaluate bas and varance n small samples under more realstc scenaros. The results from ths smulaton study are presented n secton 5. In secton 6, we dscuss the applcaton of Monte Carlo methods to selfcontrolled case seres analyses, ncludng bootstrap estmaton and randomzaton tests. Fnally n secton 7 we dscuss our fndngs and make some recommendatons. 2

3 2 The self-controlled case seres model The self-controlled case seres model s derved from an underlyng Posson cohort model. Thus, we consder a cohort of ndvduals, ndvdual beng observed n the nterval ( a, b ]. Ths nterval s the observaton perod for ndvdual ; we shall use age as the underlyng tme lne, but other choces are possble, notably calendar tme. The observaton perod for ndvdual s parttoned nto ntervals ndexed by j = 0,1,..., J (for age groups) and k = 0,1,..., K (for rsk perods). The age groups are pre-determned, as are the duratons of the post-exposure rsk perods. Rsk perods k = 1,..., K correspond to ncreased or reduced rsks relatve to the baselne control perod, whch s coded k = 0. The age groups are typcally of the form (0, A ],( A, A ],...,( A, A ],( A, ). Post-exposure rsk perods are typcally of J 2 J 1 J 1 the form ( E + Bk 1, E + Bk ] where E s the age at exposure of ndvdual and B0 <... < BK, the remander of the observaton tme beng allocated to the control perod. Let e jk denote the duraton of tme that ndvdual spends n age group j and n rsk perod k durng the course of hs or her observaton perod. Condtonng on the exposure hstory over the entre observaton perod ( a, b ], we assume that events of nterest for ndvdual arse as a non-homogeneous Posson process wth rate λ jk njk k, then denotes the number of events arsng for ndvdual n age group j and rsk perod n Posson( λ e ). jk jk jk. If Condtonng on the total number of events n = Σ j, knjk arsng n ( a, b ], whch s possble by vrtue of the assumpton that the exposure s an exogenous varable [2, 7], the log-lkelhood contrbuton of ndvdual s multnomal wth kernel 3

4 l λjk ejk = njk log j, k λrse. (1) rs r, s We assume a log-lnear model for the Posson rate of the form log( λ ) = ϕ + α + (2) jk j k where ϕ s an ndvdual effect, α j s the age effect assocated wth age group j, and s the exposure effect assocated wth rsk group k, wth α0 = 0 = 0. The k parameters α j and k are thus log relatve ncdences. Substtutng (2) n (1), and summng over ndvduals, we obtan a product multnomal log-lkelhood kernel: exp( α j + k ) ejk l( α, ) = njk log j, k exp( αr s ) e. (3) + rs r, s Ths s the self-controlled case seres log-lkelhood. The model s self-controlled because the ndvdual effects ϕ cancel out. Thus multplcatve confounders that do not vary over the ndvdual s observaton perod whch mght nclude, for example, genetc effects, soco-economc status, locaton, underlyng state of health, ndvdual fraltes are necessarly adjusted n the analyss. It s a case seres model because only ndvduals who have experenced one or more events, that s ndvduals for whom n 1, contrbute non-trvally to the log-lkelhood (3). Thus, only cases need to be sampled. These features make the self-controlled case seres method an attractve alternatve to other methods n some settngs. The effcency of the case seres model relatve to the underlyng cohort model, and the assumptons requred, n partcular the mportant assumpton that the exposure varable s exogenous, are dscussed n [7]. 4

5 3 Asymptotc bas, varance and mean square error In ths secton we study n greater detal the asymptotc propertes of the estmators of the log relatve ncdence. 3.1 A smplfed scenaro Our am s to obtan qualtatve nsght nto the factors whch affect bas and varance. So as to obtan smple explct expressons, we make the followng assumptons. All cases have the same observaton perod ( a, b ] = ( a, b]. There are no underlyng age effects, that s, α j = 0 for all j. There s at most one post-exposure rsk perod, that s, K = 1. All cases experence an exposure rsk perod of common duraton e 1 and a control perod of common duraton e 0, wth e0 + e1 = b a. The age parameters may thus be dropped from the model. We denote = 1. Under these assumptons, the log-lkelhood (3) for n events reduces to the expresson ( 1 0 ) l( ) = x n log e e + e (4) where x s the number of events occurrng n the exposure rsk perod. The maxmum lkelhood estmator of s x r = log log n x 1 r where r = e1 /( e0 + e1 ) s the rato of the length of the rsk perod to the observaton perod. Expandng as a functon of x by Taylor seres to fourth order, we obtan the followng expressons for the asymptotc bas and varance, to second order. 5

6 bas( ) = E( ) = ( re (1 r) ) + 2n re 1 r 5( re ) + 4 re (1 r) + 5(1 r) nre (1 r) O( n ) (5) re + (1 r) re re r r n re (1 r) 2 nre (1 r) ( ) ( ) 2 (1 ) + 3(1 ) 3 var( ) = O( n ). (6) Combnng expressons (5) and (6), we obtan the asymptotc mean squared error: re + (1 r) re re r r n re (1 r) 4 nre (1 r) ( ) ( ) 6 (1 ) + 7(1 ) 3 AMSE( ) = O( n ). (7) 3.2 Asymptotc propertes Consder frst the asymptotc bas. The expresson n square brackets n (5) s always greater than 1, so that sgn( bas( )) = sgn ( re (1 r) ) and the second-order bas s always greater n magntude than the frst-order bas. The asymptotc bas s zero when re = 1 r, whch occurs when the expected number of cases n the rsk perod equals the expected number of events n the control perod. The asymptotc bas s negatve (respectvely, postve) when the expected number of events n the rsk perod s less (respectvely, greater) than that n the control perod. In practce, the rsk perod s determned by the scentfc queston of nterest, and the observaton perod s determned both by the age range at whch exposures occur and by the practcaltes of data collecton. For a gven value of r, the asymptotc bas s mnmzed when 6

7 e 1 r =. r For a fxed value of, the asymptotc bas ncreases n magntude as r tends to 0 or 1. Smlarly, for a fxed value of r, the asymptotc bas ncreases n magntude as tends to ±. Fgure 1 shows the value of the second-order asymptotc bas for n = 50, for dfferent values of r and e. The asymptotc bas s neglgble unless the rato of the rsk perod to observaton tme s very close to 0 or 1, but ncreases sharply n these regons for smaller sample szes. Turnng now to the asymptotc varance, ts value to second-order s always greater than to frst order. Regardng expresson (6) as a functon of r, ts mnmum s attaned when re = 1 r. Thus, the asymptotc varance s smallest when the expected number of events n the rsk perod equals the expected number n the control perod. Fgure 2 shows var( ) for n = 50, for dfferent values of r and e. As for the bas, the asymptotc varance ncreases as r tends to 0 or 1 and as tends to ±. The second-order asymptotc mean squared error (7) s close to the second-order varance. It s mnmzed when re = 1 r, but s typcally very flat for values r n the range (0.1, 0.9) and < log(10). 4. Smulaton study In ths secton we study the propertes of the maxmum lkelhood estmator by smulaton, n more realstc scenaros than that descrbed n secton 3. In partcular, we no longer assume that there s no effect of age, or that all ndvduals have the same exposure rsk perod. Our am s to nvestgate the lmts of valdty of asymptotc theory n fnte samples. Because s the logarthm of a rato estmator, t takes values ± wth postve probablty n fnte samples. Thus, rather than the bas per se, whch s undefned, we nvestgate the medan m ( ) n of the estmator n samples of sze n. Ths provdes an 7

8 approprate measure of central tendency of the estmator n fnte samples. Note that lm m ( ) = E( ) snce s asymptotcally normally dstrbuted. From now on, the n n term bas refers to m ( ) n. We also nvestgate the coverage probablty of the Wald % confdence nterval calculated from ± 1. se( ) where se( ) s the asymptotc standard error (for unbounded estmates the confdence nterval s n effect (, + ) ). The smulatons were set up to mmc those scenaros that typcally occur n studes of paedatrc vaccnes. The smulaton experments are descrbed n the followng sectons Structure of the smulaton study Each smulaton requred the followng parameters to be specfed. Observaton perod, always taken to be 500 days for all ndvduals. Length of the rsk perod followng exposure (days): 1, 5, 10, 25, 50, 100, 200, ndefnte (descrbed n secton 4.4). True relatve ncdence RI = e = 0.5, 1, 1.5, 2, 5, 10. Dstrbuton for age at exposure E (secton 4.3). Age groups and age-specfc relatve ncdences (secton 4.2, Fgure 4). Baselne rate, always taken to be 7 ϕ = 2 10 per day, or one per hundred thousand over 500-day observaton perod. Thus the event s assumed to be rare, and wth hgh probablty a case has only a sngle event. Sample sze n = 10, 20, 50, 100, 200, 500, 1000 cases. Fgure 3 shows the structure of the smulaton study n graphcal form. For a gven set of parameters (lsted above) and random seed, a set of n exposure tmes were generated, together wth n margnal total number of events per ndvdual. These margnal totals were generated usng a truncated Posson dstrbuton (excludng zero), condtonally on the exposure hstory. 8

9 The exposures and margnal totals were resampled between runs. however, n each run of 10,000 smulatons, the exposures and margnal totals were kept fxed. Ths s to mmc the fact that the case seres method s condtonal on exposures and margnal totals. Wthn a run, the events for each ndvdual were randomly reallocated 10,000 tmes to the age and rsk categores wthn each ndvdual s person tme. Ths was done based on the case seres model, usng a multnomal dstrbuton. The run sze of 10,000 ensures that the coverage probablty for a % confdence nterval s estmated wth Monte Carlo standard error of about , and hence s accurate to wthn about (or 0.5% when expressed as a percentage). 4.2 Age effects In most self-controlled case seres analyses, t s necessary to control for age. We vared the effect of age on the event ncdence accordng to four practcally realstc scenaros. These four types of age effect are defned as follows; n each case the age groups are gven, along wth the assocated age-specfc relatve ncdences brackets) j e α (n Weak symmetrc age effect: (1), (1.2), (1.5), (1.2), and (1). Strong symmetrc age effect: 1-50 (1), (2), (3), (4), (5), (5), (4), (3), (2), and (1). Weak monotone ncreasng age effect: (1), (1.1), (1.2), (1.3), (1.4) Strong monotone ncreasng age effect: 1-50 (1), (1.5), (2), (2.5), (3), (3.5), (4), (4.5), (5), and (5.5). Fgure 4 shows bar charts representng each of the above four choces of age groups and age-specfc relatve ncdences. 9

10 4.3 Exposure dstrbuton The precson of the relatve ncdence estmator depends on the extent of betweenndvdual varaton n age at exposure. We used the followng four beta dstrbutons on [0,500] to generate age at exposure. Mean age 250 days and standard devaton 100 days. Mean age 250 days and standard devaton 50 days. Mean age 125 days and standard devaton 100 days. Mean age 125 days and standard devaton 50 days. These dstrbutons are shown n Fgure 5. For some smulatons, much more hghly peaked dstrbutons of age at exposure were also consdered, wth mean age of 125 days and standard devaton of 10, 20, 30, and 40 days. 4.4 Rsk perods Before carryng out a self-controlled case seres analyss, a major ssue to consder s how to defne the rsk perods. Generally speakng the rsk perods are elcted from experts. Dfferent studes need dfferent rsk perods. These range from very short (a few days) to very long (several months), and occasonally may be ndefnte. We smulated data wth rsk perods of 1, 5, 10, 25, 50, 100 and 200 days. We also nvestgated ndefnte rsk perods. Owng to potentally strong confoundng between age and exposure effects wth ndefnte rsk perods, we consdered these separately and vared the proporton of cases exposed (n other smulatons we assumed all cases were exposed). 5 Results of the smulaton study The presentaton of results s organsed n fve subsectons. In subsecton 5.1 we present results for our standard scenaro. In subsecton 5.2 we vary the rsk perod. 10

11 In subsecton 5.3 we vary the age effect. In subsecton 5.4 we vary the age at exposure. Fnally, n subsecton 5.5 we consder ndefnte rsk perods. 5.1 The standard scenaro For our standard scenaro the rsk perod was 25 days, all cases experenced the exposure, the age effect was weak symmetrc (see Fgure 4) and the dstrbuton of age at exposure has mean age 250 days and standard devaton 100 days (see Fgure 5). Table 1 shows the results for the standard scenaro. For very small samples ( n 20 ) and low relatve ncdences (RI 1), there s consderable bas: effectvely, n most samples there were zero events wthn a rsk perod, yeldng unbounded estmates of. For relatve ncdences greater than 1, the bas s moderate even for sample szes as small as 10. For sample szes n excess of 20, the bas s small for most values of the relatve ncdence (the excepton beng RI = 0.5). The bas tends to be negatve for low relatve ncdences, and postve for large relatve ncdences. Ths reflects the asymptotc results obtaned n secton 3, namely that, n the absence of age effects, the asymptotc bas s negatve when e < (1 r) / r and postve when e > (1 r) / r. Here, r = 25 / 500 = Thus, asymptotcally, and provded that age effects are not too strong, one mght expect zero bas at e 20. In fnte samples, ths pont appears to be reached for lower relatve ncdences: for example, wth n = 50, t s reached at e 5 n the standard scenaro. Fnally, note from Table 1 that the coverage probabltes of the Wald % confdence ntervals are close to ther nomnal values for all combnatons of sample sze and relatve ncdence, though tend to be conservatve especally for low sample szes. Smlar results (not shown) were obtaned for 90% and 99% confdence ntervals. 5.2 Rsk perod of fxed length The fxed-length rsk perods were: 1, 5, 10, 50, 100 and 200 days. Table 2 shows the results (wth n = 20, 100 and 500) for the short rsk perods of 1 and 5 days, and Table 3 shows the results for longer rsk perods of 50 and 100 days. 11

12 As expected from the asymptotc calculatons, the bas ncreases n absolute value as r, the rato of the rsk perod to the observaton perod (500 days), tends towards zero. Wth a 1-day rsk perod, the bas s consderable n small or moderate samples, unless the relatve ncdence s hgh: t s possble to estmate wth lttle bas for a 1-day rsk perod wth sample szes of 100 cases provded that the relatve ncdence s n excess of 5. A slght ncrease n the length of the rsk perod has a bg effect: there s lttle bas wth sample szes as small as 20 for relatve ncdences n excess of 5 when the rsk perod s 5 days. For longer rsk perods (50 and 100 days), Table 3 shows that there s lttle bas even for sample szes as small as 20, when the relatve rsk s greater than 1. The results for the 10 day rsk perod were broadly smlar to those for 25 days (the standard scenaro), whle the results for the 200 day rsk perod were smlar to those for the 100 day rsk perod (not shown). 5.3 Age at event In ths secton, we summarze the results we obtaned by varyng the underlyng age effect. We nvestgated sample szes 20, 100 and 500 and rsk perods of 10, 25 and 50 days, wth relatve ncdences of 1, 2 and 5. The dstrbuton of age at exposure was as n the standard scenaro, namely mean 250 days and standard devaton 100 days. Table 4 gves the results for sample sze 100 wth rsk perod 25 days. Varyng the age effect has lttle nfluence on the magntude of the bas or on the coverage probabltes, for any of the rsk ntervals consdered here. Smlar results were obtaned for other sample szes (not shown). 5.4 Age at exposure In the standard scenaro, the dstrbuton of age at exposure was a symmetrcal beta dstrbuton wth 250 days and standard devaton 100 days. Here we evaluate the performance of the model when we vary the mean and standard devaton. In vew of possble confoundng between age and exposure effects, nterest focuses partcularly on the bas when a postvely skewed dstrbuton of age at exposure s combned wth a strong monotone ncreasng age at event effect. 12

13 Table 5 presents the results for samples of 100 cases, rsk perods 25 and 50 days, relatve ncdences of 1 and 5, and both the weak symmetrc and the strong monotone age effects. There s lttle evdence that the mean or standard devaton of the age at exposure have any dscernble mpact on the bas or coverage probabltes. Smlar results were obtaned for the 10 day rsk perod, and for RI = 2 (not shown). 5.5 Indefnte rsk perods The self-controlled case seres method can be used even when the rsk perod followng an exposure s ndefnte [5, 11]. However, exposure and age effects may be confounded. Ths can be controlled by ncludng unexposed cases, whch contrbute exclusvely to the estmates of the age effects. For age at event, we used the weak symmetrc, and the strong monotone ncreasng age dstrbutons. We nvestgated sx beta dstrbutons of age at exposure: mean 250 days and standard devaton 100 days, mean 125 days and standard devaton 50 days, and four more peaked dstrbutons wth mean 125 days and standard devatons 40, 30, 20 and 10 days. We studed relatve rsks of 1, 2 and 5. We used samples of 100 exposed cases, augmented by 0%, 20%, 50% and 100% unexposed cases. For example, the sample augmented by 20% unexposed cases contaned 100 exposed cases and 20 unexposed cases. Table 6 shows the results for the strong symmetrc age effect and dstrbutons of age at exposure wth mean 125 days and standard devatons 10, 30 and 50 days. When the relatve ncdence s 1, s estmated wthout substantal bas even wth no unexposed cases. The greater the relatve ncdence and the more peaked the dstrbuton of age at exposure, the greater the bas: when the relatve ncdence s 5, the estmate s swamped by bas. However, ncluson of just 20 unexposed cases s suffcent to greatly reduce the bas. Interestngly, ncluson of more than 20 unexposed cases has lttle further benefcal effect. The coverage probabltes of the % confdence ntervals are unaffected. 13

14 When the dstrbuton of age at exposure s more evenly spread over the observaton perod (mean 250 and standard devaton 100), there s lttle bas even when only exposed cases were ncluded (not shown). 6 Monte Carlo methods In ths secton we descrbe the applcaton of Monte Carlo methods to the selfcontrolled case seres method, wth reference to two example data sets relatng to measles, mumps and rubella (MMR) vaccne. 6.1 The data In the frst data set the outcome s aseptc menngts, whch s occasonally assocated wth recept of MMR vaccnes contanng the Urabe mumps stran. There are 10 events n 10 chldren observed from ages 366 to 730 days of age nclusve. The analyss uses two age groups (366 to 547 days, and 548 to 730 days) and a sngle rsk perod days post-mmr. There were 5 events n the rsk perod. For further detals, see [9, 11]. In the second data set, the outcome s dopathc thrombocytopenc purpura (ITP), an uncommon bleedng dsorder occasonally assocated wth MMR vaccnaton. The observaton perod s 366 to 730 days of age. There are 35 chldren wth 44 ITP events. The analyss uses three age groups ( , , and days of age) and three rsk perods: 0 14 days, and days post-mmr. There were 2 events n the 0 14 day, 8 n the day, and 3 n the day rsk perods. For further detals see [10, 11]. In both data sets, the small number of events n the rsk perods calls nto queston the valdty of the asymptotc theory underpnnng the calculaton of confdence ntervals and p values. 6.2 Bootstrap The most readly applcable bootstrap method for self-controlled case seres studes s the non-parametrc method based on resamplng of cases. Ths s preferred to resamplng of resduals, snce t s far from clear what an approprate resdual, or set of resduals, would be n ths context. Note that the unts to be resampled are the 14

15 cases, not the events (an ndvdual who has experenced several events consttutes one case). As prevously noted, the bas of s undefned n fnte samples. We thus nvestgate the medan m ( ) B of the bootstrap samples; t s desrable that should le close to ths value. We also obtan percentle and bas-corrected percentle confdence ntervals [8]. All results are based on 4999 bootstrap samples. The results are shown n Table 7. Fgure 6 shows the centres of the dstrbutons of the bootstrap replcates for the two data sets; unbounded estmates have been excluded from the fgure. The pont estmates are close to the medan bootstrap values, suggestng that the bas s mld, but there are substantal dscrepances between asymptotc and bootstrap % confdence ntervals. Wth the possble excepton of Fgure 6(c), the bootstrap dstrbutons dsplay marked evdence of non-normalty. The multple modes correspond to estmates based on dstnct numbers of events wthn the rsk perod. 6.2 A randomzaton test Throughout ths paper the emphass has been on pont and nterval estmaton. In some crcumstances, however, t s requred to test the null hypothess of no assocaton between the exposure and event of nterest. For ths purpose, the lkelhood rato test s readly applcable when the sample sze s suffcently large that asymptotc theory can be reled upon. When ths s not the case, however, other methods may be requred. We descrbe a sutable randomzaton test, mplemented by Monte Carlo methods. Under the null hypothess of no assocaton, exposure hstores and event hstores are ndependent. A randomzaton test may thus be obtaned by randomly parng event tmes and exposures. More specfcally, consder a sample of n cases, case havng n events at tmes t,..., 1 tn and exposure hstory E. We then permute the exposure hstores from { E1,..., E n } and allocate the permuted values E σ ( ) to obtan new data of the form {( a, b ]; t 1,..., tn ; E σ ( ) }. These data are then analysed usng the selfcontrolled case seres method to produce a value of the log-lkelhood rato statstc 15

16 D σ. The dstrbuton of the D σ over all permutatons (whch thus ncludes the observed value D 0, say) consttutes the null dstrbuton, from whch the p value may be calculated from #{ Dσ : Dσ D0}. In practce, t s usually not feasble to obtan all permutatons, n whch case a random sample s used, augmented by D 0. Ths randomzaton test s standard [8]. The only specal pont to note s that the test requres that exposure hstores are collected n the range (mn{ a }, max{ b }] to ensure that reallocated hstores are relevant to all the observaton perods ( a, b ]. For the aseptc menngts data, none of 999 randomly sampled values of D σ exceeded D 0 = Thus, the estmated p-value s (0+1) / (999+1) = The p value based on the asymptotc χ 2 (1) dstrbuton s For the ITP data, 9 values of D σ out of 999 exceeded D 0 = Thus the estmated p value s (9+1) / (999+1) = The p value based on the asymptotc χ 2 (3) dstrbuton s Fgure 7 shows the randomzaton and asymptotc dstrbutons under the null hypothess. There s a substantal dfference between the randomzaton and asymptotc dstrbutons n each case, though the randomzaton and asymptotc tests lead to dentcal conclusons n these examples. 7 Dscusson The am of ths paper was to study the bas and varance of the maxmum lkelhood estmator of the relatve ncdence n self-controlled case seres studes. We were partcularly nterested n two aspects: determnng whch factors most substantally affect the bas and the varance, and the performance of the estmators n small to medum samples. The asymptotc expressons we obtaned n a smple scenaro suggest that the bas n s small unless (a) the rsk perod s short n relaton to the observaton perod and the relatve rsk s low, and (b) the rsk perod s long n relaton to the observaton perod and the relatve rsk s hgh. Specfcally, the drecton and magntude of the bas s 16

17 governed by the quantty re (1 r), where r s the rato of the rsk perod to the observaton perod and e s the relatve ncdence. Ths qualtatve concluson was confrmed n smulatons. Thus, we found that the bas s small when there are 50 or more cases, the relatve ncdence s not less than 1, and r s at least For sample szes of 20, the bas s large when the relatve ncdence s less than 2 or r s less than Varaton n age at exposure and age at event have only margnal effect on the bas for fnte rsk perods. For ndefnte rsk perods, confoundng between exposure and age effects may be controlled by ncluson of about 20% of unexposed cases. The asymptotc Wald confdence ntervals are generally slghtly conservatve, but perform well whatever the sample sze. When the estmate of the log relatve ncdence s unbounded, a confdence nterval obtaned by profle lkelhood methods [8] s preferable. When there s doubt about the valdty of asymptotcs, smulaton nference methods may be used. These nclude non-parametrc bootstrap methods based on resamplng complete cases (that s, ndvduals rather than events), and randomzaton tests. Note, however, that the use of randomzaton tests requres that exposures over the entre perod (mn{ a }, max{ b }] are obtaned. The scenaros we chose to nvestgate relate to those that are lkely to arse n studes of vaccne safety wth a sngle post-vaccnaton rsk perod. For smplcty, we dd not consder multple exposures, long but fxed rsk perods (wth r close to 1), semparametrc estmaton of the age effect, between-ndvdual varaton n observaton perods, and contnuous exposures. Most of these more general scenaros can nevertheless be related to those used here. Thus, dstnct rsk perods can be consdered separately, usng a value of r calculated as the rato of the rsk perod of nterest to the sum of the rsk perod and control perod; long fxed rsk perods wll yeld results smlar to those obtaned wth ndefnte rsk perods; between-ndvdual varaton n observaton perods may be accommodated by takng r to be the rato of the rsk perod to the average observaton perod; and age effects were shown to have only moderate mpact, though of course sem-parametrc estmaton wll necessarly yeld less precse estmates. 17

18 Our fndngs are thus broadly relevant to case seres studes of pont exposures. For contnuous tme-varyng exposures, further nvestgatons n small samples are requred. In such settngs, the noton of rsk perod s no longer relevant, and the wthn-ndvdual standard devaton of the exposure varable must be consdered nstead. To date, the only applcaton of self-controlled case seres methods wth contnuous exposure varables of whch we are aware s to envronmental tme seres. We have argued elsewhere that tme seres methods are generally more approprate than case seres methods for the analyss of such data [12], and n any case the sample szes used n such studes are usually large. 18

19 References [1] Andrews, N.J., Statstcal assessment of the assocaton between vaccnaton and rare adverse events post lcensure. Vaccne 20 S49-S53. [2] Dggle, P.J., Heagerty, P., Lang, S.L. and Zeger, S.L., Analyss of Longtudnal Data, 2 nd edton. Oxford Unversty Press, New York. [3] Farrngton, C.P., 19. Relatve ncdence estmaton from case seres for vaccne safety evaluaton. Bometrcs, [4] Farrngton, C.P., Nash, J., and Mller, E., 19. Case seres analyss of adverse reactons to vaccnes: a comparatve evaluaton. Amercan Journal of Epdemology, (Erratum ). [5] Farrngton, C.P., Mller, E. and Taylor, B., MMR and autsm: further evdence aganst a causal assocaton. Vaccne, [6] Farrngton, C.P., Control wthout separate controls: Evaluaton of vaccne safety usng case-only methods. Vaccne, [7] Farrngton, C.P. and Whtaker, H.J., Semparametrc analyss of case seres data (wth Dscusson). Journal of Royal Statstcal Socety, Seres C, In Press. [8] Garthwate, P.H., Jollffe, I.T. and Jones, B., Statstcal Inference, 2 nd edton. Oxford Unversty Press, New York. [9] Mller, E., Goldacre, M., Pugh, S., Colvlle, A., Farrngton, P., Flower, A., Nash, J., MacFarlane, L. and Tettmar, R., Rsk of aseptc menngts after measles, mumps and rubella vaccne n UK chldren. The Lancet, [10] Mller, E., Waght, P., Farrngton, P., Stowe, J. and Taylor, B., Idopathc thrombocytopenc purpura and MMR vaccne. Archves of Dsease n Chldhood, [11] Whtaker, H.J., Farrngton, C.P., Spessens, B. and Musonda, P., Tutoral n Bostatstcs: The self-controlled case seres method. Statstcs n Medcne, [12] Whtaker, H.J., Hocne, M.N. and Farrngton, C.P., On case-crossover methods for envronmental tme seres data. Envronmetrcs, n press. 19

20 Table 1 Standard scenaro. Frst row: medan estmate of = log( RI ). Second row: percentage coverage of % confdence nterval. True value RI n = 10 n = 20 n = 50 n = 100 n = 200 n = 500 n = Table 2 Short rsk perods. Frst row: medan estmate of = log( RI ). Second row: percentage coverage of % confdence nterval. 1 day rsk perod 5 day rsk perod True value RI n = 20 n = 100 n = 500 n = 20 n = 100 n =

21 Table 3 Longer rsk perods. Frst row: medan estmate of = log( RI ). Second row: percentage coverage of % confdence nterval. 50 day rsk perod 100 day rsk perod True value RI n = 20 n = 100 n = 500 n = 20 n = 100 n = Table 4 Effect of age at event for samples of sze 100. Frst row: medan estmate of = log( RI ). Second row: percentage coverage of % confdence nterval. Rsk perod (days) True value RI Weak symmetrc age effect Strong symmetrc age effect Weak monotone ncreasng age effect Strong monotone ncreasng age effect

22 Table 5 Effect of age at exposure for samples of sze 100. Frst row: medan estmate of = log( RI ). Second row: percentage coverage of % confdence nterval. 25 day rsk perod 50 day rsk perod Exposure dstrbuton Mean SD Weak True value symmetrc age effect RI E Strong monotone ncreasng age effect Weak symmetrc age effect Strong monotone ncreasng age effect Table 6 Indefnte rsk perods. Frst row: medan estmate of = log( RI ). Second row: percentage coverage of % confdence nterval. Exposure dstrbuton Mean SD True value RI 100 exposed cases 100 exposed cases and 20 unexposed 100 exposed cases and 50 unexposed 100 exposed cases and 100 unexposed

23 Table 7 Asymptotc and bootstrap results for aseptc menngts and ITP data Data set Rsk perod Asymptotc Bootstrap (days) Estmate % CI Medan Percentle Bas m ( B % CI corrected % CI Menngts , , , , , 1.494, ITP , , , , , 2.092,

24 Fgure 1 bas( ) for n = 50 aganst r, the rato of the rsk perod to the observaton perod, for dfferent values of the relatve ncdence (RI). Fgure 2 var( ) for n = 50 aganst r, the rato of the rsk perod to the observaton perod, for dfferent values of the relatve ncdence (RI). 24

25 Fgure 3 Structure of the smulaton study. Fx parameter values Generate exposure perods Generate margnal totals Dstrbute events across ndvdual s observaton tme Iterate 10,000 tmes Ft case seres model Output results 25

26 Fgure 4 The four effects of age at event used n the smulatons Fgure 5 Four dstrbutons of age at exposure used n the smulatons 26

27 Fgure 6 Bootstrap dstrbuton of relatve ncdence for aseptc menngts and ITP data, by rsk perod. (a) Aseptc menngts (15-35 days) (b) ITP (0-14 days) Densty Densty Bootstrap Values Bootstrap Values (c) ITP (15-28 days) (d) ITP (29-42 days) Densty Densty Bootstrap Values Bootstrap Values Fgure 7 Randomzaton and asymptotc dstrbutons of the lkelhood rato statstc Densty/y (a) Aseptc menngts Densty/y (b) ITP lkelhood rato statstc lkelhood rato statstc 27