A Comparison of Statistical Methods in Interrupted Time Series Analysis to Estimate an Intervention Effect

Transcription

1 Transport and Road Safety (TARS) Research Joanna Wang A Comparson of Statstcal Methods n Interrupted Tme Seres Analyss to Estmate an Interventon Effect Research Fellow at Transport & Road Safety (TARS) Unversty of New South Wales Road Safety Research, Polcng and Educaton (RSRPE) Conference August, 2013

2 Co-authors Scott Walter, Centre for Health Systems and Safety Research, Unversty of New South Wales Raphael Grzebeta, Transport and Road Safety, Unversty of New South Wales Jake Olver, School of Mathematcs and Statstcs, Unversty of New South Wales 2

3 Outlne Mandatory Helmet Legslaton n NSW Interrupted Tme Seres Analyss Full Bayesan and Emprcal Bayes methods Conclusons 3

4 MHL n NSW Mandatory Helmet Law (MHL) n NSW Apples to all age groups Came nto effect n two stages Adults (>16): January 1, 1991 Chldren: July 1, 1991 MHL led to much greater helmet wearng rate (>80% post law) Assocated wth fewer bcycle related head njures Sold evdence for helmet wearng n lowerng bcycle related head njures from bomechancal and epdemologcal studes 4

5 ITS A Smple Analyss Comparng sngle pre- and post-nterventon does not gve the full pcture: Ignores any trends before and after nterventon Ignores any cyclcal patterns Varances around the mean before and after nterventon may be dfferent Interventon may be mmedate or delayed Doesn t take nto account any possble autocorrelaton 5

6 ITS Interrupted Tme Seres (ITS) A type of tme seres where we know the specfc pont at whch an nterventon (nterrupton) occurred No randomsaton Defntve pre- and post-nterventon perods Type of quas-expermental desgn Strongest, quas-expermental approach for evaluatng longtudnal effects of nterventon (Wagner et al. 2002) Important comparsons can be made between pre- and post nterventon Change n level (mmedate effect) Change n slope (gradual effect)

7 ITS A Basc Model Usng segmented regresson (Wagner et al., 2002), Y t T I 0 1 t 2 3 T t I t Where - s the outcome of nterest Y t - =, -2.5, -1.5, -0.5, 0.5, 1.5, 2.5, (nterventon occurs T t when = 0 I - s a dummy varable = t T t - s the random error 0 pre nterventon 1 post nterventon

8 ITS A Basc Model (cont d) Usng segmented regresson (Wagner et al., 2002), Y t T I 0 1 t 2 3 T t I t Where - estmates the base level of the outcome estmates the base trend of the outcome (change wth tme n pre-nterventon perod) 2 - estmates the change n level n the post-nterventon perod (H 0 : = 0) estmates the change n trend n the post-nterventon perod (H 0 : = 0) 3

9 ITS Threats to Internal Valdty Factors other than the nterventon may nfluence the dependent varable For nstance, changes n head njury rate may due to Declne n the number of cyclsts Constructon of cyclng nfrastructure Use of a dependent, non-equvalent, no-nterventon control group to account for unmeasured confoundng We use arm njury More dscusson found n Olver et al. (2013)

10 ITS Model Specfcaton for Injury Counts Log-lnear negatve bnomal regresson model expressed usng Posson-Gamma mxture Y ~ log( ) ~ Gamma(, ) Posson ( ) TIME INJURY INJURY LAW 2 7 LAW 3 TIME INJURY TIME INJURY LAW 4 log( exposure), TIME LAW 5 (1) where - TIME= -17.5,, -0.5, 0.5,, INJURY s a dummy varable = 0 arm njury 1 head njury 0 pre MHL - LAW s a dummy varable = 1 post MHL - NSW populaton sze s used as exposure

11 ITS Model Specfcaton for Injury Counts Log-lnear negatve bnomal regresson model expressed usng Posson-Gamma mxture Y log( ) ~ Posson ( ) ~ Gamma(, ) TIME INJURY INJURY LAW 2 TIME INJURY LAW log( exposure), 7 LAW 3 TIME INJURY where - 6 : any dfferental changes n head njures as compared to arm njures from pre- to post MHL - 7 : any dfferences n the rate of change of head and arm njures between pre- and post MHL 4 TIME LAW 5

12 FB and EB methods Full Bayesan Method The above model can be estmated usng maxmum lkelhood approach n SAS (Walter et al., 2011) Full Bayesan (FB) method as a powerful alternatve Combnng lkelhood and pror belef to generate posteror dstrbuton of unknown parameters A non-nformatve pror dstrbuton s adopted n the absence of specfc pror nformaton Advantages 1. Allow estmaton of models wth smaller sample szes snce Bayesan methods do not depend on ther asymptotc propertes 2. Ablty to nclude pror knowledge on parameter values nto the model 3. Enables one to mplement very complex herarchcal models where the lkelhood functon s ntractable

13 FB and EB methods Dsadvantages Full Bayesan Model Estmaton 1. Posteror dstrbutons only analytcally tractable for only a small number of smple models smulaton-based Markov chan Monte Carlo (MCMC) methods 2. MCMC methods can be computatonally ntensve We wll use MCMC technques, n partcular, Gbbs samplng, by mplementng the model n the WnBUGS package The Gbbs sampler generates a sample from the jont posteror dstrbuton by teratvely samplng from each of the unvarate full condtonal dstrbutons For our model (1), we assgn the followng non-nformatve pror dstrbutons: Regresson coeffcents: Dsperson parameter: ~ Normal ~ Unform (0,1000), (0.5, 200) 0,1,..., 7

14 FB and EB methods Emprcal Bayes Method Related to the FB method n combnng current data wth pror nformaton to obtan an estmate The parameters of the pror dstrbuton are estmated from exstng data and then used assumng there s no uncertanty Extensvely used n the analyss of traffc safety data, partcularly for before-after evaluaton of road safety treatments; need to evaluate Safety Performance Functons Advantage: Easy to mplement Less computatonally costly than FB Dsadvantages: Do not fully account for all uncertantes as n FB May result n unrealstcally small standard errors

15 FB and EB methods Emprcal Bayes Method We apply a partcular EB procedure by French and Heagerty (2008) Ft a regresson model to data pror to polcy nterventon and use the model to form a trajectory of outcomes n perods after nterventon Y ~ ~ Gamma(, ) log( ) Posson ( ) 0 1TIME PRE 2INJURY 3TIMEPRE INJURY log( exposure) Contrast post-nterventon observatons wth ther expected outcomes under the absence of a polcy nterventon log( ) log( ˆ Y )

16 FB and EB methods Emprcal Bayes Method We then model ths contrast to obtan estmates for a polcy effect and assess ts sgnfcance by usng standard statstcal test 4 TIME 5 INJURY POST 6 7 TIME POST INJURY 5 4 and : baselne level and slope change for arm njures after nterventon respectvely 6 and 7 : dfferental level and slope changes n head and arm njures post-law respectvely

17 FB and EB methods Results Table 1. Negatve bnomal model estmates usng MLE, FB and EB methods Frequentst MLE Full Bayesan Emprcal Bayes Varable Intercept (β 0 or δ 0 ) TIME (β 1 or δ 1 ) INJURY (β 2 or δ 2 ) LAW (β 3 or δ 4 ) TIME INJURY (β 4 or δ 3 ) TIME LAW (β 5 or δ 5 ) INJURY LAW (β 6 or δ 6 ) TIME INJURY LAW (β 7 or δ 7 ) Estmate (95% CI) ( , ) (-0.019,0.009) (-0.128,0.272) (-0.318,0.093) (-0.022,0.016) (-0.005,0.034) (-0.618,-0.027) (-0.018,0.038) Estmate (95% CI) ( , ) (-0.020,0.009) (-0.136,0.287) (-0.330,0.106) (-0.023,0.017) (-0.006,0.036) (-0.635,-0.014) (-0.021,0.040) Estmate (95% CI) ( , ) (-0.018,0.007) (-0.111,0.255) (-0.314,0.024) (-0.021,0.014) (0.000,0.033) (-0.514,-0.064) (-0.016,0.030)

18 FB and EB methods Results Table 1. Negatve bnomal model estmates usng MLE, FB and EB methods Frequentst MLE Full Bayesan Emprcal Bayes Varable Intercept (β 0 or δ 0 ) TIME (β 1 or δ 1 ) INJURY (β 2 or δ 2 ) LAW (β 3 or δ 4 ) TIME INJURY (β 4 or δ 3 ) TIME LAW (β 5 or δ 5 ) INJURY LAW (β 6 or δ 6 ) TIME INJURY LAW (β 7 or δ 7 ) Estmate (95% CI) ( , ) (-0.019,0.009) (-0.128,0.272) (-0.318,0.093) (-0.022,0.016) (-0.005,0.034) (-0.618,-0.027) (-0.018,0.038) Estmate (95% CI) ( , ) (-0.020,0.009) (-0.136,0.287) (-0.330,0.106) (-0.023,0.017) (-0.006,0.036) (-0.635,-0.014) (-0.021,0.040) Negatve estmate of β 3 and δ 4 ndcate overall njury counts decreased after the law Estmate (95% CI) ( , ) (-0.018,0.007) (-0.111,0.255) (-0.314,0.024) (-0.021,0.014) (0.000,0.033) (-0.514,-0.064) (-0.016,0.030)

19 FB and EB methods Results Table 1. Negatve bnomal model estmates usng MLE, FB and EB methods Frequentst MLE Full Bayesan Emprcal Bayes Varable Intercept (β 0 or δ 0 ) TIME (β 1 or δ 1 ) INJURY (β 2 or δ 2 ) LAW (β 3 or δ 4 ) TIME INJURY (β 4 or δ 3 ) TIME LAW (β 5 or δ 5 ) INJURY LAW (β 6 or δ 6 ) TIME INJURY LAW (β 7 or δ 7 ) Estmate (95% CI) ( , ) (-0.019,0.009) (-0.128,0.272) (-0.318,0.093) (-0.022,0.016) (-0.005,0.034) (-0.618,-0.027) (-0.018,0.038) Estmate (95% CI) ( , ) (-0.020,0.009) (-0.136,0.287) (-0.330,0.106) (-0.023,0.017) (-0.006,0.036) (-0.635,-0.014) (-0.021,0.040) Sgnfcant and negatve estmate of β 6 and δ 6 : head njures dropped by more than arm njures post-law Estmate (95% CI) ( , ) (-0.018,0.007) (-0.111,0.255) (-0.314,0.024) (-0.021,0.014) (0.000,0.033) (-0.514,-0.064) (-0.016,0.030)

20 FB and EB methods Results Fgure 1. Head vs. arm njury counts and ftted model (FB) for 18 months pror and post MHL

21 FB and EB methods Emprcal Bayes Results Parameter estmates smlar to MLE and FB Standard errors are not drectly comparable Mean contrasts for arm njures does not sgnfcantly dffer from zero( ˆ 4 = , s.e.= 0.086, p value = 0.102) Mean contrasts for head njures s sgnfcantly dfferent from zero ( ˆ + ˆ = , s.e.= 0.086, p value = ) 4 6 Sgnfcant dfference between head and arm contrasts ( = , s.e.=0.122, p=0.019) ˆ 6

22 FB and EB methods Results Fgure 2. Head and arm njury counts wth pre-polcy estmaton (sold lne) and post-polcy predcton (dashed lne)

23 Conclusons Summary of Analyss Three estmaton methods gve smlar results Statstcally sgnfcant drop n cyclng head njures after MHL Estmated legslaton attrbutable drop n head njures are 27.6% and 26.1% usng FB and EB methods, comparable to 27.5% n the study by Walter et al. (2011) Comparng to frequentst maxmum lkelhood and EB approaches, the FB method Better accounts for uncertanty n the sample Models negatve bnomal dstrbuton as a herarchcal Posson- Gamma mxture dstrbuton; allows other dstrbutons (eg. Posson- Lognormal) to be mplemented May be computatonally costly and may have convergence ssues

24 References French, B., Heagerty, P.J. (2008) Analyss of longtudnal data to evaluate a polcy change. Statstcs n Medcne 27, Walter, S., Olver, J., Churches, T., Grzebeta, R. (2011). The mpact of compulsory cycle helmet legslaton on cyclst head njures n New South Wales, Australa. Accdent Analyss and Preventon 43, Wagner, A.K., Soumera, S.B., Zhang, F., Ross-Degnan, D. (2002). Segmented regresson analyss of nterrupted tme seres studes n medcaton use research. Journal of Clncal Pharmacy and Therapeutcs 27,

25 Thank you! 25 25