Introduction to Meta-Analysis

Size: px

Start display at page:

Download "Introduction to Meta-Analysis"

Brooke Jones
6 years ago
Views:

1 Introduction to Meta-Analysis by Michael Borenstein, Larry V. Hedges, Julian P. T Higgins, and Hannah R. Rothstein PART 2 Effect Size and Precision Summary of Chapter 3: Overview Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) Chapter 6: Effect Sizes Based on Correlations Sarah McCann

2 Chapter 3: Overview Treatment Effects and Effect Sizes Terminology: Effect size relationship between two variables e.g. difference between males and females Treatment effect only appropriate when quantifying the impact of a deliberate intervention e.g. the difference between treated and control groups Single group summary estimating the mean or risk or rate of a single population e.g. prevalence of a disease in a particular region or mean scores on a test CAMARADES: Bringing evidence to translational medicine

3 Chapter 3: Overview How to Choose an Effect Size Index 1. The effect sizes from the different studies should be comparable i.e. they measure (at least approximately) the same thing. 2. Estimates of the effect size should be computable from the information in the published reports (should not require re-analysis of raw data, unless they re known to be available). 3. Need to be able to compute variance and confidence intervals. CAMARADES: Bringing evidence to translational medicine

4 Chapter 3: Overview Summary Data Reported by Primary Studies Means and standard deviations (Chapter 4) Binary data e.g. events vs. non-events (Chapter 5) Correlation between two variables (Chapter 6) Appropriate Effect Size Raw difference in means Standardised difference in means Response ratio Risk ratio Odds ratio Risk difference Correlation coefficient

5 RAW (UNSTANDARDIZED) MEAN DIFFERENCE D The outcome is reported on a meaningful scale and all studies in the analysis use the same scale. Meta-analysis is performed directly on the raw difference in means.

6 Let µ 1 and µ 2 be the true (population) means of two groups. The population mean difference is defined as We can compute an estimate, D, of the population mean using the sample means from studies that use two independent groups, paired groups or matched designs.

7 Computing D from studies that use independent groups The variance is calculated using the sample standard deviations (S) and sample sizes (n) of the two groups.

8 Computing D from studies that use matched groups or pre-post scores Pairs of participants are matched in some way (e.g. siblings or patients at the same stage of disease) or the same subject acts as their own control (e.g. scores pretreatment are compared to scores pot-treatment)

9 STANDARDIZED MEAN DIFFERENCE, d and g The scale of measurement differs across studies and it is not meaningful to combine raw mean differences. The mean difference in each study is divided by that study s standard deviation to create an index that is comparable across studies.

10 Let µ 1 and σ 1 and µ 2 and σ 2 be the true (population) means and standard deviations of two groups. The population mean difference is defined (assuming σ 1 = σ 2 ) as: We can compute an estimate, d, of the population mean using the sample means from studies that use two independent groups, paired groups or matched designs.

11 Computing d and g from studies that use independent groups By pooling the two estimates of the standard deviation, we obtain a more accurate estimate of their common value.

12 d has a slight bias, tending to overestimate the absolute value of δ in small samples. This bias can be removed with a simple correction, resulting in an unbiased estimate sometimes called Hedges g. To convert from d to Hedges g, a correction factor called J is used: And then:

13 Computing d and g from studies that use pre-post scores or matched groups

14 Including different study designs in the same analysis A systematic review may include some studies that use independent groups and others that used matched groups. The effect size from each study can be computed using the appropriate formula and all the studies can be combined in the same analysis. Direction of effect is arbitrary, the researcher must decide on a convention and apply it to all studies, regardless of design.

15 RESPONSE RATIOS When the outcome is measured on a physical scale (e.g. length, area, mass) and is unlikely to be zero, the ratio of the two means can be used as the effect size. Computations are carried out on a log scale.

16 CAMARADES: Bringing evidence to translational medicine

17 The response ratio is computed as: The log response ratio is computed as:

18 SUMMARY POINTS The raw mean difference (D) may be used as the effect size when the outcome scale is either inherently meaningful or well known due to widespread use. This effect size can only be used when all studies in the analysis used precisely the same scale. The standardized mean difference (d or g) transforms all effect sizes to a common metric, and thus enables us to include different outcome measures in the same synthesis.

19 SUMMARY POINTS The response ratio (R) is often used in ecology. This effect size is only meaningful when the outcome has a natural zero point, but when this condition holds, it provides a unique perspective on the effect size. It is possible to compute an effect size and variance from studies that used two independent groups, from studies that used matched groups (or pre-post designs) and from studies that used clustered groups. These effect sizes may then be included in the same meta-analysis.

20 Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) CAMARADES: Bringing evidence to translational medicine

21 RISK RATIO The ratio of two risks. Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) Risk of death in treated group is 5/100 and the risk of death in the control group is 10/100, so the ratio of the two risks is 0.50.

22 Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) CAMARADES: Bringing evidence to translational medicine

23 ODDS RATIO The ratio of two odds. Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) Odds of death in the treated group would be 5/95 or and the odds of death in the control group would be 10/90 or , so the ratio of the two odds would be / or

24 Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) Why use log transformations? The log transformation is needed to maintain symmetry in the analysis. Assume that one study reports that the risk is twice as high in Group A while another reports that it is twice as high in Group B. Assuming equal weights, these studies should balance each other, with a combined effect showing equal risks (a risk ratio of 1.0). However, on the ratio scale these correspond to risk ratios of 0.50 and 2.00, which would yield a mean of By working with log values we can avoid this problem. In log units the two estimates are and , which yield a mean of We convert this back to a risk ratio of 1.00, which is the correct value for this data. CAMARADES: Bringing evidence to translational medicine

25 RISK DIFFERENCE Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) The difference between two risks. Risk in the treated group is 0.05 and the risk in the control group is 0.10, so the risk difference is No log transformation.

26 Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) SUMMARY POINTS We can compute the risk of an event (such as the risk of death) in each group (for example, treated versus control). The ratio of these risks then serves as an effect size (the risk ratio). The difference in these risks can also serve as an effect size (the risk difference). We can compute the odds of an event (such as ratio of dying to living) in each group (for example, treated versus control). The ratio of these odds then serves as the odds ratio. CAMARADES: Bringing evidence to translational medicine

27 Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) SUMMARY POINTS To work with the risk ratio or odds ratio we transform all values to log values, perform the analyses, and then convert the results back to ratio values for presentation. To work with the risk difference we work with the raw values. When choosing an effect size index, consider: The risk and odds ratios are relative measures relatively insensitive to differences in baseline events. The risk difference is an absolute measure very sensitive to the baseline risk. CAMARADES: Bringing evidence to translational medicine

28 Chapter 6: Effect Sizes Based on Correlations Computing r Studies report a correlation between two continuous variables. The correlation coefficient can serve as the effect size index. Standardised to take into account the different metrics in original scales.

29 Chapter 6: Effect Sizes Based on Correlations CAMARADES: Bringing evidence to translational medicine

* Point estimate for P is: x n

* Point estimate for P is: x n Estimation and Confidence Interval Estimation and Confidence Interval: Single Mean: To find the confidence intervals for a single mean: 1- X ± ( Z 1 σ n σ known S - X ± (t 1,n 1 n σ unknown Estimation