Is the meta-analysis of correlation coefficients accurate when population correlations vary?

Transcription

1 Is the meta-analysis of correlation coefficients accurate when population correlations vary? Article (Unspecified) Field, A. P. (2005) Is the meta-analysis of correlation coefficients accurate when population correlations vary? Psychological Methods, 10 (4). pp ISSN X This version is available from Sussex Research Online: This document is made available in accordance with publisher policies and may differ from the published version or from the version of record. If you wish to cite this item you are advised to consult the publisher s version. Please see the URL above for details on accessing the published version. Copyright and reuse: Sussex Research Online is a digital repository of the research output of the University. Copyright and all moral rights to the version of the paper presented here belong to the individual author(s) and/or other copyright owners. To the extent reasonable and practicable, the material made available in SRO has been checked for eligibility before being made available. Copies of full text items generally can be reproduced, displayed or performed and given to third parties in any format or medium for personal research or study, educational, or not-for-profit purposes without prior permission or charge, provided that the authors, title and full bibliographic details are credited, a hyperlink and/or URL is given for the original metadata page and the content is not changed in any way.

2 !" #$ % &'#(

3 Abstract * + + $, -!. / $ 0 0! $ 1 %1' $ ' $0!,-. % $ ' 0 $ 2,)- $ %1 '00 $ 1 0 ' + $0 ',3- & 0! )

4 / $ 0 $ $ $ 04 $ 0$, 5' 678-! 04 $0, & '0 $ $ 0 0 -'",)99')993'0- :0 ;,6<8-' *:,67-' 0,6<7-'%,6<<-' '% >:,67)-' %,669-0 : 0! $ ' $ 0,!!' " ' )993'02? %' )999-," ' )99)2 >'?%'662%+ ')99A-!. $ $ 0 $! Methods of Meta-Analysis ' + 0 +! $ + $0,!!' '2 1 + &'22: 2: -' ' $ ' 0 $, ' 662 5' 678-!. : + 0 3

5 : + $ '$ ' $ +!. $ 0 +, $ + -'! $0 $ 0! 0 +,? *:' 67- $,? %' 6692)99A-! Fixed- and Random-Effects Methods * $," ')993'02?%')999-! ' &, '66)2? $ '6672?%')999-!. & & ' +,?%')999-!. $ + $2' + : $! + ' '0 0 $ 0 BC 1,@ : '6682 '66)-! 5 $ ' : A

6 !* ' 0 : $ $0," ')9932? %' 6690' )9992 D ' 66)2 *0? '66)- $ &,@:?'66-!" 0 0 $, $0-0 66<)99)!. 0 %, / /,-' 0 -!. $ 0 $ 99!3'$ 90 $! $ ' + $0 ' / $ 9!3 $ /! 5 ' $,667- : & &, - 0, -! : + 0," ')9932?%')999-!. 0 0 & 0 0 & /!E $ ' &!" & '

7 %,)999- ) & & 4 $ 4, -!. / ' %,)999-' $ +. 0 F)7G!'",)993-. G 0 A379G!%' ) 0 %,) < : $ $ &, " ')993-! * + + : 0 & $0 ' $0 +,? *:' 67-!. + /$ + : 0, & 0 -!H$ $ :,!!'< ) 0? %' )999' & -' $ 0 0 $ 0 $ $ +,?')992%: '?*'67820 " ')99' -! / ' 0 + & '' $0! 8

8 Comparing Methods % : ;,6<8- $ $ 0 ' %,669')99A- $!,?*:'672 ' 66)2? $ ' 667- $ $ 0 & 0 + ' %, ' %?>:'67)-0,? %')99A2 D ' 66)2%? ' 666- &,? %' 669-!. $ 0!. : 0 + & '! Hedges and Colleagues Method ' $," 1-0 $!" 1,6)- $ /,- + ( ) ),,- & 0 ρ'$ I, 3-' 0!. 0: <

9 () ) () ) +.,)-. + $ 0 $ $," $ + ' ' -B H/,3- $ ' 0,? *:'67'!)3-, 3- ( 3).,3-.$ / 0 10 $, /,3--' 0 $ '!.$ H/,A-'/ 0 F +,?*:' /8'!)3- ( 3)( ). $ ' $ 0 0 $ $!. 0 $ 0τ ), Jτ ) - $!. $,0? $ '667' /)- ).,A- 7

10 K K K,,- K ( ) A-,0? $ '667' / ( ) J ) K + τ 3.,8-. 0 $ 0 $, " ' )9992? $ ' 6672 *$ ' 6672.:: ' %L +? % '666-' $ ' $,667' /9- H/,<--'0, / /,A--'' '' ( ) ) J τ,,<- '' ( ) ),,7- '0 M F3' ) ( 3 ( 3) ( 3).,6-0 $ ' Jτ ) ' $ $ +,0 $ 0 0 $ -!. ) Jτ '0 /,8- ' 6

11 /,- $!.$ $ 0: /,)-0 0!. $ $ $ / $!@ /,8-,? $ '667'!A63- K ( ) K.,9- *:,67- $ $ + ' α ' $ 0,!68 6G $-!. 0 0: $ + /,-0 0!68!" +!" K ( ) K ( ) K!68, K!68.,-. $ 0: /,)-0 0! Hunter and Schmidt Method. 0! 0 1 ' 9

12 ! %,)99A'!7- + ' ' ' + '.,)- %,)99A- $ 0 $ 2 ' $ $ $ 0!. $ / $ /!H/,3-' %,)99A'!7?76-' Jσ ) ( ) ).,3-. $ $ '' $ + '# ',?%')99A'!77- ) ( ) ) ) J σ #.,A- $ 0 0 $ $,?%')99A'!77-

13 ) ) ) J σ ρ J σ J σ.,- %,?%')99A?@:')99) - 0 $!. $ 0 : $, /,)--0 / $ /,- 0 α,!686g $- 0 $ 0 $ # N ) ρ +!68 J σ, ) ρ!68 J σ.,8- $ /, 0 $ !. 0 $ $,$ /, '': / Differences between Methods σ!" +!68 ), σ!"!68 ).,<- : 0 0 $ %, 0 0$ -' ' 0 1,- ',)-, $ -! )

14 . $ 0 0! " & '%$ E,67<- 0 " 0 ' ' 0!%0, "!' )9,0 -! $ ' & )9! %+,)99A- & $ 0 $ $0 0!. $! $,667-$ %1 0 $ 2 ' 0 $ ' 0, / )- %,? $ '667-! $ ' 0 $ 0 $ +, $ $ ' + -': % ' $ 1,667-0 $! / ' $ 3

15 0 B 0,? $ '667-! % $ $!>!,66- *:,& -' 0 % 0 : 0 ' ' 0 + 0!. $ + ' 0 0 % $$ >! 0! ' %,666- >! ' ' ' 0 *: 0! ",)99- >! *:1,0$ - 0,$ -' 0 ' % 0 1 $! ",)99-0 % *:,& - 0 & ' $0!. 0 ' $ + ' + $!" A

16 % $! $ ' % 1. ' 0 0 $, % 0 : & $ % $!%+,)99A- & $ 0 " 1 C1 ' '0 0! $ ' %+ 0 & $ 1, & 0 0 $ '667- The Current Study $ / $ '!" $00 + & 39, $ $! " ' $ 0!%+,)99A-: $

17 ! &$ $ 1,667-!' & " %+ 1,)99A- : 0,- $ ',3-$ 0 0 $! 0,- $ 0 0 $00 +! $ +!,)- %1 + 0 : $00 +,? $ ' 667-' $ $00 $00 +!,3-0 0 ' % $ C1,$ ' 0 % -!,A- + % 0, - $,0 ' 00 $ : :-!$ ' 1 '0 8

18 " 1'0 $ 0 :! $ ''%+,)99A- ' " 1 : 0 $ %!,- % $ $ 0 $ %! Monte Carlo Simulations General Method. 0 : $' 0 0 :!. $ 0 :$00 +, $ $$ -!. 0 : $ $ '! $ + :!* 0, 0-0 : ' /, $ 6G $ -!. 0 <

19 ' $ + ' $ $ :! ' 0 : $ 0 0 6G$!' $ $ 0 0 )!G $ 99'999! The Superpopulation ;#%%A!9!00, -! 0 ' & 0ρ 0 $ $!., " 3-!. +, ρ - 0 9, ρ M!-', ρ M!3-', ρ M!-' +, ρ M!7-0 1,677'66)-! $ '$ 0 &, 0 -!. $ '!. 0, /,--' 0! %' +,9'!'!3'!!7-' 0 $,9!9A'9!97'9!89!3)- $! 7

20 00:, /,--,0 -! )!. 0%'0 $ 0 $!.,9'!'!3'!!7- $,9!9A'9!97'9!89!3)- 0 ' 0! 00$ ' 4! * 0 ' C1!. $ & 0 0,66<- 0 0 $0 : & &!. & $ '%' 0, 0 0 -$ & 0!. 0 $0! " 0 : $ $ $ 0!. $ + $ 99'999 ' $, -! 6

21 The Mean and Standard Deviation of Population Correlations 5 0 %', ρ - 0 9, ρ M!-', ρ M!3-', ρ M!-', ρ M!7-!. $ $ 0 9!9A'9!97'9!8'9!3)!. 0 $ $ 66<)99), " -!. $ $0,9!9A- / $0,9!9A9!8- $0,9!3)-!. $ $ & 0,!!'9!3)-! 5 0 ' $ $ ' $ 0: 0, 0%-! $0 H& 9999!. $ 0.0! Number of Studies. 0 '' 0$ 89'8$ '9')9'A9'7989!",)99-$ 0 39' 0! )9

22 Average Sample Size. $ + $! + / ' 00 $!% + : 0&,)9'A9'7989- $ /!0: + $ $ ' : 0 + $, 0 0 -! +," ' )993' )99) -! $ + ' ' + ' 0!. + ' M!7 & 9'9') $, ' 677-!'$ M)9'A9'7989!. + A 0 1 / + +! Design ', 9'!'!3'!'!7-OA, $ 9!9A'9!97' 9!8'9!3)-OA,$ + )9'A9'79'89-O8,0 ' 9')9'A9'79'89-'!" $ A79099'999 )

23 ,99 0 ' 66<-!H 99'999,A7'999'999 -! $ ' 0 $ 0! '$ % $ &$ 0 $! H$ & 0,!!'!6-' & 0 0 ' &$ % 0 I) [!6!6 ( )]! 6. '$ 0$!6 $!@$!6 0 0,!!'" ')99- ' & '!7 $9!3),M,!6!7-I9!3)M9!3-3<!73G% 0 0$!6! G% 0$!62' $ $!3,&0 %0$!6M3!9AG-'0 $$ 9!8! $ $ 9!9A &!7,0 9!8)G%0$!6-! ))

24 Results Simulation 1: Comparing the Methods. $ $ $, 0 0$ -!" ) $ 0 0 6G$ 20 & $,$.0-0! 1,- $!93, & $0!3 $,9!3)-B $!9!. ",)99-0$!. %,%- $ & $0 F!99, & $!79-! $ 1, %1 0 $ $ ' $ 0!5 + % '!'!3'!'!7'!999!993'!999!99<'!999!99'!99!99 $! )3

25 " ) $ 6G 0 $ +!. $ 0! ; ' $ 0 $ 0 00 $!" & ' '$ ' & '!3P!3 $ &,σ ρ M9!3)- 0, M-!"0 ' $,σ ρ M9!89!3)-0 0 $ 0A9,σ ρ M9!8-79,σ ρ M9!3)-! $ ' $,σ ρ M9!9A9!97- $ 0 )9!5 + ': $! $ ' A9 0, ρ M!!3-' $ '& % ' /9!8!, ρ M!-' $,σ ρ M9!97-9)9 / :! $! $ ' $,σ ρ M9!9A- $ $!. 0 $ + $ + $,σ ρ M 9!3)- )A

26 $ $ 0 + '0 $ + ' $0 +! Simulation 2: Does the Shape of the Superpopulation Influence the Results? $ ' 0 0 : 0 & ' !*, /,--' 0!" % 0:, /,--0! $ % ' : 0,0 0 -' % '!%) 0 0 ' 0! ' $, 0 ) 0$ -'00 4 $ 0!. 0 " 3 & ' / 0, 0 $ ' $ $ 0 0: - $!. 0, ρ M! - )

27 $ ' 0 0 :' $ ' 0! $ 9!8 ' 0 $ 0,4 -!.0 ) $ : : 0 &! " 3 0, -' 0 0! * / 0 $,!!' 0 9'!'!3'!!7-!. ' $ % $!%'?#,6<8- / $ 0!" &, 0-0$ 0 /,7-,'- D00,-'','- $ 0,- D0'' ρ σ ρ $ $ 00!.' % 0 /,7-! )8

28 ρ ( ρ σ ρ σ ρ + ρ ρ σ ρ,7- " A $ 0 0 6G$ & $! " ' ' ' $ '0 $!,0-9'!'!3'!!7 $!999!99)'!999!97'!99!9)'!99)!93'!99A!9) $!. % $!,0-9'!'!3'!!7 $!999!99)'!999 F!993'!999 F!99<' F!99 F!9'!999F!997 $! Confidence Intervals from Simulations 1 and $ /, $.0 -!.!7A!68)!5 $,σ ρ M9!9A- $ + 0!73!68!. $4 0!6A!68,!9!6-' & 0 0,M89-!3 )<

29 !' A9 0!79! $ 9!97' $!5, ρ M9!-' 0!6A!68' $ ' 0!6A!68 $ + A9 79 0!5 $ 9!8' $ 6A68G, ρ M9!- A9 0! 6AG $! "' $,σ ρ M9!3)-' 6AG $ & + A9 0!.0 A $ / 0 %1,0.0 -! 0 %$ 0 $' $,$ $-!D $ ' %,!739!6A6-!. 0$!6A 79!3!5!3 '!6 0 2 $ '! A9 '!6!. )7

30 !6 $!.0,-8,%-.0 3A0 0 $, /,7--! " ' 0!. 0! '79 $ 9!97!5 $ 9!8' 0!3! 5 $ 9!3)' 0! 79!" % 0 $ $ 0 $! ', 0 $ -' 0 $ %!.'$ %1 $!$ 2 $ ' $0 + $ %! )6

31 Simulations 3 and 4: The Effect of the Weights * 0 % 0! " ) A '3A 0 0 )'0!" K /,-,9-'!" % ' K /,)-,3-!.,- $ B $$! " $ 0 0 6G$ & $ +!. $, " )- $!.0 < $ /! % 0 &!6,.0 < 3-! ' & 0 0 $ 0 +!.0 % 0 ' ' /, ' 0 $!. 0 0 $0, ) Jτ - 39

32 ' +! ;$ % $' & / $ % $!.0 7! 0.0 A' % $: $ 0! $ ' 0 %) & $ ' $ ' $, 0 $ -' % 0 K $!! Discussion Estimates of the True Correlation. & $ 0!E & :," ')992?@:')99)2%+ ')99A-$ 0 0 ' $ 0 $ %!%! "' 0 0! $ ' $ $ 3

33 $,σ ρ 9! ' $,σ ρ 9!8- $ 0! )' % 0 $0! ' 0$0! 0' $ 0 : $ 9!80$!' $ $00!. % '0! 3' 0 0 0! $ ' 0 + $, -!. $ % 0 $ $! A' % $ 0 $ +! % $ $, 1 -$ $0! $ ' 0 0 3)

34 0 $ $ $ + 0!. 0 & 0, 0$ - %+ 1,)99A- ", $ -! "' $ $ 0! $ ' % ' 0 % 0!. $ 0$ 0 $ '0, B 0 - % $ 0 $!. 0 ' % $,σ ρ 9!97-! $ '1 $, ρ Q!3- $,σ ρ Q9!8-', ρ Q!- $ &$,σ ρ M9!3)-!. % $,!9 $ -! 33

35 . $0 + $0,σ ρ M9!3)-!@:,)99)-0 $ $ $0 C 1! $ '" $0+ '0 $ / ' $ $ B 0 $ &! 95% Confidence Intervals $ 6G $ ' $ %!$ %1 $!$ '0 0,M- ' + $0'0!. : 0 0 & 0 $ $, ' & '" -! ' $0 % 0 $ '0 0 ' + $0 ' $ % : $ $!. % 0 0 3A

36 ' + $0 ' 0'!. % 0!. 0 $ 0 $, /3- + k n i i 1, -' /$ 0 0 $ +, kn -! ' 0 $ 0 0 $, & ' $ F -! ' 0 0 $ 0 + ' ' /30,F- N k N!. 0 $ 0 $ ' ' $, /<-!.0 $ & $%,-!. 0 ' 0 & 0!' 0 0, + $ $ $-! 3

37 Population Confidence Intervals. $, 0 6G$ - 0 ' 0 $ $ 0 $ $,ρq!- 0 ±! $ )9,& % R9!8-'A9,& % S9!8-! $,ρm!!3- $ 0 ±! $ A9,& % R9!8-79,& % S9!8- '0 $0A9 0 / %!5 $ &,& % Q9!8-' $ 0A979,-A9,% -! $ + :, + $-' 0 $! Comparisons with Previous ",)99- $ $!@:,)99)-$ 9!98, $ 38

38 $ -! ",)99- $ $ 9!)9! ' &$!93, 0 $ -!9), 0 -!. $ 0 % $ & & ",)99-0!" & ' & % M9!8' 0 39' 0 $ ' ",)99-!'!3'!!AA'!A38'!<98 $!. $!9AA'!38!)98 $! ' & % M 9!8' 0 A9' $ ' $!'!3'!',$ $ + -!99A'!93'!9)! # 0 0$ ' ",)99-!'!3'!' %!967'!)63'!A<7!. $ $!99)'!99<'!9)) $! ' 0!99'!993'!99A!.'",)99- $ % B@:,)99)-!. 2 $ '0 $ ",)99-' & : & 0 $! ' 0 3<

39 P 4,!!' $ 0 P0 -!",)99-$ 2' &$,9!666-!@:,)99)- ",)99- & $ ±!6A 9!6A,%!' ') ')99A-!. 0 $ & 0' 0 $!. 0&$ 0 $ 0 4!. %!. $ &!'0 ",)99- & 0 0 & $ ' & %, $ -!. 0 0'0: ' %+ 10, $ 0 $ + 0 %! % $ $! 37

40 @:1,)99)- $ $ + : ) * 0 ' % $! $ ' $,667- $ $ $ / $!. : $ $0,'$ ' %$ $ % '00 0 ' + $0 ' $ % 0 : 0! $ ' $ %6G $,$ - 6AG $! Conclusions & / $ 5 ' '0!,ρ Q!3- $,σ ρ Q 9!8-', ρ Q!- $ &$,σ ρ M9!3)- $ ' $,!9)0$ $ -!. % 0, 36

41 !90 $ -! 6G $' % $ $ 6AG ',73G 0 -! 6G $1 ' '6G + ' $ 0, 68!)G -, 7!AG88G -!' : ' $, $ -' $ +! A9

42 0$ : >!,668-!. 0 E! N0:,H!-' ", '@ >:#$ '383373! ' >!,677-! ) *,) H!-! D T:! '>!,66)-!! '--'F6! " '!!,)99-! &!.'/' 8F7! " '!!,)993-!. 0 &! ''<<68! " '!!,)9930-! 0 ('-/'8A)8A! " '!!,)99-!0* 12!N%! " '!!,6)-!* 00!.'-'3F3)! " 'N!,)999-!H $!3 * '45)! ;#%% 5 A!9 U V!,67AF)99)-! ' 5 % '! ;' ;!,6<8-! ' '! 6 '4'3F7! '%!!'?@:'!.!,)99)-!!3 7 '89'3<<F376! 'N!!,66)-!!3 '-9')<6)68! 'N!!'?*:'!,67-! $$ :!*'"N! A

43 'N!!?'.!E!,)99-!.!.'/')93)<! 'N!!'? $ '>!N!,667-!"&!.';'A789A! '>!H!'?%'"!N!,669-!.$ :,! )!D 0:'%! '>!H!'?%'"!N!,6690-!E+$0.!3 7 '94'33AF3A6! '>!H!'?%'"!N!,)999-!"& $! $ :! " 3 7$'8')<F)6)! '>!H!'?%'"!N!,)99A-!.$ :,! ) 12!D 0:'%! ' >! H!'%'"! N!'? >:';!@!,67)-!. :,!$!@ $ '%! >'@!.!' '@!'?%'H!,66-! 4!3 7 '8<'6A98! '! W!,66<-!.! $,% #$ X$ %% '!9<8-!. *:'%! D,66)-!!$) $,!5'E!!D! *0'!;!'? '>!,66)-! $ $ +! 3 7 '99'))! *$ '!!,667-! & &, - $0!.';'3A3<6! '!,66-!. :, $ -!D 0 :'%! A)

44 '!'?0'E!,6<7-! &. 3A! * ';'3<<A! %: '!!''!!'?*'>!!,678-!* : $0 $.!3 7 '9-'39)F 39! %'"!N!'? '>!H!,6<<-!E $ 0 $ +!3 7 '/')6A9! %'"! N!'? ' >! H!,666-! $ >' ' %,66-! 3 7 '8+,-'AAA7! %'"!N!' '>!H!'?#'!5!,6<8-!% $!3 7 '/-'A<3A7! %+ '!,)99A-!. :, $! 0 '?0! %$ ' D!!'? E' 5!!,67<-! $ % " 1W0 3 7 '9'A8A7! %0 '!>!,677-!$ +!3 7 '9;'687!.:: '@!' % +'!'?% ' E!,666-! H$!7$ 3 $'-4<')98)! 5' "!!,678-!. :! % $ /$ '9<F98!D 0:'%! A3

45 Author note 0!" ' E '#$ % &'" '@'H% &'@D 6X!H 0 Y &!!:! % N :! % $ $ $ $ $!'":%1 $ $ ' 2 0 & ' $!., - $ $ 'M'0! AA

46 .0 %E 0 % 0. %E,"+- % ρ 9!9A 9!97 9!8 9!3) E0!9!9999!9999!9999!999)!!966!96699!96<)!96)!3!)696!)699!)7A7!)8<6!!A8!A67!A3!7!883A %EE0!9 9!9366 9!9<6 9!8 9!)63)! 9!936 9!9<7< 9!A< 9!)69!3 9!9388 9!9<)6 9!AA9 9!)<A6! 9!93 9!98)6 9!)3!7 9!9))A A

47 .0 ) : : 0,K Z!99' %: M!9' > & % ρ %: ( %: (!9 9!9) 9!996 9!99 9!9)! 9!999 9!9) 9!93) 9!98 9!9A!3 9!997 9!98) 9!98<K 9!997! 9!99) 9!9)A 9!9K 9!9)3!7 9!99A 9!9< 9!8<K 9!9)!9 9!99A 9!9A3 9!99) 9!9A< 9!97! 9!93 9!9A< 9!9)6 9!9<9K!3 9!93 9!933 9!AK 9!93)! 9!99) 9!996 9!)AK 9!9)<!9 9!9) 9!993 9!99 9!7AK! 9!99 9!99) 9!968K 9!6K 9!8!3 9!99< 9!99 9!)A<K 9!987! 9!9)A 9!9<8K 9!A)K 9!)<K!9 9!99 9!38K 9!99 9!)K! 9!9K 9!A)K 9!33K 9!A87K 9!3)!3 9!8K 9!)89K 9!377K 9!)6)K A8

48 .0 3 $0 0 $,. H M - $ % %+. M9. M!. M!3. M!. M!88 D0 % )9 A )9 A )9 A )9 A )9 A !68)!67!6!6A!68)!66!6A!6A8!68)!67!6A!6A8!689!66!6!6AA!67!6<!6A!6A 9!66!6<!6A!6A!66!67!6A!6A!66!68!6A!6A<!68!6<!6)!6A<!6A!6!6!6A )9!67!68!6)!6A<!6<!68!6)!6A8!6!6A!63!6A<!6)!63!69!6A!6A8!6A6!6A6!6AA A9!6<!63!69!6A<!68!6!69!6A<!6)!6!6A6!6A8!6A3!6A7!6A<!6A!6)7!636!6A3!6A9 79!6A!63!69!6A<!6A!63!6A6!6A<!6AA!6A8!6A8!6A!6)<!637!6A)!6A)!768!6))!63)!63< 89!63!6!69!6A<!6!6A6!6A7!6A<!639!636!6A3!6AA!768!6))!633!63<!733!776!6)!6)A σ ρ M9!97 σ ρ 0.04!6!6A<!633!6A!6!6A8!63)!6A!6A!6A<!633!6A!63!6AA!63)!6A 9!6!6A8!638!6)8!6A!6A<!638!6)!6)!6A8!638!6)8!69!6AA!63!6)8 )9!6A!6A7!6A)!638!6)!6A<!6A9!638!6A6!6AA!637!63A!6AA!6A)!638!63A A9!6!6A7!6A3!6AA!69!6A<!6A3!6A3!6A!6A3!6A9!6A!63A!63<!63A!638 79!69!6A<!6A8!6A!6A6!6A8!6A8!6A!638!637!6A9!6A)!68!6)A!6)6!633 89!6)!6A7!6A<!6A7!6A8!6A8!6A8!6A<!6))!63!63!637!7<8!69)!6!6) σ ρ M9!8!638!6!76!773!63!6!768!77A!63A!63!76!77!63!6)!76A!77 9!637!6)8!66!6<!63<!6)<!66!67!638!6)3!6<!67!6)6!66!6)!6 )9!6A9!63!63!63A!636!63!63A!63A!633!63)!639!639!6)!6))!6))!6)3 A9!6A3!6A)!6A3!6A3!6A)!6A!6A!6A)!63!63)!633!633!69!68!67!67 79!6A8!6A<!6A!6A<!6AA!6AA!6A!6A!6)9!6)!6)8!6)7!7<6!769!768!766 89!6A7!6A7!6A7!6A6!6A)!6A3!6A)!6A!76<!69<!69!6!7A!73<!7A!7 σ ρ M9!3)!76<!778!7<6!7<6!768!773!779!7<!76A!7<7!7<A!7<3 9!6)!67!66!67!67!68!6<!67!69!697!69<!696 )9!638!63!63!637!633!63A!633!638!6<!6)9!66!66 A9!6A)!6A)!6A)!6A3!637!636!636!6A!69!6)!63!6A 79!6A8!6A<!6A<!6A<!6A9!636!6A9!6A9!77!77A!77<!77 89!6A6!6A7!6A7!6A6!63A!63A!63!63!7))!7)!7)!7)< A<

49 .0 A $0 % 0 $,. H M - $ % %+. M9. M!. M!3. M!. M!88 D0 % )9 A )9 A )9 A )9 A )9 A σ ρ M9!9A!7A3!7A8!7AA!7A3!7A!7A8!7AA!7AA!7A!7A!7A!7A8!7A3!7A3!7A!7A9!7A)!7A3!7A!7A 9!693!693!693!699!69A!69!693!69!69A!69A!69A!69)!693!69A!693!693!69!693!69!69 )9!6)6!6)7!6)<!6)<!6)7!6)6!6)<!6)7!6)7!6)7!6)7!6)6!6)7!6)7!6)<!6)<!6)6!6)7!6)<!6)7 A9!6A!637!637!637!6A!6A!637!636!637!637!637!636!63!63<!636!637!63)!63A!63<!63< 79!6A!6A!6AA!6A3!6A!6A8!6A3!6A3!636!6A!6A)!6A3!6)6!637!6A!6A!6)!6)<!638!6A9 89!6A<!6A8!6A<!6A!6A<!6A!6A8!6A!63)!636!6A)!6AA!697!6)7!637!6A)!78!698!6)7!636 σ ρ M9!97!7A<!7AA!7A)!7A!7AA!7A)!7A)!7A9!7A)!7A!7A!7A!7A)!7A!7A)!7A 9!69A!693!699!766!693!693!699!766!69)!69)!69)!69!69!69)!69)!699 )9!6)6!6)7!6)7!6)!6)7!6)7!6)8!6)!6)<!6)8!6)!6)A!6)8!6)7!6)8!6)< A9!636!6A9!638!63<!637!636!63<!63<!637!637!63<!638!638!637!638!638 79!6AA!6A3!6A)!6A9!6AA!6A3!6A)!6A!6A9!6A!6A)!6A)!63!63<!6A9!6A) 89!6A6!6A8!6AA!6AA!6A!6A8!6AA!6AA!633!6A9!6A3!6A3!6!639!637!6A) σ ρ M9!8!7A!7A9!73<!73!7A9!7A9!737!738!7A9!736!737!737!737!73<!738!73A 9!69)!766!768!76<!69!766!76<!767!69!766!76<!767!699!76<!76!76 )9!6)8!6)!6)3!6))!6)!6)!6)3!6)!6)!6)8!6)3!6)3!6)<!6)A!6))!6) A9!638!638!63A!63A!63<!63!63A!63A!63<!638!63!63A!63!638!63!63A 79!6A)!6A)!636!636!6A)!6A)!6A9!6A9!636!6A!637!6A9!63A!636!636!6A9 89!6A8!6AA!6A)!6A3!6AA!6A3!6A!6A)!63<!6A!6A!6A)!6)!63<!6A9!6A) σ ρ M9!3)!738!738!73!73!73!73A!738!73!73A!739!739!73 9!767!76!76<!76!768!76!76!768!76!76A!76!76 )9!6)A!6))!6)!6))!6)3!6)3!6))!6)A!6))!6)!6)9!66 A9!63A!63A!63)!633!63A!63A!63A!633!63!63!633!63) 79!6A9!636!636!637!6A9!636!636!636!636!636!63<!637 89!6A3!6A)!636!6A!6A!6A!6A9!6A!6A!6A)!6A9!6A9 A7

50 .0 $0 0 $ D0 % $ % %+. M9. M!. M!3 )9 A )9 A )9 A )9 A )9 A !68)!67!6A!6A<!68)!66!63!6A!68!67!6)!6A3!66!6A!6A7!63A!6A!639!6!76A 9!689!6<!63!6A8!689!67!63!6A8!67!68!6)!6AA!6A!6)!6A<!63<!63)!6)!66!6< )9!66!6!6)!6A<!6<!68!6)!6A8!68!6A!6A6!6AA!69!6A6!6AA!6A!63!6<!6<!6) A9!6<!6!6!6A7!68!63!69!6A<!69!6!6A7!6AA!637!6A3!6A)!636!7<!76!69A!6 79!6A!63!69!6A<!6A!6!69!6A<!6A3!6A!6A!6AA!67!63!63!637!79!7A3!78)!7<A 89!6A!6)!6A6!6A<!69!6A6!6A7!6A<!6)6!636!6A!6A)!77)!696!6)3!639!8A6!<3)!<<9!<69 σ ρ 0.08 σ ρ 0.04!6!6A8!633!6A!6!6A<!639!6A!63!6A!6)7!69!6A<!63A!68!767 9!6A!6A<!63<!6)8!6A!6A<!638!6)8!6!6A3!633!6)A!6A3!63!6)!6) )9!63!6A7!636!63<!63!6A!6A!63!6A<!6A)!63<!638!63A!639!6)6!639 A9!6)!6A8!6A3!6A)!69!6A8!6A3!6A)!6A3!6A9!636!6A9!6))!6)<!6)7!63) 79!6)!6A<!6A8!6A<!6A7!6A<!6AA!6A8!633!638!636!6A!76A!696!68!6)9 89!69!6A<!6A7!6A<!6A<!6A8!6A<!6A<!68!6)8!63!63!7A9!7<!77!76A σ ρ 0.16!63A!6A!76<!77A!633!6)!76A!77A!6)7!69!763!77!6!767!777!773 9!63<!6)8!6))!66!637!6)8!6)9!6)9!639!6)3!66!6<!6<!6A!6A!68 )9!6A!638!638!63!6A9!63A!63!638!639!6)6!63!63!69!6A!66!6)9 A9!6A3!6A)!6A3!6A3!6A)!6A!6A)!6A3!6)8!6)6!63!63)!779!776!768!699 79!6A<!6A<!6A8!6A<!6A)!6AA!6A!6A!6!67!66!6))!79<!7)!73!73< 89!6A7!6A7!6A7!6A6!6A!6A3!6A3!6AA!7<8!777!777!76!889!878!86A!<99 σ ρ 0.32!76<!769!77!77A!766!777!773!773!76!77<!778!77 9!6)!6)9!66!6))!66!67!6)!6)!6A!6!6<!6< )9!63!638!638!638!633!63A!63!63A!68!67!6)9!6)9 A9!6A3!6AA!6A!6AA!637!637!636!636!76)!763!768!76< 79!6A7!6A8!6A<!6A<!633!63A!63A!638!7)3!7)8!7)6!73 89!6A7!6A6!6A7!6A7!6))!6)9!6))!6))!879!87!877!876. M!. M!7 A6

51 .0 8 $0 % 0 $ D0 $ % +. M9. M!. M!3 )9 A )9 A )9 A )9 A )9 A !7A8!7AA!7A3!7A8!7A<!7A8!7A3!7A3!7A!7A3!7A3!7A3!7A3!7AA!7A3!7A!737!7A!737!73< 9!693!69A!693!693!69!693!69!69!69A!69)!69A!69!69)!69A!69)!69)!69)!699!767!766 )9!6)6!6)<!6)7!6)<!6)7!6)6!6)7!6)<!6)7!6)6!6)7!6)8!6)6!6)6!6)7!6)7!6)8!6)8!6)A!6) A9!6A9!6A9!636!6A9!6A!636!637!636!637!636!636!637!63!63<!637!63<!6)6!633!63!63 79!6A!6A!6AA!6A3!6A!6AA!6AA!6AA!636!6A!6A)!6A3!6)8!638!6A9!6A)!697!6)7!63<!636 89!6A7!6A7!6A8!6A!6A8!6A!6A8!6A!639!6A9!6A3!6AA!69)!6)8!638!6A3!7A7!69<!6)6!637 σ ρ 0.08!7A8!7A3!7A)!7A9!7A!737!737!738!7AA!7A)!7A3!7A)!736!7A)!737!73< 9!69A!693!69)!699!69!69!767!766!693!69!69!699!69)!69!699!699 )9!6)6!6)6!6)!6)8!6)8!6)A!6)3!6)A!6)7!6)8!6)8!6)<!6)8!6)!6)A!6)A A9!636!63<!63<!638!63<!63<!63A!63!637!638!63<!63!63!63<!638!638 79!6A8!6A3!6A)!6A)!6A)!6A)!6A9!636!63<!6A!6A)!6A!6)6!638!636!6A9 89!6A<!6A!6A!6A)!6AA!6A3!6A!6A)!63)!636!6A!6A3!698!639!63<!6A) σ ρ 0.16 σ ρ 0.04!7AA!7A9!736!73<!7A!737!737!738!7A9!7A9!737!737!73<!738!73<!738 9!699!699!699!767!69!69!767!766!766!766!766!76<!766!767!768!768 )9!6)<!6)A!6)A!6)!6)8!6)A!6)3!6)A!6)!6)A!6)3!6)!6)!6))!6)3!6)9 A9!638!638!63!63!63<!63<!63A!63!638!63!63!63A!63!63!63A!63A 79!6A3!6A!6A9!6A9!6A)!6A)!6A9!636!636!636!636!637!63!637!636!636 89!6A!6AA!6A)!6A3!6AA!6A3!6A!6A)!63<!6A!6A9!6A!6)A!638!636!6A σ ρ 0.32!73!737!738!738!737!738!73A!73!738!73!738!73 9!76<!76<!76!76<!768!76!768!76!76!76!76!76 )9!6))!6))!6)!6)9!6)3!6)!6)!6)9!6))!6)!6))!6)9 A9!63A!633!63A!633!63!63)!633!633!63!63)!63!63) 79!6A)!637!637!637!636!636!637!636!6A9!636!637!63 89!6A!6A!6A9!6A!6AA!6A!6A!6A!636!6A)!6A)!636. M!. M!7 9

52 .0 < $ 0 ' 0 % ' 0 $,. H M - D0 % $ % %+. M9. M!. M!3 )9 A )9 A )9 A )9 A )9 A !6)A!63!639!6!6)!63)!6)6!68!6)3!63)!639!68!6))!63)!6)6!6A!6)!639!639!6A 9!6)3!633!639!6A!6)A!633!6)6!6!6)A!639!6)6!6<!66!63!6)7!6!6<!6)6!6)<!6A )9!6)!63)!6)6!6!6)!63)!6)7!6!6)9!639!639!68!6<!6)7!6)8!6A!697!6)A!6)A!6) A9!6)8!63!6)7!6!6)A!633!6)7!6A!67!6)6!6)8!6A!69<!6)!6)A!63!778!63!6)9!69< 79!6)!63)!6)6!6!6)3!633!6)7!6!6!6)!6)!63!77<!6!66!697!7A7!768!697!69 89!6)!63!639!6A!6)!6)6!6)7!63!76A!67!6)!69!7A6!76<!69!699!<<!7<!77!77A σ ρ 0.08 σ ρ 0.04!6!696!77!7)3!6!698!77!7)3!69!69<!7<6!7)!697!69!7<7!7) 9!63!69<!779!7)3!6)!697!7<6!7))!697!69<!779!7)A!69!69A!7<7!7) )9!63!697!77)!7)3!6!697!779!7))!698!693!7<<!7)9!69!699!7<A!77 A9!6)!696!779!7)A!696!69<!779!7))!69)!69!7<8!76!778!763!78<!7 79!6)!698!77!7)3!69!698!7<6!7))!76)!76A!7<!7A!78A!7<!7!79 89!63!698!7<7!7)3!69<!69A!7<<!7)!7<A!773!78!793!7!7A!7)7!<<3 σ ρ 0.16!783!78!<)3!89A!78!78!<)!89A!78!7A!<)3!89)!7<!7)!<)!899 9!783!7A!<)A!89!789!7!<)3!89A!7<!79!<)9!66!79!79)!<3!6) )9!78)!7!<)A!893!76!7!<)!893!79!798!<A!6A!73<!<6!<9!73 A9!78!73!<)8!899!77!7)!<))!66!7A!<68!<98!7A!79!<8<!8<8!< 79!78)!7!<)!89!78!79!<7!66!7))!<<<!87<!8!<6!<))!83A!7 89!78!7!<)A!89)!7!79!<3!6)!<7!<A!83!33!883!83<!)!AA σ ρ 0.32!<97!67!A<3!33!<97!67!A<!3A!<93!6)!A8A!3A6 9!<97!68!A<3!3A!<97!68!A87!33!86!7!A8!3A9 )9!<97!66!A87!3!<9<!6<!A8A!39!8<7!87!AA<!338 A9!<9!6<!A8<!33!866!6!A8A!3A8!8)!AA!A))!3 79!<96!6!A8<!3)!86!7A!A8!3A3!63!A63!3<7!)7A 89!<9!6!A88!33!87!<9!AA8!333!99!A93!39!))8. M!. M!88

53 .0 7 $ 0 % ' 0 ' 0 $ D0 $ % %+. M9. M!. M!3 )9 A )9 A )9 A )9 A )9 A !7A!7A!7AA!7A3!7A3!7A!7AA!7A!7A3!7AA!7A!7A8!7A!7A)!7A!7A!7A9!7A3!7A!7A 9!69!69)!693!699!69)!69A!69A!69)!69)!69A!69A!693!69)!69A!69A!69A!693!69)!69!69) )9!6)7!6)<!6)7!6)7!6)8!6)7!6)<!6)7!6)<!6)<!6)6!6)6!6)<!6)<!6)<!6)<!6)<!6)<!6)7!6)6 A9!636!637!636!636!636!6A!637!6A9!638!637!637!636!633!63<!636!636!639!633!63<!637 79!6A3!6A!6A!6AA!6A3!6A8!6A3!6A!637!6A!6A3!6A3!6)<!637!6A!6A)!6!6)8!63<!6A 89!6A8!6A8!6A7!6A8!6A!6A!6A8!6A8!639!636!6A3!6A!69<!6)7!637!6A3!78!698!6)7!6A9 σ ρ 0.08 σ ρ 0.04!7AA!7AA!7A3!7A3!7A!7A!7A)!7A)!736!7AA!7A)!7AA!736!7A!7A3!7AA 9!69)!69)!69!69)!69!693!69!69!69!69)!693!693!766!69)!693!69) )9!6)7!6)7!639!6)7!6)8!6)7!6)7!6)7!6)!6)8!6)8!6)<!6)!6)7!6)<!639 A9!637!6A9!637!6A9!63<!636!636!6A9!638!637!636!6A9!63!637!637!636 79!6A3!6A3!6AA!6A3!6A3!6A3!6AA!6AA!637!6A!6AA!6A!6)6!63<!6A)!6A 89!6A7!6A8!6A8!6A<!6AA!6A8!6A8!6A<!63)!6A!6A8!6A8!696!639!6A9!6A σ ρ 0.16!7A9!7A!7A!7A)!736!7A!7A!7A)!736!7A9!7A)!7AA!73<!737!7A9!7A9 9!69!69)!69!693!69!69!69!693!69!699!69)!69A!766!766!766!699 )9!6)8!6)<!6)7!6)7!6)!6)<!6)7!6)7!6)A!6)7!6)<!6)7!6)<!6)8!6)<!6)7 A9!63<!636!636!6A9!63<!636!636!6A9!63<!636!636!6A9!63!636!6A9!636 79!6A3!6A8!6AA!6A!6A3!6AA!6A!6A!6A9!6AA!6A3!6A8!63!6A)!6AA!6A 89!6A8!6A<!6A<!6A7!6A!6A<!6A8!6A7!637!6AA!6A8!6A7!6))!636!6AA!6A7 σ ρ 0.32!73<!7A!7A3!7AA!73<!7A9!7A3!7A9!738!73!737!736 9!69!69)!69A!693!766!69!69)!69!767!766!766!699 )9!6)<!6)7!6)7!63!6)<!6)6!6)6!63!6)!6)<!6)<!6)< A9!637!6A9!636!6A9!636!6A9!6A9!6A!636!6A9!636!636 79!6AA!6A!6A!6A!6A!6A!6A!6A!6A3!6A!6AA!6A 89!6A7!6A<!6A<!6A7!6A!6A<!6A7!6A<!6A8!6A7!6A8!6A<. M!. M!88 )

54 FIGURES " / $ $ <)99)!. $ %1, / /,--! " ). $ $ $, - 0 6G $, - $ % 0 $!. $ + 0 +, $ + -!$ $ $.0! " 3" / $ $$!; + $ ' 0 $ 00 $! " A. $ $ $, - 0 6G $, - $ % 0 $!. $ + 0 +, $ + -! 3

55 ". $ $ $, - 0 6G $, - $ %, 0 $ -!. $ + 0 +, $ + -!$ $ $.0! A

56 Frequency Estimate of Population Standard Deviation

57 Hedges & Vevea Method Hunter-Schmidt Method Deviation From Population Effect Size ρ.66 Deviation From Population Effect Size Deviation From Population Effect Size Deviation From Population Effect Size Deviation From Population Effect Size ρ.50 Number of Studies in the Meta-Analysis ρ.30 Number of Studies in the Meta-Analysis ρ.10 Number of Studies in the Meta-Analysis ρ.00 Number of Studies in the Meta-Analysis σ ρ.04 σ ρ.08 σ ρ.16 σ ρ.32 σ ρ.04 σ ρ.08 σ ρ.16 σ ρ Number of Studies in the Meta-Analysis 8

58 Mean Effect Size in Superpopulation 0.0 Mean Effect Size in Superpopulation 0.1 Mean Effect Size in Superpopulation 0.3 Mean Effect Size in Superpopulation 0.5 Mean Effect Size in Superpopulation 0.8 Superpopulation Based on r Superpopulation Based on z Standard Deviation in Superpopulation Frequency Standard Deviation in Superpopulation Frequency Standard Deviation in Superpopulation Frequency Standard Deviation in Superpopulation Frequency Effect Size in Population Effect Size in Population <

59 Hedges & Vevea Method Hunter-Schmidt Method Deviation From Population Effect Size ρ!79 Deviation From Population Effect Size Deviation From Population Effect Size Deviation From Population Effect Size Deviation From Population Effect Size ρ.50 Number of Studies in the Meta-Analysis ρ.30 Number of Studies in the Meta-Analysis ρ.10 Number of Studies in the Meta-Analysis ρ.00 Number of Studies in the Meta-Analysis σ ρ.04 σ ρ.08 σ ρ.16 σ ρ.32 σ ρ.04 σ ρ.08 σ ρ.16 σ ρ Number of Studies in the Meta-Analysis 7

60 Hedges & Vevea Method Hunter-Schmidt Method Deviation From Population Effect Size ρ.66 Deviation From Population Effect Size Deviation From Population Effect Size Deviation From Population Effect Size Deviation From Population Effect Size ρ.50 Number of Studies in the Meta-Analysis ρ.30 Number of Studies in the Meta-Analysis ρ.10 Number of Studies in the Meta-Analysis ρ.00 Number of Studies in the Meta-Analysis σ ρ.04 σ ρ.08 σ ρ.16 σ ρ.32 σ ρ.04 σ ρ.08 σ ρ.16 σ ρ Number of Studies in the Meta-Analysis 6

61 FOOTNOTES. $ $! & & + 0 $0 & 0 $0 C& 1, *$ ' 667-! % $ $ $ 0 $$ $! $ $ 0 0, '66-0 && 1 +, " ')99-! $. $ + &! $ 0 /, /9- % ', /8-0 $ 0 $, /3-'! $ $ 0 $ '0 0,.0 -! $ ":%! & $ 0 $ 0 9!* 89

62 / $ 0 9'0 $ $ $ 0! &. $ '",)99- ' ' ' $! &. $ ',0 $0 -! 8