Characteristics of measures of directional dependence - Monte Carlo studies

Transcription

1 Characteristics of measures of directional dependence - Monte Carlo studies Alexander von Eye Richard P. DeShon Michigan State University

2 Characteristics of measures of directional dependence - Monte Carlo studies abstract Recent results (Dodge & Rousson, 000; von Eye & DeShon, 008) show that, in the context of linear models, the response variable will always have less skew than the explanatory variable. This applies accordingly to the kurtosis of the two variables. These facts can be used to determine the direction of dependence. Specifically, using third and fourth order moments, and information concerning the deviation of variables from normality, it can be ascertained which of two variables is the response and which is the explanatory variable. In this article, the ratio of two skewness measures, the ratio of two kurtosis measures, and one omnibus test of deviation from normality are examined. Simulation studies are reported that allow one to answer the question whether, these measures (1) are sensitive to various data distributions, () sample size, and (3) a simple correlation structure. Results suggest that all three measures are highly sensitive to these factors. The ratio of two kurtosis measures is sensitive in particular to the correlation structure. It is concluded that (1) directional dependence can be based on various types of deviation from normality, () measures that respond to deviations based on skewness and kurtosis have characteristics that make them prime candidates for determining directional dependence, and (3) each of the measures proposed thus far is highly sensitive to either specific or omnibus deviations from normality.

3 Characteristics of measures of directional dependence - A Monte Carlo study It is widely believed that the tools of statistics cannot be used to determine the direction of effect in data collected using observational research methods. One reason for this is that it is easy to show that standard statistical methods such as regression analysis do not allow one to derive conclusions about directional dependence (von Eye & DeShon, 008). Another reason is that, as a widely used adage states, correlation does not imply causation (but it sure is a hint; see Lynd- Stevenson, 007; Tufte, 006). Recently, matters have taken a most interesting and dramatic turn. To be able to derive conclusions about directional dependence, authors (e.g., Dodge & Rousson, 000) have proposed moving away from standard methods of analysis which involve employing no more than first and second order moments. Instead, the authors propose using at least third order moments, that is indicators of skewness. Recently, von Eye and DeShon (008) proposed to also use fourth order moments, that is, indicators of kurtosis. Based on recent work, it is now possible to determine which variable in a linear model functions as the response variable and which functions as the explanatory variable (i.e., directional dependence). This evidence is made possible through a fundamental shift in the representation of linear models. Virtually all statistical analyses of continuous variables in psychological research are based on knowledge of the first two moments (i.e., means, variances, and covariances) of the joint distribution of two or more variables. This is entirely reasonable under the common normality assumption. However, it is also well known that the variables in empirical research are not normally distributed (Micceri, 1989) and, therefore, many of the higher order moments such as skew and kurtosis deviate from expectancy. Thus, higher order moments provide critical information that can be used to determine directional dependency. Specifically, under standard assumptions for linear models, the response variable is a linear combination of an explanatory variable - that need not be normally distributed - and a normally distributed error. If the explanatory variable has a nonnormal distribution, and the error term is normally distributed, then the response variable is a linear convolution of these two distributions that must have a distribution closer to normality than the explanatory variable. In this article, we review a selection of the new approaches to determining directional

4 directional dependence - simulations; p. 4 dependency. Specifically, we focus on those approaches that use third and fourth order moments (see, for example, Dodge & Rousson, 000, 001; Muddapur, 003; Shimizu, Hoyer, Hyvärinen, & Kerminen, 006; Shimizu & Kano, 006; Sungur, 005; von Eye & DeShon, 008). We then ask questions concerning the characteristics of two measures used to derive conclusions about directional dependence. We answer these questions based on results from Monte Carlo simulation studies. 1. Assessing Directional Dependence The ideas that guided the recent and very active research area of devising methods for the determination of directional dependence are based on the proposition that, if one random variate is determined by one or several others, it will have describable distributional characteristics. Statistical methods that are based on first and second order moments, for example, regression analysis, are unable to reflect these characteristics (for an illustration, see von Eye & DeShon, 008). Therefore, it appears reasonable to conclude that statistical evidence cannot be used to determine the direction of effects or the direction of causality. However, higher order moments provide information that can be used to determine the direction of effects. Dodge and Rousson (000, 001) presented a proof for the proposition that the response variable will always have less skew than the explanatory variable. Based on this result, they proposed comparing the skewness of the response and the explanatory variables. Von Eye and DeShon (008) discussed a generalization of this proposition, according to which any deviation from normality will be less pronounced in the response variable than in the explanatory variable. Therefore, the widely used Mardia tests of skewness and kurtosis (Mardia, 1970, 1980; cf. von Eye, 005, 006; von Eye & Gardiner, 004) as well as omnibus tests of normality such as the one proposed by D Agostino (1971; D Agostino, Belanger, & D Agostino, 1990; D Agostino & Pearson, 1973) can be used to determine direction of dependence. In the following sections, we provide an overview of three approaches to determining the direction of effects. The first is based on Dodge and Rousson s (000) proposal of using skewness. The second approach, proposed by von Eye and DeShon (008), is based on the omnibus measure of deviation from normality proposed by D Agostino (1971). The third approach, also proposed by

5 directional dependence - simulations; p. 5 von Eye and DeShon (008), involves using the ratio of two kurtosis measures. 1.1 Skewness- and Kurtosis-Based Approaches Dodge and Rousson s idea (000) of using skewness to determine direction of effect (details follow below) goes back to Karl Pearson. In an article that was published in 1895, Pearson noted that skewness and kurtosis can be used to appraise deviations from normality. Tests of (multi)normality that are based on skewness have been proposed, for example, by Mardia (1970, 1980). Dodge and Rousson s procedure involves the following steps. Let Y be the response variable and X the explanatory variable. Then, the skewness of X and the skewness of Y are defined by the third moment, or and respectively. The fourth moment, the kurtosis of X and Y, is and respectively. For X, Y N(0; 1), ã = 0 and ä = 3.0. Definition and computational aspects of central moments can be found in many textbooks (e.g., Hogg & Tanis, 1993; see also D Agostino & Pearson, 1973; Walwyn, 005). To come to a conclusion about the direction of effect, Dodge and Rousson (000) propose using the Bravais-Pearson correlation where cov X,Y is the covariance between X and Y, and ó X and ó Y are the standard deviations of X and Y, respectively (for interpretations of ñ, see, for example, Dodge & Rousson, 001; Falk & Well, 1996; Rodgers & Nicewander, 1988; Rovine & von Eye, 1997). Now, using ã and ã, Dodge and Rousson (000, 001) showed that the cube of the Y X correlation coefficient can be given by the ratio of the skewness of the response variable to the

6 directional dependence - simulations; p. 6 skewness of the explanatory variable, or if ã X 0. This equation implies that the skewness of the response variable is always smaller than the skewness of the explanatory variable (in absolute values, because the range of the correlation is -1 ñ +1). If, however, the cube of the correlation is greater than 1.0, (1) the result is incorrect by definition, () researchers may redefine the status of X and Y as response and explanatory variables, or (3) X and Y are not involved in a directed or causal relationship at all. According to von Eye and DeShon (008) there are five interesting implications of this relationship: (1) If a symmetric error, for example, normally distributed noise, is added to a skewed explanatory variable, the response variable will be less skewed; () the cube of the correlation coefficient can be interpreted as the percentage of skewness that is left after a linear model was applied to describe the relationship between X and Y; (3) if the ratio of the two skewness scores lies outside [-1; +1], the cube of the correlation suggests that not X but Y may be the explanatory variable; (4) if X is perfectly symmetric as is the case in normal or in uniform distributions, directional dependence cannot be determined; and (5) above and beyond what is generally known about the Bravais-Pearson s correlation, ñ does, in the context of linear modeling, possess asymmetric properties (Dodge & Rousson, 001). In a way similar to using third order moments, one can also use fourth order moments to determine the direction of effect (von Eye & Deshon, 008; cf. Dodge & Rousson, 001). 3 Specifically, in addition to ñ, one can calculate 3 for ä X > 0. As for ñ, Y can be interpreted as the response or dependent variable if î < 1.0. If î 1.0, either X is the response variable, or X and Y are not involved in a directed relationship. The relationship of î with the Bravais-Pearson correlation still needs to be explored in detail. However, the use of î is based on the original idea that variates that are completely determined will show no

7 more than random deviation from normality. 1. von Eye and DeShon s Deviation-from-Normality Approach directional dependence - simulations; p. 7 An alternative approach to the one presented in Section 1.1 was proposed von von Eye and DeSon (008). This approach is based on a statistic proposed by D Agostino (1971; cf. D Agostino et al., 1990; D Agostino & Pearson, 1973). This statistic is omnibus to both skewness and kurtosis violations of normality. It uses skewness as well as kurtosis information, and creates a score that is distributed as with degrees of freedom. Specifically, consider the third standardized moment, which, for the normal distribution assumes the value of 0, and the fourth standardized moment, which, for the normal distribution, assumes the value of 3. Based on these two moments, D Agostino (1971) proposes a test that enables researchers to detect deviations from normality that are due to either skewness or kurtosis. The test statistic is where z( b 1) and z(b ) are the normal approximations to b 1and b (detailed equations can be found in D Agostino et al., 1990). The K statistic is approximately distributed as with df =, when the population is normally distributed. To determine the direction of effect, the K scores of two variables are compared in a fashion parallel to the comparison of two skewness scores. 3 In the following sections, we present results from simulation studies for the measures ñ and 3 K. We begin with ñ.. Monte Carlo Simulation Studies on the Direction of Effects The following Monte Carlo studies were designed to answer questions concerning the behavior of ñ 3 and K under various conditions (the detailed questions will be asked at the beginning of Sections.1 and.). In each of the Monte Carlo studies, the following factors were varied.

8 Simulation results for d; p. 8 Shape of distribution. The following five distribution shapes were realized (cf. Von Eye, 005; von Eye, von Eye, & Bogat, 006): (1) Normal distribution. The generator GASDEV from the Numerical Recipes FORTRAN collection (Press, Flannery, Teukolsky, & Vetterling, 1989) was used to create N(0;1)- distributed data (see also Sicking, 1994). The data created using GASDEV were expected not to deviate from multinormality. () Uniform distribution. The generator RANDOM, available in the Power Station s PortLib function pool, was used to create pseudo random numbers, z, from the interval 0 z < 1. The data created using RANDOM were expected to deviate from multinormality. They are symmetric, that is, skew will be about zero, but they show kurtosis characterized by heavy tails. (3) Logarithmic distribution. Uniform variates x were subjected to the logarithmic transformation log(x). Specifically, a transformed variable x was created as The resulting data were expected to exhibit some skewness and elevated kurtosis. (4) Inverse Laplace-transformed. The Laplace probability distribution, also known as double exponential distribution, is for x < á, - < x <, and â > 0. This distribution has a mean of zero, a skewness of zero, and a kurtosis of 0. For â = 1 and x centered scores, the probability distribution becomes A uniform distribution has no skew but exhibits increased kurtosis. Performing an inverse Laplace transformation on a uniform distribution should, therefore, result in a distribution with reduced kurtosis and possibly elevated skewness. The Laplace function has no inverse. Therefore, the transformation introduced by von Eye and von Eye (005) was performed. Application of this transformation to the uniformly distributed random numbers results in a distribution with both slightly elevated skewness and elevated kurtosis. The kurtosis of the

9 Simulation results for d; p. 9 transformed uniform distribution has a positive sign. The kurtosis of the uniform distribution was negative. In other words, this transformation changed the distribution from being heavytailed to heavy around the belt line. 1/3 (5) Cube root transformation. This transformation was used to create x = ½ x from the uniform x scores. Considering that the uniform scores that were cube root-transformed had no skewness and an only slightly elevated kurtosis, the resulting scores should have elevated skewness and elevated kurtosis. 3 For the simulations of ñ and î, two random variates were created for the calculation of the ratio of the two skewness and the two kurtosis scores. Each of these variates was subjected to the same transformations. Crossed, 5 x 5 patterns resulted. 3 In brief, for the simulation of ñ, the following data sets were created: (1) normally distributed variates; () uniformly distributed variates; (3) logarithmically-transformed variates (from a uniform distribution); (4) uniformly distributed variates that were subjected to the substitute of the inverse Laplace transformation; and (5) cube root-transformed variates. Magnitude of variable intercorrelation. The correlation that was taken into account varied between 0.0 and 0.6, in increments of 0.. The correlation was created between the original, normally distributed variable and the variates that resulted from the transformations described above (for details on how the correlations were created, see von Eye, 005). Sample size (N): The sample size in the simulation runs varied from 00 to 100 objects, in increments of Direction of Effects - A Monte Carlo Study on the Ratio of Two Skewness Scores To follow the notation proposed by Dodge and Rousson (000) and also used by von Eye and DeShon 3 (008), we will, from now on, use the character d to denote ñ. The Monte Carlo study for d was designed to address the following questions: 1. Is the measure d sensitive to various shapes of distributions?. What is the á curve of d; that is, how many null hypotheses are rejected? The design that was used to study the behavior of d had 5 (distribution shape) x 5 (distribution shape of the second random variate) x 4 (correlation levels) x 6 (sample sizes) = 600 cells. Each run was repeated 100 times so that a total of 60,000 samples was available for analysis. The repetitions

10 Simulation results for d; p. 10 were used as a covariate. In the following sections, we answer the two questions that were posed above. We begin with Question Is the Measure d Sensitive to Various Transformations of Data? To answer the first question, the sensitivity of the d values to the five independent variables of the simulation was examined using a standard ANOVA. The ratio of the two skewness scores, d, was used as the dependent variable. Results are summarized in Table 1. The overall multiple R was Table 1: 5 (Distribution Shape; ITRANS1) x 5 (Distribution Shape of second Random Variate; ITRANS) x 4 (Correlation Levels; RHO) x 6 (Sample Sizes;N) ANOVA with d as Dependent Measure and Repetitions (REP) as Covariate Source Type III SS df Mean Squares F-ratio p-value ITRANS1 5.63E E ITRANS 1.586E RHO N 1.938E ITRANS1*ITRANS 3.075E ITRANS1*RHO ITRANS1*N 1.63E ITRANS*RHO ITRANS*N RHO*N ITRANS1*ITRANS*RHO ITRANS1*ITRANS*N ITRANS1*RHO*N ITRANS*RHO*N ITRANS1*ITRANS*RHO*N REP Error The results in Table 1 show that, with the exception of the repetition variable, all effects are significant. By far the strongest effect was that of Shape of Distribution (ITRANS1). Figure 1 displays the distribution of d, by Shape of Distribution.

11 simulation results ; p. 11 Figure 1: Distribution of d for (1) Normal, () Uniform, (3), Inverse-Laplace Transformed, (4) Cube Root-transformed, and (5) Logarithmic Distributions Figure 1 suggests that d (labeled D in Figure 1) resembles a leptokurtic normal distribution for normal random variates (Panel 1). In contrast, for uniform distributions, d varies within a wide range (Panel ). As expected, many of the d scores are negative, thus reflecting the platykurtic shape of the distribution. The inverse Laplace transformation creates distributions that are skewed (Panel 3). In addition, d is mostly negative and shows a wider range than for the normal distribution. This applies accordingly to the distribution that is created by the cube root transformation (Panel 4). This distribution is asymmetric and leptokurtic. In contrast, the logarithmic transformation results in an asymmetric and platykurtic distribution of d. The second transformation variable (ITRANS) has the second strongest effect. It results in similar distributions as the first. Therefore, it is not illustrated here.

12 simulation results ; p. 1 The third strongest effect is that of the main effect of N. It is not depicted here. N has the effect that the range of d values increases, and the distribution becomes more asymmetric. The interaction of Shape of Distribution with N shows that the standard deviation of the distribution of d increases in particular for normally distributed data (also not illustrated here). All of the other effects, although significant, explain only small portions of the variance of d. Therefore, they will not be discussed in detail in the present context. More interesting is the effect of the interaction of the two transformation variables. In the following five figures (Figure through 6), we demonstrate the effect that the five distribution shapes have on the distribution of d, given a particular distribution. We begin with the distribution of d when the first of the two distributions (numerator of the d ratio) is normal (Figure ). Figure : Distribution of d When the First Variable Is Normal and the Second Is (1) Normal, () Uniform, (3), Inverse Laplace-transformed, (4) Cube Root-transformed, and (5) Logarithmic Figure shows that the distribution of d is leptokurtic normal when the first of the two variables is normal. In the ideal case, d will, under each of these conditions, be zero because normal distribution show no skew. Random variations yield normally distributed scores for d.

13 simulation results ; p. 13 Figure 3: Distribution of d When the First Variable Is Uniform and the Second Is (1) Normal, () Uniform, (3), Inverse Laplace-transformed, (4) Cube Root-transformed, and (5) Logarithmic Figure 3 shows that, when the first variable is uniformly distributed, d is skewed with a bias for negative scores, unless the second variable is normally distributed (Panel 1) or cube root-transformed. In these cases, the distribution of d is asymmetric but with a mean of zero. Figure 4 shows that d assumes extreme scores, most of them negative, when the first distribution is inverse Laplace-transformed (cf. Panel 3 in Figure 1). This applies even when the second variable is normally distributed (Panel 1).

14 simulation results ; p. 14 Figure 4: Distribution of d When the First Variable Is Inverse Laplace-transformed and the Second Is (1) Normal, () Uniform, (3), Inverse Laplace-transformed, (4) Cube Roottransformed, and (5) Logarithmic Figure 5: Distribution of d When the First Variable Is Cube Root-transformed and the Second Is (1) Normal, () Uniform, (3), Inverse Laplace-transformed, (4) Cube Roottransformed, and (5) Logarithmic

15 simulation results ; p. 15 Figure 5 shows that, when the first variable is cube root-transformed, d can assume a wide range of scores, in particular when the second variable is uniformly or inverse Laplace-transformed. The range is much narrower when the second variable is normally distributed. The range is narrow and most scores are positive when the second variable is also cube root-transformed. Figure 6: Distribution of d When the First Variable Is Logarithmic and the Second Is (1) Normal, () Uniform, (3), Inverse Laplace-transformed, (4) Cube Root-transformed, and (5) Logarithmic Figure 6 shows that d can assume a wide range of mostly negative scores when the first variable was subjected to a logarithmic transformation. The only exception is the case in which the second variable was cube root-transformed (Panel 4). In this case, the range is narrow, the distribution is asymmetric, and the scores vary about zero..1. á Curves of d In this section, we ask what proportion of cases coefficient d identifies as extreme, that is, outside of the 5% interval. The alpha curves depicted in Figure 7 were created based on the normally distributed data (ITRANS = 1) in the simulation.

16 simulation results ; p. 16 Figure 7: Left Panel: á-curve of d, by N; Logarithmic Smoother; Right Panel: Histogram of d; Normal Smoother The left Panel of Figure 7 shows that d is very sensitive, in particular for large samples. It identifies about 5% of the cases as extreme as long as the sample size is 400 cases or less. For more than 400 cases, far more than 5% of the cases are identified as extreme. We conclude that the use of d can come with the risk of an increased á error. The right panel of 7 shows why d tends to suggest non-conservative decisions. The distribution of d is both lepto- and platykurtic. More cases appear at the extreme end of the distribution than one would expect for a (leptokurtic) normal distribution. For each of the transformations, far more cases than 5% were identified as extreme. This was as expected. For the inverse Laplace-transformed data, all of the cases were identified as extreme.. Direction of Effects - A Monte Carlo Study on the Measure K In this section, we present results from a simulation study on D Agostino s measure K of deviation from normality (see also D Agostino et al., 1990; D Agostino & Pearson, 1973; von Eye & DeShon, 008). This measure is omnibus to deviations due to either skewness or kurtosis. Specifically, the following questions are addressed:

17 1. Is the measure K sensitive to various shapes of distributions?. Is the measure K distributed as with df =? 3. What is the á curve of K ; that is, how many null hypotheses are rejected? simulation results ; p. 17 To address these questions, the same factors were varied as in the Monte Carlo study for K. In addition, the factors covered the same range of scores. However, the design for the study was different. The dependent measure was the measure K. Therefore, there was no need to include two dependent variables in the simulations. The resulting design thus had 5 (distribution shape) x 4 (correlation levels) x 6 (sample sizes) = 10 cells. Each run was repeated 500 times so that a total of 60,000 samples was available for analysis. In the following sections, we answer the three questions that were posed above...1 Is the measure K sensitive to various transformations of data? In a first analysis, the sensitivity of the K values to the three factors of the simulation was examined using ANOVA. The repetitions were taken into account as a covariate. Results are summarized in Table. Table : 5 (Distribution Shape; ITRANS) x 4 (Correlation Level; RHO) x 6 (Sample Size; N) ANOVA with K as Dependent Measure and Repetitions (REP) as Covariate Source Type III SS df Mean Squares F-ratio p-value ITRANS RHO N ITRANS1*RHO ITRANS1*N RHO*N ITRANS1*RHO*N REP Error The ANOVA table shows that, with the exception of the repetitions, each of the effects is significant. Clearly the strongest effect is observed for shape of distribution (ITRANS). Figure 8 illustrates the

18 1 effect. simulation results ; p. 18 Figure 8 shows that, for normal distributions, K approximates a -distribution with degrees of freedom (Panel 1; this issue will be taken up in more detail, below). For uniform distributions, that is, for distributions with no skew, the variation of K shows more extreme scores (Panel ). In contrast and as expected, the K for the inverse-laplace transformed distributions (Panel 3) hovered about zero, with few extreme scores. The cube root-transformed scores as well as the scores after the logarithmic transformation resulted in skewed distributions with the largest numbers of extreme scores (Panels 4 and 5, respectively 3). Figure 8: Distribution of K for (1) Normal, () Uniform, (3) Inverse-Laplace Transformed, (4) Cube Root Transformed, and (5) Logarithmic Distributions 1 To obtain a more detailed look of the distributions under the conditions of the simulation, we excluded all values x > 30 for Figures 8 and 9. The percentage of eliminated scores was 1.9. For the following Figures, all scores were used.

19 simulation results ; p. 19 Figure 9: Distribution of K by Magnitude of Correlation Figure 9 shows that, although significant, ñ hardly changes the nature of the distribution. None of the remaining significant effects is illustrated here. They reflect, for example, the anticipated increase in deviations from normality that comes with the sample size increase. Compared to the effect of shape of distribution, each of the remaining effects is rather weak. Although significant, none of these effects explained large portions of variance. Therefore, they will not be discussed in the remainder of this article.

20 simulation results ; p. 0.. Distributional Characteristics of K To answer the second question, a probability plot of the measure K was created with reference to a -distribution with df =? Figure 10 shows this plot, with box plots of the observed and expected distributions at the margins. Figure 10: Probability Plot of K with Reference to a -Distribution with df = Clearly, the approximation of the -distribution with df = is miserable. However, one has to consider that the data used for the plot in Figure 10 represent a mixture of five different distributions and four magnitudes of correlation. Therefore, and in order to do justice to the measure K, we now examine the probability plot exclusively for normally distributed. Figure 11 displays this plot, with box plots of the observed and expected distributions at the margins.

21 simulation results ; p. 1 Figure 11: Probability Plot of K with Reference to a -Distribution with df =, for Normally Distributed Data Figure 11 shows that the distribution of K is reasonably close to a -distribution with df =. The measure seems slightly biased such that (1) more cases with values close to zero are observed than expected based on the -distribution with df =, and () fewer cases with large scores are observed than expected. The correlation structure does not make a difference that would be visible in the printed graphs. We conclude that K can be used to examine the direction of effects. The characteristics of K in the context of hypothesis testing are discussed in the next section...3 K as a Significance Test In a first step towards answering the question concerning the characteristics of K in the context of hypothesis testing, we create curves representing the proportion of rejected null hypotheses for varying sample sizes for each of the examined distributions. The null hypothesis for the K test is that the sample is drawn from an underlying distribution that is normal. Therefore, the rejection rate curves for samples from a normal distribution reflect the ability of the test to maintain the stated á

22 simulation results ; p. level. For all other distributions, the rejection rate reflects the power of the test to detect samples drawn from non-normal distributions. Figure 1 displays the rejection rates of the K test based on 5000 replications for samples of size 0 to 500 from the normal distribution, the uniform distribution, inverse-laplace transformed, cube root transformed, and logarithmic distributions. Figure 1: Rejection Rate Curves for K for (1) Normal, () Uniform, (3), Inverse-Laplace Transformed, (4) Cube Root-transformed, and (5) Logarithmic Distributions As can be seen in Figure 1, the K test maintains the desired á level of.05 across the range of samples sizes investigated for the normal distribution. It does appear that there is a very slight elevation in rejection rates for small sample sizes. With respect to power, the K test has low to

23 simulation results ; p. 3 moderate power to detect deviations from normality consistent with the uniform and cubed-root distributions with sample sizes smaller than 60. With sample sizes of approximately 80 all distributions are detected at acceptable levels (e.g., rejection rates greater than 80%) and the power to detect deviations from normality in all nonnormal distributions studies is near 100% with samples larger than Direction of Effects - A Monte Carlo Study on the Ratio of Two Kurtosis Scores The Monte Carlo study for î was designed to address the following questions: 1. Is the measure î sensitive to various shapes of distributions?. What is the á curve of î; that is, how many null hypotheses are rejected? The design that was used to study the behavior of î had 5 (distribution shape) x 5 (distribution shape of the second random variate) x 4 (correlation levels) x 6 (sample sizes) = 600 cells. Each run was repeated 00 times so that a total of 10,000 samples was available for analysis. The repetitions were used as a covariate. In the following sections, we answer the two questions that were posed above. We begin with Question 1. That is, we ask whether the measure î is sensitive to various transformations of data? To answer this question, the sensitivity of the î values to the five independent variables of the simulation was examined using a standard ANOVA. The ratio of the two kurtosis scores, î, was used as the dependent variable. Results are summarized in Table 3. The overall multiple R was

24 simulation results ; p. 4 Table 3: 5 (Distribution Shape; ITRANS1) x 5 (Distribution Shape of second Random Variate; ITRANS) x 4 (Correlation Levels; RHO) x 6 (Sample Sizes;N) ANOVA with î as Dependent Measure and Repetitions (REP) as Covariate Source Type III SS df Mean Squares F-ratio p-value ITRANS ITRANS RHO N ITRANS*ITRANS ITRANS*RHO ITRANS*N ITRANS*RHO ITRANS*N RHO*N ITRANS*ITRANS*RHO ITRANS*ITRANS*N ITRANS*RHO*N ITRANS*RHO*N ITRANS*ITRANS*RHO*N REP Error The results in Table 3 show that all effects are significant. In contrast to the ANOVAs of d and K, however, the strongest effect on kurtosis was that of Correlation (RHO). Figure 13 displays the distribution of î, by RHO.

25 simulation results ; p. 5 Figure 13: Distribution of î (Labeled as RATIOKURT), by Magnitude of Correlation Figure 13 suggests that the spread of î, the ratio of the two kurtosis measures, increases as the correlation between the two variables increases. Note that, for ñ = 0 and ignoring the outliers, the distribution of î is almost symmetric. This effect is dependent on both the sample size and the shape of the distribution. Considering only the normally distributed data (ITRANS = 1 and ITRANS = 1) and ñ = 0, we find not only that the spread of ñ shrinks as the sample size increases. We also find that the distribution of î becomes more symmetric as the sample size increases. This is illustrated in Figure 14.

26 simulation results ; p. 6 Figure 14: Distribution of î, by Sample Size, for Two Normally Distributed Variates The second strongest effect was that of the sample size, N. It is not depicted here (see, however, Figure 14). The effect is that the spread of î increases with the sample size. Figure 15: Distribution of î for (1) Normal, () Uniform, (3) Inverse-Laplace Transformed, (4)

27 Cube Root Transformed, and (5) Logarithmic Distributions simulation results ; p. 7 The third largest effect was found for distribution shape. Figure 15 displays the distribution of î, by shape of distribution (ITRANS). It shows that the spread of î varies with the shape of the distribution. Specifically, î has the widest spread for the cube root transformed (Panel 4) and the uniform distributions (Panel ). The smallest spread was observed for the normal distribution (Panel 1). None of the remaining significant main effects and interactions will be depicted. Although significant, the effects make only little contributions to the magnitude (or distribution) of î. 3. Comparing d, î, and K In this section, we compare the three measures d, î, and K. Considering that d and î are based on different types of deviation from normality, skewness and kurtosis, and K is omnibus to both, one would expect the correlation between d and î to be very low, and the correlations of either with K higher. Table 4 shows the correlation table for these three measures for all 10,000 simulation runs in Section.3. Table 4: Correlations Among d, î, and K, Aggregated over All Conditions Varied in Section 3. Measure d î î.18 K The correlation results in Table 4 disconfirm our expectations in the sense that the correlation between î and K is smaller than the correlation between d and î. However, one has to consider that

28 simulation results ; p. 8 4 out of the five distributions that went into the data pool were non-normal. Therefore, we now ask whether this correlation pattern is dependent upon the shape of the simulated distributions. Table 5 displays the correlation pattern, by distribution. Table 5: Correlations Among d, î, and K, by Shape of Distribution (Each Correlation is Based on 4,000 Runs) Measure d î Normal Distribution î -.07 K Uniform Distribution î.19 K Inverse Laplace-Transformed î.3 K Cube-Root Transformed î.08 K Logarithmic î.6 K Clearly, the correlations in Table 5 match our expectations, in particular for the normal (and the cube root-transformed) distribution. In addition, these correlations reflect the characteristics of the

29 simulation results ; p. 9 generated distributions. 4. Discussion The development of methods for the determination of the direction of effects using the means of statistics has, recently, taken an interesting turn (for recent theoretical developments in the discussion of causality, see Lynd-Stevenson, 007). Farthest developed are the approaches that resort to using third and fourth order moments. In the present research, we report results from Monte Carlo simulation studies on the behavior of the ratio, d, of two skewness measures, the ratio, î, of two kurtosis measures, and an omnibus statistic, K, of deviation from normality. The factors varied in the simulations were shape of distribution [(1) Normal, () Uniform, (3), Inverse Laplace-transformed, (4) Cube Root-transformed, and (5) Logarithmic], sample size, and correlation structure. Each of these factors had significant effects. As expected, the strongest effects came with shape of distribution. Only for the ratio î, the strongest effect was that for the correlation of the two simulated variates. The á curves that were based on the normally distributed data suggest that the omnibus measure is sensitive to both skewness and kurtosis violations of normality. In addition, it tends to suggest statistical decisions quite close to the nominal level á, and that regardless of sample size. The distributional characteristics of the ratio of two skewness measures and the ratio of two krtosis measures still need to be determined. The question as to which of the three measures discussed here, d, î, and K, can be recommended can be answered based on their characteristics. If researchers consider differences between measures of skewness, kurtosis or, in more general terms, measures of deviation from normality only if the deviation from normality is significant, K or any other omnibus test will be the measure of choice. This includes the simultaneous use of Mardia s (1970) tests, even von Eye and Gardiner s (004) local tests of deviation from normality. If, however, the focus is on the effects of an explanatory variable as they reflect in particular in either skewness or kurtosis, d and î are the measures of choice.

30 simulation results ; p. 30 A most important issue to discuss is the status of the methods presented here in the process of determining direction of effects. From our perspective, there can be no doubt that theory must exist before variables are subjected to the procedures discussed in the literature. Clearly, it makes no sense to conclude that just any two variables that differ in skewness are in a causal relationship with each other. The status of the methods discussed here is, therefore, that of tools that can be used to build theory. These methods do not replace conceptual and theoretical thinking.

31 simulation results ; p. 31 References D Agostino, R.B. (1971). Transformation to normality of the null distribution of g 1. Biometrika, 57, D Agostino, R.B., Belanger, A., & D Agostino, R.B. Jr. (1990). A suggestion for using powerful and informative tests of normality. The American Statistician, 44, D Agostino, R.B., & Pearson, E.S. (1973). Testing departures from normality. 1. Fuller empirical results for the distribution of b and b 1. Biometrika, 60, Dodge, Y., & Rousson, V. (000). Direction dependence in a regression line. Communications in Statistics - Theory and Methods, 3, Dodge, Y., & Rousson, V. (001). On asymmetric properties of the correlation coefficient in the regression setting. The American Statistician, 55, Everitt, B.S. (1998). The Cambridge dictionary of statistics. Cambridge, UK: Cambridge University Press. th Hogg, R.V., & Tanis, E.A. (1993). Probability and statistical inference (4 ed.). New York: Macmillan. Falk, R., and Well, A. D. (1996). Correlation as Probability of Common Descent. Multivariate Behavioral Research, 31, Lynd-Stevenson, R.M. (007). Concerns regarding the traditional paradigm for causal research: The unified paradigm and causal research in scientific Psychology. Review of General Psychology, 11, Mardia, K.V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57, Mardia, K.V. (1980). Tests of univariate and mutivariate normality. In P.R. Krishnaiah (ed.), Handbook of statistics (vol. 1; pp 79-30). Amsterdam: North Holland. Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105,

32 simulation results ; p. 3 Muddapur, M.V. (003). On directional dependence in a regression line. Communications in Statistics. Theory and Methods, 3, Pearson, K. (1895). Contributions to the mathematical theory of evolution. Philosophical Transactions of the Royal Society of London, 91, Press, W.H., Flannery, B.P., Teukolsky, S.A., & Vetterling, W.T. (1989). Numerical recipes. The art of scientific computing (FORTRAN version). Cambridge: Cambridge University Press. Rodgers, J.L., & Nicewander, W.A. (1988). Thirteen ways to look at the correlation coefficient. The American Statistician, 4, Rovine, M.J., & von Eye, A. (1997). A 14th way to look at a correlation coefficient: Correlation as the proportion of matches. The American Statistician, 51, Shimizu, S., Hoyer, P.O., Hyvärinen, A., & Kerminen, A. (006). A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7, Shimizu, S., & Kano, Y. (006). Use of non-normality in structural equation modeling: Application to direction of causation. Journal of Statistical Planning and Inference, 136 (in press). Sicking, F.J. (1994). GASDEV. Sungur, E.A. (005). A note on directional dependence in regression setting. Communications in Statistics. Theory and Methods, 34, Tufte, E.R. (006). The cognitive style of PowerPoint: Pitching out corrupts within. Cheshire, CT: Graphics Press. von Eye, A. (005). Comparing tests of multinormality - A Monte Carlo study. InterStat, (October, 005). von Eye, A. (006). Comparing Tests of Multinormality under Sparse Data Conditions - a Monte Carlo Study. InterStat, (May, 006). von Eye, A., & DeShon, R.P. (008). Moment Approaches to Directional Dependency. Under editorial review. von Eye, A., & Gardiner, J.C. (004). Locating deviations from multivariate normality.

33 simulation results ; p. 33 Understanding Statistics, 3, von Eye, A., & von Eye, M. (005). Can one use Cohen s kappa to examine disagreement? Methodology, 1, von Eye, A.,von Eye, M., & Bogat, G.A. (006). Multinormality and symmetry: A comparison of two statistical tests. Psychology Science, 48, Walwyn, R. (005). Moments. In B.S. Everitt, & D.C. Howell (eds.), Handbook of Statistics in Social Science (pp ). Chichester, UK: John Wiley.