The rth moment of a real-valued random variable X with density f(x) is. x r f(x) dx

Transcription

1 1 Cumulants 11 Definition The rth moment of a real-valued random variable X with density f(x) is µ r = E(X r ) = x r f(x) dx for integer r = 0, 1, The value is assumed to be finite Provided that it has a Taylor expansion about the origin, the moment generating function M(ξ) = E(e ξx ) = E(1 + ξx + + ξ r X r /r! + ) = µ r ξ r /r! r=0 is an easy way to combine all of the moments into a single expression The rth moment is the rth derivative of M at the origin: µ r = M (r) (0) The cumulants κ r are the coefficients in the Taylor expansion of the cumulant generating function about the origin K(ξ) = log M(ξ) = r κ r ξ r /r!, so that κ r = K (r) 0) Evidently µ 0 = 1 implies κ 0 = 0 The relationship between the first few moments and cumulants, obtained by extracting coefficients from the expansion, is as follows κ 1 = µ 1 κ 2 = µ 2 µ 2 1 κ 3 = µ 3 3µ 2 µ 1 + 2µ 3 1 κ 4 = µ 4 4µ 3 µ 1 3µ µ 2 µ 2 1 6µ 4 1 κ 5 = µ 5 5µ 4 µ 1 10µ 3 µ µ 3 µ µ 2 2µ 1 60µ 2 µ µ 5 1 In the reverse direction µ 2 = κ 2 + κ 2 1 µ 3 = κ 3 + 3κ 2 κ 1 + κ 3 1 µ 4 = κ 4 + 4κ 3 κ 1 + 3κ κ 2 κ κ 4 1 µ 5 = κ 5 + 5κ 4 κ κ 3 κ κ 3 κ κ 2 2κ κ 2 κ κ 5 1 1

2 In particular, κ 1 = µ 1 is the mean of X, κ 2 is the variance, and κ 3 = E((X µ 1 ) 3 ) Higher-order cumulants are not the same as moments about the mean This definition of cumulants is nothing more than the formal relation between the coefficients in the Taylor expansion of one function M(ξ) with M(0) = 1, and the coefficients in the Taylor expansion of log M(ξ) For example Student s t distribution on five degrees of freedom has finite moments up to order four, with infinite moments of order five and higher The moment generating function does not exist for real ξ 0, but the characteristic function M(iξ) = e ξ (1 + ξ + ξ 2 /3) is real and finite for all real ξ Both M(iξ) and K(iξ) have Taylor expansions about ξ = 0 up to order four only: M(iξ) = (1 ξ + ξ 2 /2 ξ 3 /3! + ξ 4 /4! + o(ξ 4 )) (1 + ξ + ξ 2 /3) = 1 + ξ 2 ( ) + ξ 3 ( ) + ξ4 ( 1 4! 1 3! ) + o(ξ4 ) = 1 ξ 2 /6 + ξ 4 /4! + o(ξ 4 ); K(iξ) = ξ + log(1 + ξ + ξ 2 /3) = ξ 2 /3 ( ξ + ξ 2 /3) 2 /2 + ( ξ + ξ 2 /3) 3 /3 ( ξ + ξ 2 /3) 4 /4 + = ξ 2 /6 + ξ 4 /36 + o(ξ 4 ), which means µ 2 = κ 2 = 1/3, µ 4 = 1 and κ 4 = 2/3 The normal distribution N(µ, σ 2 ) has cumulant generating function ξµ+ ξ 2 σ 2 /2, a quadratic polynomial implying that all cumulants of order three and higher are zero Marcinkiewicz (1935) showed that the normal distribution is the only distribution whose cumulant generating function is a polynomial, ie, the only distribution having a finite number of non-zero cumulants The Poisson distribution with mean µ has moment generating function exp(µ(e ξ 1)) and cumulant generating function µ(e ξ 1) Consequently all the cumulants are equal to the mean Two distinct distributions may have the same moments, and hence the same cumulants This statement is fairly obvious for distributions whose moments are all infinite, or even for distributions having infinite higherorder moments But it is much less obvious for distributions having finite moments of all orders Heyde (1963) gave one such pair of distributions with densities f 1 (x) = exp( (log x) 2 /2)/(x 2π) f 2 (x) = f 1 (x)[1 + sin(2π log x)/2] 2

3 for x > 0 The first of these is called the log normal distribution To show that these distributions have the same moments it suffices to show that x k f 1 (x) sin(2π log x) dx = 0 0 for integer k 1, which can be shown by making the substitution log x = y + k Cumulants of order r 2 are called semi-invariant on account of their behaviour under affine transformation of variables (Thiele 188?, Dressel 1942) For r 2, the rth cumulant of the affine transformation a + bx is b r κ r, independent of a This behaviour is considerably simpler than that of moments However, moments about the mean are also semi-invariant, so this property alone does not explain why cumulants are so natural for statistical purposes The notion of a cumulant can be traced to the work of Thiele (18??), who called them semi-invariants, but the moderm theory of cumulants and the associated k-statistics begins with the remarkable 1929 paper by Fisher Fisher used the term cumulative moment function for what we now call the cumulant generating function on account of its behaviour under convolution of independent random variables For the coefficients in the expansion, the term cumulant was suggested by Hotelling in a letter to Fisher, who approved of the coinage Let S = X + Y be the sum of two independent random variables The moment generating function of the sum is the product M S (ξ) = E(e ξ(x+y ) ) = E(e ξx e ξy ) = M X (ξ)m Y (ξ), and the cumulant generating function of the sum is the sum of the cumulant generating functions K S (ξ) = K X (ξ) + K Y (ξ) Consequently, the rth cumulant of the sum is the sum of the rth cumulants By extension, if X 1, X n are independent real-valued random variables, the rth cumulant of the sum is the sum of the rth cumulants If they are also identically distributed, the rth cumulant is nκ r, and the rth cumulant of the standardized sum n 1/2 (X 1 + +X n ) is n 1 r/2 κ r Provided that the cumulants are finite, all cumulants of order r 3 of the standardized sum tend to zero, which is a simple demonstration of the central limit theorem Good (195?) obtained an expression for the rth cumulant of X as the rth moment of the discrete Fourier transform of an independent and identically distributed sequence as follows Let X 1, X 2, be independent copies of X 3

4 with rth cumulant κ r, and let ω = e 2πi/n be a primitive nth root of unity The discrete Fourier combination Z = X 1 + ωx ω n 1 X n is a complex-valued random variable whose distribution is invariant under rotation Z ωz through multiples of 2π/n The rth cumulant of the sum is κ r nj=1 ω rj, which is equal to nκ r if r is a multiple of n, and zero otherwise Consequently E(Z r ) = 0 for integer r < n and E(Z n ) = nκ n 12 Multivariate cumulants Somewhat surprisingly, the relation between moments and cumulants is simpler and more transparent in the multivariate case than in the univariate case Let X = (X 1,, X k ) be the components of a random vector in R k In a departure from the univariate notation, we write κ r = E(X r ) for the components of the mean vector, κ rs = E(X r X s ) for the components of the second moment matrix, κ rst = E(X r X s X t ) for the third moments, and so on It is convenient notationally to adopt Einstein s summation convention in which ξ r X r denotes the linear combination ξ 1 X ξ k X k, the square of the linear combination is (ξ r X r ) 2 = ξ r ξ s X r X s a sum of k 2 terms, and so on for higher powers Technically speaking, the components of a vector X in V = R k are denoted by X 1,, X k, which is abbreviated to X r using a dummy superscript ranging over the index set [k] A linear functional ξ: V R is a vector in the dual space V of linear functionals The rth component of ξ is typically denoted by ξ r using subscripts for the coefficients, so that the value of ξ at X is a the real number ξ(x) = ξ 1 X ξ k X k which is abbreviated to ξ r X r In Einstein s notation, each repeated index should occur exactly twice, once as a subscript indexing the linear functional, and once as a superscript indexing the components of a vector in V The tensor product of X with Y is a vector or tensor in V 2 whose (r, s)-component is (X Y ) rs = X r Y s An arbitrary vector A in V 2 is a k k array of components A rs, which can be decomposed as the sum of a symmetric array and a skew-symmetric array A rs = (A rs + A sr )/2 + (A rs A sr )/2 = sym 2 (A) + alt 2 (A) 4

5 In this setting sym 2 and alt 2 are projections V 2 V 2, which are linear and complementary (the image of one is the kernel of the other) The dimensions are k(k + 1)/2 for sym 2 (V) and k(k 1)/2 for alt 2 (V) Note that (X X) rs = (X X) sr implies that the tensor product of a vector X V with itself is symmetric, so X X is a vector in the symmetric tensor product space sym 2 (V) The tensor product map X X X defines a transformation V sym 2 (V) from one vector space into another, and although 0 0, the transformation is not linear: (X + Y ) 2 is not equal to X 2 + Y 2 The image of the tensor product transformation is the set of rank-one [symmetric] tensors or matrices, and the span of these matrices is the whole space sym 2 (V) The moment generating function of [the distribution of] a random variable X taking values in V is a function M(ξ) = E(exp(ξ r X r )) on the dual space of linear functionals Its Taylor expansion is M(ξ) = 1 + ξ r κ r + 1 2! ξ rξ s κ rs + 1 3! ξ rξ s ξ t κ rst +, where each of the joint moments κ r = E(X r ), κ rs = E(X r X s ), κ rst = E(X r X s X t ), is a symmetric tensor in V, V 2, V 3 and so on The cumulants are defined as the coefficients κ r,s, κ r,s,t, in the Taylor expansion log M(ξ) = ξ r κ r + 1 2! ξ rξ s κ r,s + 1 3! ξ rξ s ξ t κ r,s,t + This notation does not distinguish first-order moments from first-order cumulants, but commas separating the superscripts serve to distinguish higherorder cumulants from moments: κ r,s = cum 2 (X r, X s ), κ r,s,t = cum 3 (X r, X s, X t ), Each superscript in this setting denote a vector component r [k], not a power Comparison of coefficients reveals that the each moment κ rs, κ rst, is a sum over partitions of the superscripts, each term in the sum being a product of cumulants: κ rs = κ r,s + κ r κ s κ rst = κ r,s,t + κ r,s κ t + κ r,t κ s + κ s,t κ r + κ r κ s κ t = κ r,s,t + κ r,s κ t [3] + κ r κ s κ t κ rstu = κ r,s,t,u + κ r,s,t κ u [4] + κ r,s κ t,u [3] + κ r,s κ t κ u [6] + κ r κ s κ t κ u 5

6 Each parenthetical number indicates a sum over distinct partitions having the same block sizes The fourth-order moment is a sum of 15 distinct cumulant products of which three are of type 2 2 with two blocks of size two, and six are of type 21 2 with three blocks: rs tu[3] = {rs tu, rt su, ru st} κ r,s κ t,u [3] = κ r,s κ t,u + κ r,t κ s,u + κ r,u κ s,t rs t u[6] = {rs t u, rt s u, ru s t, st r u, su r t, tu r s} κ r,s κ t κ u [6] = κ r,s κ t κ u + + κ t,u κ r κ s The blocks are unlabelled, so tu rs rs tu, and each block is a subset of the four elements or labels rstu, so rs tu = sr tu = rs ut = sr tu In the reverse direction, each cumulant is also a sum over partitions of the indices Each term in the sum is a product of moments, but with coefficient ( 1) ν 1 (ν 1)! where ν is the number of blocks: κ r,s = κ rs κ r κ s κ r,s,t = κ rst κ rs κ t [3] + 2κ r κ s κ t κ r,s,t,u = κ rstu κ rst κ u [4] κ rs κ tu [3] + 2κ rs κ t κ u [6] 6κ r κ s κ t κ u Partition notation serves one additional purpose It establishes moments and cumulants as special cases of generalized cumulants, which includes objects of the type κ r,st = cov(x r, X s X t ), κ rs,tu = cov(x r X s, X t X u ), and κ rs,t,u with incompletely partitioned indices These objects arise very naturally in statistical work involving asymptotic approximation of distributions They are intermediate between moments and cumulants, and have characteristics of both Every generalized cumulant can be expressed as a sum of certain products of ordinary cumulants Some examples are as follows: κ rs,t = κ r,s,t + κ r κ s,t + κ s κ r,t = κ r,s,t + κ r κ s,t [2] κ rs,tu = κ r,s,t,u + κ r,s,t κ u [4] + κ r,t κ s,u [2] + κ r,t κ s κ u [4] κ rs,t,u = κ r,s,t,u + κ r,t,u κ s [2] + κ r,t κ s,u [2] Each generalized cumulant is associated with a partition τ of the given set of indices For example, κ rs,t,u is associated with the partition τ = rs t u of four indices into three blocks Each term on the right is a cumulant product associated with a partition σ of the same indices The coefficient is one if the least upper bound σ τ has a single block, otherwise zero Thus, 6

7 with τ = rs t u, the product κ r,s κ t,u does not appear on the right because σ τ = rs tu has two blocks As an example of the way these formulae may be used, let X be a scalar random variable with cumulants κ 1, κ 2, κ 3, By translating the second formula in the preceding list, we find that the variance of the squared variable is var(x 2 ) = κ 4 + 4κ 3 κ 1 + 2κ κ 2 κ 2 1, reducing to κ 4 + 2κ 2 2 if the mean is zero Exercises Let V = R n, and let A be a tensor in V 2 with components A ij Show that sym 2 (A) ij = (A ij + A ji )/2 alt 2 (A) ij = (A ij A ji )/2 are linear projections V 2 V 2 Show also that the projections are complementary What are the dimensions of the image spaces sym 2 (V) and alt 2 (V)? 122 Let A be a tensor in V 3 with components A ijk Show that sym 3 (A) ijk = (A ijk + A jik + A kji + A ikj + A jki + A kij )/6 alt 3 (A) ijk = (A ijk A jik A kji A ikj + A jki + A kij )/6 res 3 (A) ijk = (2A ijk A jki A kij )/3 are linear projections V 3 V 3, ie, satisfying T (T (A)) = T (A) Show also that the projections are complementary in the sense that the kernel of each one is the direct sum of the images of the other two What are the dimensions of the image spaces sym 3 (V), alt 3 (V) and res 3 (V)? 13 Partition lattice 131 Poset A Boolean function S S {0, 1}, ie, a subset of S 2, is called a partial order if the subset or relationship ( ) is reflexive and transitive Reflexive means a a [is true] for every a S; transitive means that a b and b c implies a c To simplify matters, a third condition is imposed such that a b and b a together imply a = b; otherwise it is necessary to reduce the discussion to equivalence classes 7

8 The real line is a partially ordered set that is also a total order (either a b or b a for every pair) The real plane or complex plane ordered componentwise (a b if a 1 b 1 and a 2 b 2 ) is partially ordered but not completely ordered: there exist pairs for which a b and b a are both false For any set A each S 2 A is a collection of subsets of A, which is partially ordered by subset inclusion The set of subspaces of a vector space V is partially ordered by subspace inclusion The set of factorial models (factorial subspaces) generated by factors A, B, C is also partially ordered by subspace inclusion 132 Set partition Let n 1 be a positive integer, and let [n] = {1,, n} be a finite set A partition of the set [n] is a collection of disjoint non-empty subsets, called blocks, whose union is [n] The partition type is the set of block sizes counted with multiplicity Since the sum of the block sizes is n, the partition type is a partition of the integer n For example {{1, 3}, {2, 5}, {4}} and {{1, 5}, {2, 3}, {4}} are two distinct partitions of [5], usually abbreviated to and Since a partition is a set of subsets, the order of the blocks, and the order within blocks are ignored All told, there are 15 distinct partitions of [5] of the same type or A partition of [n] is also an equivalence relation B: [n] 2 {0, 1} In other words, B [n] 2 is a symmetric Boolean matrix that is also reflexive and transitive The matrix representations of and are = , = Evidently the expressions and determine the same subset B [5] 2, and therefore the same partition Let Pn be the set of partitions of [n] For n 5 the elements of Pn grouped by partition type are P1 : 1 P2 : 12, 1 2 P3 : 123, 12 3, 13 2, 23 1, P4 : 1234, [4], [3], [6],

9 P5 : 12345, [5], [10], [10], [15], [10], Thus, [3] = {12 34, 13 24, 14 23} P4 is the subset consisting of all distinct set partitions of type 2 2, and [15] P5 is the subset of 15 partitions of type 12 2 (five ways to choose the singleton, and three ways to split the remaining four into two pairs) Each subset of a given type is an orbit of the symmetric group [n] [n] acting on partitions Pn in the obvious way by permutation or re-labelling of elements When we say that B = is a partition of [5], we make no distinction between different representations: (i) as a subset B [5] 2 ; (ii) as the symmetric binary matrix displayed above with rows and columns indexed by [n]; (iii) as a Boolean function B: [n] 2 {0, 1}; (iv) as the set of disjoint non-empty subsets B = {{1, 5}, {2, 3}, {4}} The symbol #A applied to a set A means the number of its elements; the same symbol #B applied to a partition B denotes the number of blocks (which is the same as the rank of the matrix of B) 133 Sub-partition Let B, B be two partitions of the same finite set [n] If each block of B is a subset of some block of B we write B B, which is a partial order on Pn As subsets of the square [n] 2, B B if and only if B B The maximal partition is the one-block partition {[n]}, conventionally denoted by 1 or 1 n ; the minimal partition is the n-block partition by singletons, conventionally denoted by 0 or 0 n As subsets of the n-square, 1 n = [n] 2 is the entire square or the n n matrix whose components are all one; 0 n = diag([n] 2 ) is the diagonal subset or the identity matrix I n or the Kronecker function δ(b, B ) For every partition B we have 0 n B 1 n To each pair of partitions B, B Pn there corresponds a least upper bound B B = B B and a greatest lower bound B B = B B The greatest lower bound is the intersection B B of B and B as subsets of [n] 2 ; equivalently, the blocks of B B are the non-empty intersections of the blocks of B with the blocks of B The least upper bound is the intersection of all partitions that contain B B as subsets of [n] 2 For example, the least upper bound of and is The partition lattices P2 P4 are illustrated as graphs in Figure 1, with each partition as a node, and an edge joining two nodes B < B only if 9

10 Figure 1: Hasse diagrams for smaller partition lattices B is a parent of B, ie, there is no intermediate partition B such that B < B < B 134 Zeta and Möbius functions For any partially ordered set S, the partial order is encoded in the zeta function ζ(a, b) = 1 if a b and zero otherwise In particular, this implies ζ(a, a) = 1 for every a S If the elements of S are listed in non-decreasing order, then ζ is ab upper triangular matrix with unit values along the diagonal For example, the zeta function for P3 with elements listed in decreasing number of blocks is S = {1 2 3, 12 3, 13 2, 23 1, 123} ζ = 1 0 1, with all entries below the diagonal equal to zero, and also some of those above the diagonal 10

11 Let S be finite and let f be a real-valued function on S, ie, f R S Then F = ζ f is also a real-valued function, so ζ is a linear function R S R S In fact F (a) = (ζ f)(a) = ζ(b, a)f(b) = f(b) b b a is the cumulative sum of f-values over the subset b a If S is a finite lattice, it has a minimal element 0, and the sum is a sum over the lattice interval [0, a] The inverse function m(a, b) is also upper-triangular, and satisfies m(a, b)ζ(b, c) b S ζ(a, b)m(b, c) = δ(a, c) = b S and the sum may be restricted to the interval [a, c] = {b: a b c} In particular, for a < c, the sum over the interval [a, c] is zero m(b, c) = 0 = m(a, b) a b c a b c More explicitly, back-substitution gives m(a, a) = 1 for every a and m(a, c) = m(b, c) a<b c for a < c The Möbius function depends on the structure of the lattice For the partition lattice Pn, it is sufficient to know that the value relative to the maximal one-block partition is m(σ, 1 n ) = ( 1) #σ 1 (#σ 1)! = ( 1) #σ 1 Γ(#σ), the gamma function with alternating sign, independent of the block sizes More generally, if σ is a partition of [n], the restriction of σ to b [n] is a partition of the subset b denoted by σ[b] For example, the restriction of σ = to b = {2, 3, 4, 5} is σ[b] = consisting of two complete blocks and one partial block of σ In general, the restriction σ[b] is not a subset of the blocks of σ For any interval [σ, τ] with σ τ, each block b τ is the union of certain blocks σ[b] σ, so the restriction σ[b] is a subset consisting of certain blocks of σ (with no partial blocks) The Möbius function is m(σ, τ) = m(σ[b], 1 b ) = ( 1) #σ #τ Γ(#σ[b]) b τ b τ for σ τ, and zero otherwise 11

12 14 Cumulants and generalized cumulants The relationship between moments and cumulants is most conveniently described by summation over the partition lattice Let [n] = {1,, n} be the index set For any subset b [n] let µ b = E( i b Y i ) be the moment and let κ b be the joint cumulant or order #b of the variables Y [b] = {Y i : i b} Then µ [n] = κ [n] = σ Pn b σ κ b σ Pn( 1) #σ 1 (#σ 1)! b σ More generally, for any partition τ Pn, the interval [0, τ] is isomorphic with the Cartesian product b τ [0, b] of smaller lattices Consequently, the moment product F (τ) = b τ µb is expressible as a sum of cumulant products f(σ) = b σ κb over partitions σ in [0, τ] F (τ) = µ b = κ b = f(σ) b τ σ τ b σ σ τ f(τ) = κ b = m(σ, τ) µ b b τ σ τ b σ µ b Now consider a different sort of mixed cumulant of order less than the number of variables A partition τ splits the random variables Y 1,, Y n into disjoint subsets, one subset Y [b] for each block b τ Now consider the joint cumulant κ(τ) of order k = #τ of the variables X 1 = Y i,, X k = Y i i b 1 i b k By definition, κ(τ) = cum k (X 1,, X k ) = ς τ m(ς, 1 n )F (ς) where F (ς) is the moment product over the blocks of ς Now substitute the expression F (ς) = σ ς f(σ) for moment products in terms of cumulant products κ(τ) = m(ς, 1 n ) f(σ) ς τ σ ς = σ Pn f(σ) ς τ ς σ 12 m(ς, 1 n )

13 κ(τ) = = f(σ) σ Pn σ:τ σ=1 n b σ ς τ σ κ b m(ς, 1 n ) This result is fundamental for computing means, variances and higherorder cumulants of quadratic forms and higher-order polynomial functions of random variables To understand how it works, we list, for various small integers n and selected partitions τ Pn, the subset of partitions σ Pn satisfying the connectivity condition σ τ = 1 n A short list of some connected set-partitions τ {σ : σ τ = 1} , 12 3, 13 2, 23 1, , 13 2, , [3], [3], [3] , [4], [2], [4] , [2], [2] excluding partitions having a singleton block , [6], [9], [9], [9], [6] , [12], [6], [4] [8] For a scalar random variable Y having zero mean, it follows from lines 5, 7, 8 of the table that var(y 2 ) = κ 4 + 2κ 2 2 var(y 3 ) = κ κ 4 κ 2 + 9κ κ 3 2 cum 3 (Y 2 ) = κ κ 4 κ κ κ Gaussian moments and the Isserlis-Wick formulae Suppose that the random vector ψ = (ψ 1,, ψ n ) is zero-mean Gaussian in R n with covariance matrix K whose (i, j) component is K i,j We ask for the joint cumulant of order n of the n squared variables κ(τ) = cum n ( ψ 1 2,, ψ n 2 ) 13

14 If the component variables are all equal, ψ 1 = = ψ n, then ψ 2 is distributed as κ 2 χ 2 1 The cumulant generating function is log(1 2κ 2ξ)/2, and the cumulants are cum n ( ψ 2 ) = 2 n 1 (n 1)!κ n 2 The moment generating function is (1 2κ 2 ξ) 1/2, so the moment of order n is proportional to the ascending factorial ( 1/2) n ( 2κ 2 ) n = (1/2) n (2κ 2 ) n = (2n)! κn 2 2 n n! For the more general setting, τ is a partition of the set [2n] of type 2 n τ = 1, 1 2, 2 n, n in which [n] and [n] are duplicate copies of the same index set, and ψ r 2 gives rise to two matching indices r, r, distinguished by primes but numerically equal Since the only non-zero cumulants are of order two, the expression for κ(τ) is a sum over all partitions σ of type 2 n such that σ τ = 1 2n Consider a cyclic permutation π: [n] [n], with the same permutation also acting on the second copy π: [n] [n], ie, 1 π(1) π 2 (1) π n 1 (1) π n (1) = 1 1 π(1 ) π 2 (1 ) π n 1 (1 ) π n (1 ) = 1 We associate with π a partition σ of [2n] as follows σ = 1π(1 ) π(1)π 2 (1 ) π 2 (1)π 3 (1 ) π n 1 (1)1 so that each block contains a pair j, π(j) with j [n] and π(j) [n] Clearly, σ τ = 1 2n, and the contribution of σ to κ(τ) is n K j,π(j) j=1 There are 2 n 1 essentially distinct ways of assigning the primes to one element in each block of τ, so the total contribution to the joint cumulant is κ(τ) = 2 n 1 n K j,π(j) = 2 1 cyp(2k), π:#π=1 j=1 where the sum runs over (n 1)! permutations π: [n] [n] having a single cycle 14

15 The expected value of the n-fold product µ(τ) = E ψ j 2 is the sum over partitions of [n] of cumulant products µ(τ) = κ(τ[b]) σ Pn b σ = 2 #b 1 cyp(k[b]) σ Pn b σ = n 2 n #σ κ j,σ(j) σ Π n j=1 = per 1/2 (2K) = 2 n per 1/2 (K) This derivation implies that per 1/2 (K) 0 for all positive semi-definite symmetric matrices, and a simple extension shows that per α/2 (K) 0 for all positive integers α For any real number α, the α-permanent per α (K) of a square matrix K of order n is the sum over n! permutations per α (K) = cyp(k) = α #σ n K j,σ(j) σ:[n] [n] j=1 n Σ:#σ=1 j=1 K j,σ(j) = lim α 0 α 1 per α (K) Note that per α (K) is a polynomial of degree n in α; it is homogeneous of total degree n in K In addition, per 1 (K) = ( 1) n det(k) In applications to quantum mechanics, ψ arises as a wave function, which is regarded as a zero-mean complex-valued Gaussian process Each component is distributed symmetrically with respect to rotation in the complex plane, and the joint distribution is invariant with respect to scalar multiplication by a unit complex number For this setting, all the algebraic joint moments and cumulants of ψ are zero The only non-zero cumulants are those of the form E(ψ r ψs ) = K rs = K sr, and higher-order products involving an equal number of conjugated and non-conjugated terms The argument given above for Gaussian cumulants applies equally to complex-valued Gaussian process The details are a little simpler in the complex case because the label-assignment factor 2 n 1 does not arise: [n] is the set of indices, [n ] is the conjugated copy, and there are no symmetries from switching elements As a result, κ(τ) = cum n ( ψ 1 2,, ψ n 2 ) = cyp(k) µ(τ) = E( ψ 1 2 ψ n 2 ) = per 1 (K) = per(k) 15

16 This derivation implies that per(k) 0 for all positive semi-definite Hermitian matrices, and a simple extension shows that per α (K) 0 for all positive integers α Exercises Let ψ be a complex-valued zero-mean process on some space with covariance function K(x, x ) = E(ψ(x) ψ(x )) Let x = {x 1,, x n } and x = {x 1,, x n} be two subsets of n points in the domain, possibly but not necessarily equal Use the matching argument given above to compute the joint cumulants cum 2n (ψ(x 1 ),, ψ(x n ), ψ(x 1),, ψ(x n)), cum n (ψ(x 1 ) ψ(x 1),, ψ(x n ) ψ(x n)), and the joint moment ( E ψ(x) ) ψ(x ) x x x x Both of these are complex numbers 152 Explain how the answers to the preceding exercise are modified if ψ is a real-valued Gaussian process 16 Exponential families 2 Generating functions and formal power series 21 The vector spaces Seq and Seq n To any sequence of scalars f = (f 1, f 2, ), in which f n is a scalar, we may associate a formal polynomial in a variable t by f(t) = f 1 t + f 2 t 2 /2! + f 3 t 3 /3! + + f n t n /n! + Thus f(t) is the generating function for the sequence f; we say it is a generating function of exponential type because f n is the coefficient of t n /n! in the polynomial In particular, the constant sequence with f n = 1 corresponds to the function exp(t) 1, while the sequence m n = ( 1) n 1 (n 1)! corresponds to the inverse function m(t) = log(1 + t) Some further examples are 16

17 given below Function Coefficient sequence f n f(t) (f 1, f 2, ) (1 t) 1 1 n! log(1 t) (n 1)! exp(t) 1 1 exp(e t 1) 1 B n (Bell number) log(1 + t) ( 1) n 1 (n 1)! (1 + t) α 1 α n = α(α 1) (α n + 1) Whether f is regarded as a sequence of scalars or as a formal polynomial, the set of such sequences or formal polynomials is denoted by Seq To say that f is a formal polynomial is to imply that certain operations on sequences f, g are carried out as if these were polynomials The simplest such operations are addition and scalar multiplication Thus if α is a scalar, αf is a sequence whose components are (αf 1, αf 2, ), and the associated polynomial is (αf)(t) = αf(t) Likewise, addition operates component-wise, so that f + g = (f 1 + g 1, f 2 + g 2,, f n + g n, ) (f + g)(z) = f(z) + g(z) The set of sequences with these operations of addition and scalar multiplication is a vector space denoted by Seq + The restriction to finite sequences of n components is a vector space of dimension n denoted by Seq + n 22 Composition of series Let g(ξ) be a formal power series of exponential type in the variable ξ g(ξ) = g r ξ r /r! r=1 Evidently each monomial g 2 (ξ), g 3 (ξ), is also a formal power series The coefficient of ξ n /n! in the monomial g 2 is g 2 (ξ)[ξ n /n!] = r+s=n n! r! s! g rg s where the sum runs over ordered pairs (r, s) in the square [n] 2 subject to the restriction that they add to n If n is odd, every pair occurs twice, once as 17

18 (r, s) and once as (s, r); if n is even, every pair occurs twice except for the pair on the diagonal with r = s = n/2 For example, if n = 6, the sum runs over pairs (1, 5), (2, 4), (3, 3), (4, 2), (5, 1) with transpose pairs contributing equally For combinatorial accounting purposes, it is more convenient to combine the symmetric pairs, in effect to ignore the order of the two parts Instead of summing over strictly positive integer vectors adding to n (compositions of n), we combine similar terms and sum over integer partitions having two parts For n = 6, the total contribution of the {1, 5} terms is 2! 6 g 1 g 5, the total contribution of the {2, 4} terms is 2! 15g 2 g 4 and the contribution of the (3, 3) term is 1! 20g3 2 = 2! 10g2 3 The number of partitions of [6] of type is 6, the number of type is 15, and the number of type 3 2 is 10 (not 20) For that reason, is most convenient to write the quadratic monomial as a sum over partitions of the set [n] into two blocks g 2 (ξ)[ξ n /n!] = r+s=n n! r! s! g rg s = 2! σ Pn:#σ=2 b σ g #b, where σ is a partition of [n], #σ is the number of elements (blocks), b σ is a block, and #b is the block size The extension to higher-order monomials follows the same pattern g 3 (ξ)[ξ n /n!] = g k (ξ)[ξ n /n!] = r+s+t=n r 1 + +r k =n n! r! s! t! g rg s g t = 3! σ P (3) n g #b b σ n! r 1! r k! g r 1 g rk = k! σ P (k) n g #b, b σ where P n (k) Pn is the subset of partitions having k blocks The first sum runs over k-part compositions of n, which are strictly positive integer vectors r = (r 1,, r k ) whose components add to n Two compositions consisting of the same components in a different order determine the same integer partition If the components of r are distinct integers, there are k! permutations giving rise to the same product, and n!/(r 1! r k!) is the number of partitions of the set [n] having k blocks of sizes r 1,, r k Otherwise, if some of the components of r are equal, the number of compositions giving rise to a particular product g r1 g rk is reduced by the product of the factorials of the multiplicities For example, if n = 10 and k = 4, the point r = (1, 2, 3, 4) gives rise to the product g 1 g 2 g 3 g 4, and there are k! compositions giving rise to the same product However, if n = 8 and k = 5, the 18

19 composition r = (2, 2, 2, 1, 1) gives rise to g2 3g2 1, but there are only k!/(3!2!) similar compositions giving rise to the same product To each integer partition m = 1 m 1 2 m2 n mn having k = m parts, there correspond k!/(m 1!m 2! ) compositions with parts r 1,, r k in some order In the expansion of g k (ξ), each composition r of n has a combinatorial factor n!/r! This means that the product associated with the integer partition m has a combinatorial factor g m = g m 1 1 gm 2 2 gn mn n! r 1! r k! k! m 1! m n! = n! k! j j!m j m j!, which is k! times the number of set partitions of [n] corresponding to the given integer partition In other words, these awkward combinatorial factors are automatically accommodated by summation over set partitions: g k (ξ) k! = n k ξ n n! σ P (k) n g #b Now let f be another formal power series of exponential type The compositional product fg is a sequence whose polynomial is f(g(ξ)) Note that fg is ordinarily different from gf, so the product operation is not commutative However, it is associative, so fgh is well defined By definition, the series expansion of the composition (f g)(ξ) is a linear combination of the monomials g k (ξ)/k! with coefficients f k, giving (fg)(ξ) = = k=1 n=1 (fg) n = σ Pn f k g k (ξ) k! ξ n n! σ Pn b σ f #σ f #σ g #b With this operation, the space of sequences or formal exponential generating functions is a vector space with a non-commutative compositional product (f, g) fg that is linear in the first argument but not in the second Exercises Use Faà di Bruno s formula to derive the coefficient (fg) n in the Taylor expansion of the composition b σ b σ g #b 19

20 23 Inverse function The unit sequence e = (1, 0, 0, ), ie, e(ξ) = ξ is the compositional identity satisfying eg = ge = g for every g If g is given with g 1 0, the compositional equation fg = e has a left-inverse solution, satisfying f 1 = g1 1, and recursively f n = g n 1 1 k<n for n > 1 This produces f 1 = g 1 1, f 2 g 3 1 = g 2, f 3 g 5 1 = g 1 g 3 + 3g 2 2 f 4 g 7 1 = g 4 g g 1 g 2 g 3 15g 3 2 σ P (k) n f k f 5 g 9 1 = g 5 g g 4 g 2 g g 2 3g g 3 g 2 2g g 4 2 f 6 g 11 1 = g 6 g g 5 g 2 g g 4 g 3 g g 4 g 2 2g g 2 3g 2 g g 3 g 3 2g 1 945g 5 2 Conversely, the equation gf = e has a right-inverse solution satisfying f 1 = g1 1, and subsequently fn = g1 1 g k f #b 1<k n σ P (k) n b σ b σ g #b for n > 1 This sequence produces the same solution, which is k 1 f k g1 k = ( 1) ν g ν ν=1 1 σ P ( ν) b σ k+ν 1 g #b for k > 1, where the sum runs over partitions of [k + ν 1] having ν nonsingleton blocks (Why?) Note that if g k = 1 for every k, then f = (1, 1, 2, 6, 24, 120, ), and the extension to general k 1 is f k = ( 1) k 1 (k 1)!; conversely, if f k = 1 for every k, then g k = ( 1) k 1 (k 1)! For f(ξ) = log(1 + ξ), the coefficients are f n = ( 1) n 1 (n 1)!, in which case log(1 + g(ξ))[ξ n /n!] = σ Pn( 1) n 1 (n 1)! b σ is the expression for the cumulant of order n in terms of moment products of total order n Conversely, for f(ξ) = exp(ξ) 1 with f n = 1, we obtain the expression for the moment of order n in terms of cumulant products of total order n 20 g #b

21 24 The vector spaces Seq and Seq n Consider now a different operation (convolution) on the space of sequences, one that is more readily understood as an operation on polynomials than as an operation on sequences For any scalar α and sequences f, g, define the sequences f g and α g by (f g)(ξ) = f(ξ) + g(ξ) + f(ξ)g(ξ), (α g)(ξ) = (1 + g(ξ)) α 1 Equivalently, 1+(f g)(ξ) = (1+f(ξ))(1+g(ξ)), showing that this operation is commutative: f g = g f In addition 1 f = f and f f = 2 f for every f, the zero vector is the zero sequence, and so on In terms of the augmented sequences with f 0 = g 0 = 1, the components of f g and α g are n ( ) n (f g) n = f j g n j j j=0 (α g) n = σ Pn α #σ b σ where σ Pn is a partition of [n] containing #σ blocks, and for each block b σ, #b is the number of elements The descending factorial product α r = α(α 1) (α r + 1) is the rth Taylor coefficient associated with the function (1 + ξ) α 1 The space of sequences with these operations is a vector space denoted by Seq, the vector-space properties being more obvious from the polynomial representation than the sequence representation The restriction to finite sequences of n components, or to polynomials of degree less than or equal to n, is an n-dimensional vector space denoted by Seq n, which may be identified with the subspace of Seq having zero components for r > n In Seq n the product 1 + (f g)(ξ) is the restriction of the polynomial product (1 + f(ξ))(1 + g(ξ)) to terms of degree n, so that (f g) r = 0 for r > n Likewise for α f This restriction by polynomial degree is a linear projection Seq Seq n, ie, it commutes with vector-space operations By a homomorphism, we mean a linear transformation T : Seq Seq that also acts on the finite-dimensional restrictions, ie T Seq n Seq n The set of such linear transformations is itself a vector space, closed under addition and scalar multiplication It is readily seen that the logarithmic transformation acting on polynomials 1+f(ξ) log(1+f(ξ)) carries polynomials in Seq to polynomials in Seq, and also carries Seq n to Seq n preserving 21 g #b

22 vector-space operations It is also apparent that there is essentially only one such transformation Thus Hom(Seq, Seq) is the one-dimensional space of linear transformations that are scalar multiples of (T g) 1 = g 1 (T g) 2 = g 2 g 2 1 (T g) 3 = g 3 3g 2 g 1 + 2g1 3 (T g) n = m #σ σ Pn where m r = ( 1) r 1 (r 1)! is the coefficient defining the logarithmic function m(t) = log(1+t) This T is of course, the transformation that generates cumulants from moments Despite the occurrence of multiplicative terms, T is a linear transformation Seq Seq on vector spaces b σ g #b 3 Approximation of distributions 31 Edgeworth approximation 32 Saddlepoint approximation 4 Samples and sub-samples A function f: R n R is symmetric if f(x 1,, x n ) = f(x P(1),, x P(n) ) for each permutation P of the arguments For example, the total T n = x x n, the average T n /n, the min, max and median are symmetric functions, as are the sum of squares S n = x 2 i, the sample variance s2 n = (S n T 2 n/n)/(n 1) and the mean absolute deviation x i x j /(n(n 1)) A vector x in R n is an ordered list of n real numbers (x 1,, x n ) or a function x: [n] R where [n] = {1,, n} For m n, a 1 1 function ϕ: [m] [n] is a sample of size m, the sampled values being xϕ = (x ϕ(1),, x ϕ(m) ) All told, there are n(n 1) (n m + 1) distinct samples of size m that can be taken from a list of length n A sequence of functions f n : R n R is consistent under subsampling if, for each f m, f n, f n (x) = ave ϕ f m (xϕ), where ave ϕ denotes the average over samples of size m For m = n, this condition implies only that f n is a symmetric function Although the total and the median are both symmetric functions, neither is consistent under subsampling For example, the median of the numbers 22

23 (0, 1, 3) is one, but the average of the medians of samples of size two is 4/3 However, the average x n = T n /n is sampling consistent Likewise the sample variance s 2 n = (x i x) 2 /(n 1) with divisor n 1 is sampling consistent, but the mean squared deviation (x i x n ) 2 /n with divisor n is not Other sampling consistent functions include Fisher s k-statistics, the first few of which are k 1,n = x n, k 2,n = s 2 n for n 2, k 3,n = n (x i x n ) 3 /((n 1)(n 2)) k 4,n = defined for n 3 and n 4 respectively For a sequence of independent and identically distributed random variables, the k-statistic of order r n is the unique symmetric function such that E(k r,n ) = κ r Fisher (1929) derived the variances and covariances The connection with finite-population sub-sampling was developed by Tukey (1954) 23