Covariance and Correlation. Def: If X and Y are JDRVs with finite means and variances, then. Example Sampling

Transcription

1 Definitions Properties E(X) µ X Transformations Linearity Monotonicity Expectation Chapter 7 xdf X (x). Expectation Independence Recall: µ X minimizes E[(X c) ] w.r.t. c. The Prediction Problem The Problem: Let X Y be JDRVs with finite means variances: find a b to minimize MSE E[(Y ax b) ] The Solution: For any a, (*) is minimized by in which case b E(Y ax) µ Y aµ X, MSE E[(Ỹ a X) ], Ỹ Y µ Y, X X µ X. ( ) MSE E(Ỹ ) ae( XỸ ) + a E( X ) σ Y aσ XY + a σ X, σ XY E( XỸ ) E[(X µ X )(Y µ Y )]. Here d da MSE σ XY + aσx. MSE is minimum when a σ XY σ X, b µ Y aµ X, if σx > 0. With this a b, Ŷ ax + b is called the best linear predictor of Y. Note: The minimum MSE is σ Y aσ XY + a σ X σ Y σ XY σ X. In this example Galton s Data X father s height, Y son s height, µ X 68, µ Y 69, σ X σ Y, σ XY.5. a, b , Ŷ 35 + X 69 + (X 68). Regression Effect:

2 Covariance Correlation Def: X Y are JDRVs with finite means variances, σ XY E[(X µ X )(Y µ Y )] is called the covariance between X Y ; ρ ρ XY σ XY σ X σ Y is called correlation between X Y Interpretation: The minimum MSE is σ Y σ XY σ X σy [ ρ ]. Identity: σ XY E(XY ) µ X µ Y. Special Cases: a) σ XX σ X. b) X Y, σ XY 0. Sampling Suppose that a SRS of n is drawn w.o.r. from R red N R white tickets; let for i,. p R/N, A i {red on the i th draw, X i Ai E(X i ) P (A i ) R N p, V ar(x) pq, E(X X ) R(R ) N(N ), Note: Measures of dependence. σ E(X X ) µ µ R(R ) N(N ) ( R N ) NR(R ) (N )R N (n ) R(N R) N (N ) pq N. pq/(n ) ρ pq N. X Unif[, ], Y X, E(X) xdx 0, E(XY ) E(X 3 ) x 3 dx 0. σ XY E(XY ) E(X)E(Y ) 0 0 0, but X Y are not independent.

3 Linear Functions X ax + b, Y cy + d, E(X ) ae(x) + b, E(Y ) ce(y ) + d, C(X, Y ) E[(X µ X )(Y µ Y )] ρ X Y E[ac(X µ X )(Y µ Y )] acc(x, Y ). ac ac ρ XY ±ρ XY Sums Theorem. Let X,, X m, Y,, Y n be JDRVs with finite means variances, let Proof. S X + + X m, T Y + + Y n. E(S) E(X ) + + E(X m ), E(T ) E(Y ) + + E(Y n ), C(S, T ) m i j n C(X i, Y j ). Note: σ X σ X,X a σ X. Corollary D (S) The Variance of Sum m i D (X i ) + i j Proof. D (S) C(S, S). C(X i, X j ). Def: X,, X m are uncorrelated if C(X i, X j ) 0 for all i j. Corollary. X,, X m are uncorrelated, D (S) D (X ) + + D (X m ). In particular, (*) holds if X,, X m are independent. : Binomial. ( ) Suppose that The Signal Plus Noise Problem X Y + Z, Y Z E(Z) 0. E(Y ) E(X), σ X σ Y + σ Z, σ XY σ Y Y + σ Y Z σ Y, ρ σ XY σ X σ Y Best Linear Predictor: Find σ Y. σ Y + σz Ŷ ( ρ )µ Y + ρ X.

4 Conditional Expectation X, Y f f X (x) > 0, E(Y X x) y Y yf Y X (y x), in the discrete case, or E(Y X x) yf Y X (y x)dy in the continuous case, f Y X (y x) f(x, y) f X (x), provided that the sum or integral converges. In a bridge game, South has five spades. How many spades does North have? Let X #spades in South s h for y 0,, 8. Y #spades in North s h ( 8 3 ) y)( 3 y f Y X (y X 5) ) ( 39 3 E(Y X 5) 3 ( 8 39 ). Y Unif[0, ] X Y Unif[0, y] what is E(Y X )? Here 0 < y for 0 < x y. for 0 < x y, f X (x) f Y (y) f X Y (x y) y f(x, y) f Y (y)f X Y (x y) y x dy y log(y) yx log(x), f Y X (y x) y log(x), E(Y X ) yf Y X (y )dy dy log( ) log( ) log()

5 Smoothing Notation: Write E(Y X) when x is replaced by X. Theorem. Y has a finite expectation, Proof. See the text. E(Y ) E[E(Y X)]. : The Trapped Miner. Two doors: One leads to safety after 3 hours; Two leads back to the mine after 5. Let Y be the time required to get back. E(Y X ) 3, E(Y X ) the miner chooses a door at rom, E(Y ) E(Y X ) + E(Y X ) 5.5. Moment Generating Functions X F, M(t) E(e tx ) e tx df (x) is called the moment generating function of X / or F, provided that it converges in some non-degenerate interval. : Exponential. X Exp(λ), for M(t) λ 0 0 e tx λe λx dx e (λ t)x dx λ e (λ t)x x0 λ t λ λ t, t < λ. Poisson. for all < t <. Normal. for < t <. Other s See The Text X Poisson(λ), M(t) e λ(et ) X Normal(µ, σ ), M(t) e µt+ σ t Moments the MGF Moments: Recall µ k E(X k ) x k df (x). Theorem. M(t) < for t < h for some h > 0, M(0), M (0) µ, M (0) µ, M (k) (0) µ k, for all k 0,,,.

6 Exponential Proof-Outline-The Discrete Case. X has PMF f, M(t) x X M(0) x X e tx f(x), f(x), M (t) x X xe tx f(x), M (0) x X xf(x) µ, for k,,. X Exp(λ), M(t) λ λ t, M λ (t) (λ t), M λ (t) (λ t) 3,, M (k) k!λ (t) (λ t). k+ µ k k! λ k Sums Let Corollary m(t) log[m(t)]. m (t) M (t) M(t), m (t) M(t)M (t) M (t) M(t). m (0) M (0) M(0) µ m (0) µ µ σ. Cumulants: κ j m (j) (0). Theorem. X,, X n are independent with MGFs M,, M n, the MGF of is S X + + X n M S (t) M (t) M n (t). Proof. We have M S (t) E(e ts ) E( e tx i ) i E(e tx i ) i i M i (t). Note: Product, not convolution.

7 Unicity Theorem: M X (t) M Y (t) for all t in some non-degerate interval, F X (z) F Y (z) for all z.. M(t) cosh(t) et + e t, P [X ±]. Normal X,, X n are independent since X i ind Normal(µ i, σ i ), S Normal(µ, σ ), µ µ + + µ n, σ σ + + σ n, M S (t) M i (t) i i e µ it+ σ i t e µt+ σ t.