Soc 709 Lec 2 Inferences from Regression

Transcription

1 Soc 709 Lec 2 Inferences from Regression Ted Mouw tedmouw@ .unc.edu Department of Sociology University of North Carolina, Chapel Hill January 2, 2008

2 Outline Basics Properties of the regression line breview of estimating the coecients Variance of b 1 and b 0 The variance of b 0 Sampling distribution of b 1 Inferences concerning b 1 Partitioning the total sum of squares

3 Properties of the regression line breview of estimating the coecients Reading Weisberg, Chapter 2

4 Properties of the regression line breview of estimating the coecients Survey Collect data on education and income of parents to use in lab 2

5 Properties of the regression line breview of estimating the coecients properties of the regression line (from NWK 2.6) 1. e i = 0 (this will also be satised by any line passing throught the point x, ȳ) 2. e 2 i is at a minimum 3. ŷ i = y i 4. x i e i = 0 5. y i e i = 0 6. the regression line always passes throught the point x, ȳ

6 Properties of the regression line breview of estimating the coecients proof of #4 x i e i = x i (y i b 0 b 1 x i ) = (x i y i x i (Ȳ b 1 X ) b1 x 2 i ) v = [x i (y i ȳ) b 1 x i (x i x)] = [(x i x)(y i ȳ) b 1 (x i x) 2 ] = 0

7 Properties of the regression line breview of estimating the coecients Review From the last lecture, we know that B 1 = (x x)(y i ȳ) = S (x i x) 2 XY S xx and B 0 = Ȳ B 1 X 2 i s 2 = RSS = e = N 2 N 2 MSE bs 2 is our estimate of the variance of ε i, the error term of the regression

8 Properties of the regression line breview of estimating the coecients Variance of the error term We are assuming that the error term is normally distributed with unknown variance, i.e., ε i N(0,σ 2 ) this assumption is not important for the point estimates of the coecients, but it is important for inferences about the variances of the coecients.

9 variance of b 1 The estimated variance of B 1 is s 2 {B 1 } = s 2 Let's think about this formula: s 2 = (1) S xx (x i x) 2 uncertainty in the model comes from the residuals, e i. larger residuals mean more uncerntainty about our estimate of the slope of the regression line, b 1. the eect of the residuals is relative to the overall variation of x, S xx.

10 Proof start with the formula for b 1, b 1 = (x x)(y ȳ) = k i Y i where k i = x i x (x i x) 2 (x x) 2 k i is xed, the variation in b i comes from y i via the error term ε i k 2 = i x i x x i x 1 = (x i x) 2 (x i x) 2 (x i x) 2 Hence, σ 2 {b 1 } = σ 2 { k i y i } = k 2σ 2 {y i i } = σ 2 1 (x i x) 2

11 Proof start with the formula for b 1, b 1 = (x x)(y ȳ) = k i Y i where k i = x i x (x i x) 2 (x x) 2 k i is xed, the variation in b i comes from y i via the error term ε i k 2 = i x i x x i x 1 = (x i x) 2 (x i x) 2 (x i x) 2 Hence, σ 2 {b 1 } = σ 2 { k i y i } = k 2σ 2 {y i i } = σ 2 1 (x i x) 2

12 Proof start with the formula for b 1, b 1 = (x x)(y ȳ) = k i Y i where k i = x i x (x i x) 2 (x x) 2 k i is xed, the variation in b i comes from y i via the error term ε i k 2 = i x i x x i x 1 = (x i x) 2 (x i x) 2 (x i x) 2 Hence, σ 2 {b 1 } = σ 2 { k i y i } = k 2σ 2 {y i i } = σ 2 1 (x i x) 2

13 Proof start with the formula for b 1, b 1 = (x x)(y ȳ) = k i Y i where k i = x i x (x i x) 2 (x x) 2 k i is xed, the variation in b i comes from y i via the error term ε i k 2 = i x i x x i x 1 = (x i x) 2 (x i x) 2 (x i x) 2 Hence, σ 2 {b 1 } = σ 2 { k i y i } = k 2σ 2 {y i i } = σ 2 1 (x i x) 2

14 Proof start with the formula for b 1, b 1 = (x x)(y ȳ) = k i Y i where k i = x i x (x i x) 2 (x x) 2 k i is xed, the variation in b i comes from y i via the error term ε i k 2 = i x i x x i x 1 = (x i x) 2 (x i x) 2 (x i x) 2 Hence, σ 2 {b 1 } = σ 2 { k i y i } = k 2σ 2 {y i i } = σ 2 1 (x i x) 2

15 Proof start with the formula for b 1, b 1 = (x x)(y ȳ) = k i Y i where k i = x i x (x i x) 2 (x x) 2 k i is xed, the variation in b i comes from y i via the error term ε i k 2 = i x i x x i x 1 = (x i x) 2 (x i x) 2 (x i x) 2 Hence, σ 2 {b 1 } = σ 2 { k i y i } = k 2σ 2 {y i i } = σ 2 1 (x i x) 2

16 Proof start with the formula for b 1, b 1 = (x x)(y ȳ) = k i Y i where k i = x i x (x i x) 2 (x x) 2 k i is xed, the variation in b i comes from y i via the error term ε i k 2 = i x i x x i x 1 = (x i x) 2 (x i x) 2 (x i x) 2 Hence, σ 2 {b 1 } = σ 2 { k i y i } = k 2σ 2 {y i i } = σ 2 1 (x i x) 2

17 The variance of b 0 the estimated variance of b 0 is: s 2 {b 0 } = s 2 [ 1 x n + 2 (x i x) ] (2) 2

18 sampling distribution of b 1 (NWK p. 67) given our assumption about the disribution of the error term, b 1 is normally distributed (why?) with mean β 1 and variance σ 2 {b 1 }. I.e., b 1 N(β 1,σ 2 {b 1 }) So, we can convert the distribution of b 1 to a standard normal variable: b 1 β 1 σ{b 1 } N(0,1) (3)

19 sampling distribution of b 1 (NWK p. 67) given our assumption about the disribution of the error term, b 1 is normally distributed (why?) with mean β 1 and variance σ 2 {b 1 }. I.e., b 1 N(β 1,σ 2 {b 1 }) So, we can convert the distribution of b 1 to a standard normal variable: b 1 β 1 σ{b 1 } N(0,1) (3)

20 sampling distribution of b 1 (NWK p. 67) given our assumption about the disribution of the error term, b 1 is normally distributed (why?) with mean β 1 and variance σ 2 {b 1 }. I.e., b 1 N(β 1,σ 2 {b 1 }) So, we can convert the distribution of b 1 to a standard normal variable: b 1 β 1 σ{b 1 } N(0,1) (3)

21 sampling distribution of b 1 (NWK p. 67) given our assumption about the disribution of the error term, b 1 is normally distributed (why?) with mean β 1 and variance σ 2 {b 1 }. I.e., b 1 N(β 1,σ 2 {b 1 }) So, we can convert the distribution of b 1 to a standard normal variable: b 1 β 1 σ{b 1 } N(0,1) (3)

22 sampling distribution of b1 continued we don't know the true variance, so we used the estimated variance s 2 {b 1 }. with N-2 degrees of freedom. Big question: why is this result so important? Why N-2? Why a t-distribution? b 1 β 1 s{b 1 } t (4)

23 sampling distribution of b1 continued we don't know the true variance, so we used the estimated variance s 2 {b 1 }. with N-2 degrees of freedom. Big question: why is this result so important? Why N-2? Why a t-distribution? b 1 β 1 s{b 1 } t (4)

24 sampling distribution of b1 continued we don't know the true variance, so we used the estimated variance s 2 {b 1 }. with N-2 degrees of freedom. Big question: why is this result so important? Why N-2? Why a t-distribution? b 1 β 1 s{b 1 } t (4)

25 sampling distribution of b1 continued we don't know the true variance, so we used the estimated variance s 2 {b 1 }. with N-2 degrees of freedom. Big question: why is this result so important? Why N-2? Why a t-distribution? b 1 β 1 s{b 1 } t (4)

26 sampling distribution of b1 continued we don't know the true variance, so we used the estimated variance s 2 {b 1 }. with N-2 degrees of freedom. Big question: why is this result so important? Why N-2? Why a t-distribution? b 1 β 1 s{b 1 } t (4)

27 sampling distribution of b1 continued we don't know the true variance, so we used the estimated variance s 2 {b 1 }. with N-2 degrees of freedom. Big question: why is this result so important? Why N-2? Why a t-distribution? b 1 β 1 s{b 1 } t (4)

28 condence internvals for b 1 reference: NWK Table A.1, Cumulative Probabilities of the Standard Normal Distribution, NWK Table A.2 Percentiles of the t distribution if the number of degrees of freedom (# of cases - parameters estimated) is greater than 60, then the standard normal is a good approximation for the t-distribution. So, for the sake of simplicity we are going to assume that β N(b, ˆσ) (remember, this is just a close approximation...it is really a t distribution, but the shape is very similar) Question: did you cover the normal distribution in class last semester? Draw the shape of the sampling of distribution of β if b=3 and σ(b) ˆ = 5.

29 Example (draw on board) if b=3 and s(b) (the estimated standard error) is 5, what is the probability that β > w where w=5? Steps 1) convert w to a point on the standard normal distribution, z = w b s(b) 2) nd the cumulative probability from -innity to z=p 3) calculate 1-p Question: if b=5 and s=10, what is the probability thatβ > 10? β < 12?

30 condence interval the 1 α condence interval for b is b ± t(1 α, n p)s(b) (5) 2 where n is the number of cases, p is the number of parameters estimated, and s(b) is the estimated s.e. of b Example: b=5, s=10, n=1000, and p=2. What is the 95% c.i. for b? (do in class)

31 Hypothesis tests concerning b Hypothesis tests >this continues our discussion from two slides back. We have an estimate, b. Can we reject the hypothesis that the true value = 0 (i.e., β =0)? H o : β = 0 H a : β 0 Here is the way to think about it. If the true value was 0, what is the chance of observing a coecient of magnitude b or greater? (add a pdf showing this visually), draw on board

32 T-test t = b s(b) decision rule: if t t(1 α 2, n p)conclude H 0 otherwise conclude H a Example: b=5, s(b)=3. t = 1.67, don't reject H 0. Let's think about this: What is the chance that if β = 0, we would estimate b 5 in this case? (in lab 5 we are going to work more on the intuition of the sampling distribution of b.)

33 Partitioning the total sum of squares analysis of variance review from lecture 2 SSTO=total sum of squares= (Y i Ȳ ) 2 e = Y i Ŷ i SSE=sum of squared errors=rss= e 2 i SSR=regression sum of squares= (Ŷ i Ȳ ) 2 explain what SSR is

34 Partitioning the total sum of squares partioning of the deviation reference for this slide: NWK p Y i Ȳ = (Ŷ i Ȳ ) + (Y i Ŷ i ) total deviation = deviation of tted regression value around mean + deviatioin around tted regression line (Y i Ȳ ) 2 = (Ŷ i Ȳ ) 2 + (Y i Ŷ i ) 2 SSTO=SSR+SSE

35 Partitioning the total sum of squares R-squared What % of the variation in Y (SSTO) did we explain with our model? R 2 = (Ŷ i Ȳ ) 2 (Y i Ȳ ) 2 = SSR SSTO = 1 SSE SSTO (6)

36 Partitioning the total sum of squares F-test Weisberg, Ch.2 p.30, NWK p.240 The F-test tests whether the regression as a whole is an improvement over a model with just a constant term (the mean) F = SSR/(p 1) SSE/(n p) H o : β 1 = β 2 =... = β p 1 = 0 If F F (1 α, p 1, n p), conclude H o.