Business Statistics 41000: Probability 3

Transcription

1 Business Statistics 41000: Probability 3 Drew D. Creal University of Chicago, Booth School of Business February 7 and 8,

2 Class information Drew D. Creal Office: 404 Harper Center Office hours: me for an appointment Office phone:

3 Course schedule Week # 1: Plotting and summarizing univariate data Week # 2: Plotting and summarizing bivariate data Week # 3: Probability 1 Week # 4: Probability 2 Week # 5: Probability 3 Week # 6: In-class exam Week # 7: Statistical inference 1 Week # 8: Statistical inference 2 Week # 9: Simple linear regression Week # 10: Multiple linear regression 3

4 Outline of today s topics I. Continuous Random Variables (AWZ p ) II. The Uniform Distribution (AWZ p. 411) III. The Normal Distribution (AWZ p ) IV. Modeling Stock Returns: the i.i.d. Normal Model V. The Random Walk Model (again) VI. Cumulative Distribution Functions (AWZ p. 538) VII. Expected Value and Variance of Continuous Random Variables (AWZ p ) VIII. The Chi-square and Student s t distributions (AWZ p. 438, 453, 434) 4

5 Continuous Random Variables 5

6 Continuous Random Variables In the last two lectures, we considered discrete random variables. We used them to model uncertainty in situations where there were a countable number of outcomes of the random event. In many situations, a random event can take on a continuous range of values. This is a continuous random variable! 6

7 Continuous Random Variables Suppose we have a machine that cuts cloth. Each time the machine cuts the cloth, there are threads. We believe the length of a thread could be anything between 0 and 0.5 inches and any value in the interval is equally likely. The machine is about to cut and it will leave a new thread. In advance of the machine cutting the cloth, the length of the thread is a number we are uncertain about. It is a random variable. How do we describe our beliefs? 7

8 Continuous Random Variables This is an example of a continuous random variable. It can take on any value in an interval. In this example each value in the interval is equally likely. But in general, we might want to express beliefs that some values are more likely than others. 8

9 Continuous Random Variables In the discrete case, we listed the possible outcomes and assigned a probability to each outcome. Now, the number of possible outcomes is so large that we can t make a list and give each outcome a probability! Instead we talk about the probability of intervals. In the discrete case, we had P(X = x) = 0.1 whereas in the continuous case we have P(a < X < b) = 0.1 (NOTE: Here, the interval is (a, b), i.e. all the real numbers between a and b.) 9

10 Continuous Random Variables One convenient way to specify the probability of an interval is with a probability density function (pdf). The probability of an interval is the area under the pdf over that interval. This is an example of a pdf. Values close to zero are more likely because the area under the curve in that region is larger. f(x) x 10

11 Continuous Random Variables f(x) x For this random variable the probability that it is in the interval [0, 2] is (47.7 percent of the time it will fall in this interval). The area under the curve within the blue region is

12 Continuous Random Variables f(x) x Here is another example of a probability density function. This random variable can only take on positive values because it gives zero probability mass to values less than zero. This kind of distribution is called skewed right. 12

13 Continuous Random Variables f(x) x For this random variable, consider the interval (4, + ). What is the probability that it takes a value greater than 4? The probability is

14 Continuous Random Variables For a continuous random variable X, the probability that it takes a value in the interval (a, b) is the area under the probability density function from a to b. By definition, the area under the curve (pdf) must be equal to 1. (NOTE: Why?) The probability density function is a function that you may see written as f (x). 14

15 Continuous Random Variables For those of you who like math, an equivalent way of saying this is to use calculus. (NOTE: This will not be on an exam.) If the random variable X has a probability density function f (x), then P(a < X < b) = b a f (x)dx An integral allows us to compute the area under the curve f (x) over the interval (a, b). 15

16 Continuous Random Variables An integral is just a sum! P(a < X < b) = b a f (x)dx It is a fancy way of saying OR means ADD that we saw earlier. P(a < X < b) = a<x <b p(x) 16

17 The Uniform Distribution 17

18 The Uniform Distribution Let s return to our example of the machine that cuts cloth and leaves threads. Any value between 0 and 0.5 is equally likely. If X is the length of a thread, what is its pdf? 18

19 The Uniform Distribution The pdf is flat in the interval (0, 0.5). Its height must be 2 so that the total area is 1. Outside of (0, 0.5), the pdf is equal to zero. f(x) x Technically, any particular value has probability 0! But, we still express the idea that they are equally likely in (0,.5). 19

20 The Uniform Distribution In general, we have a family of uniform distributions describing the situation where any value in the interval (a, b) is equally likely. 1/(b a) a b The height must be 1/(b a) so that the area of the rectangle = 1. 20

21 The Uniform Distribution A random variable X is said to have a uniform distribution if it has probability density function f (x) = { 1 (b a) a < x < b 0 otherwise We write X U(a, b) The parameters of the Uniform distribution are the two endpoints a and b. 21

22 The Normal Distribution 22

23 The Normal Distribution This pdf describes the standard normal distribution. We often use the letter Z to denote the random variable which has this pdf. 4 f(x) x The normal distribution has a bell-shaped curve. Any value in the interval (, ) is possible. It is the standard normal because it has mean 0 and variance 1. 23

24 The Standard Normal Distribution Some properties of the standard normal distribution, which has µ Z = 0 and σ Z = 1. P(0 < Z < 1) = 0.34 P( 1 < Z < 1) = 0.68 P( 2 < Z < 2) = P( 1.96 < Z < 1.96) = 0.95 P( 3 < Z < 3) = (NOTE: In these notes I will usually act as if 1.96 = 2) 24

25 The Normal Distribution If X is a normal random variable, we write: X N(µ, σ 2 ) The normal distribution has two parameters µ and σ 2. It turns out that µ is the mean, σ 2 is the variance, and σ is the standard deviation. The standard normal is the special case when µ = 0 and σ 2 = 1. Here, we really have a family of normal distributions. One for each value of the parameters (µ, σ 2 ). The normal distribution is also known as the Gaussian distribution. 25

26 The Normal Distribution The normal family has two parameters µ : determines where the curve is centered. σ : determines how spread out the curve is. pdf X N(0,4) Z N(0,1) Y N(1,0.5 2 ) X,Y, and Z are all normally distributed. 26

27 Interpreting the Normal Distribution X N(µ, σ 2 ) There is a 95% chance X is in the interval µ ± 2σ. µ describes what you think will happen on average. ±2σ describes how wrong you could be. µ determines the location of the distribution. σ determines the spread. 27

28 The Empirical Rule In lecture #1, we suggested that for mound-shaped data an empirical rule was Approximately 68% of the data is in the interval (x s x, x + s x ) = x ± s x Approximately 95% of the data is in the interval (x 2s x, x + 2s x ) = x ± 2s x That s right. We are just using a normal distribution as an approximation. 28

29 Identically distributed normal random variables Consider two normal random variables: X N(1.2, 1) Y N(1.201, 1) Are X and Y identically distributed? Consider two more normal random variables: X N(µ, σ 2 ) Y N(µ, 2σ 2 ) Are X and Y identically distributed? 29

30 Example: Interpreting the Normal Distribution Let the random variable R t be the return next month on a mutual fund. Assume that R t N(0.1, 0.01) There is a 95% chance that R t will be in the interval (µ 2σ, µ+2σ) = ( 0.1, 0.3) f(x) % area x What is P( 0.1 < R t < 0.3)? (NOTE: The distribution is symmetric around µ.) 30

31 Interpreting the Normal Distribution Just so you know, if then f (x) = X N(µ, σ 2 ) ( 1 exp 1 ) (x µ)2 2πσ 2 2σ2 is the probability density function (pdf). In this class, we will not need this. But, we will see the normal distribution many more times throughout the course. It is probably the most important distribution in statistics. 31

32 One More Property of the Normal Distribution Any linear combination of normal random variables is also normally distributed. In statistics, many models are based on linear relationships Y = c 0 + c 1 X c k X k. In words, if each X i is normally distributed, then Y will be normally distributed as well. This is another reason why the normal distribution is so important. 32

33 Building Models Part II 1. The i.i.d. Normal Model 2. The random walk model 33

34 The i.i.d. Normal Model Remember how we used the idea of i.i.d draws from the Bernoulli(.5) distribution to model coin tosses and i.i.d. draws from the Bernoulli(.2) to model a manufacturing process that gives faulty and good parts? Now, we want to use the normal distribution to model real data. Surprisingly often, data looks like i.i.d. draws from a normal distribution. 34

35 The i.i.d. Normal Model We can have i.i.d. draws from any distribution. When we write X 1, X 2,..., X n N(µ, σ 2 ) i.i.d. we mean that each random variable X i for i = 1,..., n will be an independent draw from the same normal distribution. We have not formally defined independence for continuous distributions, but our intuition is the same as for discrete random variables. Each draw has no effect on the others, and the same normal distribution describes what we think each variable will turn out to be. 35

36 The i.i.d. Normal Model What do i.i.d. normal draws look like? We can simulate them using a computer. In this plot, there are 100 draws from a standard normal distribution There is no pattern in the draws. They look random

37 The i.i.d. Normal Model This is the same graph only now I have added lines indicating µ = 0 and the interval (µ 2σ, µ + 2σ). 3 95% of the data will be in between these lines (in the long run)

38 The i.i.d. Normal Model Here are draws from a different normal distribution. These are draws from N(5, 4) How would you predict the next draw?

39 The i.i.d. Normal Model Data that is approximately normal is very common Histogram of the daily returns on the Dow Jones from 1928 to

40 The i.i.d. Normal Model Example: Consider the return data for Canada. We have monthly returns from Feb 1988 to Dec Do the data look i.i.d.? 2. Do the data look normally distribution? Look at a time series plot. There appears to be no pattern Canadian returns

41 The i.i.d. Normal Model Canada 2. Look at a histogram. They appear approximately normal. The returns look i.i.d. normal (at least as a first approximation). 41

42 The i.i.d. Normal Model The sample mean and stand. dev. are R t = 0.01 and s Rt = If we use these to approximate µ and σ, then our prediction for the next return is 0.01 and a 95% interval is (-0.07,0.09) R t = Canadian returns Our model is R t N(0.01, ) i.i.d

43 The i.i.d. Normal Model It is important to realize that not all data is normal. 40 Remember the data set on arrival times of customers at a bank Histogram of Interarrival Times for Bank Customers

44 The i.i.d. Normal Model Not all time series are independent... Dow Jones Industrial Average in levels Dow Jones Industrial Average Index Sept. 29, (NOTE: we can often transform data such that the i.i.d. Normal model is a better approximation.)

45 The i.i.d. Normal Model Here, I have transformed the DJ Index into returns. If X t measures the level of the series, then the returns R t are computed as R t = log(x t) log(x t 1 ) Dow Jones Returns The normal approximation appears to be better. 45

46 The i.i.d. Normal Model But, appearances can still be deceiving. Are the returns really even i.i.d.? Does it look like they have the same variance σ 2? Returns on the Dow Jones Industrial Average Index (NOTE: Robert Engle won the 2003 Nobel prize for modeling volatility clustering, which is a fancy phrase to describe a time-varying variance σ 2 t.) 46

47 The i.i.d. Normal Model Not all time series are independent Levels of Lake Michigan from 1872 to (see lakemichigan.xls ) year 47

48 The Random Walk Model (again) In Lecture # 4, we looked at the random walk model. Let P t denote the price of an asset. The equation for a random walk is: P t+1 = P t + E t+1 (..., E t 1, E t, E t+1,...) are a sequence of i.i.d. random variables. Recall that in the previous lecture, we modeled E t+1 as a series of discrete random variables. Each random variable E t+1 could take on only two possible values or with equal probabilities. 48

49 The Random Walk Model (again) Here is the same graph of P t from our previous lecture. We can see that prices move by taking discrete jumps

50 The Random Walk Model (again) P t+1 = P t + E t+1 In a random walk model, it is possible for E t+1 to be a random variable having a continuous distribution. Consider letting E t+1 be a sequence of i.i.d normal random variables. E t N(0, 0.04) for t = 1, 2,... 50

51 The Random Walk Model (again) Here is a plot of data from this model. At each point in time, prices changes are normally distributed

52 Cumulative Distribution Functions 52

53 Cumulative Distribution Function The cumulative distribution function (c.d.f.) is another way (besides the p.d.f.) to specify the probability of intervals. For a random variable X, the c.d.f. is defined by F X (x) = P(X x) (NOTE: it is common to use capital F to denote the c.d.f. and lower case f to denote the p.d.f.) 53

54 Cumulative Distribution Function F X (x) = P(X x) = Area under the pdf to the left of the point x Two examples for the standard normal distribution F( 1) = F(1.96) =

55 Cumulative Distribution Function The cumulative distribution function allows us to compute the probability of any interval (a, b) P(a < x < b) = P(X < b) P(X < a) = F X (b) F X (a) By taking the difference between two c.d.f.s we get the probability of the interval (a, b). 55

56 Cumulative Distribution Function As an example, consider a random variable Z that has a standard normal distribution. P( 1 < Z < 1) = P(Z < 1) P(Z < 1) = F Z (1) F Z ( 1) = = The area under the curve between -1 and 1 is

57 Cumulative Distribution Function Instead of plotting the p.d.f., we can plot the c.d.f.. In the c.d.f., the probability of an interval is the jump in the c.d.f. over that interval. Probability is plotted on the vertical axis. Here is an example of Z N(0, 1). 1.0 Note that F Z (1) = 0.84 and F Z ( 1) = 0.16 as on the last slide. F(1) F( 1) x 57

58 Cumulative Distribution Function For x big enough, F (x) must go to 1. (Why?) For x small enough, F (x) must go to 0. (Why?) Here is an example of a c.d.f. where we consider a generic interval (a, b). 1.0 F(b) 0.8 F(a) x a b 58

59 Example: let R denote the return on our portfolio next month. We do not know the return in advance so it is random. Let s assume that R N(0.01, )

60 R N(0.01, ) What is the probability of a negative return? P(R < 0) = F R (0) = NOTE: in Excel, =NORMDIST(0,0.01,0.04,TRUE) What is the probability of a return between 0 and 0.05? P(0 < R < 0.05) = F R (0.05) F R (0) = = 0.44 NOTE: in Excel, =NORMDIST(0.05,0.01,0.04,TRUE) 60

61 CDF of the Uniform Distribution Let X U(a, b). Recall that its p.d.f. is: f (x) = { 1 (b a) a < x < b 0 otherwise What is the c.d.f. of F X (x)? F X (x) = P(X < x) = area under f (x) to the left of x. Can you write down a formula for this area? 61

62 Expected Value and Variance of Continuous Random Variables 62

63 Expected Value and Variance of Continuous RV s If X is a continuous random variable with p.d.f. f (x) then the expected value is E [X ] = x f (x)dx This is basically the same thing as we had in the discrete case E [X ] = all x x p(x) 63

64 Expected Value and Variance of Continuous RV s If X is a continuous random variable with p.d.f. f (x) then the variance is Var [X ] = E [(X µ) 2] = (x µ) 2 f (x)dx This is basically the same thing as we had in the discrete case Var [X ] = all x p(x) (x µ) 2 64

65 Expected Value and Variance of Continuous RV s If you know calculus, those formulas are intuitive. If you do not know calculus, they are basically jibberish. But remember: An integral is just a sum. Just replace with all x and then replace f (x)dx with p(x). If you did this, then the formulas would be the same as for discrete random variables from Lecture # 3. 65

66 Expected Value and Variance of Continuous RV s Practically, what does this mean? First, recognize that the intuition we developed for discrete random variables is the same. 1. The mean is a best guess or a measure of the center of the distribution. 2. The variance is the expected squared distance of X from its mean. It measures how spread out the distribution is. 66

67 Expected Value and Variance of Continuous RV s The formulas about means and variances of linear functions that we had for discrete random variables are also the same. Suppose Then Y = c 0 + c 1 X 1 + c 2 X 2 E [Y ] = c 0 + c 1 E [X 1 ] + c 2 E [X 2 ] V [Y ] = c 2 1 V [X 1] + c 2 2 V [X 2] + 2c 1 c 2 COV [X 1, X 2 ] 67

68 Expected Value and Variance of Continuous RV s We could also define joint probabilities, conditional probabilities, independence, covariance, and correlation for continuous random variables. The ideas behind them are the same. It is still true that for two rv s X and Y : ρ XY = σ XY σ X σ Y and it is still true that 1 ρ XY 1. The only thing that s different is how we specify the probabilities. We specify probabilities of intervals instead of individual outcomes. 68

69 The Random Walk Model (again) Example: Remember the random walk model P t+1 = P t + E t+1 E t+1 N(0, 0.04) E t N(0, 0.04) for t = Given we know the price today P t = p t, what is the expected value of the price tomorrow based on this model? In other words, what is the mean of the conditional probability distribution P(P t+1 P t = p t )? 69

70 The Random Walk Model (again) We take the (conditional) expected value of each side: E[P t+1 P t = p t ] = E[P t + E t+1 ] = p t + E[E t+1 ] = p t Given that we know today s price, the price P t = p t is a constant. The expectation of E t+1 is E[E t+1 ] = 0. The best guess of tomorrow s price is today s price. 70

71 The Chi-Square and Student s t Distributions 71

72 The Chi-Square and Student s t Distributions We will take a brief look at two continuous distributions: 1. the Chi-squared distribution: χ 2 2. the Student s t distribution: t Both of these distributions are related to the normal distribution. 72

73 The Chi-Square Distribution Suppose that we have a sequence of standard normal random variables Z 1, Z 2, Z 3,..., Z k i.i.d. N(0, 1). Next, define a new random variable X to be the sum of squared Z i s. X = Z Z Z 2 k The random variable X has a Chi-squared distribution: X χ 2 (k) The χ 2 (k) distribution is a family of distributions for different values of the parameter k. The parameter k is known as the degrees of freedom. 73

74 The Chi-Square Distribution What does the χ 2 (k) density look like? χ 2 (3) χ 2 (10) The χ 2 distribution is skewed to the right. However, as k gets larger, the density flattens out and becomes symmetric. 74

75 The Chi-Square Distribution Why is the χ 2 distribution useful? Look at the sample variance: s 2 x = 1 n 1 n (x i x) 2 i=1 If we assume our data x 1, x 2,..., x n are i.i.d. normal, then the sample variance looks chi-square! 75

76 The Chi-Square Distribution Just so you know, if then X χ 2 (k) f (x) = 1 2 k/2 Γ ( )x k 2 1 e x 2 k 2 is the probability density function (pdf). In this class, we will not need this. But, we will see the chi-squared distribution later in the course when we build confidence intervals for the sample variance. 76

77 The Chi-Square Distribution Finally, we note that the mean of a χ 2 (k) random variable is: E[X ] = k and the variance is V[X ] = 2k 77

78 The Student s t Distribution Suppose we have two independent random variables Z and X whose distributions are Z N(0, 1) and X χ 2 (k). Define a new random variable T as: T = Z X /k The random variable T has a Student s t distribution written as: T t(k) The parameter k is known as the degrees of freedom. 78

79 The Student s t Distribution The random variable T with a Student s t distribution T t(k) describes a standard normal random variable divided by the square root of a normalized chi-square (a chi-square with k d.f., divided by k). 79

80 The Student s t Distribution What does a Student s t distribution look like? N(0,1) in red t 30 in green t The t-distribution has fatter tails (higher kurtosis) than the normal distribution. 80

81 The Student s t Distribution N(0,1) in red t 30 in green t At about 30 degrees of freedom and higher, though, the t and the normal are pretty much the same. 81

82 The Student s t Distribution Why is the Student s t distribution useful? There are many uses of a Student s t distribution and in particular 1. It is used as a model for asset returns. 2. It is used to construct confidence intervals (we will see this in Lecture # 8). 82

83 An i.i.d. Student s t Model Earlier we stated that it is common for data to appear approximately normal Histogram of the daily returns on the Dow Jones from 1928 to

84 An i.i.d. Student s t Model However, returns often have heavy tails. Occasionally, we observe very large (negative) returns, making the i.i.d. normal model unrealistic

85 An i.i.d. Student s t Model A Student s t distribution with a small degrees of freedom, say ν 4, assigns more probability mass to the tails. Consequently, it may be a better approximation than the normal model