4.3 Normal distribution

Transcription

1 43 Normal distribution Prof Tesler Math 186 Winter 216 Prof Tesler 43 Normal distribution Math 186 / Winter / 4

2 Normal distribution aka Bell curve and Gaussian distribution The normal distribution is a continuous distribution Parameters: µ = mean (center) σ = standard deviation (width) ( ) PDF: f X () = 1 σ ep ( µ)2 for < < 2π 2σ 2 Normal distribution N(2, 5): µ = 2, σ = Normal µ µ ± σ The normal distribution is symmetric about = µ, so median = mean = µ Prof Tesler 43 Normal distribution Math 186 / Winter / 4

3 Applications of normal distribution Applications Many natural quantities are modelled by it: eg, a histogram of the heights or weights of everyone in a large population often follows a normal distribution Many distributions such as binomial, Poisson, are closely approimated by it when the parameters are large enough Sums and averages of huge quantities of data are often modelled by it Coverage in DNA sequencing Illumina GA II sequencing of E coli at 6 coverage Chitsaz et al (211), Nature Biotechnology % of positions with coverage Empirical distribution of coverage Coverage Prof Tesler 43 Normal distribution Math 186 / Winter / 4

4 Cumulative distribution function The integral for total probability is tricky, but does equal 1: ) 1 ( σ 2π ep ( µ)2 2σ 2 d = 1 See the details in the class tetbook or in a Calculus tetbook in the section on double integrals in polar coordinates The cumulative distribution function is the integral F X () = P(X ) = ) 1 ( σ 2π ep (t µ)2 2σ 2 dt This integral cannot be done symbollically in terms of the usual functions (polynomials, eponentials, logs, trig functions, etc) It can be done via numerical integration or Taylor series We ll learn how to do it with a look-up table Prof Tesler 43 Normal distribution Math 186 / Winter / 4

5 Standard normal distribution Standard normal distribution N(, 1): µ =, σ = 1 CDF of standard normal distribution 2 4 Normal µ µ ± σ cdf 4 8 Normal µ µ ± σ z The standard normal distribution is the normal distribution for µ =, σ = 1 Use the variable name Z: PDF: φ(z) = f Z (z) = e z2 /2 2π for < z < CDF: Φ(z) = F Z (z) = P(Z z) = 1 z e t2 /2 dt 2π Compute Φ(z) with the lookup table in the book (pages 697 8) z Prof Tesler 43 Normal distribution Math 186 / Winter / 4

6 Standard normal distribution tables Table A1 in the back of the book (pages 697 8) is similar to this Cumulative Area Under the Standard Normal Distribution Area = Φ(z) Cumulative Area Under the Standard Normal Distribution Area = Φ(z) z z z z Prof Tesler 43 Normal distribution Math 186 / Winter / 4

7 Standard normal distribution tables Table A1 in the back of the book (pages 697 8) is similar to this Cumulative Area Under the Standard Normal Distribution z Area = Φ(z) z Φ(151) 9345 Φ(162) 9474 Φ( 151) 655 Prof Tesler 43 Normal distribution Math 186 / Winter / 4

8 Standard normal distribution areas 4 Standard Normal Curve 3 2 1!5 a b 5 z The area between z = a and z = b is P(a Z b) = 1 2π b a e t2 /2 dt = Φ(b) Φ(a) Use table in back of book: P(151 Z 162) = Φ(162) Φ(151) = = 129 Prof Tesler 43 Normal distribution Math 186 / Winter / 4

9 Standard normal distribution symmetries of areas 4 Area! on the right "#$ &'()*!*+,*-(*/( "#! z!!2 2 z "!%!!! " %! Area right of z is P(Z > z) = 1 Φ(z) By symmetry, the area left of z and the area right of z are equal: Φ( z) = 1 Φ(z) Φ( 151) = 1 Φ(151) = = 655 Area between z = ±151 is Φ(151) Φ( 151) = 2Φ(151) Prof Tesler 43 Normal distribution Math 186 / Winter / 4

10 Central area Area between z = ±1: Φ(1) Φ( 1) = 6827 = 6827% Area between z = ±1 is 6827% Area between z = ±2 is 9545% Area between z = ±3 is 9973%!2 2 Find the center part containing 95% of the area 4 2 Area! split half on each tail!z!/2 z!/2 Put 25% of the area at the upper tail, 25% at the lower tail, and 95% in the middle The value of z putting 25% at the top gives Φ(z) = 1 25 = 975 Using the table in the book, z 196 Notation: z 25 = 196 The area between z = ±196 is about 95% For 99% in the middle, 5% on each side, use z z Prof Tesler 43 Normal distribution Math 186 / Winter / 4

11 Central area Find z with Φ(z) 975 Cumulative Area Under the Standard Normal Distribution z Area = Φ(z) z Φ(196) 975 Φ( 196) 25 z 25 = 196 Prof Tesler 43 Normal distribution Math 186 / Winter / 4

12 Areas on normal curve for arbitrary µ, σ P(a X b) = b a ) 1 ( σ 2π ep ( µ)2 2σ 2 d Substitute z = µ σ (or = σz + µ) into the integral to turn it into the standard normal integral: ( a µ P X µ b µ ) ( a µ = P Z b µ ) σ σ σ σ σ ( ) ( ) b µ a µ = Φ Φ σ σ The z-score of is z = µ σ Prof Tesler 43 Normal distribution Math 186 / Winter / 4

13 Binomial distribution Compute P(43 X 51) when n = 6, p = 3/4 Binomial: n = 6, p = 3/4 k P(X = k) = ( ) 6 k (75) k (25) 6 k Total 7544 Mean µ = np = 6(3/4) = 45 Standard deviation σ= np(1 p) = 6(3/4)(1/4) = Mode (k with ma ) np + p = 6(3/4) + (3/4) = = 45 Prof Tesler 43 Normal distribution Math 186 / Winter / 4

14 Mode of a distribution The mode of random variable X is the value k at which the is maimum Mode of binomial distribution when < p < 1 The mode is (n + 1)p Eception: If (n + 1)p is an integer then (n + 1)p and (n + 1)p 1 are tied as the mode The mode is within 1 of the mean np When np is an integer, the mode equals the mean Prof Tesler 43 Normal distribution Math 186 / Winter / 4

15 Binomial and normal distributions Binomial Normal approimation to binomial k P(X = k) 15 Binomial: n=6, p=3/4 Binomial P(43! X! 51) Normal: µ=45,"= Total P(X = k) shown as a rectangle: height P(X = k), etent k ± 1/2 The binomial distribution is only defined at the integers, and is very close to the normal distribution shown We will approimate the probability P(43 X 51) we had above by the corresponding one for the normal distribution Prof Tesler 43 Normal distribution Math 186 / Winter / 4

16 Normal approimation to binomial, step 1 Compute corresponding parameters We want to approimate P(a X b) in a binomial distribution We ll use n = 6, p = 3/4 and approimate P(43 X 51) Determine µ, σ: µ = np = 6(3/4) = 45 σ = np(1 p) = The normal distribution with those same values of µ, σ is a good approimation to the binomial distribution provided µ ± 3σ are both between and n Check: µ 3σ 45 3(3354) = µ + 3σ (3354) = 5562 are both between and 6, so we may proceed Note: Some applications are more strict and may require µ ± 5σ or more to be between and n Since µ + 5σ 61771, this would fail at that level of strictness Prof Tesler 43 Normal distribution Math 186 / Winter / 4

17 Normal approimation to binomial, step 2 Continuity correction Normal approimation to binomial Binomial: n=6, p=3/4 Binomial P(43! X! 51) Normal: µ=45,"=335 The binomial distribution is discrete (X = integers) but the normal distribution is continuous The sum P(X = 43) + + P(X = 51) has 9 terms, corresponding to the area of the 9 rectangles in the picture The area under the normal distribution curve from 425 X 515 approimates the area of those rectangles Change P(43 X 51) to P(425 X 515) Prof Tesler 43 Normal distribution Math 186 / Winter / 4

18 Normal approimation to binomial, step 3 Convert to z-scores For random variable X with mean µ and standard deviation σ, The z-score of a value is z = E(X) SD(X) The random variable Z is Z = X E(X) SD(X) = µ σ = X µ σ Convert to z-socres: P(425 X 515) = P ( X ) = P( Z ) Prof Tesler 43 Normal distribution Math 186 / Winter / 4

19 Normal approimation to binomial, step 4 Use the normal distribution We re at P(43 X 51) = P( Z ) Approimate this by the standard normal distribution cdf: P( Z ) Φ( ) Φ( ) Look in the standard normal table in the back of the book: It only has z s to two decimal places, so round them: Φ(194) 9738 and Φ( 75) 2266 So Φ( ) Φ( ) 7472 On a calculator/computer with more digits, it s These are close to the true answer (apart from rounding errors) P(43 X 51) = 7544 we got from the binomial distribution Prof Tesler 43 Normal distribution Math 186 / Winter / 4

20 Estimating fraction of successes instead of number of successes What is the value of p in the binomial distribution? Estimate it: flip a coin n times and divide the # heads by n Let X = binomial distribution for n flips, probability p of heads Let X = X/n be the fraction of flips that are heads X is discrete, with possible values n, 1 n, 2 n,, n n {( n ) P(X = k n ) = P(X = k) = k p k (1 p) n k for k =, 1,, n; otherwise Mean E(X) = E(X/n) = E(X)/n = np/n = p Variance Var(X) = Var ( ) X n = Var(X) n 2 Standard deviation SD(X) = p(1 p)/n = np(1 p) n 2 = p(1 p) n Prof Tesler 43 Normal distribution Math 186 / Winter / 4

21 Normal approimation for fraction of successes n flips, probability p of heads, X=observed fraction of heads Mean E(X) = p Variance Var(X) = p(1 p)/n Standard deviation SD(X) = p(1 p)/n The Z transformation of X is Z = X E(X) SD(X) = X p p(1 p)/n and value X = has z-score z = p p(1 p)/n For k heads in n flips, The z-score of X = k is z 1 = k np np(1 p) The z-score of X = k/n is z 2 = (k/n) p p(1 p)/n These are equal! Divide the numerator and denominator of z 1 by n to get z 2 Prof Tesler 43 Normal distribution Math 186 / Winter / 4

22 Normal approimation for fraction of successes For n = 6 flips of a coin with p = 3 4, we ll estimate P ( 43 6 The eact answer equals P(43 X 51) 7544 X 51 6) Step 1: Determine mean and SD E(X) = p = 75 SD(X) = p(1 p)/n = (75)(25)/6 = Verify approimation is valid: Mean ± 3 SD between and 1 Mean 3 SD = Mean + 3 SD = 9177 Both are between and 1 Step 2: Continuity correction P ( ) ( X 6 = P Step 3: z-scores X Step 4: Evaluate using table or calculator ) Prof Tesler 43 Normal distribution Math 186 / Winter / 4

23 Normal approimation for fraction of successes P ( 43 6 ) ( 51 X 6 = P X = P(7833 X 85833) = P ( 7833 E(X) SD(X) ) X E(X) SD(X) = P ( Z = P( Z ) E(X) SD(X) ) ) = 7472 with table in book or with a calculator/computer Prof Tesler 43 Normal distribution Math 186 / Winter / 4

24 Mean and SD of sums and averages of iid random variables Let X 1,, X n be n iid (independent identically distributed) random variables, each with mean µ and standard deviation σ Let S n = X X n be their sum and X n = (X X n )/n = S n /n be their average Means: Sum: E(S n ) = E(X 1 ) + + E(X n ) = n E(X 1 ) = nµ Avg: E(X n ) = E(S n /n) = nµ/n = µ Variance: Sum: Var(S n ) = Var(X 1 ) + + Var(X n ) = n Var(X 1 ) = nσ 2 Avg: Var(X n ) = Var(S n )/n 2 = nσ 2 /n 2 = σ 2 /n Standard deviation: Sum: SD(S n ) = σ n Avg: SD(X n ) = σ/ n Terminology for different types of standard deviation The standard deviation (SD) of a trial (each X i ) is σ The standard error (SE) of the sum is σ n The standard error (SE) of the average is σ/ n Prof Tesler 43 Normal distribution Math 186 / Winter / 4

25 Z-scores of sums and averages For sum S n For average X n Mean: E(S n ) = nµ E(X n ) = µ Variance: Var(S n ) = nσ 2 Var(X n ) = σ 2 /n Standard Deviation: SD(S n ) = σ n SD(X n ) = σ/ n Z-scores: Z = S n E(S n ) SD(S n ) = S n nµ σ n Z = X n E(X n ) SD(X n ) = X n µ σ/ n Z-scores of sum and average are equal! Divide the numerator and denominator of Z of the sum by n to get Z of the average Z sum = (S n nµ)/n (σ n)/n = X n µ σ/ n = Z avg Prof Tesler 43 Normal distribution Math 186 / Winter / 4

26 Theorem (Central Limit Theorem abbreviated CLT) For n iid random variables X 1,, X n with sum S n = X X n and average X n = S n /n, and any real numbers a < b, P ( a S n nµ σ n ) b = P ( a X ) n µ σ/ n b Φ(b) Φ(a) if n is large enough As n, the approimation becomes equality Interpretation of Central Limit Theorem As n increases, the closely resembles a normal distribution However, the is defined as in-between the red points shown (on upcoming slides), if it s a discrete distribution The cdfs are approimately equal everywhere on the continuum Probabilities of intervals for sums or averages of enough iid variables can be approimated with the normal distribution Prof Tesler 43 Normal distribution Math 186 / Winter / 4

27 Repeated rolls of a die One roll: µ = 35, σ = 35/ Average of 1 roll of die; µ=35,!=171 2 Average of 2 rolls of die; µ=35,!= Average of 3 rolls of die; µ=35,!= Average of 1 rolls of die; µ=35,!=17 Die average Normal dist µ µ±! Prof Tesler 43 Normal distribution Math 186 / Winter / 4

28 Repeated rolls of a die Find n so that at least 95% of the time, the average of n rolls of a die is between 3 and 4 ( ) P(3 X 4) = P 3 µ σ/ X µ n σ/ 4 µ n σ/ n Plug in µ = 35 and σ = 35/12 ( P(3 X 4) = P 1/2 Z 35/(12n) ) 1/2 35/(12n) Recall the center 95% of the area on the standard normal curve is between z = ±196 1/2 196 n (196) 2 35/ /(12n) (1/2) 2 n is an integer so n 45 Prof Tesler 43 Normal distribution Math 186 / Winter / 4

29 Prof Tesler 43 Normal distribution Math 186 / Winter / 4

30 Sawtooth distribution (made up as demo) One trial: µ = 4, σ 224 Average of 1 trial; µ=4,!=224 Average of 2 trials; µ=4,!= Average of 3 trials; µ=4,!= Average of 1 trials; µ=4,!= Prof Tesler 43 Normal distribution Math 186 / Winter / 4

31 Binomial distribution (n, p) A Bernoulli trial is to flip a coin once and count the number of heads, { 1 probability p; X 1 = probability 1 p Mean E(X 1 ) = p, standard deviation SD(X 1 ) = p(1 p) The binomial distribution is the sum of n iid Bernoulli trials Mean µ = np, standard deviation σ = np(1 p) The binomial distribution is approimated pretty well by the normal distribution when µ ± 3σ are between and n Prof Tesler 43 Normal distribution Math 186 / Winter / 4

32 Binomial distribution (n, p) One flip: µ = p = 75, σ = p(1 p) = Binomial n=1,p=75; µ=75,!= Binomial n=6,p=75; µ=45,!= Binomial n=3,p=75; µ=225,!= Binomial n=6,p=75; µ=45,!= Prof Tesler 43 Normal distribution Math 186 / Winter / 4

33 Poisson distribution (µ or µ = λ d) Mean: µ (same as the Poisson parameter) Standard deviation: σ = µ It is approimated pretty well by the normal distribution when µ 5 The reason the Central Limit Theorem applies is that a Poisson distribution with parameter µ equals the sum of n iid Poissons with parameter µ/n The Poisson distribution has infinite range =, 1, 2, and the normal distribution has infinite range < < (reals) Both are truncated in the plots Prof Tesler 43 Normal distribution Math 186 / Winter / 4

34 Poisson distribution (µ) Poisson µ=1;!= Poisson µ=6;!= Poisson µ=3;!= Poisson µ=6;!= Prof Tesler 43 Normal distribution Math 186 / Winter / 4

35 Geometric and negative binomial distributions Geometric distribution (p) X is the number of flips { until the first heads, (1 p) 1 p if = 1, 2, 3, ; p X () = otherwise The plot doesn t resemble the normal distribution at all Mean: µ = 1/p Negative binomial distribution (r, p) Standard deviation: σ = 1 p/p r = 1 is same as geometric distribution r > 2: The has a bell -like shape, but is not close to the normal distribution unless r is very large Mean: µ = r/p Standard deviation: σ = r(1 p)/p Prof Tesler 43 Normal distribution Math 186 / Winter / 4

36 Geometric and negative binomial distributions Heads with probability p = 1 Geometric p=1; µ=1,!= Neg bin r=6,p=1; µ=6,!= Neg bin r=3,p=1; µ=3,!= Neg bin r=6,p=1; µ=6,!= ! Prof Tesler 43 Normal distribution Math 186 / Winter / 4

37 Eponential and gamma distributions Eponential distribution (λ) The eponential distribution doesn t resemble the normal distribution at all Mean: µ = 1/λ Standard deviation: σ = 1/λ Gamma distribution (r, λ) The gamma distribution for r = 1 is the eponential distribution The gamma distribution for r > 1 does have a bell -like shape, but it is not close to the normal distribution until r is very large There is a generalization to allow r to be real numbers, not just integers Mean: µ = r/λ Standard deviation: σ = r/λ Prof Tesler 43 Normal distribution Math 186 / Winter / 4

38 Eponential and gamma distributions Rate λ = 1 Eponential!=1; µ=1,"=1 1 5 Gamma r=6,p=1; µ=6,!= Gamma r=3,p=1; µ=3,!= ! Gamma r=6,p=1; µ=6,!= ! Prof Tesler 43 Normal distribution Math 186 / Winter / 4

39 Geometric/Negative binomial vs Eponential/Gamma p = λ gives same means for geometric and eponential p = 1 e λ gives same eponential decay rate for both geometric and eponential distributions 1 e λ λ when λ is small This corespondence carries over to the gamma and negative binomial distributions Prof Tesler 43 Normal distribution Math 186 / Winter / 4

40 Geometric/negative binomial vs Eponential/gamma This is for p = 1 vs λ = 1; a better fit for λ = 1 would be p = 1 e λ 95 Geometric p=1; µ=1,!=949 Eponential!=1; µ=1,"= Neg bin r=3,p=1; µ=3,!= Gamma r=3,p=1; µ=3,!= ! Prof Tesler 43 Normal distribution Math 186 / Winter / 4