Exam P September 2014 Study Sheet! copylefted by Jared Nakamura, 9/4/2014!

Transcription

1 Exam P September 2014 Study Sheet copylefted by Jared Nakamura, 9/4/2014 I. General Probability (15-30%) A. Set functions including set notation and basic elements of probability 1. Set notation: A = {a1, a2,, an} = {ai i = 1, 2,, n}; ai are called elements of A 2. Subset: A B means Every element of A is an element of B 3. Set union: A B is the set of elements in A or B 4. Set intersection: A B is the set of elements in A and B 5. Set difference: A \ B is the set of elements in A and not in B 6. Set complement: A is the set of elements not in A 7. Disjoint sets: A and B are disjoint iff no element of A is an element of B 8. Cardinality: A is the number of elements in A 9. Rule of Sum: If A and B are disjoint, then A + B = A B 10. Probability notation: P(A) is the probability that an event occurs within the set of events A B. Mutually exclusive events 1. Mutual exclusion: The set of events A and B are mutually exclusive iff A and B are disjoint C. Addition and multiplication rules 1. Addition rule: A and B are mutually exclusive iff P(A B) = P(A) + P(B) 2. Multiplication rule: A and B are independent iff P(A B) = P(A)P(B) D. Independence of events 1. Independence: The set of events A and B are independent iff P(A) is not affected by restricting the universe of events to B (and vice versa) E. Combinatorial probability 1. Factorial operation: n = 1*2*3* *n = (product from i = 1 to n) i 2. Number of ordered subsets of cardinality r from a set of cardinality n: npr = n/(n-r) 3. # of subsets of cardinality r from a set of cardinality n: ncr = n/[r(n-r)] 4. # of ways to order a multiset of cardinality n with multiplicities {mi}: n/[(product over i) mi] F. Conditional probability 1. Conditional probability: P(A B) is the probability that an event in A occurs after restricting the universe of events to B 2. Conditional axiom of probability: P(A B)P(B) = P(A B) G. Bayes' Theorem / Law of total probability 1. Bayes Theorem: P(A B)P(B) = P(B A)P(A) 2. Law of total probability: If {Bi} are mutually exclusive and (sum over i) P(Bi) = 1, then P(A) = (sum over i) P(A Bi) 3. Law of total conditional probability: If {Bi} are mutually exclusive, (sum over i) P(Bi) = 1, and C is independent of {Bi}, then P(A C) = (sum over i) P(A (C Bi))P(Bi)

2 II. Univariate Probability Distributions (including binomial, negative binomial, geometric, hypergeometric, Poisson, uniform, exponential, gamma, and normal) (30-50%) A. Probability functions and probability density functions 1. Discrete pf: px(n) = P(X = n) 2. Continuous pdf: fx(x) is such that P(a X b) = (integral from x = a to b) fx(x)dx 3. Binomial pf: p(r) = ncrp r (1 - p) n - r 4. Negative Binomial pf: p(n) = n - 1Cr - 1p r (1 - p) n - r 5. Geometric pf: p(n) = p(1 - p) n Hypergeometric pf: p(k) = nck(n - ncr - k)/ncr 7. Poisson pf: p(k) = e -λ λ k /k 8. Uniform pdf: f(x) = 1/(b-a) 9. Exponential pdf: f(x) = λe -λx 10. Gamma pdf: f(x) = λe -λx (λx) α-1 /Γ(α) 11. Normal pdf: f(x) = exp[-(x - μ) 2 /(2σ 2 )]/((2π) 1/2 σ) B. Cumulative distribution functions 1. Cumulative distribution function: FX(x) is such that P(X x) = FX(x) 2. Discrete cdf: FX(x) = (sum from n = - to floor(x)) px(n) 3. Continuous cdf: F(x) = (integral from t = - to x) f(t)dt 4. Binomial cdf: No elementary form 5. Negative Binomial cdf: No elementary form 6. Geometric cdf: F(x) = 1 - (1 - p) floor(x) 7. Hypergeometric cdf: No elementary form 8. Poisson cdf: No elementary form 9. Uniform cdf: F(x) = (x-a)/(b-a) 10. Exponential cdf: F(x) = 1 - e -λx 11. Gamma cdf: No elementary form 12. Normal cdf: No elementary form, however F(x) Z(t (x - μ)/σ) where Z(t) is the Z table 13. Survival function: S(x) = 1 - F(x) C. Mode, median, percentiles, and moments 1. Mode: The modes are the values of x for which p(x) or f(x) is the greatest 2. Informal definition of median: The median is the value of x for which FX(x) = Informal definition of percentile: The nth percentile is the value of x for which FX(x) = n / Discrete expected value: E(g(X)) = (sum from x = - to ) g(x)px(x) 5. Continuous expected value: E(g(X)) = (integral from x = - to ) g(x)fx(x)dx 6. Binomial ev: E(X) = np 7. Negative Binomial ev: E(X) = r(1 - p)/p 8. Geometric ev: E(X) = 1/p 9. Hypergeometric ev: E(X) = nr/n 10. Poisson ev: E(X) = λ 11. Uniform ev: E(X) = (b + a)/2 12. Exponential ev: E(X) = 1/λ 13. Gamma ev: E(X) = α/λ 14. Normal ev: E(X) = μ 15. Affine expected value: E(aX + b) = ae(x) + b 16. Discrete raw moment: The nth raw moment is (sum from x = - to ) x n p(x) = E(X n ) 17. Continuous raw moment: The nth raw moment is (integral from x = - to ) x n f(x)dx = E(X n )

3 D. Variance and measures of dispersion 1. Variance: Var(X) = E[(X-E(X)) 2 ] 2. Binomial var: Var(X) = np(1 - p) 3. Negative Binomial var: Var(X) = r(1 - p)/p 2 4. Geometric var: Var(X) = (1 - p)/p 2 5. Hypergeometric var: Var(X) = nr(n - r)(n - n)/(n 3 - N 2 ) 6. Poisson var: Var(X) = λ 7. Uniform var: Var(X) = (b - a) 2 /12 8. Exponential var: Var(X) = 1/λ 2 9. Gamma var: Var(X) = α/λ Normal var: Var(X) = σ Standard deviation: The standard deviation σx is the square root of the variance 12. Affine variance: Var(aX + b) = a 2 Var(X) 13. Variance as expected values: Var(X) = E(X 2 ) - E(X) 2 E. Moment generating functions 1. Moment generating function: MX(t) = E(e tx ) 2. Moment generation: [d n (MX(t))/dt n ](0) = E(X n ) 3. Binomial mgf: M(t) = (pe t p) n 4. Negative Binomial mgf: M(t) = (1 - p) r /(1 - pe t ) r 5. Geometric mgf: M(t) = pe t /(1 - (1 - p)e t ) 6. Hypergeometric mgf: No elementary form 7. Poisson mgf: M(t) = exp(λ(e t - 1)) 8. Uniform mgf: M(t) = (e bt - e at )/(t(b - a)) 9. Exponential mgf: M(t) = λ/(λ - t) 10. Gamma mgf: M(t) = (1 - t/λ) -α 11. Normal mgf: M(t) = exp(tμ+σ 2 t 2 /2) F. Transformations 1. Univariate Transformation: If g is one-to-one, fg(x)(x) = fx(g -1 (x)) d(g -1 (x))/dx 2. Affine transformation: fax + b(x) = fx((x - b)/a)/a 3. Inverse transformation: f1 / X(x) = fx(1/x)/x 2

4 III. Multivariate Probability Distributions (including the bivariate normal) (30-45%) A. Joint probability functions and joint probability density functions 1. Discrete joint probability function: pxy(m, n) = P(X = m Y = n) 2. Continuous joint probability density function: fxy(x, y) is such that P(a X b c Y d) = (integral from x = a to b)(integral from y = c to d) fxy(x, y)dydx 3. Mixed joint probability density function: fxy(x, y) is such that P(X = m a Y b) = (integral from y = a to b) fxy(m, y)dy 4. Bivariate normal: fxy(x, y) = exp[-[((x - μx)/σx) 2 + ((y - μy)/σy) 2-2ρ(x - μx)(y - μy)/(σxσy)]/(2-2ρ 2 )]/[2πσxσy(1 - ρ 2 ) 1/2 ] 5. Bivariate uniform: fxy(x, y) = 1 / A where A is the area of the domain 6. Joint independent variables: X and Y are independent iff FXY(x, y) = FX(x)FY(y) B. Joint cumulative distribution functions 1. Joint cumulative distribution function: F(x, y) is such that P(X x Y y) = F(x, y) 2. Discrete joint cdf: F(x, y) = (sum from m = - to floor(x))(sum from n = - to floor(y)) pxy(m, n) 3. Continuous joint cdf: F(x, y) = (integral from s = - to x)(integral from t = - to y) fxy(s, t)dtds 4. Mixed joint cdf: F(x, y) = (sum from m = - to floor(x))(integral from t = - to y) fxy(m, t)dt C. Central Limit Theorem 1. Informal definition of Central Limit Theorem: The arithmetic mean of a large number of iterates of independent random variables is approximately normally distributed with a mean equal to the mean of the expected values of the random variables and a variance equal to the sum of the variances of the random variables D. Conditional and marginal probability distributions 1. Discrete marginal distribution: px(m) = (sum from n = - to ) pxy(m, n) 2. Continuous marginal distribution: fx(x) = (integral from t = - to ) fxy(x, t)dt 3. Mixed marginal distribution, discrete: px(m) = (integral from t = - to ) fxy(m, t)dt 4. Mixed marginal distribution, continuous: fy(y) = (sum from m = - to ) fxy(m, y) 5. Discrete conditional distribution: px Y(x y) = pxy(x, y)/py(y) 6. Continuous conditional distribution: fx Y(x y) = fxy(x, y)/fy(y) E. Moments for joint, conditional, and marginal probability distributions 1. Discrete joint expected value: E(g(X, Y)) = (sum from x = - to )(sum from y = - to ) g(x, y) pxy(x, y) 2. Continuous joint expected value: E(g(X, Y)) = (integral from x = - to )(integral from y = - to ) g(x, y)fxy(x, y)dydx 3. Mixed joint expected value: E(g(X, Y)) = (sum from x = - to )(integral from y = - to ) g(x, y)fxy(x, y)dy 4. Joint raw moment: The (m, n)th raw moment is equal to E(X m Y n ) 5. Conditional raw moment: The nth conditional raw moment of X is equal to E(X n Y) 6. Marginal raw moment: The nth marginal raw moment of X is equal to E(X n ) 7. Law of total expectation: E(X) = E(E(X Y)) F. Joint moment generating functions 1. Joint moment generating function: MXY(s, t) = E(e sx + ty ) 2. Joint moment generation: [d m+n (MXY(s, t))/(ds m dt n )](0) = E(X m Y n )

5 G. Variance and measures of dispersion for conditional and marginal probability distributions 1. Conditional variance: Var(X Y) = E((X - E(X Y)) 2 X) 2. Marginal variance: Var(X) of the marginal distribution is equal to Var(X) of the joint distribution 3. Law of total variance: Var(X) = E(Var(X Y)) + Var(E(X Y)) H. Covariance and correlation coefficients 1. Discrete covariance: Cov(X, Y) = (sum from x = - to )(sum from y = - to ) (x - μx)(y - μy)pxy(x, y) 2. Continuous covariance: Cov(X, Y) = (integral from x = - to )(integral from y = - to ) (x - μx)(y - μy)fxy(x, y)dydx 3. Mixed covariance: Cov(X, Y) = (sum from x = - to )(integral from y = - to ) (x - μx)(y - μy)fxy(x, y)dydx 4. Symmetric property of covariance: Cov(X, Y) = Cov(Y, X) 5. Covariance as expected values: Cov(X, Y) = E(XY) - E(X)E(Y) 6. Affine covariance: Cov(aX + b, Y) = acov(x, Y) 7. Correlation coefficient: ρ(x, Y) = Cov(X, Y)/(Var(X)Var(Y)) 1/2 I. Transformations and order statistics 1. Product distribution: fx*y(x) = (integral from t = - to ) [fxy(t, x/t)/ t ]dt 2. Ratio distribution: fx / Y(x) = (integral from t = - to ) fxy(xt, t) t dt 3. Multivariate transformation: If U = g(x, Y) and V = h(x, Y) such that g and h are one-to-one, then fuv(u, v) = fxy(g (u, v), h (u, v)) det(j), J being the Jacobian matrix [ (u, v)/ (x, y)] 4. Min. independent distribution: fmin over {X_i}(x) = -d[(product over i) (1 - FX_i(x))]/dx 5. Max. independent distribution: fmax over {X_i}(x) = d[(product over i) FX_i(x)]/dx 6. Min. dependent distribution: fmin over {X_i}(x) = -d[(integrals from {ti} = x to ) f{x_i}({ti})d{ti}]/dx 7. Max. dependent distribution: fmax over {X_i}(x) = d(f{x_i}(x, x,, x))/dx J. Probabilities and moments for linear combinations of independent random variables 1. Discrete convolution: px + Y(n) = (sum from m = - to ) pxy(n - m, m) 2. Continuous convolution: fx + Y(x) = (integral from t = - to ) fxy(x - t, t)dt 3. Moments of linear combination: E((aX + by) n ) = (sum from i = 0 to n) ncia i b n - i E(X i )E(Y n - i ) 4. Expected value of sum: E(X + Y) = E(X) + E(Y) 5. Variance of sum: Var(X + Y) = Var(X) + 2Cov(X, Y) + Var(Y) 6. Covariance of sum: Cov(X + Y, Z) = Cov(X, Z) + Cov(Y, Z)

6 IV. Additional topics, formulae, shortcuts, etc. (?%) A. General Probability 1. Set membership: a A means a is an element of A 2. Contains: A a means A contains the element a 3. Symmetry properties: A B = B A, A B = B A 4. Set difference as intersection: A \ B = A B 5. Empty set: is the set of no elements 6. Inclusion-Exclusion Principle: (union over i)ai = (sum from k = 1 to n) [(-1) k + 1 (sum over all B of size k such that B {Ai}) (intersection over all elements of B) ] 7. Method of partitions: P(A B) + P(A B ) + P(A B) + P(A B ) = 1 8. Method of decomposition: 1 = P(A B ) + P(A B), P(A B) = P(B \ A) + P(A), P(A) = P(A \ B) + P(A B) B. Univariate Probability Distributions 1. Darth vader rule: If Y is a random variable X after deductible d and payout cap u, then E(Y) = (integral from x = d to d + u) S(x)dx and E(Y 2 ) = 2(integral from x = d to d + u) (x - d)s(x)dx 2. Bernoulli pf: p(r) = {1 - p, r = 0; p, r = 1} 3. Bernoulli ev: E(X) = p 4. Bernoulli var: Var(X) = p - p 2 5. Bernoulli mgf: MX(t) = pe t 6. Chi-squared pdf: The chi-squared distribution is a special case of the gamma distribution where λ = 1/2 and α = n/2. In particular, f(x) = x n/2-1 e -x/2 /(2 n/2 Γ(n/2)) 7. Chi-squared cdf: No elementary form 8. Chi-squared ev: E(X) = n 9. Chi-squared var: Var(X) = 2n 10. Chi-squared mgf: (1-2t) -n/2 11. Memorylessness of exponential: If X is exponential, E(X x > d) = E(X) + d 12. Memorylessness of geometric: If X is geometric, E(X x > d) = E(X) + floor(d) Poisson approximation to the binomial: A binomial variable with n >> 1 and p << 1 is approximately Poisson with parameter λ = np 14. Normal approximation to the binomial: A binomial variable with n >> 1 is approximately normal with parameters μ = np and σ = (np(1 - p)) 1 / Better definition of median: The median is E(X F(x) = 0.5) 16. Better definition of percentile: The nth percentile is E(X F(x) = n / 100) 17. Dirac delta: δ(x) allows mixed distributions to be expressed as continuous; it is the limiting normal distribution as σ approaches 0; also, (integral from x = - to ) δ(x)f(x) = f(0) C. Multivariate Probability Distributions 1. Shortcuts for independent variables: If X and Y are independent, E(XY) = E(X)E(Y), Cov(X, Y) = 0, ρ(x, Y) = 0, fxy(x, y) = fx(x)fy(y), and Var(X + Y) = Var(X) + Var(Y) 2. Moment generating function of a sum: MX + Y(t) = MX(t)MY(t) 3. Bivariate normal convolution: The convolution of a bivariate normal distribution is normal with mean μx + μy and variance (σx) 2 + (σy) 2 + 2ρσXσY