CS 237: Probability in Computing Wayne Snyder Computer Science Department Boston University Lecture 10: o Cumulative Distribution Functions o Standard Deviations Bernoulli Binomial Geometric
Cumulative Distribution Functions One more topic before lab next week and before considering the standard distributions... The Cumulative Distribution Function (CDF) for a random variable X shows what happens when we keep track of the sum of the probability distribution from left to right over its range: Example: X = The number of dots showng on a thrown die Probability Distribution Function f X Cumulative Distribution Function F X
Standard Distributions Any random variable X has a distribution which can be characterized by o R X o f X o F X o E(X) -- The range of the random variable -- The probability distribution function (sometimes this is called the probability mass function or PMF) -- The CDF -- Expected value o Var(X) -- Variance o! X -- Standard Deviation In addition, we are interested in o Formulae for calculating f X and F X, if such exist (hopefully efficient!) o Algorithms for generating random variates o Any special properties of the distribution (e.g., the memoryless property ) o Applications (random experiments which follow that distribution)
Standard Distributions We will look at the following distributions in the next week or so: Discrete Distributions o Bernoulli o Binomial o Geometric o Poisson Continuous Distributions o Normal o Exponential These are summarized, with useful code for displaying the PMF and CDF in the notebook Distributions.ipynb on the class web site. Wikipedia has very good pages on all these distributions. Your textbook is a little spotty on organizing this material.
Bernoulli Distribution Suppose you have a coin where the probability of a heads is p and we define the random variable X = the number of heads showing on a flipped coin Then we say that X is distributed according to the Bernoulli Distribution with parameter p, and write this as: where and where Among other accomplishments, Bernoulli discovered the number e (but Euler got the credit for Euler s Number ). e =
Bernoulli Distribution Each poke of such a random variable is called a Bernoulli Trial, and the outcomes are sometimes labelled as 1 = Success 0 = Failure Bernoulli Trials, and Bernoulli random variables, describe simple random experiments where there are two possible outcomes, and the probability of the outcomes is fixed by p and (1-p); the notion of success and failure is just a convenience and does not always correspond to the desirability of the outcome: o Will I pass this course or not? o Am I pregnant or not? o Do I have cancer or not?
Bernoulli Distribution The probability distribution is easy to visualize: Not much more to say about this, except as a foundation for more complicated distributions such as the Binomial...
Binomial Distribution The Binomial Distribution occurs when you count the number of successes in N independent and identically distributed Bernoulli Trials (i.e., p is the same each time). Formally, if Y ~ Bernoulli(p), and N times X = The number of successes in N trials of Y = Y + Y +... + Y then we say that X is distributed according to the Binomial Distribution with parameters N and p, and write this as: Note: k successes and N-k failures: where SSS...FFFF... has probability p k (1-p) N-k and there are C(N,k) such sequences. and where
Binomial Distribution For the visual display of this distribution, there are two parameters, and hence two dimensions along which the distribution may vary, and we will look briefly at the Distributions notebook to get a sense for this... The motivation for this distribution comes from the fact that many complex phenomena are composed of the additive effect of many small binary choices or events (Bernoulli Trials!); a vivid illustration of this can be seen in the Galton Board or Quicunx: https://www.mathsisfun.com/data/quincunx.html Phenomena explained by the binomial are widespread throughout ordinary life, biology, engineering, and business: o You go through 10 traffic lights; what is the probability that you stop at 4 of them? o The probability of any individual in this class having a tattoo is 0.2; what is the probability that at least 40 people have a tattoo? o Suppose you have not studied for a multiple-choice test and you randomly guess at each problem; what is the probability that you pass the test? o Suppose 5% of tax returns are submitted with fraudulent data and the IRS examines 1% of returns; what is the probability that they will detect 3% of all fraudulent returns?
Binomial Distribution The binomial distribution is of widespread applicability, but it has a disadvantage: the only way to compute probabilities is to use the formula and this can involve some very large numbers... for example, is:
Binomial Distribution The binomial distribution is of widespread applicability, but it has a disadvantage: the only way to compute probabilities is to use the formula and this can involve some very large numbers... for example, is: 25306860788368395425009996518100609077068723635393882065500412722438406556357 59019339192864845058090054648588847516373207436678575382395143428849429230095 48452463962038917241970441315347251288293140458085203641975429015396451120702 90054172676007088840418254863094094181054099386605927195740729486409692056617 18423204598807405832967747589281702974353796541551881065990864294962746741105 93266103962513789521968236641998601608992292111723427844080079206219850932895 94893598742945434967990132009169690766577216007716147870782048492669875259433 87372044495991001950014092399556127332333759349165922429479347018286914614395 87257149876168747889795948911236702108322879910187970022052679201806406384025 98651617475613895716421284464721781536343381888676773053733597398324984189651 96131689184732588697226168384517309056386524294344846582784027540865916271808 12730710494080 or about 10 861. There are about 10 87 atoms in the universe...
Binomial Distribution The CDF also has no easy, efficient formula, you can only sum up the numbers involved: Therefore, before modern computers, it was important to find more efficient ways to compute the binomial; we will see that many results in the history of probability were motivated by the desire to find easy-to-compute approximations to the binomial.
Geometric Distribution The Geometric Distribution occurs when you count the number of independent and identically distributed Bernoulli trials until the first success. Formally, if Y ~ Bernoulli(p), and X = The number of trials of Y until the first success then we say that X is distributed according to the Geometric Distribution with parameter p, and write this as: Note: k-1 failures and 1 success: where FFFF... S has probability (1-p) k-1 p and where
Geometric Distribution Fortunately, the probability mass function and the CDF are both easy to compute: