Probability Binomial Distributions SOCY601 Alan Neustadtl
In the 1870s, Sir Francis Galton created a device he called a quincunx for studying probability. The device was made up of a vertical board with a chute at the top. The chute was filled with marbles which were dispensed into an array of pegs. The pegs acted as obstructions, forcing the marbles to change direction, the choice of direction being random. At the bottom of the quincunx was a set of bins for catching the marbles. Galton's original sketch of the quincunx illustrates the setup and the idea of studying and explaining that final distribution of marbles among bins. He also shows a possible arrangement of marbles in their bins after completing their journey. Quincunx
Quincunx Notice the arrangement of pegs as alternating rows so that between two pegs in one row there is a peg in the next row. This falling marble will strike a peg in each row as it progresses. The word quincunx refers to any arrangement of five objects in a rectangle, one object at each corner and one in the middle. The word is often generalized to mean anything made up of such patterns of five. The goal is to describe, mathematically and probabilistically, the possible resting places for a marble passing through the quincunx.
Quincunx Galton s interest was not in the mean but in the distribution of deviations from it. he came independently to view the Gaussian distribution not primarily as a way of differentiating true values from false ones but as a tool for analyzing populations in terms of their members variations from a mean the kind of variations inevitably manifest in, for example, the heights or weights of a large, randomly selected group of people. Eventually, he concluded that there was scarcely anything so apt to impress the imagination as the wonderful form of cosmic order expressed by the Law of Frequency of Error, (Kevles 1985, 13-4)
Quincunx Concerning the quincunx, Stigler wrote, its use to provide an analogue proof that a normal mixture of normal distributions was itself normal was a stroke of genius The crucial point is that Galton s conceptual use of the result was new and ingenious and represents the most important step in perhaps the single major breakthrough in statistics in the last half of the nineteenth century. This concept freed Galton of the restrictions of naive error theory without requiring that he go beyond that theory. He could conceive of his data as a mixture of very different populations, notwithstanding the unity apparent in its normal outline. (Stigler 1986, 281)
Continuous vs. Discrete Distributions Unlike normal distributions which are continuous, the binomial distributions is discrete. Discrete distributions provide nice bridges linking frequency histograms with probability, on the way to understanding probability with continuous distributions. The binomial distribution is one kind of discrete probability distribution.
Binomial Distribution The most basic element of a binomial distribution is a Bernoulli experiment. A Bernoulli experiment is a simple experiment with only two possible outcomes. This could be something like flipping a coin (heads or tails). It could also be something like rolling a die and getting a three. Here, our experimental outcome is three (3) or everything else (1,2,4,5,6). Almost any phenomenon can be reduced to a binomial outcome. We denote these outcomes success and failure, although there is no implied normative meaning. So, for instance, taking a treatment for cancer either leads to an improvement or not. If we are interested in the likelihood that a treatment does not improve a cancer patients condition, success is failure to treat the disease.
Binomial Distribution Success is normally noted as p; failure as q. Since there are only two outcomes, we know that the sum of the two probabilities must equal 1, or p + q = 1. Furthermore, q = 1-p and p = 1-q. The binomial distribution is known as a one parameter distribution because if you know one outcome, you implicitly know the other
Binomial Distribution A binomial experiment is a number of independent Bernoulli experiments. Specifically, a binomial experiment has the following properties. The experiment consists of n identical trials. Each trial results on one of two possible outcomes. The probability of success on a single trial, p, is invariant across trials, and q = 1 - p. The trials are independent. We are interested in r, the number of successes in n trials. The number of successes, r, represents a binomial variable. What is interesting about this variable is that we can associate a probability with each outcome, r, and derive a probability distribution.
Binomial Distribution Example Let's look at an easy example. If we toss a fair coin, defining getting a head on a toss as success, we know a priori that p = 0.5 and q = 0.5. We propose to flip this coin four times; this is our binomial experiment. Each flip of the coin, or trial, constitutes a Bernoulli experiment or trial. How do we enumerate success? We know that in four flips of the coin that there are five possible values of r; 0, 1, 2, 3, and 4. The binomial distribution associates a probability with each of these outcomes. So, for instance, what is the probability that r equals 4 [p(r=4)]? This is asking what is the probability of p(h and H and H and H). This is equal to (p)(p)(p)(p) = p 4 = 0.5 4 = 0.0625. Thus, there is a 6.25% chance of tossing a coin four times and observing a head on each toss.
Binomial Distribution Example What about getting one head and three tails [p(r=1)]? How many ways are there to get one head and three tails? We can list them as follows: H T T T T H T T T T H T T T T H There are 4 ways to get 1 success and they have equal probability
Binomial Distribution Example These are the probabilities associated with each of the possible outcomes: p(httt) = (p)(q)(q)(q) = pq 3 = (0.5)(0.5 3 ) = 0.0625 p(thtt) = (q)(p)(q)(q) = pq 3 = (0.5)(0.5 3 ) = 0.0625 p(ttht) = (q)(q)(p)(q) = pq 3 = (0.5)(0.5 3 ) = 0.0625 p(ttth) = (q)(q)(q)(p) = pq 3 = (0.5)(0.5 3 ) = 0.0625 Summing these independent probabilities we get 0.0625+0.0625+0.0625+0.0625 = 0.25. Thus, there is a 25% chance that in four flips, we observe a single head.
Binomial Distribution Example We can also use this method to calculate all the other probabilities for the values of r. But, there is a shortcut. What we are actually asking is, given 4 trials, how many ways can we distribute r successes across these trials. This then becomes a simple combination problem. The formula for combinations is: n n! = r r! ( n r)!
Binomial Distribution Example r 0 Combinations 4 4! = = 1 0 0! ( 4 0 )! 1 4 4! = = 1 1! ( 4 1 )! 4 2 4 4! = = 6 2 2! ( 4 2 )! 3 4 4! = = 3 3! ( 4 3 )! 4 4 4 4! = = 1 4 4! ( 4 4 )!
Binomial Distribution Example We can also use a general formula to calculate the individual probabilities associated with any one outcome of r. This formula is: pq r n r Putting both this formula and that for combinations together, we get: n pq r n r r
Binomial Distribution Example p(r) Binomial Probability 4 0.5 0.5 0 0 4 p(0) ( )( ) 4 0.5 0.5 1 1 3 p(1) ( )( ) 4 0.5 0.5 2 2 2 p(2) ( )( ) 4 0.5 0.5 3 3 1 p(3) ( )( ) 4 0.5 0.5 4 4 0 p(4) ( )( ) 0.0625 0.2500 0.3750 0.2500 0.0625
Binomial Distribution Example Binomial Distribution b(4, 0.5) p(r) 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 0 1 2 3 4 Number of Successes
Binomial Probability Beer Example Let's look at an example of using binomial probability to solve a problem close to the hearts of many of us; beer testing. Let s set up an experiment to see if three people can identify a premium beer by taste and taste alone. Here is what we will do: Pour three glasses of beer where two glasses are pedestrian beers and one is a premium beer. Each of three evaluators will sample from each beer and identify the one beer that tastes best to them. We will compare these empirical results with what chance could produce as shown in a theoretical model provided by the binomial distribution.
Is this really a binomial problem? The experiment consists of n independent trials. We satisfy this requirement since each experiment is exactly the same. Three beers are tasted; two are low-end beers and one is a premium beer Each trial results in one of two possible outcomes. We satisfy this requirement since each subject either identifies the premium beer as their favorite (success) or they identify one of the other two (failure). The probability of success on a single trial p is invariant across trials and q=1-p. A priori, the probability of success due solely to chance for any given trial is failure is equal to or 1-p. The trials are independent of each other. Since the judgments of one evaluator does not affect later evaluations, the trials are independent. We are interested in r, the number of successes in n trials. We don t yet know r, but will derive it from the results of our experiment.
Here is what we know: The probability of success is 1/3 (p=0.33) The probability of failure is 2/3 (p=0.67) The total number of successes (r) is equal to 0, 1, 2, or 3.
Longhand Solution 0 1 2 3 Outcome fff sff fsf ffs fss sfs ssf sss 0 0.296 1 0.148 1 0.148 1 0.148 2 0.074 2 0.074 2 0.074 3 0.370 0.296 0.444 0.222 0.037 While p(fff) and p(sss) can occur in only one way, the other two possible outcomes can occur three sequences each. One success is observed by p(sff), p(fsf), and p(ffs). Two success can are observed by p(ssf), p(sfs), and p(fss). In other words, there are three sequences for each of these outcomes and their probabilities have to be summed to get the probability of the combination of, respectively, one success and two successes. p p p p ( fff ) ( sff ) ( ssf ) = = = 2 2 2 = 3 3 3 1 2 2 = 3 3 3 2 1 1 = 3 3 3 1 1 1 3 3 3 8 27 4 27 2 27 1 27 0.296 0.148 0.074 ( sss) = = = 0. 037 = = =
Longhand Solution This can be summarized in tabular and graphical form: p(0) 0.296 p(1) 0.444 p(2) 0.222 p(3) 0.037 4 p 1.000 i= 1 Probability of Successe 0.50 0.40 0.30 0.20 0.10 0.00 Binomial Distribution (n=3, p=0.33, q=0.67) 0 1 2 3 Number of Successes
Longhand Solution And now we can answer a number of typical questions such as what is the probability that at least 2 of the three people correctly identify the correct beer by chance: p 2 3 = p 2 + p 3 ( ) ( ) ( ) = ( 0.222) + ( 0.37) = 0.259 Probability of Successe 0.50 0.40 0.30 0.20 0.10 Binomial Distribution (n=3, p=0.33, q=0.67) Or, the probability of one or two successes: ( 1 2) = ( 1) + ( 2) = ( 0.444) + ( 0.222) p p p = 0.666 0.00 0 1 2 3 Number of Successes
A More Realistic Example It turns out that this experiment was actually done and reported in the April 1978 issue of Consumer Reports. They used three samples of beer, but tested 24 individuals. In other words:, p=0.33, q=0.67, and n=24. Their results indicated that 11 people, or 45.8%, successfully identified the correct beer. As a percentage, this appears high, but we must always measure this against chance.
A More Realistic Example Using the formula for binomial probability, we can produce the following table and graphic: Binomial Probability Distribution (p=1/3, q=2/3, n=24) Cumulative r p(r) p(r) 0 0.000059 1.000000 1 0.000713 0.999941 2 0.004099 0.999228 3 0.015029 0.995129 4 0.039451 0.980100 5 0.078902 0.940649 6 0.124929 0.861746 7 0.160623 0.736818 8 0.170661 0.576195 9 0.151699 0.405534 10 0.113774 0.253835 11 0.072402 0.140061 12 0.039218 0.067659 13 0.018100 0.028441 14 0.007111 0.010341 15 0.002370 0.003230 16 0.000667 0.000859 17 0.000157 0.000193 18 0.000031 0.000036 19 0.000005 0.000005 20 0.000001 0.000001 21 0.000000 0.000000 22 0.000000 0.000000 23 0.000000 0.000000 24 0.000000 0.000000 Probability of Successe 0.20 0.15 0.10 0.05 0.00 Binomial Distribution (n=24, p=0.33, q=0.67) 0 2 4 6 8 10 12 14 16 18 20 22 24 Number of Successes
A More Realistic Example Now, the question we must answer is how likely is it to record 11 successes out of 24 given the likelihood of success due to chance alone? To do this we use hypothesis testing. Consider what could produce these findings: There is actually a measurable taste difference among these two beers. random error, sampling error, or chance.
Hypothesis Testing Assume the null hypothesis of no difference --In this case, we assume that there is really no difference in the beers. If this is the case, then we expect success approximately one-third of the time, and failure twothirds of the time by chance and chance alone. We have to have an idea of the range of outcomes In this example, the appropriate distribution is the binomial distribution, and we need to apply the binomial formula to the relevant information. How much evidence do we need? Here, we are talking about setting an alpha (α) level. To be generous with this test, we will use an alpha equal to 0.05, and use a one-tailed test. We need to go to the real world and collect data--by conducting the experiment 24 times, we have collected our data. We have to compare the real world with our theoretical distribution In other words, if our real and theoretical data are similar, we fail to reject the null hypothesis and conclude that chance produced this outcome. Alternatively, if we find this a rare event, we would reject the null hypothesis and conclude that a better product served to differentiate it from its competitors. From the above probability distribution we find that p(r 11) = 0.14. This does not fall in the critical region, and so we fail to reject the null hypothesis.
Large Sample Binomial Approximation Clearly, it can be tedious to calculate all the exact probabilities when there are lots of trials. We can take a shortcut when we have a large sample. We can use the large sample binomial approximation, which is basically a z-score type of problem. We can define the mean and standard deviation of a binomial distribution as follows: X s B B = = np npq Based on this we can define a z-score as: z B X X X np B = = s B npq
Large Sample Binomial Approximation In this example, this equals: z B ( )( ) ( 24)( 0.33)( 0.67) 11 24 0.33 11 8 = = = 1.299 2.309 z critical is equal to 1.96. As before, we find that we do not reject the null hypothesis. But notice, that the area beyond z B is approximately 0.0968, which obviously does not equal 0.1407. Remember, this is only an approximation which in this example leads to the same conclusion; it need not be so! If possible, compute the actual probability.
Heart Surgery Example From past experience a surgeon knows that the probability that a patient recovers from a delicate heart operation is 0.87. What is the probability that at least 90% of the next 150 patients having this operation survive? So, p=0.87, q=0.13, and n=150. The number of survivors we are interested in is: z 1 ( )( ) 0.9 150 = 135 The z-score for this value is equal to: ( )( ) ( 150)( 0.87)( 0.13) 135 150 0.87 135 130.5 = = = 1.17 3.84 The areas up to and including the mean is equal to 0.5000. The area between the mean and a z-score of 1.17 is equal to 0.3790. Therefore, the probability that between 0 and 135 patients survive (less than 95%) is equal to 0.8790. The area beyond, 1.000-0.8790 is equal to the probability that at least 90% of the next 150 patients will survive; p(x 135)=0.1210.
Public Opinion Example General Social Survey Question: Please tell me whether or not you think it should be possible for a pregnant woman to obtain a legal abortion if the woman wants it for any reason? In 2000, 38.9% of the GSS respondents responded yes. Take a random sample of n=8 people and let Y equal the number of who respond yes. The distribution if Y is b(8, 0.389) Calculate the probability that at least three of these eight people agree with this statement, or p(y 3)
Public Opinion Example ( 3) = p( 3 4 5 6 7 8) = p( 3) + p( 4) + p( 5) + p( 6) + p( 7) + p( 8) = 1 py ( 3) = 1 p( 0) + p( 1) + p( 2) py p p p 8 0 = 0.389 0.611 = 0.0194 0 ( ) ( 0)( 8) 8 1 = 0.389 0.611 = 0.0989 1 () ( 1)( 7) 8 2 = 0.389 0.611 = 0.2204 2 ( ) ( 2)( 6) ( ) py 3 = 1 0.3388 = 0.6612
Public Opinion Example ( ) py 3 = 1 0.3388 = 0.6612
Binomial Approximation Example Title VII of the Civil Rights Act of 1964 prohibits discrimination on the basis of race, color, religion, sex, or national origin. Statistical methods were used in the case Hazelwood School District v. United States (433 U.S. 299, 1977). In a Supreme Court decision it was noted that of 405 teachers hired by Hazelwood during the 1972-73 and 1973-74 school years, 15 or 3.7% were black. In the relevant labor market defined as St. Louis County the percentage of black teachers was 5.7%. Do these two percentages differ significantly?
Binomial Approximation Example In a Supreme Court decision it was noted that of 405 teachers hired by Hazelwood during the 1972-73 and 1973-74 school years, 15 or 3.7% were black. In the relevant labor market defined as St. Louis County the percentage of black teachers was 5.7%. Do these two percentages differ significantly? X s B X B = 15 ( 0.057)( 405) ( )( )( ) 405 0.057 0.943 = 15 2 4.666 = 1.71 α = 0.05 z = 1.71 area between mean and z = 0.4564 area beyond z = 0.0436
Binomial Example r 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Probability 0.00000000005 0.00000000116 0.00000001422 0.00000011543 0.00000070120 0.00000339924 0.00001369790 0.00004719456 0.00014192149 0.00037840714 0.00090576945 0.00196600782 0.00390178540 0.00712977361 0.01206693392 0.01901277881 ( 15) = ( 0) + ( 1) + ( 15) p r p p p 0.0456 α = 0.05 z = 1.71 area between mean and z = 0.4564 area beyond z = 0.0436
Binomial Distribution