Random Samples. Mathematics 47: Lecture 6. Dan Sloughter. Furman University. March 13, 2006

Random Samples Mathematics 47: Lecture 6 Dan Sloughter Furman University March 13, 2006 Dan Sloughter (Furman University) Random Samples March 13, 2006 1 / 9

Random sampling Definition We call a sequence X 1, X 2,..., X n of random variables a sample. We call a sample X 1, X 2,..., X n a random sample if X 1, X 2,..., X n are independent and identically distributed. If X 1, X 2,..., X n is a random sample, g : R n R, and W = g(x 1, X 2,..., X n ), we call W a statistic. Dan Sloughter (Furman University) Random Samples March 13, 2006 2 / 9

Random sampling Definition We call a sequence X 1, X 2,..., X n of random variables a sample. We call a sample X 1, X 2,..., X n a random sample if X 1, X 2,..., X n are independent and identically distributed. If X 1, X 2,..., X n is a random sample, g : R n R, and W = g(x 1, X 2,..., X n ), we call W a statistic. Note: A random sample corresponds to repeated independent experiments. Dan Sloughter (Furman University) Random Samples March 13, 2006 2 / 9

Random sampling Definition We call a sequence X 1, X 2,..., X n of random variables a sample. We call a sample X 1, X 2,..., X n a random sample if X 1, X 2,..., X n are independent and identically distributed. If X 1, X 2,..., X n is a random sample, g : R n R, and W = g(x 1, X 2,..., X n ), we call W a statistic. Note: A random sample corresponds to repeated independent experiments. When sampling from a finite population (such as balls in an urn, or voters in a state), this would mean sampling with replacement. In particular, sampling without replacement does not, by this definition, produce a random sample. Dan Sloughter (Furman University) Random Samples March 13, 2006 2 / 9

Random sampling Definition We call a sequence X 1, X 2,..., X n of random variables a sample. We call a sample X 1, X 2,..., X n a random sample if X 1, X 2,..., X n are independent and identically distributed. If X 1, X 2,..., X n is a random sample, g : R n R, and W = g(x 1, X 2,..., X n ), we call W a statistic. Note: A random sample corresponds to repeated independent experiments. When sampling from a finite population (such as balls in an urn, or voters in a state), this would mean sampling with replacement. In particular, sampling without replacement does not, by this definition, produce a random sample. However, sampling without replacement from a finite population is often referred to as random sampling, even if it does not produce a random sample. Dan Sloughter (Furman University) Random Samples March 13, 2006 2 / 9

Sampling The question of how one actually obtains a random sample, or performs random sampling in the case of a finite population, is not a simple one. Dan Sloughter (Furman University) Random Samples March 13, 2006 3 / 9

Sampling The question of how one actually obtains a random sample, or performs random sampling in the case of a finite population, is not a simple one. For scientific procedures, such as measuring the length of the lifetime of a light bulb, the process is a matter of carefully replicating a given experiment. Dan Sloughter (Furman University) Random Samples March 13, 2006 3 / 9

Sampling The question of how one actually obtains a random sample, or performs random sampling in the case of a finite population, is not a simple one. For scientific procedures, such as measuring the length of the lifetime of a light bulb, the process is a matter of carefully replicating a given experiment. Sampling a human population is far more difficult. People who have to do this routinely, such as the Bureau of Labor Statistics, develop sophisticated multiple layer sampling procedures, the properties of which may be analyzed mathematically. Dan Sloughter (Furman University) Random Samples March 13, 2006 3 / 9

Sample (cont d) Samples which are drawn from voluntary response, such as call-in surveys or mail response surveys, are not examples of random sampling. Dan Sloughter (Furman University) Random Samples March 13, 2006 4 / 9

Sample (cont d) Samples which are drawn from voluntary response, such as call-in surveys or mail response surveys, are not examples of random sampling. Famous examples of large samples producing erroneous results abound, such as the 1936 Literary Digest prediction that Alf Landon would defeat Franklin Roosevelt in the presidential election. Dan Sloughter (Furman University) Random Samples March 13, 2006 4 / 9

Sample (cont d) Samples which are drawn from voluntary response, such as call-in surveys or mail response surveys, are not examples of random sampling. Famous examples of large samples producing erroneous results abound, such as the 1936 Literary Digest prediction that Alf Landon would defeat Franklin Roosevelt in the presidential election. On the other hand, when done properly, random sampling can produce results which are even better than attempts at complete population enumeration. The current problems with producing an accurate census of the population of the United States is an example. Dan Sloughter (Furman University) Random Samples March 13, 2006 4 / 9

Likelihood Definition Suppose X 1, X 2,..., X n is a sample with joint probability function f having parameters θ 1, θ 2,..., θ m. We call L a likelihood function for X 1, X 2,..., X n if for some constant k. L(θ 1, θ 2,..., θ m ) = kf (x 1, x 2,..., x n θ 1, θ 2,..., θ m ), Dan Sloughter (Furman University) Random Samples March 13, 2006 5 / 9

Likelihood Definition Suppose X 1, X 2,..., X n is a sample with joint probability function f having parameters θ 1, θ 2,..., θ m. We call L a likelihood function for X 1, X 2,..., X n if for some constant k. L(θ 1, θ 2,..., θ m ) = kf (x 1, x 2,..., x n θ 1, θ 2,..., θ m ), Note: given observations x 1, x 2,..., x n, L contains all the information about θ 1, θ 2,..., θ m. Dan Sloughter (Furman University) Random Samples March 13, 2006 5 / 9

Likelihood Definition Suppose X 1, X 2,..., X n is a sample with joint probability function f having parameters θ 1, θ 2,..., θ m. We call L a likelihood function for X 1, X 2,..., X n if for some constant k. L(θ 1, θ 2,..., θ m ) = kf (x 1, x 2,..., x n θ 1, θ 2,..., θ m ), Note: given observations x 1, x 2,..., x n, L contains all the information about θ 1, θ 2,..., θ m. Moreover, if X 1, X 2,..., X n is a random sample and each X i, i = 1, 2,..., n, has probability function f, then L(θ 1, θ 2,..., θ m ) n f (x i θ 1, θ 2,..., θ m ). i=1 Dan Sloughter (Furman University) Random Samples March 13, 2006 5 / 9

Example Dan Sloughter (Furman University) Random Samples March 13, 2006 6 / 9

Example Suppose X 1, X 2,..., X n is a random sample from an exponential distribution { λe λx, if x > 0, f (x λ) = 0, otherwise. Dan Sloughter (Furman University) Random Samples March 13, 2006 6 / 9

Example Suppose X 1, X 2,..., X n is a random sample from an exponential distribution { λe λx, if x > 0, f (x λ) = 0, otherwise. Then n f (x i λ) = i=1 {λ n e λ P n i=1 x i, if x i > 0, i = 1, 2,..., n, 0, otherwise. Dan Sloughter (Furman University) Random Samples March 13, 2006 6 / 9

Example Suppose X 1, X 2,..., X n is a random sample from an exponential distribution { λe λx, if x > 0, f (x λ) = 0, otherwise. Then Hence n f (x i λ) = i=1 L(λ) = is a likelihood function. {λ n e λ P n i=1 x i, if x i > 0, i = 1, 2,..., n, 0, otherwise. {λ n e λ P n i=1 x i, if λ > 0, 0, otherwise, Dan Sloughter (Furman University) Random Samples March 13, 2006 6 / 9

Example Dan Sloughter (Furman University) Random Samples March 13, 2006 7 / 9

Example Suppose X 1, X 2,..., X n is a random sample from N(µ, σ 2 ) (that is, a normal distribution with mean µ and variance σ 2 ). Dan Sloughter (Furman University) Random Samples March 13, 2006 7 / 9

Example Suppose X 1, X 2,..., X n is a random sample from N(µ, σ 2 ) (that is, a normal distribution with mean µ and variance σ 2 ). Then n i=1 ( ) n 1 ( ) e 1 2σ 2 (x i µ) 2 1 2 1 = 2πσ 2π σ n e 1 P n 2σ 2 i=1 (x i µ) 2. Dan Sloughter (Furman University) Random Samples March 13, 2006 7 / 9

Example Suppose X 1, X 2,..., X n is a random sample from N(µ, σ 2 ) (that is, a normal distribution with mean µ and variance σ 2 ). Then Hence n i=1 L(µ, σ 2 ) = ( ) n 1 ( ) e 1 2σ 2 (x i µ) 2 1 2 1 = 2πσ 2π σ n e 1 P n 2σ 2 i=1 (x i µ) 2. is a likelihood function. { ( 1 σ 2 ) n 2 e 1 2σ 2 P n i=1 (x i µ) 2, if σ 2 > 0, < µ <, 0, otherwise, Dan Sloughter (Furman University) Random Samples March 13, 2006 7 / 9

Example Dan Sloughter (Furman University) Random Samples March 13, 2006 8 / 9

Example Suppose X 1, X 2,..., X n is a random sample from a uniform distribution on the interval (0, θ). Dan Sloughter (Furman University) Random Samples March 13, 2006 8 / 9

Example Suppose X 1, X 2,..., X n is a random sample from a uniform distribution on the interval (0, θ). That is, X 1, X 2,..., X n is a random sample from a distribution with probability function 1, if 0 < x < θ, f (x θ) = θ 0, otherwise, for some θ > 0. Dan Sloughter (Furman University) Random Samples March 13, 2006 8 / 9

Example Suppose X 1, X 2,..., X n is a random sample from a uniform distribution on the interval (0, θ). That is, X 1, X 2,..., X n is a random sample from a distribution with probability function 1, if 0 < x < θ, f (x θ) = θ 0, otherwise, for some θ > 0. Then n 1 f (x i θ) = θ n, if 0 < x i < θ, i = 1, 2,..., n, 0, otherwise. i=1 Dan Sloughter (Furman University) Random Samples March 13, 2006 8 / 9

Example (cont d) Dan Sloughter (Furman University) Random Samples March 13, 2006 9 / 9

Example (cont d) Hence 1 L(θ) = θ n, if θ > x (n), 0, otherwise, is a likelihood function. Dan Sloughter (Furman University) Random Samples March 13, 2006 9 / 9