Chapter 4: Asymptotic Properties of MLE (Part 3)
|
|
- Christal Wilkerson
- 5 years ago
- Views:
Transcription
1 Chapter 4: Asymptotic Properties of MLE (Part 3) Daniel O. Scharfstein 09/30/13 1 / 1
2 Breakdown of Assumptions Non-Existence of the MLE Multiple Solutions to Maximization Problem Multiple Solutions to Score Equations Number of Parameters Increase with the Sample Size Support of p(x; θ) depends on θ Non-I.I.D. Data 2 / 1
3 Non-Existence of the MLE The non-existence of the MLE may occur for all values of x n or for only some of them. In general, this is due either to the fact that the parameter space is not compact or that the log-likelihood is discontinuous in θ. Example 4.1: Suppose that X Bernoulli(1/(1 + exp(θ)), where Θ = R. If we observe x = 1, then L(θ; 1) = 1/(1 + exp(θ)). The likelihood function is a decreasing function of θ and the maximum is not attained on Θ. If Θ were closed, i.e., Θ = R, the MLE would be. Example 4.2: Suppose that X Normal(µ, σ 2 ). So, θ = (µ, σ 2 ) and Θ = R R +. Now, l(θ; x) log σ 1 (x µ) 2. Take 2σ 2 µ = x. Then as σ 0, l(θ; x) +. So, the MLE does not exist. 3 / 1
4 Multiple Solutions One reason for multiple solutions to the maximization problem is non-identification of the parameter θ. Example 4.3: Suppose that Y Normal(X θ, I ), where X is an n k matrix with rank smaller than k and θ Θ R k. The density function is p(y; θ) = (2π) n/2 exp( 1 2 (y X θ) (y X θ)) Since X is not full rank, there exists an infinite number of solutions to X θ = 0. That means that there exists an infinite number of θ s that generate the same density function. So, θ is not identified. Furthermore, note that the likelihood is maximized at all values of θ satisfying X X θ = X y. 4 / 1
5 Multiple Roots to the Score Equations Even though the score equations may have multiple roots for fixed n, we can still use our theorems to show consistency and asymptotic normality. This will work provided that as n gets large there is a unique maximum with large probability. Example 4.4: Suppose that X n = (X 1,..., X n ), where the X i s are i.i.d. Cauchy(θ, 1). We assume that θ 0 lies in the interior of a compact set Θ R. So, p(x; θ) = 1 π(1 + (x θ) 2 ) So, the log-likelihood for the full sample is l(θ; x) = n log π log(1 + (x i θ) 2 ) Note that as θ ±, l(θ; x). 5 / 1
6 Multiple Roots to the Score Equations The score for θ is given by dl(θ; x) dθ = 2(x i θ) 1 + (x i θ) 2 There can be multiple roots to the score equations. Regardless, the MLE is consistent (see Homework 2). 6 / 1
7 Number of Parameters Increase with the Sample Size Up to now, we have implicitly assumed that the number of parameters is equal to a fixed constant k. In some cases the number of parameters increases naturally with the number of observations. In such cases, the MLE may i. no longer converge ii. may converge to a parameter value different than θ 0 iii. may still converge to θ 0. In general, the outcome depends on the importance of the number of parameters relative to the number of observations. 7 / 1
8 Example 8.5: (Neyman-Scott, Econometrika, 1948) Suppose that X n = (X 1,..., X n ), where the X i s are independent with X i = (X i1, X i2 ), X i1 independent of X i2 and X ip N(µ i, σ 2 ) for p = 1, 2. We are interested in estimating the µ i s and σ 2. In this problem, we have n + 1 parameters. The likelihood function is L(µ 1,..., µ n, σ 2 ; x n ) = n It is easy to show that the MLE s are 1 2πσ 2 exp( 1 2σ 2 2 (X ip µ i ) 2 ) p=1 ˆµ i = 1 2 (X i1 + X i2 ) for i = 1,..., n ˆσ 2 = 1 2n p=1 2 (X ip ˆµ i ) 2 8 / 1
9 Example 4.5: (Neyman-Scott, Econometrika, 1948) Note that ˆµ i doesn t converge to µ i and we can show that ˆσ 2 converges in probability to σ 2 /2. To show this latter fact, note that we can express ˆσ 2 as 1 n 4n (X i1 X i2 ) 2. Let Z i = X i1 X 2σ i2. Then Z i N(0, 1) and Zi 2 is χ 2 1. Since we have an i.i.d. sample of Z 2 i s, we can employ the WLLN to show that 1 n This implies that ˆσ 2 = σ2 2 1 n Z 2 i P σ2 2 n Z 2 i P 1. 9 / 1
10 Example 4.6 Suppose that X n = (X 1,..., X n ), where the X i s are independent with X i = (X i1, X i2,..., X in ), X ip s are independent N(µ i, σ 2 ) random variables for p = 1, 2,..., n. We are interested in estimating the µ i s and σ 2. Again, we have n + 1 parameters. The likelihood function is L(µ 1,..., µ n, σ 2 ; x n ) = n 1 2πσ 2 exp( 1 2σ 2 (X ip µ i ) 2 ) p=1 It is easy to show that the MLE s are ˆµ i = 1 X ip for i = 1,..., n n p=1 ˆσ 2 = 1 n 2 p=1 (X ip ˆµ i ) 2 By the WLLN, we know that ˆµ i converges in probability to µ i and we can also show that ˆσ 2 converges in probability to σ / 1
11 Support of p(x; θ) depends on θ In this case, the MLE is frequently consistent, but not asymptotically normal. Example 4.7: Suppose X n = (X 1,..., X n ), where the X i s are i.i.d. from a shifted exponential. That is, p(x; θ) = exp( (x θ))i (x θ) Then, the likelihood for the full sample is L(θ; x n ) = exp( (x i θ))i (min x i θ) 11 / 1
12 Support of p(x; θ) depends on θ The MLE for θ is min X i or the first order statistic X (1). Note that the likelihood is not differentiable at the MLE. This violates condition (iv) of Theorem 4.6. We can show that the MLE is consistent. P θ0 [ X (1) θ 0 > ɛ] = P θ0 [X (1) θ 0 > ɛ] + P θ0 [X (1) θ 0 < ɛ] = P θ0 [X (1) > θ 0 + ɛ] + P θ0 [X (1) < θ 0 ɛ] n = P θ0 [X i > θ 0 + ɛ] = exp( nɛ) 0 12 / 1
13 Support of p(x; θ) depends on θ It is obvious that n(x (1) θ 0 ) cannot be centered at zero since X (1) is always greater than θ 0. W can show that n(x (1) θ 0 ) D Exponential(1). To see this, not that P θ0 [n(x (1) θ 0 ) a] = P θ0 [X (1) a/n + θ 0 ] = P θ0 [X i a/n + θ 0 ] n = exp( a) Here the rate of convergence is n instead of n. 13 / 1
14 Non-I.I.D. Data Example 4.8: Consider independent random variables Y i Normal(θx i, 1), where the x i s are given constants. The MLE of θ is ˆθ = x i Y i / xi 2 Normal(θ, 1/ xi 2 ) This estimator may not be consistent. Suppose that n x i 2 1. Then, we know that ˆθ D(θ 0) N(θ 0, 1), which is not θ 0. If n x 2 i, then ˆθ is consistent. To see this, note that ˆθ is unbiased and its variance goes to zero. 14 / 1
15 Non-I.I.D. Data What about the limiting distribution of n(ˆθ θ 0 )? We know that n xi 2(ˆθ θ 0 ) D N(0, 1) If n/ n x 2 i it converges at n x 2 i 1, then ˆθ converges at n rates. In general, rates. 15 / 1
Definition 9.1 A point estimate is any function T (X 1,..., X n ) of a random sample. We often write an estimator of the parameter θ as ˆθ.
9 Point estimation 9.1 Rationale behind point estimation When sampling from a population described by a pdf f(x θ) or probability function P [X = x θ] knowledge of θ gives knowledge of the entire population.
More informationLikelihood Methods of Inference. Toss coin 6 times and get Heads twice.
Methods of Inference Toss coin 6 times and get Heads twice. p is probability of getting H. Probability of getting exactly 2 heads is 15p 2 (1 p) 4 This function of p, is likelihood function. Definition:
More informationLecture 10: Point Estimation
Lecture 10: Point Estimation MSU-STT-351-Sum-17B (P. Vellaisamy: MSU-STT-351-Sum-17B) Probability & Statistics for Engineers 1 / 31 Basic Concepts of Point Estimation A point estimate of a parameter θ,
More informationEE641 Digital Image Processing II: Purdue University VISE - October 29,
EE64 Digital Image Processing II: Purdue University VISE - October 9, 004 The EM Algorithm. Suffient Statistics and Exponential Distributions Let p(y θ) be a family of density functions parameterized by
More informationTwo hours. To be supplied by the Examinations Office: Mathematical Formula Tables and Statistical Tables THE UNIVERSITY OF MANCHESTER
Two hours MATH20802 To be supplied by the Examinations Office: Mathematical Formula Tables and Statistical Tables THE UNIVERSITY OF MANCHESTER STATISTICAL METHODS Answer any FOUR of the SIX questions.
More informationPoint Estimators. STATISTICS Lecture no. 10. Department of Econometrics FEM UO Brno office 69a, tel
STATISTICS Lecture no. 10 Department of Econometrics FEM UO Brno office 69a, tel. 973 442029 email:jiri.neubauer@unob.cz 8. 12. 2009 Introduction Suppose that we manufacture lightbulbs and we want to state
More informationExercise. Show the corrected sample variance is an unbiased estimator of population variance. S 2 = n i=1 (X i X ) 2 n 1. Exercise Estimation
Exercise Show the corrected sample variance is an unbiased estimator of population variance. S 2 = n i=1 (X i X ) 2 n 1 Exercise S 2 = = = = n i=1 (X i x) 2 n i=1 = (X i µ + µ X ) 2 = n 1 n 1 n i=1 ((X
More informationNon-informative Priors Multiparameter Models
Non-informative Priors Multiparameter Models Statistics 220 Spring 2005 Copyright c 2005 by Mark E. Irwin Prior Types Informative vs Non-informative There has been a desire for a prior distributions that
More informationLecture Notes 6. Assume F belongs to a family of distributions, (e.g. F is Normal), indexed by some parameter θ.
Sufficient Statistics Lecture Notes 6 Sufficiency Data reduction in terms of a particular statistic can be thought of as a partition of the sample space X. Definition T is sufficient for θ if the conditional
More informationApplied Statistics I
Applied Statistics I Liang Zhang Department of Mathematics, University of Utah July 14, 2008 Liang Zhang (UofU) Applied Statistics I July 14, 2008 1 / 18 Point Estimation Liang Zhang (UofU) Applied Statistics
More informationThe Bernoulli distribution
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
More informationChapter 8. Introduction to Statistical Inference
Chapter 8. Introduction to Statistical Inference Point Estimation Statistical inference is to draw some type of conclusion about one or more parameters(population characteristics). Now you know that a
More informationChapter 5. Statistical inference for Parametric Models
Chapter 5. Statistical inference for Parametric Models Outline Overview Parameter estimation Method of moments How good are method of moments estimates? Interval estimation Statistical Inference for Parametric
More informationChapter 7 - Lecture 1 General concepts and criteria
Chapter 7 - Lecture 1 General concepts and criteria January 29th, 2010 Best estimator Mean Square error Unbiased estimators Example Unbiased estimators not unique Special case MVUE Bootstrap General Question
More informationUnit 5: Sampling Distributions of Statistics
Unit 5: Sampling Distributions of Statistics Statistics 571: Statistical Methods Ramón V. León 6/12/2004 Unit 5 - Stat 571 - Ramon V. Leon 1 Definitions and Key Concepts A sample statistic used to estimate
More informationUnit 5: Sampling Distributions of Statistics
Unit 5: Sampling Distributions of Statistics Statistics 571: Statistical Methods Ramón V. León 6/12/2004 Unit 5 - Stat 571 - Ramon V. Leon 1 Definitions and Key Concepts A sample statistic used to estimate
More informationInterval estimation. September 29, Outline Basic ideas Sampling variation and CLT Interval estimation using X More general problems
Interval estimation September 29, 2017 STAT 151 Class 7 Slide 1 Outline of Topics 1 Basic ideas 2 Sampling variation and CLT 3 Interval estimation using X 4 More general problems STAT 151 Class 7 Slide
More informationPoint Estimation. Principle of Unbiased Estimation. When choosing among several different estimators of θ, select one that is unbiased.
Point Estimation Point Estimation Definition A point estimate of a parameter θ is a single number that can be regarded as a sensible value for θ. A point estimate is obtained by selecting a suitable statistic
More informationChapter 8: Sampling distributions of estimators Sections
Chapter 8 continued Chapter 8: Sampling distributions of estimators Sections 8.1 Sampling distribution of a statistic 8.2 The Chi-square distributions 8.3 Joint Distribution of the sample mean and sample
More informationChapter 7: Estimation Sections
Chapter 7: Estimation Sections 7.1 Statistical Inference Bayesian Methods: 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions Frequentist Methods: 7.5 Maximum Likelihood Estimators
More informationCS340 Machine learning Bayesian model selection
CS340 Machine learning Bayesian model selection Bayesian model selection Suppose we have several models, each with potentially different numbers of parameters. Example: M0 = constant, M1 = straight line,
More informationChapter 8: Sampling distributions of estimators Sections
Chapter 8: Sampling distributions of estimators Sections 8.1 Sampling distribution of a statistic 8.2 The Chi-square distributions 8.3 Joint Distribution of the sample mean and sample variance Skip: p.
More informationActuarial Mathematics and Statistics Statistics 5 Part 2: Statistical Inference Tutorial Problems
Actuarial Mathematics and Statistics Statistics 5 Part 2: Statistical Inference Tutorial Problems Spring 2005 1. Which of the following statements relate to probabilities that can be interpreted as frequencies?
More informationBack to estimators...
Back to estimators... So far, we have: Identified estimators for common parameters Discussed the sampling distributions of estimators Introduced ways to judge the goodness of an estimator (bias, MSE, etc.)
More informationPoint Estimation. Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage
6 Point Estimation Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage Point Estimation Statistical inference: directed toward conclusions about one or more parameters. We will use the generic
More informationA potentially useful approach to model nonlinearities in time series is to assume different behavior (structural break) in different subsamples
1.3 Regime switching models A potentially useful approach to model nonlinearities in time series is to assume different behavior (structural break) in different subsamples (or regimes). If the dates, the
More informationQualifying Exam Solutions: Theoretical Statistics
Qualifying Exam Solutions: Theoretical Statistics. (a) For the first sampling plan, the expectation of any statistic W (X, X,..., X n ) is a polynomial of θ of degree less than n +. Hence τ(θ) cannot have
More informationIntroduction to the Maximum Likelihood Estimation Technique. September 24, 2015
Introduction to the Maximum Likelihood Estimation Technique September 24, 2015 So far our Dependent Variable is Continuous That is, our outcome variable Y is assumed to follow a normal distribution having
More informationChapter 6: Point Estimation
Chapter 6: Point Estimation Professor Sharabati Purdue University March 10, 2014 Professor Sharabati (Purdue University) Point Estimation Spring 2014 1 / 37 Chapter Overview Point estimator and point estimate
More informationPosterior Inference. , where should we start? Consider the following computational procedure: 1. draw samples. 2. convert. 3. compute properties
Posterior Inference Example. Consider a binomial model where we have a posterior distribution for the probability term, θ. Suppose we want to make inferences about the log-odds γ = log ( θ 1 θ), where
More information6. Genetics examples: Hardy-Weinberg Equilibrium
PBCB 206 (Fall 2006) Instructor: Fei Zou email: fzou@bios.unc.edu office: 3107D McGavran-Greenberg Hall Lecture 4 Topics for Lecture 4 1. Parametric models and estimating parameters from data 2. Method
More informationChapter 7: Point Estimation and Sampling Distributions
Chapter 7: Point Estimation and Sampling Distributions Seungchul Baek Department of Statistics, University of South Carolina STAT 509: Statistics for Engineers 1 / 20 Motivation In chapter 3, we learned
More informationRandom Samples. Mathematics 47: Lecture 6. Dan Sloughter. Furman University. March 13, 2006
Random Samples Mathematics 47: Lecture 6 Dan Sloughter Furman University March 13, 2006 Dan Sloughter (Furman University) Random Samples March 13, 2006 1 / 9 Random sampling Definition We call a sequence
More informationHardy Weinberg Model- 6 Genotypes
Hardy Weinberg Model- 6 Genotypes Silvelyn Zwanzig Hardy -Weinberg with six genotypes. In a large population of plants (Mimulus guttatus there are possible alleles S, I, F at one locus resulting in six
More informationCSE 312 Winter Learning From Data: Maximum Likelihood Estimators (MLE)
CSE 312 Winter 2017 Learning From Data: Maximum Likelihood Estimators (MLE) 1 Parameter Estimation Given: independent samples x1, x2,..., xn from a parametric distribution f(x θ) Goal: estimate θ. Not
More informationLearning From Data: MLE. Maximum Likelihood Estimators
Learning From Data: MLE Maximum Likelihood Estimators 1 Parameter Estimation Assuming sample x1, x2,..., xn is from a parametric distribution f(x θ), estimate θ. E.g.: Given sample HHTTTTTHTHTTTHH of (possibly
More informationSTAT 830 Convergence in Distribution
STAT 830 Convergence in Distribution Richard Lockhart Simon Fraser University STAT 830 Fall 2013 Richard Lockhart (Simon Fraser University) STAT 830 Convergence in Distribution STAT 830 Fall 2013 1 / 31
More informationChapter 7: Estimation Sections
1 / 40 Chapter 7: Estimation Sections 7.1 Statistical Inference Bayesian Methods: Chapter 7 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions 7.4 Bayes Estimators Frequentist Methods:
More informationMVE051/MSG Lecture 7
MVE051/MSG810 2017 Lecture 7 Petter Mostad Chalmers November 20, 2017 The purpose of collecting and analyzing data Purpose: To build and select models for parts of the real world (which can be used for
More informationSimulation Wrap-up, Statistics COS 323
Simulation Wrap-up, Statistics COS 323 Today Simulation Re-cap Statistics Variance and confidence intervals for simulations Simulation wrap-up FYI: No class or office hours Thursday Simulation wrap-up
More informationPROBABILITY AND STATISTICS
Monday, January 12, 2015 1 PROBABILITY AND STATISTICS Zhenyu Ye January 12, 2015 Monday, January 12, 2015 2 References Ch10 of Experiments in Modern Physics by Melissinos. Particle Physics Data Group Review
More informationChapter 4: Commonly Used Distributions. Statistics for Engineers and Scientists Fourth Edition William Navidi
Chapter 4: Commonly Used Distributions Statistics for Engineers and Scientists Fourth Edition William Navidi 2014 by Education. This is proprietary material solely for authorized instructor use. Not authorized
More informationدرس هفتم یادگیري ماشین. (Machine Learning) دانشگاه فردوسی مشهد دانشکده مهندسی رضا منصفی
یادگیري ماشین توزیع هاي نمونه و تخمین نقطه اي پارامترها Sampling Distributions and Point Estimation of Parameter (Machine Learning) دانشگاه فردوسی مشهد دانشکده مهندسی رضا منصفی درس هفتم 1 Outline Introduction
More informationSTAT 111 Recitation 4
STAT 111 Recitation 4 Linjun Zhang http://stat.wharton.upenn.edu/~linjunz/ September 29, 2017 Misc. Mid-term exam time: 6-8 pm, Wednesday, Oct. 11 The mid-term break is Oct. 5-8 The next recitation class
More informationChapter 7: Estimation Sections
1 / 31 : Estimation Sections 7.1 Statistical Inference Bayesian Methods: 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions 7.4 Bayes Estimators Frequentist Methods: 7.5 Maximum Likelihood
More informationIEOR 165 Lecture 1 Probability Review
IEOR 165 Lecture 1 Probability Review 1 Definitions in Probability and Their Consequences 1.1 Defining Probability A probability space (Ω, F, P) consists of three elements: A sample space Ω is the set
More informationPractice Exercises for Midterm Exam ST Statistical Theory - II The ACTUAL exam will consists of less number of problems.
Practice Exercises for Midterm Exam ST 522 - Statistical Theory - II The ACTUAL exam will consists of less number of problems. 1. Suppose X i F ( ) for i = 1,..., n, where F ( ) is a strictly increasing
More informationConfidence Intervals Introduction
Confidence Intervals Introduction A point estimate provides no information about the precision and reliability of estimation. For example, the sample mean X is a point estimate of the population mean μ
More informationComparing the Means of. Two Log-Normal Distributions: A Likelihood Approach
Journal of Statistical and Econometric Methods, vol.3, no.1, 014, 137-15 ISSN: 179-660 (print), 179-6939 (online) Scienpress Ltd, 014 Comparing the Means of Two Log-Normal Distributions: A Likelihood Approach
More informationConjugate Models. Patrick Lam
Conjugate Models Patrick Lam Outline Conjugate Models What is Conjugacy? The Beta-Binomial Model The Normal Model Normal Model with Unknown Mean, Known Variance Normal Model with Known Mean, Unknown Variance
More informationBIO5312 Biostatistics Lecture 5: Estimations
BIO5312 Biostatistics Lecture 5: Estimations Yujin Chung September 27th, 2016 Fall 2016 Yujin Chung Lec5: Estimations Fall 2016 1/34 Recap Yujin Chung Lec5: Estimations Fall 2016 2/34 Today s lecture and
More informationHuber smooth M-estimator. Mâra Vçliòa, Jânis Valeinis. University of Latvia. Sigulda,
University of Latvia Sigulda, 28.05.2011 Contents M-estimators Huber estimator Smooth M-estimator Empirical likelihood method for M-estimators Introduction Aim: robust estimation of location parameter
More informationCSC 411: Lecture 08: Generative Models for Classification
CSC 411: Lecture 08: Generative Models for Classification Richard Zemel, Raquel Urtasun and Sanja Fidler University of Toronto Zemel, Urtasun, Fidler (UofT) CSC 411: 08-Generative Models 1 / 23 Today Classification
More informationThe method of Maximum Likelihood.
Maximum Likelihood The method of Maximum Likelihood. In developing the least squares estimator - no mention of probabilities. Minimize the distance between the predicted linear regression and the observed
More informationExam 1 Spring 2015 Statistics for Applications 3/5/2015
8.443 Exam Sprig 05 Statistics for Applicatios 3/5/05. Log Normal Distributio: A radom variable X follows a Logormal(θ, σ ) distributio if l(x) follows a Normal(θ, σ ) distributio. For the ormal radom
More informationStochastic Models. Statistics. Walt Pohl. February 28, Department of Business Administration
Stochastic Models Statistics Walt Pohl Universität Zürich Department of Business Administration February 28, 2013 The Value of Statistics Business people tend to underestimate the value of statistics.
More informationMATH 3200 Exam 3 Dr. Syring
. Suppose n eligible voters are polled (randomly sampled) from a population of size N. The poll asks voters whether they support or do not support increasing local taxes to fund public parks. Let M be
More informationLog-linear Modeling Under Generalized Inverse Sampling Scheme
Log-linear Modeling Under Generalized Inverse Sampling Scheme Soumi Lahiri (1) and Sunil Dhar (2) (1) Department of Mathematical Sciences New Jersey Institute of Technology University Heights, Newark,
More informationMuch of what appears here comes from ideas presented in the book:
Chapter 11 Robust statistical methods Much of what appears here comes from ideas presented in the book: Huber, Peter J. (1981), Robust statistics, John Wiley & Sons (New York; Chichester). There are many
More informationSTAT 111 Recitation 3
STAT 111 Recitation 3 Linjun Zhang stat.wharton.upenn.edu/~linjunz/ September 23, 2017 Misc. The unpicked-up homeworks will be put in the STAT 111 box in the Stats Department lobby (It s on the 4th floor
More informationReview for Final Exam Spring 2014 Jeremy Orloff and Jonathan Bloom
Review for Final Exam 18.05 Spring 2014 Jeremy Orloff and Jonathan Bloom THANK YOU!!!! JON!! PETER!! RUTHI!! ERIKA!! ALL OF YOU!!!! Probability Counting Sets Inclusion-exclusion principle Rule of product
More informationSTAT/MATH 395 PROBABILITY II
STAT/MATH 395 PROBABILITY II Distribution of Random Samples & Limit Theorems Néhémy Lim University of Washington Winter 2017 Outline Distribution of i.i.d. Samples Convergence of random variables The Laws
More informationComputer Statistics with R
MAREK GAGOLEWSKI KONSTANCJA BOBECKA-WESO LOWSKA PRZEMYS LAW GRZEGORZEWSKI Computer Statistics with R 5. Point Estimation Faculty of Mathematics and Information Science Warsaw University of Technology []
More informationBayesian Linear Model: Gory Details
Bayesian Linear Model: Gory Details Pubh7440 Notes By Sudipto Banerjee Let y y i ] n i be an n vector of independent observations on a dependent variable (or response) from n experimental units. Associated
More informationChapter 7. Sampling Distributions and the Central Limit Theorem
Chapter 7. Sampling Distributions and the Central Limit Theorem 1 Introduction 2 Sampling Distributions related to the normal distribution 3 The central limit theorem 4 The normal approximation to binomial
More informationA New Hybrid Estimation Method for the Generalized Pareto Distribution
A New Hybrid Estimation Method for the Generalized Pareto Distribution Chunlin Wang Department of Mathematics and Statistics University of Calgary May 18, 2011 A New Hybrid Estimation Method for the GPD
More informationStatistics for Managers Using Microsoft Excel 7 th Edition
Statistics for Managers Using Microsoft Excel 7 th Edition Chapter 7 Sampling Distributions Statistics for Managers Using Microsoft Excel 7e Copyright 2014 Pearson Education, Inc. Chap 7-1 Learning Objectives
More informationPractice Exam 1. Loss Amount Number of Losses
Practice Exam 1 1. You are given the following data on loss sizes: An ogive is used as a model for loss sizes. Determine the fitted median. Loss Amount Number of Losses 0 1000 5 1000 5000 4 5000 10000
More informationINDIAN INSTITUTE OF SCIENCE STOCHASTIC HYDROLOGY. Lecture -5 Course Instructor : Prof. P. P. MUJUMDAR Department of Civil Engg., IISc.
INDIAN INSTITUTE OF SCIENCE STOCHASTIC HYDROLOGY Lecture -5 Course Instructor : Prof. P. P. MUJUMDAR Department of Civil Engg., IISc. Summary of the previous lecture Moments of a distribubon Measures of
More informationLecture 17: More on Markov Decision Processes. Reinforcement learning
Lecture 17: More on Markov Decision Processes. Reinforcement learning Learning a model: maximum likelihood Learning a value function directly Monte Carlo Temporal-difference (TD) learning COMP-424, Lecture
More informationSTART HERE: Instructions. 1 Exponential Family [Zhou, Manzil]
START HERE: Instructions Thanks a lot to John A.W.B. Constanzo and Shi Zong for providing and allowing to use the latex source files for quick preparation of the HW solution. The homework was due at 9:00am
More informationChapter 7. Sampling Distributions and the Central Limit Theorem
Chapter 7. Sampling Distributions and the Central Limit Theorem 1 Introduction 2 Sampling Distributions related to the normal distribution 3 The central limit theorem 4 The normal approximation to binomial
More informationPoint Estimation. Copyright Cengage Learning. All rights reserved.
6 Point Estimation Copyright Cengage Learning. All rights reserved. 6.2 Methods of Point Estimation Copyright Cengage Learning. All rights reserved. Methods of Point Estimation The definition of unbiasedness
More informationPoint Estimation. Some General Concepts of Point Estimation. Example. Estimator quality
Point Estimation Some General Concepts of Point Estimation Statistical inference = conclusions about parameters Parameters == population characteristics A point estimate of a parameter is a value (based
More informationGenerating Random Numbers
Generating Random Numbers Aim: produce random variables for given distribution Inverse Method Let F be the distribution function of an univariate distribution and let F 1 (y) = inf{x F (x) y} (generalized
More informationECON 5350 Class Notes Maximum Likelihood Estimation
ECON 5350 Class Notes Maximum Likelihood Estimatio 1 Maximum Likelihood Estimatio Example #1. Cosider the radom sample {X 1 = 0.5, X 2 = 2.0, X 3 = 10.0, X 4 = 1.5, X 5 = 7.0} geerated from a expoetial
More informationIEOR E4703: Monte-Carlo Simulation
IEOR E4703: Monte-Carlo Simulation Further Variance Reduction Methods Martin Haugh Department of Industrial Engineering and Operations Research Columbia University Email: martin.b.haugh@gmail.com Outline
More informationChapter 6. Importance sampling. 6.1 The basics
Chapter 6 Importance sampling 6.1 The basics To movtivate our discussion consider the following situation. We want to use Monte Carlo to compute µ E[X]. There is an event E such that P(E) is small but
More informationProbability. An intro for calculus students P= Figure 1: A normal integral
Probability An intro for calculus students.8.6.4.2 P=.87 2 3 4 Figure : A normal integral Suppose we flip a coin 2 times; what is the probability that we get more than 2 heads? Suppose we roll a six-sided
More informationObjective Bayesian Analysis for Heteroscedastic Regression
Analysis for Heteroscedastic Regression & Esther Salazar Universidade Federal do Rio de Janeiro Colóquio Inter-institucional: Modelos Estocásticos e Aplicações 2009 Collaborators: Marco Ferreira and Thais
More informationStatistical estimation
Statistical estimation Statistical modelling: theory and practice Gilles Guillot gigu@dtu.dk September 3, 2013 Gilles Guillot (gigu@dtu.dk) Estimation September 3, 2013 1 / 27 1 Introductory example 2
More informationParameter Estimation for the Lognormal Distribution
Brigham Young University BYU ScholarsArchive All Theses and Dissertations 2009-11-13 Parameter Estimation for the Lognormal Distribution Brenda Faith Ginos Brigham Young University - Provo Follow this
More informationPoint Estimation. Edwin Leuven
Point Estimation Edwin Leuven Introduction Last time we reviewed statistical inference We saw that while in probability we ask: given a data generating process, what are the properties of the outcomes?
More information12. Conditional heteroscedastic models (ARCH) MA6622, Ernesto Mordecki, CityU, HK, 2006.
12. Conditional heteroscedastic models (ARCH) MA6622, Ernesto Mordecki, CityU, HK, 2006. References for this Lecture: Robert F. Engle. Autoregressive Conditional Heteroscedasticity with Estimates of Variance
More informationStatistics for Business and Economics
Statistics for Business and Economics Chapter 5 Continuous Random Variables and Probability Distributions Ch. 5-1 Probability Distributions Probability Distributions Ch. 4 Discrete Continuous Ch. 5 Probability
More informationThe Normal Distribution
The Normal Distribution The normal distribution plays a central role in probability theory and in statistics. It is often used as a model for the distribution of continuous random variables. Like all models,
More informationProbability & Statistics
Probability & Statistics BITS Pilani K K Birla Goa Campus Dr. Jajati Keshari Sahoo Department of Mathematics Statistics Descriptive statistics Inferential statistics /38 Inferential Statistics 1. Involves:
More informationConditional Heteroscedasticity
1 Conditional Heteroscedasticity May 30, 2010 Junhui Qian 1 Introduction ARMA(p,q) models dictate that the conditional mean of a time series depends on past observations of the time series and the past
More informationChapter 5: Statistical Inference (in General)
Chapter 5: Statistical Inference (in General) Shiwen Shen University of South Carolina 2016 Fall Section 003 1 / 17 Motivation In chapter 3, we learn the discrete probability distributions, including Bernoulli,
More informationME3620. Theory of Engineering Experimentation. Spring Chapter III. Random Variables and Probability Distributions.
ME3620 Theory of Engineering Experimentation Chapter III. Random Variables and Probability Distributions Chapter III 1 3.2 Random Variables In an experiment, a measurement is usually denoted by a variable
More informationUQ, STAT2201, 2017, Lectures 3 and 4 Unit 3 Probability Distributions.
UQ, STAT2201, 2017, Lectures 3 and 4 Unit 3 Probability Distributions. Random Variables 2 A random variable X is a numerical (integer, real, complex, vector etc.) summary of the outcome of the random experiment.
More informationModeling and Estimation of
Modeling and of Financial and Actuarial Mathematics Christian Doppler Laboratory for Portfolio Risk Management Vienna University of Technology PRisMa 2008 29.09.2008 Outline 1 2 3 4 5 Credit ratings describe
More informationDrunken Birds, Brownian Motion, and Other Random Fun
Drunken Birds, Brownian Motion, and Other Random Fun Michael Perlmutter Department of Mathematics Purdue University 1 M. Perlmutter(Purdue) Brownian Motion and Martingales Outline Review of Basic Probability
More informationWeek 2 Quantitative Analysis of Financial Markets Hypothesis Testing and Confidence Intervals
Week 2 Quantitative Analysis of Financial Markets Hypothesis Testing and Confidence Intervals Christopher Ting http://www.mysmu.edu/faculty/christophert/ Christopher Ting : christopherting@smu.edu.sg :
More informationChapter 5. Sampling Distributions
Lecture notes, Lang Wu, UBC 1 Chapter 5. Sampling Distributions 5.1. Introduction In statistical inference, we attempt to estimate an unknown population characteristic, such as the population mean, µ,
More informationECE 340 Probabilistic Methods in Engineering M/W 3-4:15. Lecture 10: Continuous RV Families. Prof. Vince Calhoun
ECE 340 Probabilistic Methods in Engineering M/W 3-4:15 Lecture 10: Continuous RV Families Prof. Vince Calhoun 1 Reading This class: Section 4.4-4.5 Next class: Section 4.6-4.7 2 Homework 3.9, 3.49, 4.5,
More informationThe Vasicek Distribution
The Vasicek Distribution Dirk Tasche Lloyds TSB Bank Corporate Markets Rating Systems dirk.tasche@gmx.net Bristol / London, August 2008 The opinions expressed in this presentation are those of the author
More informationA Regime Switching model
Master Degree Project in Finance A Regime Switching model Applied to the OMXS30 and Nikkei 225 indices Ludvig Hjalmarsson Supervisor: Mattias Sundén Master Degree Project No. 2014:92 Graduate School Masters
More informationProbability Theory and Simulation Methods. April 9th, Lecture 20: Special distributions
April 9th, 2018 Lecture 20: Special distributions Week 1 Chapter 1: Axioms of probability Week 2 Chapter 3: Conditional probability and independence Week 4 Chapters 4, 6: Random variables Week 9 Chapter
More information2.1 Probability, stochastic variables and distribution functions
Chapter 2 Probability and statistics 2.1 Probability, stochastic variables and distribution functions The defining characteristic of a stochastic experiment E is that it produces different outcomes under
More information