Conjugate Models. Patrick Lam
|
|
- Emma Cain
- 5 years ago
- Views:
Transcription
1 Conjugate Models Patrick Lam
2 Outline Conjugate Models What is Conjugacy? The Beta-Binomial Model The Normal Model Normal Model with Unknown Mean, Known Variance Normal Model with Known Mean, Unknown Variance
3 Outline Conjugate Models What is Conjugacy? The Beta-Binomial Model The Normal Model Normal Model with Unknown Mean, Known Variance Normal Model with Known Mean, Unknown Variance
4 Outline Conjugate Models What is Conjugacy? The Beta-Binomial Model The Normal Model Normal Model with Unknown Mean, Known Variance Normal Model with Known Mean, Unknown Variance
5 Conjugacy Suppose we have a Bayesian model with a likelihood p(y θ) and a prior p(θ). If we multiply our likelihood and prior, we get our posterior p(θ y) up to a constant of proportionality. If our posterior is a distribution that is of the same family as our prior, then we have conjugacy. We say that the prior is conjugate to the likelihood. Conjugate models are great because we know the exact distribution of the posterior so we can easily simulate or derive quantities of interest analytically. In practice, we rarely have conjugacy.
6 Brief List of Conjugate Models Likelihood Prior Posterior Binomial Beta Beta Negative Binomial Beta Beta Poisson Gamma Gamma Geometric Beta Beta Exponential Gamma Gamma Normal (mean unknown) Normal Normal Normal (variance unknown) Inverse Gamma Inverse Gamma Normal (mean and variance unknown) Normal/Gamma Normal/Gamma Multinomial Dirichlet Dirichlet
7 Outline Conjugate Models What is Conjugacy? The Beta-Binomial Model The Normal Model Normal Model with Unknown Mean, Known Variance Normal Model with Known Mean, Unknown Variance
8 A Binomial Example Suppose we have vector of data on voter turnout for a random sample of n voters in the 004 US Presidential election. We can model the voter turnout with a binomial model. Y Binomial(n, π) Quantity of interest: π (voter turnout) Assumptions: Each voter s decision to vote follows the Bernoulli distribution. Each voter has the same probability of voting. (unrealistic) Each voter s decision to vote is independent. (unrealistic)
9 The Conjugate Beta Prior We can use the beta distribution as a prior for π, since the beta distribution is conjugate to the binomial distribution. p(π y) p(y π)p(π) = Binomial(n, π) Beta(α, β) ( ) n = π y (n y) Γ(α + β) ( π) y Γ(α)Γ(β) π(α ) ( π) (β ) π y ( π) (n y) π (α ) ( π) (β ) p(π y) π y+α ( π) n y+β The posterior distribution is simply a Beta(y + α, n y + β) distribution. Effectively, our prior is just adding α successes and β failures to the dataset.
10 The Uninformative (Flat) Uniform Prior Suppose we have no strong prior beliefs about the parameters. We can choose a prior that gives equal weight to all possible values of the parameters, essentially an uninformative or flat prior. for all values of π. p(π) = constant For the binomial model, one example of a flat prior is the Beta(,) prior: p(π) = Γ() Γ()Γ() π( ) ( π) ( ) = which is the Uniform distribution over the [0, ] interval.
11 Since we know that a Binomial likelihood and a Beta(,) prior produces a Beta(y +, n y + ) posterior, we can simulate the posterior in R. Suppose our turnout data had 500 voters, of which 85 voted. > table(turnout) turnout Setting our prior parameters at α = and β =, > a <- > b <- we get the posterior > posterior.unif.prior <- rbeta(0000, shape = 85 + a, shape = b)
12 Outline Conjugate Models What is Conjugacy? The Beta-Binomial Model The Normal Model Normal Model with Unknown Mean, Known Variance Normal Model with Known Mean, Unknown Variance
13 Outline Conjugate Models What is Conjugacy? The Beta-Binomial Model The Normal Model Normal Model with Unknown Mean, Known Variance Normal Model with Known Mean, Unknown Variance
14 Normal Model with Unknown Mean, Known Variance Suppose we wish to estimate a model where the likelihood of the data is normal with an unknown mean µ and a known variance σ. Our parameter of interest is µ. We can use a conjugate Normal prior on µ, with mean µ 0 and variance τ 0. p(µ y, σ ) p(y µ, σ )p(µ) Normal(µ, τ ) = Normal(µ, σ ) Normal(µ 0, τ 0 )
15 Let θ represent our parameter of interest, in this case µ. p(θ y) ny i= exp (y «i θ) πσ σ p exp πτ 0! «(θ µ0) τ0 nx (y i θ) (θ µ0) exp σ τ i= 0 "!# = exp nx (y i θ) (θ µ0) + σ τ i= 0 "!# = exp nx τ σ τ0 0 (y i θ) +σ (θ µ 0) i= "!# = exp nx τ σ τ0 0 (yi θy i + θ )+σ (θ θµ 0 + µ 0) i=
16 We can multiply the θy i term in the summation by n n in order to get the equations in terms of the sufficient statistic ȳ. "!# p(θ y) exp nx τ σ τ0 0 (yi θ n n y i + θ )+σ (θ θµ 0 + µ 0) i= "!# = exp nx τ σ τ0 0 yi τ0 θnȳ + τ0 nθ +θ σ θµ 0σ + µ 0σ i= We can then factor the terms into several parts. Since µ 0 σ and τ0 n i= y i do not contain θ, we can represent them as some constant k, which we will drop into the normalizing constant.» p(θ y) exp θ σ + τ0 n θ µ 0σ + τ0 nȳ + k» = exp σ τ 0 θ σ + τ0 n σ τ0» = exp θ τ0 «θ + n σ «θ µ0σ + τ 0 nȳ µ0 τ 0 ««+ k σ τ0 + nȳ ««+ k σ
17 τ Let s multiply by 0 τ 0 p(θ y) exp 4 τ0 = exp 4 + n σ «+ n σ «in order to simplify the θ term. τ 0 = exp 4 τ n σ 0 + n «σ + n τ0 σ + n τ0 σ 0 + n «@θ σ 0 A µ 0 + nȳ τ0 σ + n τ0 σ µ 0 + nȳ τ0 σ + n τ0 σ µ 0 τ 0 τ 0 + nȳ σ + n σ 3 A + ka5 3 AA 5 3 A + ka5 Finally, we have something that looks like the density function of a Normal distribution!
18 p(θ y) exp 4 τ n «@θ σ µ 0 + nȳ τ0 σ + n τ0 σ 3 AA 5 Posterior Mean: µ = µ 0 τ 0 τ 0 + nȳ σ «+ n σ «Posterior Variance: τ = ( τ 0 + n σ ) Posterior Precision: τ = τ 0 + n σ Posterior Precision is just the sum of the prior precision and the data precision.
19 We can also look more closely at how the prior mean µ 0 and the posterior mean µ relate to each other. µ0 + nȳ τ0 µ = σ + n σ = τ 0 µ 0 σ +τ 0 nȳ τ 0 σ σ +nτ 0 τ 0 σ = µ0σ + τ0 nȳ σ + nτ0 µ 0σ = + τ 0 nȳ σ + nτ0 σ + nτ0 As n increases, data mean dominates prior mean. As τ0 decreases (less prior variance, greater prior precision), our prior mean becomes more important.
20 A Simple Example Suppose we have some (fake) data on the heights (in inches) of a random sample of 00 individuals in the U.S. population. > known.sigma.sq <- 6 > unknown.mean <- 68 > n <- 00 > heights <- rnorm(n, mean = unknown.mean, sd = sqrt(known.sigma.sq)) We believe that the heights are normally distributed with some unknown mean µ and a known variance σ = 6. Suppose before we see the data, we have a prior belief about the distribution of µ. Let our prior mean µ 0 = 7 and our prior variance τ 0 = 36. > mu0 <- 7 > tau.sq0 <- 36
21 Our posterior is a Normal distribution with Mean Variance ( τ 0 + n σ ) µ 0 τ 0 τ 0 > post.mean <- (mu0/tau.sq0 + (n * mean(heights)/known.sigma.sq))/(/tau.sq0 + + n/known.sigma.sq) > post.mean [] > post.var <- /(/tau.sq0 + n/known.sigma.sq) > post.var [] nȳ σ «+ n σ «and
22 Outline Conjugate Models What is Conjugacy? The Beta-Binomial Model The Normal Model Normal Model with Unknown Mean, Known Variance Normal Model with Known Mean, Unknown Variance
23 Normal Model with Known Mean, Unknown Variance Now suppose we wish to estimate a model where the likelihood of the data is normal with a known mean µ and an unknown variance σ. Now our parameter of interest is σ. We can use a conjugate inverse gamma prior on σ, with shape parameter α 0 and scale parameter β 0. p(σ y, µ) p(y µ, σ )p(σ ) Invgamma(α, β ) = Normal(µ, σ ) Invgamma(α 0, β 0 )
24 Let θ represent our parameter of interest, in this case σ. ny p(θ y, µ) exp (y ««i µ) βα 0 0 i= πθ θ Γ(α 0) θ (α 0+) exp β0 θ ny θ exp (y ««i µ) θ (α0+) exp β0 θ θ i= P n = θ n exp i= (y i µ) ««θ (α0+) exp β0 θ θ» P n = θ (α 0+ n +) β0 exp θ + i= (y i µ) «θ 0 Pn = θ (α 0+ n +) exp β0 + 3 i= (y i µ) A5 θ 0 Pn 3 = θ (α 0+ n +) exp β0 + i= (y i µ) A5 θ This looks like the density of an inverse gamma distribution!
25 0 Pn 3 p(θ y, µ) θ (α 0+ n +) exp β0 + i= (y i µ) A5 θ α = α 0 + n β = P n i= β 0 + i µ) Our posterior is an Invgamma(α 0 + n P, β n i= 0 + i µ) ) distribution.
26 A Simple Example Again suppose we have some (fake) data on the heights (in inches) of a random sample of 00 individuals in the U.S. population. > known.mean <- 68 > unknown.sigma.sq <- 6 > n <- 00 > heights <- rnorm(n, mean = known.mean, sd = sqrt(unknown.sigma.sq)) We believe that the heights are normally distributed with a known mean µ = 68 and some unknown variance σ. Suppose before we see the data, we have a prior belief about the distribution of σ. Let our prior shape α 0 = 5 and our prior scale β 0 = 0. > alpha0 <- 5 > beta0 <- 0
27 Our posterior is a inverse gamma distribution with shape α 0 + n and scale β 0 + P n i= (y i µ) > alpha <- alpha0 + n/ > beta <- beta0 + sum((heights - known.mean)^)/ > library(mcmcpack) > posterior <- rinvgamma(0000, alpha, beta) > post.mean <- mean(posterior) > post.mean [].8839 > post.var <- var(posterior) > post.var [] Hmm... what if we increased our sample size?
28 > n <- 000 > heights <- rnorm(n, mean = known.mean, sd = sqrt(unknown.sigma.sq)) > alpha <- alpha0 + n/ > beta <- beta0 + sum((heights - known.mean)^)/ > posterior <- rinvgamma(0000, alpha, beta) > post.mean <- mean(posterior) > post.mean [] 5.98 > post.var <- var(posterior) > post.var []
Non-informative Priors Multiparameter Models
Non-informative Priors Multiparameter Models Statistics 220 Spring 2005 Copyright c 2005 by Mark E. Irwin Prior Types Informative vs Non-informative There has been a desire for a prior distributions that
More informationBayesian Normal Stuff
Bayesian Normal Stuff - Set-up of the basic model of a normally distributed random variable with unknown mean and variance (a two-parameter model). - Discuss philosophies of prior selection - Implementation
More informationChapter 7: Estimation Sections
1 / 31 : Estimation Sections 7.1 Statistical Inference Bayesian Methods: 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions 7.4 Bayes Estimators Frequentist Methods: 7.5 Maximum Likelihood
More informationPosterior Inference. , where should we start? Consider the following computational procedure: 1. draw samples. 2. convert. 3. compute properties
Posterior Inference Example. Consider a binomial model where we have a posterior distribution for the probability term, θ. Suppose we want to make inferences about the log-odds γ = log ( θ 1 θ), where
More informationCommon one-parameter models
Common one-parameter models In this section we will explore common one-parameter models, including: 1. Binomial data with beta prior on the probability 2. Poisson data with gamma prior on the rate 3. Gaussian
More informationcontinuous rv Note for a legitimate pdf, we have f (x) 0 and f (x)dx = 1. For a continuous rv, P(X = c) = c f (x)dx = 0, hence
continuous rv Let X be a continuous rv. Then a probability distribution or probability density function (pdf) of X is a function f(x) such that for any two numbers a and b with a b, P(a X b) = b a f (x)dx.
More informationExam 2 Spring 2015 Statistics for Applications 4/9/2015
18.443 Exam 2 Spring 2015 Statistics for Applications 4/9/2015 1. True or False (and state why). (a). The significance level of a statistical test is not equal to the probability that the null hypothesis
More informationChapter 7: Estimation Sections
1 / 40 Chapter 7: Estimation Sections 7.1 Statistical Inference Bayesian Methods: Chapter 7 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions 7.4 Bayes Estimators Frequentist Methods:
More informationTwo hours. To be supplied by the Examinations Office: Mathematical Formula Tables and Statistical Tables THE UNIVERSITY OF MANCHESTER
Two hours MATH20802 To be supplied by the Examinations Office: Mathematical Formula Tables and Statistical Tables THE UNIVERSITY OF MANCHESTER STATISTICAL METHODS Answer any FOUR of the SIX questions.
More informationBayesian Hierarchical/ Multilevel and Latent-Variable (Random-Effects) Modeling
Bayesian Hierarchical/ Multilevel and Latent-Variable (Random-Effects) Modeling 1: Formulation of Bayesian models and fitting them with MCMC in WinBUGS David Draper Department of Applied Mathematics and
More informationCS 361: Probability & Statistics
March 12, 2018 CS 361: Probability & Statistics Inference Binomial likelihood: Example Suppose we have a coin with an unknown probability of heads. We flip the coin 10 times and observe 2 heads. What can
More informationSTAT 425: Introduction to Bayesian Analysis
STAT 45: Introduction to Bayesian Analysis Marina Vannucci Rice University, USA Fall 018 Marina Vannucci (Rice University, USA) Bayesian Analysis (Part 1) Fall 018 1 / 37 Lectures 9-11: Multi-parameter
More informationCS340 Machine learning Bayesian statistics 3
CS340 Machine learning Bayesian statistics 3 1 Outline Conjugate analysis of µ and σ 2 Bayesian model selection Summarizing the posterior 2 Unknown mean and precision The likelihood function is p(d µ,λ)
More informationBayesian Linear Model: Gory Details
Bayesian Linear Model: Gory Details Pubh7440 Notes By Sudipto Banerjee Let y y i ] n i be an n vector of independent observations on a dependent variable (or response) from n experimental units. Associated
More informationThis is a open-book exam. Assigned: Friday November 27th 2009 at 16:00. Due: Monday November 30th 2009 before 10:00.
University of Iceland School of Engineering and Sciences Department of Industrial Engineering, Mechanical Engineering and Computer Science IÐN106F Industrial Statistics II - Bayesian Data Analysis Fall
More informationLecture Notes 6. Assume F belongs to a family of distributions, (e.g. F is Normal), indexed by some parameter θ.
Sufficient Statistics Lecture Notes 6 Sufficiency Data reduction in terms of a particular statistic can be thought of as a partition of the sample space X. Definition T is sufficient for θ if the conditional
More information1 Bayesian Bias Correction Model
1 Bayesian Bias Correction Model Assuming that n iid samples {X 1,...,X n }, were collected from a normal population with mean µ and variance σ 2. The model likelihood has the form, P( X µ, σ 2, T n >
More informationThe Normal Distribution
The Normal Distribution The normal distribution plays a central role in probability theory and in statistics. It is often used as a model for the distribution of continuous random variables. Like all models,
More informationConjugate priors: Beta and normal Class 15, Jeremy Orloff and Jonathan Bloom
1 Learning Goals Conjugate s: Beta and normal Class 15, 18.05 Jeremy Orloff and Jonathan Bloom 1. Understand the benefits of conjugate s.. Be able to update a beta given a Bernoulli, binomial, or geometric
More informationGenerating Random Numbers
Generating Random Numbers Aim: produce random variables for given distribution Inverse Method Let F be the distribution function of an univariate distribution and let F 1 (y) = inf{x F (x) y} (generalized
More informationChapter 7: Estimation Sections
Chapter 7: Estimation Sections 7.1 Statistical Inference Bayesian Methods: 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions Frequentist Methods: 7.5 Maximum Likelihood Estimators
More informationExam STAM Practice Exam #1
!!!! Exam STAM Practice Exam #1 These practice exams should be used during the month prior to your exam. This practice exam contains 20 questions, of equal value, corresponding to about a 2 hour exam.
More informationChapter 4: Asymptotic Properties of MLE (Part 3)
Chapter 4: Asymptotic Properties of MLE (Part 3) Daniel O. Scharfstein 09/30/13 1 / 1 Breakdown of Assumptions Non-Existence of the MLE Multiple Solutions to Maximization Problem Multiple Solutions to
More informationMATH 3200 Exam 3 Dr. Syring
. Suppose n eligible voters are polled (randomly sampled) from a population of size N. The poll asks voters whether they support or do not support increasing local taxes to fund public parks. Let M be
More informationPart II: Computation for Bayesian Analyses
Part II: Computation for Bayesian Analyses 62 BIO 233, HSPH Spring 2015 Conjugacy In both birth weight eamples the posterior distribution is from the same family as the prior: Prior Likelihood Posterior
More informationMetropolis-Hastings algorithm
Metropolis-Hastings algorithm Dr. Jarad Niemi STAT 544 - Iowa State University March 27, 2018 Jarad Niemi (STAT544@ISU) Metropolis-Hastings March 27, 2018 1 / 32 Outline Metropolis-Hastings algorithm Independence
More informationChapter 3 Common Families of Distributions. Definition 3.4.1: A family of pmfs or pdfs is called exponential family if it can be expressed as
Lecture 0 on BST 63: Statistical Theory I Kui Zhang, 09/9/008 Review for the previous lecture Definition: Several continuous distributions, including uniform, gamma, normal, Beta, Cauchy, double exponential
More informationProbability Theory and Simulation Methods. April 9th, Lecture 20: Special distributions
April 9th, 2018 Lecture 20: Special distributions Week 1 Chapter 1: Axioms of probability Week 2 Chapter 3: Conditional probability and independence Week 4 Chapters 4, 6: Random variables Week 9 Chapter
More informationST440/550: Applied Bayesian Analysis. (5) Multi-parameter models - Summarizing the posterior
(5) Multi-parameter models - Summarizing the posterior Models with more than one parameter Thus far we have studied single-parameter models, but most analyses have several parameters For example, consider
More informationDistributions and Intro to Likelihood
Distributions and Intro to Likelihood Gov 2001 Section February 4, 2010 Outline Meet the Distributions! Discrete Distributions Continuous Distributions Basic Likelihood Why should we become familiar with
More informationGOV 2001/ 1002/ E-200 Section 3 Inference and Likelihood
GOV 2001/ 1002/ E-200 Section 3 Inference and Likelihood Anton Strezhnev Harvard University February 10, 2016 1 / 44 LOGISTICS Reading Assignment- Unifying Political Methodology ch 4 and Eschewing Obfuscation
More information**BEGINNING OF EXAMINATION** A random sample of five observations from a population is:
**BEGINNING OF EXAMINATION** 1. You are given: (i) A random sample of five observations from a population is: 0.2 0.7 0.9 1.1 1.3 (ii) You use the Kolmogorov-Smirnov test for testing the null hypothesis,
More informationSTA 114: Statistics. Notes 10. Conjugate Priors
STA 114: Statistics Notes 10. Conjugate Priors Conjugate family Once we get a /pmf ξ(θ x) by combining a model X f(x θ) with a /pmf ξ(θ) on θ Θ, a report can be made by summarizing the. It helps to have
More informationChapter 5: Statistical Inference (in General)
Chapter 5: Statistical Inference (in General) Shiwen Shen University of South Carolina 2016 Fall Section 003 1 / 17 Motivation In chapter 3, we learn the discrete probability distributions, including Bernoulli,
More informationRules and Models 1 investigates the internal measurement approach for operational risk capital
Carol Alexander 2 Rules and Models Rules and Models 1 investigates the internal measurement approach for operational risk capital 1 There is a view that the new Basel Accord is being defined by a committee
More information# generate data num.obs <- 100 y <- rnorm(num.obs,mean = theta.true, sd = sqrt(sigma.sq.true))
Posterior Sampling from Normal Now we seek to create draws from the joint posterior distribution and the marginal posterior distributions and Note the marginal posterior distributions would be used to
More informationChapter 7: Point Estimation and Sampling Distributions
Chapter 7: Point Estimation and Sampling Distributions Seungchul Baek Department of Statistics, University of South Carolina STAT 509: Statistics for Engineers 1 / 20 Motivation In chapter 3, we learned
More informationExtended Model: Posterior Distributions
APPENDIX A Extended Model: Posterior Distributions A. Homoskedastic errors Consider the basic contingent claim model b extended by the vector of observables x : log C i = β log b σ, x i + β x i + i, i
More informationChapter 5. Statistical inference for Parametric Models
Chapter 5. Statistical inference for Parametric Models Outline Overview Parameter estimation Method of moments How good are method of moments estimates? Interval estimation Statistical Inference for Parametric
More informationSYSM 6304 Risk and Decision Analysis Lecture 2: Fitting Distributions to Data
SYSM 6304 Risk and Decision Analysis Lecture 2: Fitting Distributions to Data M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu September 5, 2015
More informationCS340 Machine learning Bayesian model selection
CS340 Machine learning Bayesian model selection Bayesian model selection Suppose we have several models, each with potentially different numbers of parameters. Example: M0 = constant, M1 = straight line,
More informationChapter 7. Sampling Distributions and the Central Limit Theorem
Chapter 7. Sampling Distributions and the Central Limit Theorem 1 Introduction 2 Sampling Distributions related to the normal distribution 3 The central limit theorem 4 The normal approximation to binomial
More informationSTAT 111 Recitation 4
STAT 111 Recitation 4 Linjun Zhang http://stat.wharton.upenn.edu/~linjunz/ September 29, 2017 Misc. Mid-term exam time: 6-8 pm, Wednesday, Oct. 11 The mid-term break is Oct. 5-8 The next recitation class
More informationWhat was in the last lecture?
What was in the last lecture? Normal distribution A continuous rv with bell-shaped density curve The pdf is given by f(x) = 1 2πσ e (x µ)2 2σ 2, < x < If X N(µ, σ 2 ), E(X) = µ and V (X) = σ 2 Standard
More informationChapter 8: Sampling distributions of estimators Sections
Chapter 8 continued Chapter 8: Sampling distributions of estimators Sections 8.1 Sampling distribution of a statistic 8.2 The Chi-square distributions 8.3 Joint Distribution of the sample mean and sample
More informationM.Sc. ACTUARIAL SCIENCE. Term-End Examination
No. of Printed Pages : 15 LMJA-010 (F2F) M.Sc. ACTUARIAL SCIENCE Term-End Examination O CD December, 2011 MIA-010 (F2F) : STATISTICAL METHOD Time : 3 hours Maximum Marks : 100 SECTION - A Attempt any five
More informationStatistics for Business and Economics
Statistics for Business and Economics Chapter 5 Continuous Random Variables and Probability Distributions Ch. 5-1 Probability Distributions Probability Distributions Ch. 4 Discrete Continuous Ch. 5 Probability
More information1. You are given the following information about a stationary AR(2) model:
Fall 2003 Society of Actuaries **BEGINNING OF EXAMINATION** 1. You are given the following information about a stationary AR(2) model: (i) ρ 1 = 05. (ii) ρ 2 = 01. Determine φ 2. (A) 0.2 (B) 0.1 (C) 0.4
More informationINSTITUTE AND FACULTY OF ACTUARIES. Curriculum 2019 SPECIMEN EXAMINATION
INSTITUTE AND FACULTY OF ACTUARIES Curriculum 2019 SPECIMEN EXAMINATION Subject CS1A Actuarial Statistics Time allowed: Three hours and fifteen minutes INSTRUCTIONS TO THE CANDIDATE 1. Enter all the candidate
More informationChapter 4: Commonly Used Distributions. Statistics for Engineers and Scientists Fourth Edition William Navidi
Chapter 4: Commonly Used Distributions Statistics for Engineers and Scientists Fourth Edition William Navidi 2014 by Education. This is proprietary material solely for authorized instructor use. Not authorized
More informationدرس هفتم یادگیري ماشین. (Machine Learning) دانشگاه فردوسی مشهد دانشکده مهندسی رضا منصفی
یادگیري ماشین توزیع هاي نمونه و تخمین نقطه اي پارامترها Sampling Distributions and Point Estimation of Parameter (Machine Learning) دانشگاه فردوسی مشهد دانشکده مهندسی رضا منصفی درس هفتم 1 Outline Introduction
More informationRandom Variables Handout. Xavier Vilà
Random Variables Handout Xavier Vilà Course 2004-2005 1 Discrete Random Variables. 1.1 Introduction 1.1.1 Definition of Random Variable A random variable X is a function that maps each possible outcome
More information6. Genetics examples: Hardy-Weinberg Equilibrium
PBCB 206 (Fall 2006) Instructor: Fei Zou email: fzou@bios.unc.edu office: 3107D McGavran-Greenberg Hall Lecture 4 Topics for Lecture 4 1. Parametric models and estimating parameters from data 2. Method
More informationContinuous Distributions
Quantitative Methods 2013 Continuous Distributions 1 The most important probability distribution in statistics is the normal distribution. Carl Friedrich Gauss (1777 1855) Normal curve A normal distribution
More informationUQ, STAT2201, 2017, Lectures 3 and 4 Unit 3 Probability Distributions.
UQ, STAT2201, 2017, Lectures 3 and 4 Unit 3 Probability Distributions. Random Variables 2 A random variable X is a numerical (integer, real, complex, vector etc.) summary of the outcome of the random experiment.
More informationObjective Bayesian Analysis for Heteroscedastic Regression
Analysis for Heteroscedastic Regression & Esther Salazar Universidade Federal do Rio de Janeiro Colóquio Inter-institucional: Modelos Estocásticos e Aplicações 2009 Collaborators: Marco Ferreira and Thais
More informationChapter 9: Sampling Distributions
Chapter 9: Sampling Distributions 9. Introduction This chapter connects the material in Chapters 4 through 8 (numerical descriptive statistics, sampling, and probability distributions, in particular) with
More informationBack to estimators...
Back to estimators... So far, we have: Identified estimators for common parameters Discussed the sampling distributions of estimators Introduced ways to judge the goodness of an estimator (bias, MSE, etc.)
More informationExtracting Information from the Markets: A Bayesian Approach
Extracting Information from the Markets: A Bayesian Approach Daniel Waggoner The Federal Reserve Bank of Atlanta Florida State University, February 29, 2008 Disclaimer: The views expressed are the author
More informationChapter 7. Sampling Distributions and the Central Limit Theorem
Chapter 7. Sampling Distributions and the Central Limit Theorem 1 Introduction 2 Sampling Distributions related to the normal distribution 3 The central limit theorem 4 The normal approximation to binomial
More informationUseful Probability Distributions
Useful Probability Distributions Standard Normal Distribution Binomial Multinomial Hypergeometric Poisson Beta Binomial Student s t Beta Gamma Dirichlet Multivariate Normal and Correlation Standard Normal
More informationSOCIETY OF ACTUARIES EXAM STAM SHORT-TERM ACTUARIAL MATHEMATICS EXAM STAM SAMPLE QUESTIONS
SOCIETY OF ACTUARIES EXAM STAM SHORT-TERM ACTUARIAL MATHEMATICS EXAM STAM SAMPLE QUESTIONS Questions 1-307 have been taken from the previous set of Exam C sample questions. Questions no longer relevant
More information4-1. Chapter 4. Commonly Used Distributions by The McGraw-Hill Companies, Inc. All rights reserved.
4-1 Chapter 4 Commonly Used Distributions 2014 by The Companies, Inc. All rights reserved. Section 4.1: The Bernoulli Distribution 4-2 We use the Bernoulli distribution when we have an experiment which
More information(11) Case Studies: Adaptive clinical trials. ST440/540: Applied Bayesian Analysis
Use of Bayesian methods in clinical trials Bayesian methods are becoming more common in clinical trials analysis We will study how to compute the sample size for a Bayesian clinical trial We will then
More informationPractice Exam 1. Loss Amount Number of Losses
Practice Exam 1 1. You are given the following data on loss sizes: An ogive is used as a model for loss sizes. Determine the fitted median. Loss Amount Number of Losses 0 1000 5 1000 5000 4 5000 10000
More informationEE641 Digital Image Processing II: Purdue University VISE - October 29,
EE64 Digital Image Processing II: Purdue University VISE - October 9, 004 The EM Algorithm. Suffient Statistics and Exponential Distributions Let p(y θ) be a family of density functions parameterized by
More informationFinancial Risk Management
Financial Risk Management Professor: Thierry Roncalli Evry University Assistant: Enareta Kurtbegu Evry University Tutorial exercices #3 1 Maximum likelihood of the exponential distribution 1. We assume
More informationModel 0: We start with a linear regression model: log Y t = β 0 + β 1 (t 1980) + ε, with ε N(0,
Stat 534: Fall 2017. Introduction to the BUGS language and rjags Installation: download and install JAGS. You will find the executables on Sourceforge. You must have JAGS installed prior to installing
More informationOutline. Review Continuation of exercises from last time
Bayesian Models II Outline Review Continuation of exercises from last time 2 Review of terms from last time Probability density function aka pdf or density Likelihood function aka likelihood Conditional
More informationSTART HERE: Instructions. 1 Exponential Family [Zhou, Manzil]
START HERE: Instructions Thanks a lot to John A.W.B. Constanzo and Shi Zong for providing and allowing to use the latex source files for quick preparation of the HW solution. The homework was due at 9:00am
More informationMonte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMSN50)
Monte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMSN50) Magnus Wiktorsson Centre for Mathematical Sciences Lund University, Sweden Lecture 5 Sequential Monte Carlo methods I January
More informationStatistical Tables Compiled by Alan J. Terry
Statistical Tables Compiled by Alan J. Terry School of Science and Sport University of the West of Scotland Paisley, Scotland Contents Table 1: Cumulative binomial probabilities Page 1 Table 2: Cumulative
More informationBayesian course - problem set 3 (lecture 4)
Bayesian course - problem set 3 (lecture 4) Ben Lambert November 14, 2016 1 Ticked off Imagine once again that you are investigating the occurrence of Lyme disease in the UK. This is a vector-borne disease
More informationInterval estimation. September 29, Outline Basic ideas Sampling variation and CLT Interval estimation using X More general problems
Interval estimation September 29, 2017 STAT 151 Class 7 Slide 1 Outline of Topics 1 Basic ideas 2 Sampling variation and CLT 3 Interval estimation using X 4 More general problems STAT 151 Class 7 Slide
More informationCommonly Used Distributions
Chapter 4: Commonly Used Distributions 1 Introduction Statistical inference involves drawing a sample from a population and analyzing the sample data to learn about the population. We often have some knowledge
More information(5) Multi-parameter models - Summarizing the posterior
(5) Multi-parameter models - Summarizing the posterior Spring, 2017 Models with more than one parameter Thus far we have studied single-parameter models, but most analyses have several parameters For example,
More informationAppendix A. Selecting and Using Probability Distributions. In this appendix
Appendix A Selecting and Using Probability Distributions In this appendix Understanding probability distributions Selecting a probability distribution Using basic distributions Using continuous distributions
More informationRandom Samples. Mathematics 47: Lecture 6. Dan Sloughter. Furman University. March 13, 2006
Random Samples Mathematics 47: Lecture 6 Dan Sloughter Furman University March 13, 2006 Dan Sloughter (Furman University) Random Samples March 13, 2006 1 / 9 Random sampling Definition We call a sequence
More informationVersion A. Problem 1. Let X be the continuous random variable defined by the following pdf: 1 x/2 when 0 x 2, f(x) = 0 otherwise.
Math 224 Q Exam 3A Fall 217 Tues Dec 12 Version A Problem 1. Let X be the continuous random variable defined by the following pdf: { 1 x/2 when x 2, f(x) otherwise. (a) Compute the mean µ E[X]. E[X] x
More informationStatistics 431 Spring 2007 P. Shaman. Preliminaries
Statistics 4 Spring 007 P. Shaman The Binomial Distribution Preliminaries A binomial experiment is defined by the following conditions: A sequence of n trials is conducted, with each trial having two possible
More informationWeight Smoothing with Laplace Prior and Its Application in GLM Model
Weight Smoothing with Laplace Prior and Its Application in GLM Model Xi Xia 1 Michael Elliott 1,2 1 Department of Biostatistics, 2 Survey Methodology Program, University of Michigan National Cancer Institute
More informationStochastic Components of Models
Stochastic Components of Models Gov 2001 Section February 5, 2014 Gov 2001 Section Stochastic Components of Models February 5, 2014 1 / 41 Outline 1 Replication Paper and other logistics 2 Data Generation
More informationMVE051/MSG Lecture 7
MVE051/MSG810 2017 Lecture 7 Petter Mostad Chalmers November 20, 2017 The purpose of collecting and analyzing data Purpose: To build and select models for parts of the real world (which can be used for
More informationContinuous random variables
Continuous random variables probability density function (f(x)) the probability distribution function of a continuous random variable (analogous to the probability mass function for a discrete random variable),
More informationActuarial Society of India EXAMINATIONS
Actuarial Society of India EXAMINATIONS 7 th June 005 Subject CT6 Statistical Models Time allowed: Three Hours (0.30 am 3.30 pm) INSTRUCTIONS TO THE CANDIDATES. Do not write your name anywhere on the answer
More informationCalibration of Interest Rates
WDS'12 Proceedings of Contributed Papers, Part I, 25 30, 2012. ISBN 978-80-7378-224-5 MATFYZPRESS Calibration of Interest Rates J. Černý Charles University, Faculty of Mathematics and Physics, Prague,
More information4-2 Probability Distributions and Probability Density Functions. Figure 4-2 Probability determined from the area under f(x).
4-2 Probability Distributions and Probability Density Functions Figure 4-2 Probability determined from the area under f(x). 4-2 Probability Distributions and Probability Density Functions Definition 4-2
More informationConfidence Intervals Introduction
Confidence Intervals Introduction A point estimate provides no information about the precision and reliability of estimation. For example, the sample mean X is a point estimate of the population mean μ
More informationA potentially useful approach to model nonlinearities in time series is to assume different behavior (structural break) in different subsamples
1.3 Regime switching models A potentially useful approach to model nonlinearities in time series is to assume different behavior (structural break) in different subsamples (or regimes). If the dates, the
More information5.3 Interval Estimation
5.3 Interval Estimation Ulrich Hoensch Wednesday, March 13, 2013 Confidence Intervals Definition Let θ be an (unknown) population parameter. A confidence interval with confidence level C is an interval
More informationCentral limit theorems
Chapter 6 Central limit theorems 6.1 Overview Recall that a random variable Z is said to have a standard normal distribution, denoted by N(0, 1), if it has a continuous distribution with density φ(z) =
More informationBusiness Statistics 41000: Probability 3
Business Statistics 41000: Probability 3 Drew D. Creal University of Chicago, Booth School of Business February 7 and 8, 2014 1 Class information Drew D. Creal Email: dcreal@chicagobooth.edu Office: 404
More informationSOCIETY OF ACTUARIES/CASUALTY ACTUARIAL SOCIETY EXAM C CONSTRUCTION AND EVALUATION OF ACTUARIAL MODELS EXAM C SAMPLE QUESTIONS
SOCIETY OF ACTUARIES/CASUALTY ACTUARIAL SOCIETY EXAM C CONSTRUCTION AND EVALUATION OF ACTUARIAL MODELS EXAM C SAMPLE QUESTIONS Copyright 2008 by the Society of Actuaries and the Casualty Actuarial Society
More informationReview for Final Exam Spring 2014 Jeremy Orloff and Jonathan Bloom
Review for Final Exam 18.05 Spring 2014 Jeremy Orloff and Jonathan Bloom THANK YOU!!!! JON!! PETER!! RUTHI!! ERIKA!! ALL OF YOU!!!! Probability Counting Sets Inclusion-exclusion principle Rule of product
More information8.1 Estimation of the Mean and Proportion
8.1 Estimation of the Mean and Proportion Statistical inference enables us to make judgments about a population on the basis of sample information. The mean, standard deviation, and proportions of a population
More informationMachine Learning for Quantitative Finance
Machine Learning for Quantitative Finance Fast derivative pricing Sofie Reyners Joint work with Jan De Spiegeleer, Dilip Madan and Wim Schoutens Derivative pricing is time-consuming... Vanilla option pricing
More informationECE 340 Probabilistic Methods in Engineering M/W 3-4:15. Lecture 10: Continuous RV Families. Prof. Vince Calhoun
ECE 340 Probabilistic Methods in Engineering M/W 3-4:15 Lecture 10: Continuous RV Families Prof. Vince Calhoun 1 Reading This class: Section 4.4-4.5 Next class: Section 4.6-4.7 2 Homework 3.9, 3.49, 4.5,
More informationLecture 18. Ingo Ruczinski. October 31, Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Johns Hopkins University
Lecture 18 Department of Bios Johns Hopkins Bloomberg School of Public Health Johns Hopkins University October 31, 2015 1 2 3 4 5 6 1 Tests for a binomial proportion 2 Score test versus Wald 3 Exact binomial
More information3 ˆθ B = X 1 + X 2 + X 3. 7 a) Find the Bias, Variance and MSE of each estimator. Which estimator is the best according
STAT 345 Spring 2018 Homework 9 - Point Estimation Name: Please adhere to the homework rules as given in the Syllabus. 1. Mean Squared Error. Suppose that X 1, X 2 and X 3 are independent random variables
More informationSTAT 825 Notes Random Number Generation
STAT 825 Notes Random Number Generation What if R/Splus/SAS doesn t have a function to randomly generate data from a particular distribution? Although R, Splus, SAS and other packages can generate data
More information