Part II: Computation for Bayesian Analyses
|
|
- Irene Barrett
- 5 years ago
- Views:
Transcription
1 Part II: Computation for Bayesian Analyses 62 BIO 233, HSPH Spring 2015
2 Conjugacy In both birth weight eamples the posterior distribution is from the same family as the prior: Prior Likelihood Posterior Beta(a, b) Y i Bernoulli(θ) Beta(y + + a, n y + + b) Normal(m, V ) Y i Normal(µ, σ 2 ) Normal(m, V ) This is referred to as conjugacy arises because of specific choice of prior/likelihood combination Having the posterior be of a known distribution is very convenient we have the means to compute moments and quantiles for both the Beta and the Normal distribution 63 BIO 233, HSPH Spring 2015
3 Conjugacy was originally pursued and advocated because of this if you can fi it so that the posterior distribution is from a known family then life becomes much easier! particularly important in the past More generally, when the posterior is of a known form we say that the posterior is analytically tractable summary measures are available analytically For many problems, however, the posterior is not a known distribution 64 BIO 233, HSPH Spring 2015
4 Mean birth weight: µ and σ 2 unknown In Homework #1 you assumed that σ 2 was known More realistically, let s take σ 2 as unknown multiparameter setting, θ =(µ, σ 2 ) Likelihood remains the same Y i Normal(µ, σ 2 ), i =1,..., n L(y µ, σ 2 ) = ( ) { n 1 ep 1 2πσ 2 2σ 2 } n (y i µ) 2 i=1 How do we specify a bivariate prior distribution for (µ, σ 2 )? 65 BIO 233, HSPH Spring 2015
5 One option is the following noninformative prior π(µ, σ 2 ) 1 σ 2 uniform for (µ, logσ) on(, ) (, ) aprioriindependence Using Bayes Theorem, the joint posterior distribution is proportional to π(µ, σ 2 y) L(y µ, σ 2 )π(µ, σ 2 ) 66 BIO 233, HSPH Spring 2015
6 Unfortunately, this doesn t correspond to any commonly known joint distribution even if we performed the integration to get the normalizing constant, we d be stuck How do we proceed? How do summarize a distribution if it isn t available analytically? visualize the joint posterior distribution for (µ, σ 2 ) compute summary statistics We use simulation-based or Monte Carlo methods 67 BIO 233, HSPH Spring 2015
7 Monte Carlo methods Consider the Beta(85, 915) distribution posterior for the low birth weight eample To summarize this distribution, note that the mean and variance have closed form epressions but the median does not Suppose we could generate random deviates from a Beta distribution e.g., using rbeta() in R We could empirically estimate the median: > sampposterior <- rbeta(1000, 85, 915) > median(sampposterior) [1] BIO 233, HSPH Spring 2015
8 More generally, one can estimate any summary measure by eploiting the duality between a distribution and samples generated from that distribution ## Mean > round(mean(sampposterior), digits=3) [1] ## Standard deviation > round(sd(sampposterior), digits=3) [1] ## 95% credible interval > round(quantile(sampposterior, c(0.025, 0.975)), digits=3) 2.5% 97.5% ## P(theta > 0.10) > round(mean(sampposterior > 0.10), digits=3) [1] BIO 233, HSPH Spring 2015
9 One can also visual the distribution > hist(sampposterior, breaks=seq(from=0.05, to=0.12, by=0.005), main="", lab=epression(theta * " = P(LBW)"), col="blue", freq=false) Density θ = P(LBW) 70 BIO 233, HSPH Spring 2015
10 Monte Carlo refers to the use of samples or simulation as a means to learn about a distribution the term was coined by physicists working on the Manhattan Project in the 1940 s These ideas can be applied to any distribution the trick is being able to generate samples from π(θ y) Consider again the posterior for (µ, σ 2 ) π(µ, σ 2 y) 1 { ep 1 [ σn+2 (n 1)s 2 2σ 2 + n(y µ) 2] } At the outset, it isn t clear how to generate samples from this distribution 71 BIO 233, HSPH Spring 2015
11 We can, however, decompose the joint posterior distribution as π(µ, σ 2 y) = π(µ y,σ 2 )π(σ 2 y) where conditional posterior of µ y,σ 2 is a Normal(y, σ 2 /n) and the marginal posterior of σ 2 y is given by π(σ 2 y) ( σ 2) n+1 2 ep { 1σ [ ]} (n 1)s which is the kernel for an inverse-gamma distribution: ( n 1 Inv-gamma, 2 (n 1)s 2 ) 2 72 BIO 233, HSPH Spring 2015
12 The decomposition suggests generating samples using the algorithm: (1) generate a random ( n 1 σ 2(r) Inv-gamma, 2 (2) generate a random µ (r) σ 2(r) Normal ( y, (n 1)s 2 ) 2 σ 2(r) n ) each cycle generates an independent random deviate from the joint posterior, π(µ, σ 2 y) (µ (1),σ 2(1) ) (µ (2),σ 2(2) ). 73 BIO 233, HSPH Spring 2015
13 One can also easily augment this algorithm to generate samples from the posterior predictive distribution f(ỹ y) = f(ỹ θ)π(θ y) θ This representation suggests imbedding a step where we generate random ỹ (r) Normal (µ (r), σ 2(r)) at the end of the r th cycle Marginally, the {ỹ (1),...,ỹ (R) } are a random sample from the target posterior predictive distribution 74 BIO 233, HSPH Spring 2015
14 Run the algorithm... > ## > load("northcarolina_data.dat") > n <- 100 > sampy <- sample(infants$weight, n) > ## > library(mcmcpack) >?rinvgamma > ## > R < > sampposterior <- matri(na, nrow=r, ncol=3) > for(r in 1:R) + { + ## + sigmasq <- rinvgamma(1, (n-1)/2, ((n-1)*var(sampy))/2) + mu <- rnorm(1, mean(sampy), sqrt(sigmasq/n)) + ytilde <- rnorm(1, mu, sqrt(sigmasq)) + ## + sampposterior[r,] <- c(mu, sigmasq, ytilde) + } 75 BIO 233, HSPH Spring 2015
15 Visualize the results... > ## > library(ks) > fhat <- kde(=sampposterior[,1:2], H=Hpi(=sampPosterior[,1:2])) > ## > plot(fhat, lab=epression(mu), ylab=epression(sigma^2), lim=c(3,3.5), drawpoints=true, ptcol="red", pch="", lwd=3, aes=false) > ais(1, seq(from=3, to=3.5, by=0.1)) > ais(2, seq(from=0.5, to=1.2, by=0.1)) > ## > fhaty <- kde(=sampposterior[,3], h=hpi(sampposterior[,3])) > plot(fhaty, lim=c(0, 6), lab=epression("prediction, " * tilde(y)), ylab="", aes=false, col="red", lwd=3) > ais(1, seq(from=0, to=6, by=1)) > ais(2, seq(from=0, to=0.5, by=0.1)) 76 BIO 233, HSPH Spring 2015
16 Joint posterior distribution, π(µ, σ 2 y) µ σ BIO 233, HSPH Spring 2015
17 Posterior predictive distribution, f(ỹ y) relative weight assigned to possible birth weight values, averaging over the uncertainty in our knowledge about µ and σ 2 density, f(y ~ y) prediction, Y ~ 78 BIO 233, HSPH Spring 2015
18 Compute numerical summary measures... > ## > apply(sampposterior, 2, mean) [1] > t(apply(sampposterior, 2, quantile, probs=c(0.5, 0.025, 0.975))) 50% 2.5% 97.5% [1,] [2,] [3,] > cor(sampposterior[,1], sampposterior[,2]) [1] BIO 233, HSPH Spring 2015
19 Sequential decomposition This strategy can be applied to a generic θ = (θ 1,...,θ p ) decompose the p-dimensional posterior distribution as π(θ y) = π(θ p y,θ 1,...,θ p 1 ) π(θ p 1 y,θ 1,...,θ p 2 )... π(θ 1 y) cycle through and sample from the p distributions sequentially each cycle generates an independent, random deviate from the multivariate posterior, π(θ y) Intuitive in that one breaks up the problem into a series of manageable pieces can then empirically summarize the joint distribution 80 BIO 233, HSPH Spring 2015
20 The Gibbs sampling algorithm For many problems, sequential decomposition doesn t yield a set of p distributions where each is of a known form Consider the set of p full conditionals π(θ 1 y,θ 1 ) π(θ 2 y,θ 2 ). π(θ p y,θ p ) it is often the case that these are each of a convenient form The Gibbs sampling algorithm generates samples by sequentially sampling from each these p full conditionals 81 BIO 233, HSPH Spring 2015
21 For eample, for p=2 the algorithm proceeds as follows: generate θ (1) 1 from π(θ 1 y,θ (0) 2 ) generate θ (1) 2 from π(θ 2 y,θ (1) 1 ) θ (1) = (θ (1) 1,θ(1) 2 ) θ (0) 2 is the starting value generate θ (2) 1 from π(θ 1 y,θ (1) 2 ) generate θ (2) 2 from π(θ 2 y,θ (2) 1 ) θ (2) = (θ (2) 1,θ(2) 2 ). 82 BIO 233, HSPH Spring 2015
22 Helpful to visualize the way the algorithm generates samples: 83 BIO 233, HSPH Spring 2015
23 Mean birth weight: Normal likelihood Suppose Y i i.i.d. Normal(µ, σ 2 )fori =1,..., n task is to learn about the unknown θ =(µ, σ 2 ) Further, suppose we adopt a flat prior for µ and an independent Inv-Gamma(a, b) priorforσ 2 the prior density is π(µ, σ 2 ) = ba Γ(a) ( ) σ 2 a 1 ep { b } σ 2 setting a=b=0 corresponds to the non informative prior we ve been using 84 BIO 233, HSPH Spring 2015
24 Applying Bayes Theorem, the joint posterior distribution is π(µ, σ 2 y) L(µ, σ 2 y)π(µ, σ 2 ) = ( ) { n 1 ep 1 2πσ 2 2σ 2 ba Γ(a) } n (y i µ) 2 i=1 ( σ 2 ) a 1 ep { b σ 2 } ( ) ( n/2+a+1 1 σ 2 ep { 1σ )} n (y i µ) 2 + b i=1 It isn t immediately clear how to directly generate samples from this joint distribution 85 BIO 233, HSPH Spring 2015
25 Rather than directly sampling from π(µ, σ 2 y), wecanusethegibbs sampling algorithm and iterate between (i) (ii) ( µ σ 2 σ 2 ), y Normal y, n ( ) n σ 2 µ, y Inv-gamma 2 + a +1, D 2 + b where y is the sample mean and D = n i=1 (y i µ) 2 Result is a sequence of samples from the joint posterior: (µ (1),σ 2(1) ) (µ (2),σ 2(2) ). 86 BIO 233, HSPH Spring 2015
26 Markov Chain Monte Carlo The Gibbs algorithm is helpful in that a seemingly difficult problem is broken down into a series of manageable pieces But there are two problems! (1) The set of full conditionals does not jointly fully specify the target posterior distribution in contrast to the components obtained via a sequential decomposition: π(θ 1,θ 2 y) = π(θ 1 y) π(θ 2 y,θ 1 ) (2) The resulting samples are dependent dependency between θ (1) and θ (2) is generated by the value of θ (1) 2 say that the samples ehibit autocorrelation 87 BIO 233, HSPH Spring 2015
27 The result is that we don t have an independent sample from the joint posterior distribution empirical summary measures won t pertain to π(θ y) However, by construction, the sequence of samples constitute a Markov chain Further, it s possible to show that the stationary distribution for the Markov chain is the sought-after posterior distribution we say that once the Markov chain converges, oneisgeneratingsamples from the posterior distribution In practice, we need to run the chain long enough to ensure that it has converged to its stationary distribution make adjustments to remove autocorrelation in the samples 88 BIO 233, HSPH Spring 2015
28 The Gibbs algorithm is one member of a broader class of algorithms that generate samples from (arbitrary) posterior distributions The general technique is referred to as Markov Chain Monte Carlo MCMC each algorithm requires consideration of convergence and autocorrelation Other algorithms include, among many others: importance sampling adaptive rejection sampling the Metropolis algorithm the Metropolis-Hastings algorithm We are going to focus on the Metropolis-Hastings algorithm versatile and, as we ll see, generally a good algorithm in the contetof GLMs 89 BIO 233, HSPH Spring 2015
29 The Metropolis-Hastings Algorithm Mean birth weight: t-distribution likelihood So far we ve taken the continuous response, Y,tobeNormallydistributed An alternative is to use the t-distribution heavier tails provide an alternative that may be robust to unusual/outlier observations Specifically, suppose Y t ν (µ, σ 2 ) non-central, scaled t-distribution with ν degrees of freedom for y (, ), thedensityis f(y µ, σ 2,ν) = Γ ( ) ν+1 2 Γ ( ) ν 2 ( ) 1/2 1 [1+ πνσ 2 (y µ)2 νσ 2 ] ν BIO 233, HSPH Spring 2015
30 Given an i.i.d sample of size n, thelikelihoodis L(µ, σ 2 y) = n i=1 Γ ( ) ν+1 2 Γ ( ) ν 2 ( ) 1/2 1 [1+ (y i µ) 2 ] ν+1 2 πνσ 2 νσ 2 here we take ν to be fied and known Again adopt a flat prior for µ and an independent Inv-Gamma(a, b) priorfor σ 2 : π(µ, σ 2 ) = b2 ( σ 2 ) a 1 ep { b } Γ(a) σ 2 Using Bayes Theorem, the posterior distribution is π(µ, σ 2 y) L(µ, σ 2 y)π(µ, σ 2 ) 91 BIO 233, HSPH Spring 2015
31 As with the posterior based on the Normal likelihood, this joint posterior doesn t belong to any known family We could again use the Gibbs sampling algorithm and iterate between the two full conditionals: π(µ σ 2, y) π(σ 2 µ, y) Unfortunately, neither of these are of a convenient form either! Q: How can we generate samples from a distribution that does not belong to any known family? 92 BIO 233, HSPH Spring 2015
32 Metropolis-Hastings Consider the general task of sampling from π(θ y) goal is to generate a sequence: θ (1), θ (2),... use these samples to evaluate summaries of the posterior Suppose the distribution corresponding to π(θ y) is unknown the functional form of the density doesn t correspond to a known distribution we only know the kernel of the density π(θ y) L(θ y)π(θ) integral in the denominator doesn t have a closed form epression software isn t readily available 93 BIO 233, HSPH Spring 2015
33 The Metropolis-Hastings algorithm proceeds as follows: let θ (r) be the current state in the sequence (i) generate a proposal for the net value of θ, denotedbyθ denote the density of the proposal distribution by q(θ θ (r) ) (ii) either reject the proposal θ (r+1) = θ (r) accept the proposal θ (r+1) = θ the decision to reject/accept the proposal is based on the flip ofacoin with probability ( = min 1, a r referred to as the acceptance ratio accept automatically if a r 1 π(θ y) π(θ (r) y) q(θ (r) θ ) ) q(θ θ (r) ) 94 BIO 233, HSPH Spring 2015
34 The algorithm boils down to being able to perform three tasks: (1) choose a proposal distribution (2) sample from the proposal distribution (3) compute the acceptance ratio Ideally the proposal distribution is as close to the target posterior distribution as possible intuitively, why does this make sense? mathematically, why does this make sense? But we also have to choose a proposal distribution that we can actually sample from! Q: Interpretation of the acceptance ratio? 95 BIO 233, HSPH Spring 2015
35 Mean birth weight: t-distribution likelihood For the birth weight eample, we couldn t sample directly from the two full conditionals: π(µ σ 2, y) n [1+ (y i µ) 2 i=1 νσ 2 ] ν+1 2 π(σ 2 µ, y) ( ) n/2+a+1 { 1 σ 2 ep b } n [1+ (y i µ) 2 ] ν+1 2 σ 2 νσ 2 i=1 Instead, we sample from each using the Metropolis-Hasting algorithm say: implement a Gibbs sampling algorithm with two Metropolis-Hastings steps or updates Need to choose proposal distributions for both updates 96 BIO 233, HSPH Spring 2015
36 Recall, for the Normal likelihood the two full conditionals were µ σ 2, y Normal ( y, σ 2 n ) σ 2 µ, y Inv-gamma (a, b ) where a = n/2+a +1and b = D/2+b Their densities are, respectively, q 1 (µ σ 2, y) = q 2 (σ 2 µ, y) = { } 1 2πσ2 /n ep n(µ y)2 2σ 2 ( ) } σ 2 a 1 ep { b b Γ(a ) σ 2 Q: Why might these be good proposal distributions? 97 BIO 233, HSPH Spring 2015
37 Suppose the current state in the Markov chain is (µ (r), σ 2(r) ) To sample µ (r+1) : generate a proposal, µ,fromanormal(y, σ 2(r) /n) calculate a r = min ( 1, π(µ σ 2(r), y) π(µ (r) σ 2(r), y) q 1 (µ (r) σ 2(r), y) q 1 (µ σ 2(r), y) ) generate a random U Uniform(0,1) if U<a r,acceptthemove set µ (r+1) equal to µ if U>a r,rejectthemove set µ (r+1) equal to µ (r) 98 BIO 233, HSPH Spring 2015
38 Whatever the decision, we have a value for µ (r+1) To sample σ 2(r+1) : generate a proposal, σ 2,fromanInv-gamma(a, b ) calculate a r = min ( 1, π(σ 2 µ (r+1), y) π(σ 2(r) µ (r+1), y) q 2 (σ 2(r) µ (r+1), y) q 2 (σ 2 µ (r+1), y) ) generate a random U Uniform(0,1) if U<a r,acceptthemove set σ 2(r+1) equal to σ 2 if U>a r,rejectthemove set σ 2(r+1) equal to σ 2(r) 99 BIO 233, HSPH Spring 2015
39 The Metropolis sampling algorithm The Metropolis algorithm is a special case of the Metropolis-Hastings algorithm where the proposal distribution is symmetric As such, q(θ (r) θ ) = q(θ θ (r) ) and the acceptance ratio reduces to ( π(θ ) y) a r = min 1, π(θ (r) y) However, if the target distribution is not symmetric then we might epect symmetric proposal distributions to not perform as well fewer proposals will be accepted Trade-off in terms of computational ease and efficiency of the sampling algorithm 100 BIO 233, HSPH Spring 2015
40 Practical issues for MCMC Following this algorithm yields a sequence of samples that form a Markov chain one whose stationary distribution is the desired posterior distribution Practical issues include: (1) specification of starting values, (µ (0), σ 2(0) ) (2) monitoring convergence of the chain (3) deciding how many samples to generate (4) accounting for correlation in the samples 101 BIO 233, HSPH Spring 2015
41 In practice, one often runs M chains simultaneously each with differing starting values pool samples across the chains when summarizing the posterior distribution Monitoring convergence is often done via visual inspection of the chains referred to as trace plots goal is to have good coverage of the parameter space eamine miing of the chains if M>1 102 BIO 233, HSPH Spring 2015
42 Posterior for µ based on a Normal likelihood Gibbs sampling algorithm µ Scan 103 BIO 233, HSPH Spring 2015
43 Posterior for σ 2 based on a Normal likelihood Gibbs sampling algorithm σ Scan 104 BIO 233, HSPH Spring 2015
44 Posterior for ψ = νσ 2 /(ν 2) based on a t-distribution likelihood Gibbs sampling algorithm with M-H updates ψ Scan 105 BIO 233, HSPH Spring 2015
45 If M>1, onecancalculatethepotential scale reduction (PSR) factor suppose each chain is run for R iterations for a given parameter, θ, them th chain is denoted θ m (1), θ m (2),...,θ m (R) let θ m and s 2 m denote the sample mean and variance of the m th chain calculate PSR for θ as B/R + W (R 1)/R PSR = W where W is the mean of the within-chain variance of θ M W = 1 M m=1 s 2 m 106 BIO 233, HSPH Spring 2015
46 and B/R is the between-chain variance of the chain means B R = 1 M 1 M (θ m θ) 2 m=1 typically set R such that PSR is less than 1.05 should be ensured for each parameter The number of samples should ideally be based on the Monte Carlo error see Homework #1 may not be clear cut if the algorithm is computationally epensive Autocorrelation arises because of the dependent nature of the sampling as one cycles through the full conditionals will also be a problem if the proposal distribution was poorly chosen typically handled by thinning several graphical and numerical diagnostics are also available 107 BIO 233, HSPH Spring 2015
Chapter 7: Estimation Sections
1 / 40 Chapter 7: Estimation Sections 7.1 Statistical Inference Bayesian Methods: Chapter 7 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions 7.4 Bayes Estimators Frequentist Methods:
More informationConjugate Models. Patrick Lam
Conjugate Models Patrick Lam Outline Conjugate Models What is Conjugacy? The Beta-Binomial Model The Normal Model Normal Model with Unknown Mean, Known Variance Normal Model with Known Mean, Unknown Variance
More informationOutline. Review Continuation of exercises from last time
Bayesian Models II Outline Review Continuation of exercises from last time 2 Review of terms from last time Probability density function aka pdf or density Likelihood function aka likelihood Conditional
More informationGenerating Random Numbers
Generating Random Numbers Aim: produce random variables for given distribution Inverse Method Let F be the distribution function of an univariate distribution and let F 1 (y) = inf{x F (x) y} (generalized
More informationNon-informative Priors Multiparameter Models
Non-informative Priors Multiparameter Models Statistics 220 Spring 2005 Copyright c 2005 by Mark E. Irwin Prior Types Informative vs Non-informative There has been a desire for a prior distributions that
More information# generate data num.obs <- 100 y <- rnorm(num.obs,mean = theta.true, sd = sqrt(sigma.sq.true))
Posterior Sampling from Normal Now we seek to create draws from the joint posterior distribution and the marginal posterior distributions and Note the marginal posterior distributions would be used to
More informationST440/550: Applied Bayesian Analysis. (5) Multi-parameter models - Summarizing the posterior
(5) Multi-parameter models - Summarizing the posterior Models with more than one parameter Thus far we have studied single-parameter models, but most analyses have several parameters For example, consider
More informationPosterior Inference. , where should we start? Consider the following computational procedure: 1. draw samples. 2. convert. 3. compute properties
Posterior Inference Example. Consider a binomial model where we have a posterior distribution for the probability term, θ. Suppose we want to make inferences about the log-odds γ = log ( θ 1 θ), where
More informationCOS 513: Gibbs Sampling
COS 513: Gibbs Sampling Matthew Salesi December 6, 2010 1 Overview Concluding the coverage of Markov chain Monte Carlo (MCMC) sampling methods, we look today at Gibbs sampling. Gibbs sampling is a simple
More informationExam 2 Spring 2015 Statistics for Applications 4/9/2015
18.443 Exam 2 Spring 2015 Statistics for Applications 4/9/2015 1. True or False (and state why). (a). The significance level of a statistical test is not equal to the probability that the null hypothesis
More information(5) Multi-parameter models - Summarizing the posterior
(5) Multi-parameter models - Summarizing the posterior Spring, 2017 Models with more than one parameter Thus far we have studied single-parameter models, but most analyses have several parameters For example,
More informationChapter 7: Estimation Sections
1 / 31 : Estimation Sections 7.1 Statistical Inference Bayesian Methods: 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions 7.4 Bayes Estimators Frequentist Methods: 7.5 Maximum Likelihood
More informationCalibration of Interest Rates
WDS'12 Proceedings of Contributed Papers, Part I, 25 30, 2012. ISBN 978-80-7378-224-5 MATFYZPRESS Calibration of Interest Rates J. Černý Charles University, Faculty of Mathematics and Physics, Prague,
More informationBayesian Hierarchical/ Multilevel and Latent-Variable (Random-Effects) Modeling
Bayesian Hierarchical/ Multilevel and Latent-Variable (Random-Effects) Modeling 1: Formulation of Bayesian models and fitting them with MCMC in WinBUGS David Draper Department of Applied Mathematics and
More informationBayesian Normal Stuff
Bayesian Normal Stuff - Set-up of the basic model of a normally distributed random variable with unknown mean and variance (a two-parameter model). - Discuss philosophies of prior selection - Implementation
More informationA Markov Chain Monte Carlo Approach to Estimate the Risks of Extremely Large Insurance Claims
International Journal of Business and Economics, 007, Vol. 6, No. 3, 5-36 A Markov Chain Monte Carlo Approach to Estimate the Risks of Extremely Large Insurance Claims Wan-Kai Pang * Department of Applied
More informationBayesian Multinomial Model for Ordinal Data
Bayesian Multinomial Model for Ordinal Data Overview This example illustrates how to fit a Bayesian multinomial model by using the built-in mutinomial density function (MULTINOM) in the MCMC procedure
More informationA Bayesian model for classifying all differentially expressed proteins simultaneously in 2D PAGE gels
BMC Bioinformatics This Provisional PDF corresponds to the article as it appeared upon acceptance. Fully formatted PDF and full text (HTML) versions will be made available soon. A Bayesian model for classifying
More informationSubject CS1 Actuarial Statistics 1 Core Principles. Syllabus. for the 2019 exams. 1 June 2018
` Subject CS1 Actuarial Statistics 1 Core Principles Syllabus for the 2019 exams 1 June 2018 Copyright in this Core Reading is the property of the Institute and Faculty of Actuaries who are the sole distributors.
More informationBayesian course - problem set 3 (lecture 4)
Bayesian course - problem set 3 (lecture 4) Ben Lambert November 14, 2016 1 Ticked off Imagine once again that you are investigating the occurrence of Lyme disease in the UK. This is a vector-borne disease
More informationChapter 7: Estimation Sections
Chapter 7: Estimation Sections 7.1 Statistical Inference Bayesian Methods: 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions Frequentist Methods: 7.5 Maximum Likelihood Estimators
More informationSTA 114: Statistics. Notes 10. Conjugate Priors
STA 114: Statistics Notes 10. Conjugate Priors Conjugate family Once we get a /pmf ξ(θ x) by combining a model X f(x θ) with a /pmf ξ(θ) on θ Θ, a report can be made by summarizing the. It helps to have
More informationContinuous Distributions
Quantitative Methods 2013 Continuous Distributions 1 The most important probability distribution in statistics is the normal distribution. Carl Friedrich Gauss (1777 1855) Normal curve A normal distribution
More informationAdaptive Experiments for Policy Choice. March 8, 2019
Adaptive Experiments for Policy Choice Maximilian Kasy Anja Sautmann March 8, 2019 Introduction The goal of many experiments is to inform policy choices: 1. Job search assistance for refugees: Treatments:
More informationSTAT 425: Introduction to Bayesian Analysis
STAT 45: Introduction to Bayesian Analysis Marina Vannucci Rice University, USA Fall 018 Marina Vannucci (Rice University, USA) Bayesian Analysis (Part 1) Fall 018 1 / 37 Lectures 9-11: Multi-parameter
More informationApplication of MCMC Algorithm in Interest Rate Modeling
Application of MCMC Algorithm in Interest Rate Modeling Xiaoxia Feng and Dejun Xie Abstract Interest rate modeling is a challenging but important problem in financial econometrics. This work is concerned
More informationTwo hours. To be supplied by the Examinations Office: Mathematical Formula Tables and Statistical Tables THE UNIVERSITY OF MANCHESTER
Two hours MATH20802 To be supplied by the Examinations Office: Mathematical Formula Tables and Statistical Tables THE UNIVERSITY OF MANCHESTER STATISTICAL METHODS Answer any FOUR of the SIX questions.
More informationSTA 532: Theory of Statistical Inference
STA 532: Theory of Statistical Inference Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA 2 Estimating CDFs and Statistical Functionals Empirical CDFs Let {X i : i n}
More informationCS 361: Probability & Statistics
March 12, 2018 CS 361: Probability & Statistics Inference Binomial likelihood: Example Suppose we have a coin with an unknown probability of heads. We flip the coin 10 times and observe 2 heads. What can
More informationThe Bernoulli distribution
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
More informationA Practical Implementation of the Gibbs Sampler for Mixture of Distributions: Application to the Determination of Specifications in Food Industry
A Practical Implementation of the for Mixture of Distributions: Application to the Determination of Specifications in Food Industry Julien Cornebise 1 Myriam Maumy 2 Philippe Girard 3 1 Ecole Supérieure
More informationExtended Model: Posterior Distributions
APPENDIX A Extended Model: Posterior Distributions A. Homoskedastic errors Consider the basic contingent claim model b extended by the vector of observables x : log C i = β log b σ, x i + β x i + i, i
More informationBusiness Statistics 41000: Probability 3
Business Statistics 41000: Probability 3 Drew D. Creal University of Chicago, Booth School of Business February 7 and 8, 2014 1 Class information Drew D. Creal Email: dcreal@chicagobooth.edu Office: 404
More informationRelevant parameter changes in structural break models
Relevant parameter changes in structural break models A. Dufays J. Rombouts Forecasting from Complexity April 27 th, 2018 1 Outline Sparse Change-Point models 1. Motivation 2. Model specification Shrinkage
More informationAn Introduction to Bayesian Inference and MCMC Methods for Capture-Recapture
An Introduction to Bayesian Inference and MCMC Methods for Capture-Recapture Trinity River Restoration Program Workshop on Outmigration: Population Estimation October 6 8, 2009 An Introduction to Bayesian
More informationModeling skewness and kurtosis in Stochastic Volatility Models
Modeling skewness and kurtosis in Stochastic Volatility Models Georgios Tsiotas University of Crete, Department of Economics, GR December 19, 2006 Abstract Stochastic volatility models have been seen as
More informationExtracting Information from the Markets: A Bayesian Approach
Extracting Information from the Markets: A Bayesian Approach Daniel Waggoner The Federal Reserve Bank of Atlanta Florida State University, February 29, 2008 Disclaimer: The views expressed are the author
More informationStatistical Computing (36-350)
Statistical Computing (36-350) Lecture 16: Simulation III: Monte Carlo Cosma Shalizi 21 October 2013 Agenda Monte Carlo Monte Carlo approximation of integrals and expectations The rejection method and
More informationMetropolis-Hastings algorithm
Metropolis-Hastings algorithm Dr. Jarad Niemi STAT 544 - Iowa State University March 27, 2018 Jarad Niemi (STAT544@ISU) Metropolis-Hastings March 27, 2018 1 / 32 Outline Metropolis-Hastings algorithm Independence
More informationStrategies for Improving the Efficiency of Monte-Carlo Methods
Strategies for Improving the Efficiency of Monte-Carlo Methods Paul J. Atzberger General comments or corrections should be sent to: paulatz@cims.nyu.edu Introduction The Monte-Carlo method is a useful
More informationA potentially useful approach to model nonlinearities in time series is to assume different behavior (structural break) in different subsamples
1.3 Regime switching models A potentially useful approach to model nonlinearities in time series is to assume different behavior (structural break) in different subsamples (or regimes). If the dates, the
More informationLecture 17: More on Markov Decision Processes. Reinforcement learning
Lecture 17: More on Markov Decision Processes. Reinforcement learning Learning a model: maximum likelihood Learning a value function directly Monte Carlo Temporal-difference (TD) learning COMP-424, Lecture
More informationTHE UNIVERSITY OF TEXAS AT AUSTIN Department of Information, Risk, and Operations Management
THE UNIVERSITY OF TEXAS AT AUSTIN Department of Information, Risk, and Operations Management BA 386T Tom Shively PROBABILITY CONCEPTS AND NORMAL DISTRIBUTIONS The fundamental idea underlying any statistical
More informationChapter 5. Statistical inference for Parametric Models
Chapter 5. Statistical inference for Parametric Models Outline Overview Parameter estimation Method of moments How good are method of moments estimates? Interval estimation Statistical Inference for Parametric
More informationChapter 5 Univariate time-series analysis. () Chapter 5 Univariate time-series analysis 1 / 29
Chapter 5 Univariate time-series analysis () Chapter 5 Univariate time-series analysis 1 / 29 Time-Series Time-series is a sequence fx 1, x 2,..., x T g or fx t g, t = 1,..., T, where t is an index denoting
More informationProbability. An intro for calculus students P= Figure 1: A normal integral
Probability An intro for calculus students.8.6.4.2 P=.87 2 3 4 Figure : A normal integral Suppose we flip a coin 2 times; what is the probability that we get more than 2 heads? Suppose we roll a six-sided
More informationChapter 2 Uncertainty Analysis and Sampling Techniques
Chapter 2 Uncertainty Analysis and Sampling Techniques The probabilistic or stochastic modeling (Fig. 2.) iterative loop in the stochastic optimization procedure (Fig..4 in Chap. ) involves:. Specifying
More informationChapter 4: Asymptotic Properties of MLE (Part 3)
Chapter 4: Asymptotic Properties of MLE (Part 3) Daniel O. Scharfstein 09/30/13 1 / 1 Breakdown of Assumptions Non-Existence of the MLE Multiple Solutions to Maximization Problem Multiple Solutions to
More informationIEOR E4602: Quantitative Risk Management
IEOR E4602: Quantitative Risk Management Basic Concepts and Techniques of Risk Management Martin Haugh Department of Industrial Engineering and Operations Research Columbia University Email: martin.b.haugh@gmail.com
More informationEstimation Appendix to Dynamics of Fiscal Financing in the United States
Estimation Appendix to Dynamics of Fiscal Financing in the United States Eric M. Leeper, Michael Plante, and Nora Traum July 9, 9. Indiana University. This appendix includes tables and graphs of additional
More informationModel 0: We start with a linear regression model: log Y t = β 0 + β 1 (t 1980) + ε, with ε N(0,
Stat 534: Fall 2017. Introduction to the BUGS language and rjags Installation: download and install JAGS. You will find the executables on Sourceforge. You must have JAGS installed prior to installing
More informationIntroduction to Sequential Monte Carlo Methods
Introduction to Sequential Monte Carlo Methods Arnaud Doucet NCSU, October 2008 Arnaud Doucet () Introduction to SMC NCSU, October 2008 1 / 36 Preliminary Remarks Sequential Monte Carlo (SMC) are a set
More informationDown-Up Metropolis-Hastings Algorithm for Multimodality
Down-Up Metropolis-Hastings Algorithm for Multimodality Hyungsuk Tak Stat310 24 Nov 2015 Joint work with Xiao-Li Meng and David A. van Dyk Outline Motivation & idea Down-Up Metropolis-Hastings (DUMH) algorithm
More information4.1 Introduction Estimating a population mean The problem with estimating a population mean with a sample mean: an example...
Chapter 4 Point estimation Contents 4.1 Introduction................................... 2 4.2 Estimating a population mean......................... 2 4.2.1 The problem with estimating a population mean
More informationThis homework assignment uses the material on pages ( A moving average ).
Module 2: Time series concepts HW Homework assignment: equally weighted moving average This homework assignment uses the material on pages 14-15 ( A moving average ). 2 Let Y t = 1/5 ( t + t-1 + t-2 +
More informationSupplementary Material: Strategies for exploration in the domain of losses
1 Supplementary Material: Strategies for exploration in the domain of losses Paul M. Krueger 1,, Robert C. Wilson 2,, and Jonathan D. Cohen 3,4 1 Department of Psychology, University of California, Berkeley
More informationChapter 8 Statistical Intervals for a Single Sample
Chapter 8 Statistical Intervals for a Single Sample Part 1: Confidence intervals (CI) for population mean µ Section 8-1: CI for µ when σ 2 known & drawing from normal distribution Section 8-1.2: Sample
More informationدرس هفتم یادگیري ماشین. (Machine Learning) دانشگاه فردوسی مشهد دانشکده مهندسی رضا منصفی
یادگیري ماشین توزیع هاي نمونه و تخمین نقطه اي پارامترها Sampling Distributions and Point Estimation of Parameter (Machine Learning) دانشگاه فردوسی مشهد دانشکده مهندسی رضا منصفی درس هفتم 1 Outline Introduction
More informationComputer Statistics with R
MAREK GAGOLEWSKI KONSTANCJA BOBECKA-WESO LOWSKA PRZEMYS LAW GRZEGORZEWSKI Computer Statistics with R 5. Point Estimation Faculty of Mathematics and Information Science Warsaw University of Technology []
More informationSimulation Wrap-up, Statistics COS 323
Simulation Wrap-up, Statistics COS 323 Today Simulation Re-cap Statistics Variance and confidence intervals for simulations Simulation wrap-up FYI: No class or office hours Thursday Simulation wrap-up
More information**BEGINNING OF EXAMINATION** A random sample of five observations from a population is:
**BEGINNING OF EXAMINATION** 1. You are given: (i) A random sample of five observations from a population is: 0.2 0.7 0.9 1.1 1.3 (ii) You use the Kolmogorov-Smirnov test for testing the null hypothesis,
More informationConjugate priors: Beta and normal Class 15, Jeremy Orloff and Jonathan Bloom
1 Learning Goals Conjugate s: Beta and normal Class 15, 18.05 Jeremy Orloff and Jonathan Bloom 1. Understand the benefits of conjugate s.. Be able to update a beta given a Bernoulli, binomial, or geometric
More informationApproximate Bayesian Computation using Indirect Inference
Approximate Bayesian Computation using Indirect Inference Chris Drovandi c.drovandi@qut.edu.au Acknowledgement: Prof Tony Pettitt and Prof Malcolm Faddy School of Mathematical Sciences, Queensland University
More informationMaster s in Financial Engineering Foundations of Buy-Side Finance: Quantitative Risk and Portfolio Management. > Teaching > Courses
Master s in Financial Engineering Foundations of Buy-Side Finance: Quantitative Risk and Portfolio Management www.symmys.com > Teaching > Courses Spring 2008, Monday 7:10 pm 9:30 pm, Room 303 Attilio Meucci
More informationBayesian Linear Model: Gory Details
Bayesian Linear Model: Gory Details Pubh7440 Notes By Sudipto Banerjee Let y y i ] n i be an n vector of independent observations on a dependent variable (or response) from n experimental units. Associated
More informationWC-5 Just How Credible Is That Employer? Exploring GLMs and Multilevel Modeling for NCCI s Excess Loss Factor Methodology
Antitrust Notice The Casualty Actuarial Society is committed to adhering strictly to the letter and spirit of the antitrust laws. Seminars conducted under the auspices of the CAS are designed solely to
More informationPROBABILITY. Wiley. With Applications and R ROBERT P. DOBROW. Department of Mathematics. Carleton College Northfield, MN
PROBABILITY With Applications and R ROBERT P. DOBROW Department of Mathematics Carleton College Northfield, MN Wiley CONTENTS Preface Acknowledgments Introduction xi xiv xv 1 First Principles 1 1.1 Random
More informationMEASURING PORTFOLIO RISKS USING CONDITIONAL COPULA-AR-GARCH MODEL
MEASURING PORTFOLIO RISKS USING CONDITIONAL COPULA-AR-GARCH MODEL Isariya Suttakulpiboon MSc in Risk Management and Insurance Georgia State University, 30303 Atlanta, Georgia Email: suttakul.i@gmail.com,
More informationUPDATED IAA EDUCATION SYLLABUS
II. UPDATED IAA EDUCATION SYLLABUS A. Supporting Learning Areas 1. STATISTICS Aim: To enable students to apply core statistical techniques to actuarial applications in insurance, pensions and emerging
More informationComputational Statistics Handbook with MATLAB
«H Computer Science and Data Analysis Series Computational Statistics Handbook with MATLAB Second Edition Wendy L. Martinez The Office of Naval Research Arlington, Virginia, U.S.A. Angel R. Martinez Naval
More informationEE266 Homework 5 Solutions
EE, Spring 15-1 Professor S. Lall EE Homework 5 Solutions 1. A refined inventory model. In this problem we consider an inventory model that is more refined than the one you ve seen in the lectures. The
More informationAnalysis of the Bitcoin Exchange Using Particle MCMC Methods
Analysis of the Bitcoin Exchange Using Particle MCMC Methods by Michael Johnson M.Sc., University of British Columbia, 2013 B.Sc., University of Winnipeg, 2011 Project Submitted in Partial Fulfillment
More informationMonotonically Constrained Bayesian Additive Regression Trees
Constrained Bayesian Additive Regression Trees Robert McCulloch University of Chicago, Booth School of Business Joint with: Hugh Chipman (Acadia), Ed George (UPenn, Wharton), Tom Shively (U Texas, McCombs)
More informationEvidence from Large Indemnity and Medical Triangles
2009 Casualty Loss Reserve Seminar Session: Workers Compensation - How Long is the Tail? Evidence from Large Indemnity and Medical Triangles Casualty Loss Reserve Seminar September 14-15, 15, 2009 Chicago,
More informationStatistics 431 Spring 2007 P. Shaman. Preliminaries
Statistics 4 Spring 007 P. Shaman The Binomial Distribution Preliminaries A binomial experiment is defined by the following conditions: A sequence of n trials is conducted, with each trial having two possible
More informationProbability Theory. Probability and Statistics for Data Science CSE594 - Spring 2016
Probability Theory Probability and Statistics for Data Science CSE594 - Spring 2016 What is Probability? 2 What is Probability? Examples outcome of flipping a coin (seminal example) amount of snowfall
More informationStatistical Inference and Methods
Department of Mathematics Imperial College London d.stephens@imperial.ac.uk http://stats.ma.ic.ac.uk/ das01/ 14th February 2006 Part VII Session 7: Volatility Modelling Session 7: Volatility Modelling
More informationEfficiency Measurement with the Weibull Stochastic Frontier*
OXFORD BULLETIN OF ECONOMICS AND STATISTICS, 69, 5 (2007) 0305-9049 doi: 10.1111/j.1468-0084.2007.00475.x Efficiency Measurement with the Weibull Stochastic Frontier* Efthymios G. Tsionas Department of
More informationOption Pricing Using Bayesian Neural Networks
Option Pricing Using Bayesian Neural Networks Michael Maio Pires, Tshilidzi Marwala School of Electrical and Information Engineering, University of the Witwatersrand, 2050, South Africa m.pires@ee.wits.ac.za,
More informationChapter 4: Commonly Used Distributions. Statistics for Engineers and Scientists Fourth Edition William Navidi
Chapter 4: Commonly Used Distributions Statistics for Engineers and Scientists Fourth Edition William Navidi 2014 by Education. This is proprietary material solely for authorized instructor use. Not authorized
More informationRobust Regression for Capital Asset Pricing Model Using Bayesian Approach
Thai Journal of Mathematics : 016) 71 8 Special Issue on Applied Mathematics : Bayesian Econometrics http://thaijmath.in.cmu.ac.th ISSN 1686-009 Robust Regression for Capital Asset Pricing Model Using
More informationDealing with forecast uncertainty in inventory models
Dealing with forecast uncertainty in inventory models 19th IIF workshop on Supply Chain Forecasting for Operations Lancaster University Dennis Prak Supervisor: Prof. R.H. Teunter June 29, 2016 Dennis Prak
More informationExtend the ideas of Kan and Zhou paper on Optimal Portfolio Construction under parameter uncertainty
Extend the ideas of Kan and Zhou paper on Optimal Portfolio Construction under parameter uncertainty George Photiou Lincoln College University of Oxford A dissertation submitted in partial fulfilment for
More information1. You are given the following information about a stationary AR(2) model:
Fall 2003 Society of Actuaries **BEGINNING OF EXAMINATION** 1. You are given the following information about a stationary AR(2) model: (i) ρ 1 = 05. (ii) ρ 2 = 01. Determine φ 2. (A) 0.2 (B) 0.1 (C) 0.4
More informationReview for Final Exam Spring 2014 Jeremy Orloff and Jonathan Bloom
Review for Final Exam 18.05 Spring 2014 Jeremy Orloff and Jonathan Bloom THANK YOU!!!! JON!! PETER!! RUTHI!! ERIKA!! ALL OF YOU!!!! Probability Counting Sets Inclusion-exclusion principle Rule of product
More informationFinancial Econometrics (FinMetrics04) Time-series Statistics Concepts Exploratory Data Analysis Testing for Normality Empirical VaR
Financial Econometrics (FinMetrics04) Time-series Statistics Concepts Exploratory Data Analysis Testing for Normality Empirical VaR Nelson Mark University of Notre Dame Fall 2017 September 11, 2017 Introduction
More informationELEMENTS OF MONTE CARLO SIMULATION
APPENDIX B ELEMENTS OF MONTE CARLO SIMULATION B. GENERAL CONCEPT The basic idea of Monte Carlo simulation is to create a series of experimental samples using a random number sequence. According to the
More informationHigh-Frequency Data Analysis and Market Microstructure [Tsay (2005), chapter 5]
1 High-Frequency Data Analysis and Market Microstructure [Tsay (2005), chapter 5] High-frequency data have some unique characteristics that do not appear in lower frequencies. At this class we have: Nonsynchronous
More informationChapter 8: Sampling distributions of estimators Sections
Chapter 8 continued Chapter 8: Sampling distributions of estimators Sections 8.1 Sampling distribution of a statistic 8.2 The Chi-square distributions 8.3 Joint Distribution of the sample mean and sample
More informationBasic Data Analysis. Stephen Turnbull Business Administration and Public Policy Lecture 4: May 2, Abstract
Basic Data Analysis Stephen Turnbull Business Administration and Public Policy Lecture 4: May 2, 2013 Abstract Introduct the normal distribution. Introduce basic notions of uncertainty, probability, events,
More informationEstimation after Model Selection
Estimation after Model Selection Vanja M. Dukić Department of Health Studies University of Chicago E-Mail: vanja@uchicago.edu Edsel A. Peña* Department of Statistics University of South Carolina E-Mail:
More informationComparison of Pricing Approaches for Longevity Markets
Comparison of Pricing Approaches for Longevity Markets Melvern Leung Simon Fung & Colin O hare Longevity 12 Conference, Chicago, The Drake Hotel, September 30 th 2016 1 / 29 Overview Introduction 1 Introduction
More informationAnalysis of truncated data with application to the operational risk estimation
Analysis of truncated data with application to the operational risk estimation Petr Volf 1 Abstract. Researchers interested in the estimation of operational risk often face problems arising from the structure
More informationIEOR E4602: Quantitative Risk Management
IEOR E4602: Quantitative Risk Management Risk Measures Martin Haugh Department of Industrial Engineering and Operations Research Columbia University Email: martin.b.haugh@gmail.com Reference: Chapter 8
More informationEstimating a Dynamic Oligopolistic Game with Serially Correlated Unobserved Production Costs. SS223B-Empirical IO
Estimating a Dynamic Oligopolistic Game with Serially Correlated Unobserved Production Costs SS223B-Empirical IO Motivation There have been substantial recent developments in the empirical literature on
More informationMVE051/MSG Lecture 7
MVE051/MSG810 2017 Lecture 7 Petter Mostad Chalmers November 20, 2017 The purpose of collecting and analyzing data Purpose: To build and select models for parts of the real world (which can be used for
More informationConfidence Intervals Introduction
Confidence Intervals Introduction A point estimate provides no information about the precision and reliability of estimation. For example, the sample mean X is a point estimate of the population mean μ
More informationSOCIETY OF ACTUARIES EXAM STAM SHORT-TERM ACTUARIAL MATHEMATICS EXAM STAM SAMPLE QUESTIONS
SOCIETY OF ACTUARIES EXAM STAM SHORT-TERM ACTUARIAL MATHEMATICS EXAM STAM SAMPLE QUESTIONS Questions 1-307 have been taken from the previous set of Exam C sample questions. Questions no longer relevant
More informationOil Price Volatility and Asymmetric Leverage Effects
Oil Price Volatility and Asymmetric Leverage Effects Eunhee Lee and Doo Bong Han Institute of Life Science and Natural Resources, Department of Food and Resource Economics Korea University, Department
More informationFREDRIK BAJERS VEJ 7 G 9220 AALBORG ØST Tlf.: URL: Fax: Monte Carlo methods
INSTITUT FOR MATEMATISKE FAG AALBORG UNIVERSITET FREDRIK BAJERS VEJ 7 G 9220 AALBORG ØST Tlf.: 96 35 88 63 URL: www.math.auc.dk Fax: 98 15 81 29 E-mail: jm@math.aau.dk Monte Carlo methods Monte Carlo methods
More information