Bayesian Linear Model: Gory Details
|
|
- Cory Copeland
- 6 years ago
- Views:
Transcription
1 Bayesian Linear Model: Gory Details Pubh7440 Notes By Sudipto Banerjee Let y y i ] n i be an n vector of independent observations on a dependent variable (or response) from n experimental units. Associated with the y i, is a p vector of regressors, say x i, and lead to the linear regression model y X + ɛ, () where X x T i ]n i is the n p matrix of regressors with i-th row being xt i and is assumed fixed, is the slope vector of regression coefficients and ɛ ɛ i ] n i is the vector of random variables representing pure error or measurement error in the dependent variable. For independent observations, we assume ɛ MV N(0, σ I n ), viz. that each component ɛ i iid N(0, σ ). Furthermore, we will assume that the columns of the matrix X are linearly independent so that the rank of X is p. The N IG conjugate prior family A popular Bayesian model builds upon the linear regression of y using conjugate priors by specifying p(, σ ) p( σ )p(σ ) N(µ, σ V ) IG(a, b) NIG(µ, V, a, b) b a ( ) a+p/+ (π) p/ V / Γ(a) σ exp σ { b + }] ( µ )T V ( µ ) ( ) a+p/+ σ exp σ { b + }] ( µ )T V ( µ ), () where Γ( ) represents the Gamma function and the IG(a, b) prior density for σ is given by p(σ ) ( ) ba a+ ( Γ(a) σ exp b ) σ, σ > 0, where a, b > 0. We call this the Normal-Inverse-Gamma (NIG) prior and denote it as NIG(µ, V, a, b). The NIG probability distribution is a joint probability distribution of a vector and a scalar σ. If (, σ ) NIG(µ, V, a, b), then an interesting analytic form results from integrating out σ
2 from the joint density: b a ( (π) p/ V / Γ(a) σ ) a+ exp { σ b + ]} ( µ)t V ( µ) dσ NIG(µ, V, a, b)dσ b a (π) p/ V / exp { σ ( Γ(a) b + )} ( µ)t V ( µ) dσ b a Γ ( a + p ) (π) p/ V / b + ] (a+ p ) Γ(a) ( µ)t V ( µ) Γ ( a + p ) π p/ (a) b a V + ( µ)t b a V ] ] ( a+p ) ( µ). / Γ(a) a This is a multivariate t density: Γ ( ν+p) MV St ν (µ, Σ) Γ ( ) ν π p/ νσ / + ( µ)t Σ ( µ) ν with ν a and Σ ( b a) V. ] ν+p, (3) The likelihood The likelihood for the model is defined, up to proportionality, as the joint probability of observing the data given the parameters. Since X is fixed, the likelihood is given by p(y, σ ) N(X, σ I) ( ) n/ { πσ exp } σ (y X)T (y X). (4) 3 The posterior distribution from the N IG prior Inference will proceed from the posterior distribution p(, σ y) p(, σ )p(y, σ ), p(y) where p(y) p(, σ )p(y, σ )ddσ is the marginal distribution of the data. The key to deriving the joint posterior distribution is the following easily verified multivariate completion of squares or ellipsoidal rectification identity: u T Au α T u (u A α) T A(u A α) α T A α, (5)
3 where A is a symmetric positive definite (hence invertible) matrix. An application of this identity immediately reveals, σ b + { ( µ ) T V ( µ ) + (y X) T (y X) }] σ b + ] ( µ ) T V ( µ ), using which we can write the posterior as where p(, σ y) ( ) a+(n+p)/+ σ exp { σ b + ]} ( µ ) T V ( µ ), (6) µ (V + X T X) (V µ + X T y), V (V + X T X), a a + n/, b b + µt V µ + y T y µ T V µ ]. This posterior distribution is easily identified as a NIG(µ, V, a, b ) proving it to be a conjugate family for the linear regression model. Note that the marginal posterior distribution of σ is immediately seen to be an IG(a, b ) whose density is given by: ( ) p(σ y) b a a + ) Γ(a ) σ exp ( b σ. (7) The marginal posterior distribution of is obtained by integrating out σ from the NIG joint posterior as follows: p( y) ( p(, σ y)dσ σ This is a multivariate t density: MV St ν (µ, Σ ) NIG(µ, V, a, b )dσ ) a + exp { σ b + ]} ( µ ) T V ( µ ) dσ + ( µ ) T V ( µ ] (a ) +p/) b. Γ ( ν with ν a and Σ ( b a ) V. ( ) Γ ν +p ) π p/ ν Σ / + ( µ ) T Σ ( µ ) 3 ν ] ν +p, (8)
4 4 A useful expression for the N IG scale parameter Here we will prove: b b + ( y Xµ ) T ( I + XV X T ) (y Xµ ) (9) On account of the expression for b derived in the preceding section, it suffices to prove that y T y + µ T V µ µ V µ ( y Xµ ) T ( I + XV X T ) (y Xµ ) Substituting µ V (V µ + X T y) in the left hand side above we obtain: y T y + µ T V µ µ V µ y T y + µ T V µ (V µ + X T y)v (V µ + X T y) y T (I XV XT )y y T XV V µ + µ T (V V V V )µ. Further development of the proof will employ two tricky identities. The first is the well-known Sherman-Woodbury-Morrison identity in matrix algebra: (0) (A + BDC) A A B ( D + CA B ) CA, () where A and D are square matrices that are invertible and B and C are rectangular (square if A and D have the same dimensions) matrices such that the multiplications are well-defined. This identity is easily verified by multiplying the right hand side with A + BDC and simplifying to reduce it to the identity matrix. Applying () twice, once with A V and D (X T X) to get the second equality and then with A (X T X) and D V to get the third equality, we have V V V V V V (V V + (X T X) ] + XX T ) V X T X X T X(X T X + V ) X T X X T (I n XV X T )X. () The next identity notes that since V (V + X T X) I p, we have V V I p V X T X, so that XV V X XV X T X (I n XV X T )X. (3) 4
5 Substituting () and (3) in (0) we obtain y T (I n XV X T )y y T (I n XV X T )µ + µ T (I n XV X T )µ (y Xµ ) T (I n XV X T )(y Xµ ) (y Xµ ) T (I n + XV X T ) (y Xµ ), (4) where the last step is again a consequence of (): (I n + XV X T ) I n X(V + X T X) X T I n XV X T. 5 Marginal distributions the hard way To obtain the marginal distribution of y, we first compute the distribution p(y σ ) by integrating out and subsequently integrate out σ to obtain p(y). To be precise, we use the expression for b derived in the preceding section, proceeding as below: p(y σ ) p(y, σ )p( σ )d exp (πσ ) n+p V / N(X, σ I n ) N(µ, σ V )d { σ (y X) T (y X) + ( µ ) T V )} ] ( µ d (πσ ) n+p V / exp { (y Xµ σ ) T (I + XV X T ) (y Xµ ) + ( µ ) T V ( µ ) } ] d { exp } (πσ ) n+p V / σ (y Xµ ) T (I + XV X T ) (y Xµ ) exp { ( µ σ ) T V ( µ ) } ] d ( V ) / { exp } (πσ ) n V σ (y Xµ ) T (I + XV X T ) (y Xµ ) { exp } (πσ ) n I + XV X T / σ (y Xµ ) T (I + XV X T ) (y Xµ ) N(Xµ, σ (I + XV X T )). (5) Here we have applied the matrix identity A + BDC A D D + CA B (6) 5
6 to obtain I n + XV X T V V + X T X ( ) V V. Now, the marginal distribution of p(y) is obtained by integrating a N IG density as follows: p(y) p(y σ )p(σ )dσ N(Xµ, σ (I + XV X T ))IG(a, b)dσ NIG(Xµ, (I + XV X T ), a, b)dσ MV St a (Xµ, b ) a (I + XV XT ). (7) Rewriting our result slightly differently reveals another useful property of the N IG density: p(y) p(y, σ )p(, σ )ddσ N(X, σ I n ) NIG(µ, V, a, b)ddσ MV St a (Xµ, b ) a (I + XV XT ). (8) Of course, the computation of p(y) could also be carried out in terms of the NIG distribution parameters more directly as p(y) p(y, σ )p(, σ )ddσ N(X, σ I n ) NIG(µ, V, a, b)ddσ b a ( ) a +p/+ (π) p/ V / Γ(a) σ exp { σ b + ]} ( µ ) T V ( µ ) b a Γ(a)(π) (n+p)/ V Γ(a )(π) p/ V (b ) a ba Γ ( a + n ) V (π) n/ Γ(a) V b + { µ T V µ + y T y µ V µ }] (a+n/). (9) 6 Marginal distribution: the easy way An alternative and much easier way to derive p(y σ ), avoiding any integration at all, is to note that we can write the above model as: y X + ɛ, where ɛ N(0, σ I); µ + ɛ, where ɛ N(0, σ V ), where ɛ and ɛ are independent of each other. It then follows that y Xµ + Xɛ + ɛ N(Xµ, σ (I + XV X T )). 6
7 This gives p(y σ ). Next we integrate out σ to obtain p(y) as in the preceding section to obtain In fact, the entire distribution theory for the Bayesian regression with NIG priors could proceed by completely avoiding any integration. To be precise, we obtain this marginal distribution first and derive the posterior distribution: p(, σ y) p(, σ ) p(y, σ ) p(y) NIG(µ, V, a, b) N(X, σ I) MV St a (Xµ, b a (I + XV X T )), which indeed reduces (after some algebraic manipulation) to the NIG(µ, V, a, b ) density. 7 Bayesian Predictions Next consider Bayesian prediction in the context of the linear regression model. Suppose we now want to apply our regression analysis to a new set of data, where we have observed a new m p matrix of regressors X, and we wish to predict the corresponding outcome ỹ. Observe that if and σ were known, then the probability law for the predicted outcomes would be described as ỹ N( X, σ I m ) and would be independent of y. However, these parameters are not known; instead they are summarized through their posterior samples. Therefore, all predictions for the data must follow from the posterior predictive distribution: p(ỹ y) p(ỹ, σ )p(, σ y)ddσ N( X, σ I m ) NIG(µ, V, a, b )ddσ ( ) MV St a Xµ, b a (I + XV XT ), (0) where the last step follows from (8). There are two sources of uncertainty in the posterior predictive distribution: () the fundamental source of variability in the model due to σ, unaccounted for by X, and () the posterior uncertainty in and σ as a result of their estimation from a finite sample y. As the sample size n the variance due to posterior uncertainty disappears, but the predictive uncertainty remains. 7
8 8 Posterior and posterior predictive sampling Sampling from the NIG posterior distribution is straightforward: for each l,..., L, we sample { } L σ (l) IG(a + n/, b ) and (l) MV N(µ, σ (l) V ). The resulting (l), σ (l) provide l samples from the joint distribution p(, σ y) while { (l) } L l and {σ(l) } L l provide samples from the marginal posterior distributions p( y) and p(σ y) respectively. Predictions are carried out by sampling from the posterior predictive density (0). Sampling from this is easy for each posterior sample ( (l), σ (l) ), we draw ỹ (l) N( X (l), σ (l) I m ). The resulting {ỹ (l) } L l are samples from the desired posterior predictive distribution in (0); the mean and variance of this sample provide estimates of the predictive mean and variance respectively. 9 The posterior distribution from improper priors Taking V 0 (i.e. the null matrix) and a p/ and b 0 leads to the improper prior p(, σ ) /σ. The posterior distribution is NIG (µ, V, a, b ) with µ ˆ (X T X) X T y, V (X T X), a n p, b (n p)s where s n p (y X ˆ) T (y X ˆ) n p yt (I P X )y, where P X X(X T X) X T. Here ˆ is the classical least squares estimates (also the maximum likelihood estimate) of, s is the classical unbiased estimate of σ and P X is the projection matrix onto the column space of X. Plugging in the above values implied by the improper priors into the more general NIG(µ, V, a, b ) ( ) density, we find the marginal posterior distribution of σ is an IG n p, (n p)s (equivalently the posterior distribution of (n p)s /σ is a χ n p distribution) and the marginal posterior distribution of is a MV St n p (ˆ, s X T X) with density: MV St n p (µ, s X T Γ ( ) n X) Γ ( n p) + ( ˆ) T X T X( ˆ) ] n π p/ (n p)s (X T X) / (n p)s. Predictions with non-informative priors again follow by sampling from the posterior predictive distribution as earlier, but some additional insight is gained by considering analytical expressions 8
9 for the expectation and variance of the posterior predictive distribution. Again, plugging in the parameter values implied by the improper priors into (0), we obtain the posterior predictive density ( as a MV St n p X ˆ, s (I + X(X ) T X) XT ). Note that E(ỹ σ, y) EE(ỹ, σ, y) σ, y] E X σ, y] X ˆ X(X T X) X T y, where the inner expectation averages over p(ỹ, σ ) and the outer expectation averages with respect to p( σ, y). Note that given σ, the future observations have a mean which does not depend on σ. In analogous fashion, var(ỹ σ, y) Evar(ỹ, σ, y) σ, y] + vare(ỹ, σ, y) σ, y] Eσ I m ] + var X σ, y] (I m + X(X T X) XT )σ. Thus, conditional on σ, the posterior predictive variance has two components: σ I m, representing sampling variation, and X(X T X) XT σ, due to uncertainty about. 9
Non-informative Priors Multiparameter Models
Non-informative Priors Multiparameter Models Statistics 220 Spring 2005 Copyright c 2005 by Mark E. Irwin Prior Types Informative vs Non-informative There has been a desire for a prior distributions that
More informationSTART HERE: Instructions. 1 Exponential Family [Zhou, Manzil]
START HERE: Instructions Thanks a lot to John A.W.B. Constanzo and Shi Zong for providing and allowing to use the latex source files for quick preparation of the HW solution. The homework was due at 9:00am
More informationPosterior Inference. , where should we start? Consider the following computational procedure: 1. draw samples. 2. convert. 3. compute properties
Posterior Inference Example. Consider a binomial model where we have a posterior distribution for the probability term, θ. Suppose we want to make inferences about the log-odds γ = log ( θ 1 θ), where
More informationBayesian Normal Stuff
Bayesian Normal Stuff - Set-up of the basic model of a normally distributed random variable with unknown mean and variance (a two-parameter model). - Discuss philosophies of prior selection - Implementation
More informationSTAT 425: Introduction to Bayesian Analysis
STAT 45: Introduction to Bayesian Analysis Marina Vannucci Rice University, USA Fall 018 Marina Vannucci (Rice University, USA) Bayesian Analysis (Part 1) Fall 018 1 / 37 Lectures 9-11: Multi-parameter
More informationConjugate Models. Patrick Lam
Conjugate Models Patrick Lam Outline Conjugate Models What is Conjugacy? The Beta-Binomial Model The Normal Model Normal Model with Unknown Mean, Known Variance Normal Model with Known Mean, Unknown Variance
More informationBlack-Litterman Model
Institute of Financial and Actuarial Mathematics at Vienna University of Technology Seminar paper Black-Litterman Model by: Tetyana Polovenko Supervisor: Associate Prof. Dipl.-Ing. Dr.techn. Stefan Gerhold
More informationObjective Bayesian Analysis for Heteroscedastic Regression
Analysis for Heteroscedastic Regression & Esther Salazar Universidade Federal do Rio de Janeiro Colóquio Inter-institucional: Modelos Estocásticos e Aplicações 2009 Collaborators: Marco Ferreira and Thais
More informationConjugate Bayesian Models for Massive Spatial Data
Conjugate Bayesian Models for Massive Spatial Data Abhi Datta 1, Sudipto Banerjee 2 and Andrew O. Finley 3 July 31, 2017 1 Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins
More informationChapter 8: Sampling distributions of estimators Sections
Chapter 8 continued Chapter 8: Sampling distributions of estimators Sections 8.1 Sampling distribution of a statistic 8.2 The Chi-square distributions 8.3 Joint Distribution of the sample mean and sample
More informationExtended Model: Posterior Distributions
APPENDIX A Extended Model: Posterior Distributions A. Homoskedastic errors Consider the basic contingent claim model b extended by the vector of observables x : log C i = β log b σ, x i + β x i + i, i
More informationChapter 5 Univariate time-series analysis. () Chapter 5 Univariate time-series analysis 1 / 29
Chapter 5 Univariate time-series analysis () Chapter 5 Univariate time-series analysis 1 / 29 Time-Series Time-series is a sequence fx 1, x 2,..., x T g or fx t g, t = 1,..., T, where t is an index denoting
More informationModule 2: Monte Carlo Methods
Module 2: Monte Carlo Methods Prof. Mike Giles mike.giles@maths.ox.ac.uk Oxford University Mathematical Institute MC Lecture 2 p. 1 Greeks In Monte Carlo applications we don t just want to know the expected
More informationPractice Exercises for Midterm Exam ST Statistical Theory - II The ACTUAL exam will consists of less number of problems.
Practice Exercises for Midterm Exam ST 522 - Statistical Theory - II The ACTUAL exam will consists of less number of problems. 1. Suppose X i F ( ) for i = 1,..., n, where F ( ) is a strictly increasing
More informationChapter 4: Asymptotic Properties of MLE (Part 3)
Chapter 4: Asymptotic Properties of MLE (Part 3) Daniel O. Scharfstein 09/30/13 1 / 1 Breakdown of Assumptions Non-Existence of the MLE Multiple Solutions to Maximization Problem Multiple Solutions to
More informationChapter 7: Estimation Sections
1 / 31 : Estimation Sections 7.1 Statistical Inference Bayesian Methods: 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions 7.4 Bayes Estimators Frequentist Methods: 7.5 Maximum Likelihood
More informationSimulation Wrap-up, Statistics COS 323
Simulation Wrap-up, Statistics COS 323 Today Simulation Re-cap Statistics Variance and confidence intervals for simulations Simulation wrap-up FYI: No class or office hours Thursday Simulation wrap-up
More informationSTA258 Analysis of Variance
STA258 Analysis of Variance Al Nosedal. University of Toronto. Winter 2017 The Data Matrix The following table shows last year s sales data for a small business. The sample is put into a matrix format
More informationarxiv: v1 [math.st] 18 Sep 2018
Gram Charlier and Edgeworth expansion for sample variance arxiv:809.06668v [math.st] 8 Sep 08 Eric Benhamou,* A.I. SQUARE CONNECT, 35 Boulevard d Inkermann 900 Neuilly sur Seine, France and LAMSADE, Universit
More informationFinal Exam Suggested Solutions
University of Washington Fall 003 Department of Economics Eric Zivot Economics 483 Final Exam Suggested Solutions This is a closed book and closed note exam. However, you are allowed one page of handwritten
More informationBayesian Hierarchical/ Multilevel and Latent-Variable (Random-Effects) Modeling
Bayesian Hierarchical/ Multilevel and Latent-Variable (Random-Effects) Modeling 1: Formulation of Bayesian models and fitting them with MCMC in WinBUGS David Draper Department of Applied Mathematics and
More informationA potentially useful approach to model nonlinearities in time series is to assume different behavior (structural break) in different subsamples
1.3 Regime switching models A potentially useful approach to model nonlinearities in time series is to assume different behavior (structural break) in different subsamples (or regimes). If the dates, the
More informationχ 2 distributions and confidence intervals for population variance
χ 2 distributions and confidence intervals for population variance Let Z be a standard Normal random variable, i.e., Z N(0, 1). Define Y = Z 2. Y is a non-negative random variable. Its distribution is
More informationINSTITUTE AND FACULTY OF ACTUARIES. Curriculum 2019 SPECIMEN EXAMINATION
INSTITUTE AND FACULTY OF ACTUARIES Curriculum 2019 SPECIMEN EXAMINATION Subject CS1A Actuarial Statistics Time allowed: Three hours and fifteen minutes INSTRUCTIONS TO THE CANDIDATE 1. Enter all the candidate
More informationChapter 3 Common Families of Distributions. Definition 3.4.1: A family of pmfs or pdfs is called exponential family if it can be expressed as
Lecture 0 on BST 63: Statistical Theory I Kui Zhang, 09/9/008 Review for the previous lecture Definition: Several continuous distributions, including uniform, gamma, normal, Beta, Cauchy, double exponential
More informationConjugate priors: Beta and normal Class 15, Jeremy Orloff and Jonathan Bloom
1 Learning Goals Conjugate s: Beta and normal Class 15, 18.05 Jeremy Orloff and Jonathan Bloom 1. Understand the benefits of conjugate s.. Be able to update a beta given a Bernoulli, binomial, or geometric
More informationChapter 7: Estimation Sections
1 / 40 Chapter 7: Estimation Sections 7.1 Statistical Inference Bayesian Methods: Chapter 7 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions 7.4 Bayes Estimators Frequentist Methods:
More informationTwo hours. To be supplied by the Examinations Office: Mathematical Formula Tables and Statistical Tables THE UNIVERSITY OF MANCHESTER
Two hours MATH20802 To be supplied by the Examinations Office: Mathematical Formula Tables and Statistical Tables THE UNIVERSITY OF MANCHESTER STATISTICAL METHODS Answer any FOUR of the SIX questions.
More informationRESEARCH ARTICLE. The Penalized Biclustering Model And Related Algorithms Supplemental Online Material
Journal of Applied Statistics Vol. 00, No. 00, Month 00x, 8 RESEARCH ARTICLE The Penalized Biclustering Model And Related Algorithms Supplemental Online Material Thierry Cheouo and Alejandro Murua Département
More informationLecture 3: Factor models in modern portfolio choice
Lecture 3: Factor models in modern portfolio choice Prof. Massimo Guidolin Portfolio Management Spring 2016 Overview The inputs of portfolio problems Using the single index model Multi-index models Portfolio
More informationCase Study: Heavy-Tailed Distribution and Reinsurance Rate-making
Case Study: Heavy-Tailed Distribution and Reinsurance Rate-making May 30, 2016 The purpose of this case study is to give a brief introduction to a heavy-tailed distribution and its distinct behaviors in
More informationIEOR E4602: Quantitative Risk Management
IEOR E4602: Quantitative Risk Management Basic Concepts and Techniques of Risk Management Martin Haugh Department of Industrial Engineering and Operations Research Columbia University Email: martin.b.haugh@gmail.com
More informationدرس هفتم یادگیري ماشین. (Machine Learning) دانشگاه فردوسی مشهد دانشکده مهندسی رضا منصفی
یادگیري ماشین توزیع هاي نمونه و تخمین نقطه اي پارامترها Sampling Distributions and Point Estimation of Parameter (Machine Learning) دانشگاه فردوسی مشهد دانشکده مهندسی رضا منصفی درس هفتم 1 Outline Introduction
More informationChapter 7: Estimation Sections
Chapter 7: Estimation Sections 7.1 Statistical Inference Bayesian Methods: 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions Frequentist Methods: 7.5 Maximum Likelihood Estimators
More informationStochastic Volatility (SV) Models
1 Motivations Stochastic Volatility (SV) Models Jun Yu Some stylised facts about financial asset return distributions: 1. Distribution is leptokurtic 2. Volatility clustering 3. Volatility responds to
More informationPhD Qualifier Examination
PhD Qualifier Examination Department of Agricultural Economics May 29, 2015 Instructions This exam consists of six questions. You must answer all questions. If you need an assumption to complete a question,
More informationELEMENTS OF MATRIX MATHEMATICS
QRMC07 9/7/0 4:45 PM Page 5 CHAPTER SEVEN ELEMENTS OF MATRIX MATHEMATICS 7. AN INTRODUCTION TO MATRICES Investors frequently encounter situations involving numerous potential outcomes, many discrete periods
More informationThe mean-variance portfolio choice framework and its generalizations
The mean-variance portfolio choice framework and its generalizations Prof. Massimo Guidolin 20135 Theory of Finance, Part I (Sept. October) Fall 2014 Outline and objectives The backward, three-step solution
More informationEstimation after Model Selection
Estimation after Model Selection Vanja M. Dukić Department of Health Studies University of Chicago E-Mail: vanja@uchicago.edu Edsel A. Peña* Department of Statistics University of South Carolina E-Mail:
More informationStatistics 431 Spring 2007 P. Shaman. Preliminaries
Statistics 4 Spring 007 P. Shaman The Binomial Distribution Preliminaries A binomial experiment is defined by the following conditions: A sequence of n trials is conducted, with each trial having two possible
More informationIEOR E4703: Monte-Carlo Simulation
IEOR E4703: Monte-Carlo Simulation Simulation Efficiency and an Introduction to Variance Reduction Methods Martin Haugh Department of Industrial Engineering and Operations Research Columbia University
More informationLinear Regression with One Regressor
Linear Regression with One Regressor Michael Ash Lecture 9 Linear Regression with One Regressor Review of Last Time 1. The Linear Regression Model The relationship between independent X and dependent Y
More informationExam STAM Practice Exam #1
!!!! Exam STAM Practice Exam #1 These practice exams should be used during the month prior to your exam. This practice exam contains 20 questions, of equal value, corresponding to about a 2 hour exam.
More informationLecture Notes 6. Assume F belongs to a family of distributions, (e.g. F is Normal), indexed by some parameter θ.
Sufficient Statistics Lecture Notes 6 Sufficiency Data reduction in terms of a particular statistic can be thought of as a partition of the sample space X. Definition T is sufficient for θ if the conditional
More informationConfidence Intervals Introduction
Confidence Intervals Introduction A point estimate provides no information about the precision and reliability of estimation. For example, the sample mean X is a point estimate of the population mean μ
More informationThe Delta Method. j =.
The Delta Method Often one has one or more MLEs ( 3 and their estimated, conditional sampling variancecovariance matrix. However, there is interest in some function of these estimates. The question is,
More informationCausal Analysis of Economic Growth and Military Expenditure
Causal Analysis of Economic Growth and Military Expenditure JAKUB ODEHNAL University of Defence Department of Economy Kounicova 65, 662 10 Brno CZECH REPUBLIC jakub.odehnal@unob.cz JIŘÍ NEUBAUER University
More informationCSC 411: Lecture 08: Generative Models for Classification
CSC 411: Lecture 08: Generative Models for Classification Richard Zemel, Raquel Urtasun and Sanja Fidler University of Toronto Zemel, Urtasun, Fidler (UofT) CSC 411: 08-Generative Models 1 / 23 Today Classification
More informationCentral limit theorems
Chapter 6 Central limit theorems 6.1 Overview Recall that a random variable Z is said to have a standard normal distribution, denoted by N(0, 1), if it has a continuous distribution with density φ(z) =
More information12. Conditional heteroscedastic models (ARCH) MA6622, Ernesto Mordecki, CityU, HK, 2006.
12. Conditional heteroscedastic models (ARCH) MA6622, Ernesto Mordecki, CityU, HK, 2006. References for this Lecture: Robert F. Engle. Autoregressive Conditional Heteroscedasticity with Estimates of Variance
More informationChapter 8: CAPM. 1. Single Index Model. 2. Adding a Riskless Asset. 3. The Capital Market Line 4. CAPM. 5. The One-Fund Theorem
Chapter 8: CAPM 1. Single Index Model 2. Adding a Riskless Asset 3. The Capital Market Line 4. CAPM 5. The One-Fund Theorem 6. The Characteristic Line 7. The Pricing Model Single Index Model 1 1. Covariance
More information12 The Bootstrap and why it works
12 he Bootstrap and why it works For a review of many applications of bootstrap see Efron and ibshirani (1994). For the theory behind the bootstrap see the books by Hall (1992), van der Waart (2000), Lahiri
More information(5) Multi-parameter models - Summarizing the posterior
(5) Multi-parameter models - Summarizing the posterior Spring, 2017 Models with more than one parameter Thus far we have studied single-parameter models, but most analyses have several parameters For example,
More informationQuantitative Risk Management
Quantitative Risk Management Asset Allocation and Risk Management Martin B. Haugh Department of Industrial Engineering and Operations Research Columbia University Outline Review of Mean-Variance Analysis
More informationGPD-POT and GEV block maxima
Chapter 3 GPD-POT and GEV block maxima This chapter is devoted to the relation between POT models and Block Maxima (BM). We only consider the classical frameworks where POT excesses are assumed to be GPD,
More informationMTH6154 Financial Mathematics I Stochastic Interest Rates
MTH6154 Financial Mathematics I Stochastic Interest Rates Contents 4 Stochastic Interest Rates 45 4.1 Fixed Interest Rate Model............................ 45 4.2 Varying Interest Rate Model...........................
More informationlast problem outlines how the Black Scholes PDE (and its derivation) may be modified to account for the payment of stock dividends.
224 10 Arbitrage and SDEs last problem outlines how the Black Scholes PDE (and its derivation) may be modified to account for the payment of stock dividends. 10.1 (Calculation of Delta First and Finest
More informationExtend the ideas of Kan and Zhou paper on Optimal Portfolio Construction under parameter uncertainty
Extend the ideas of Kan and Zhou paper on Optimal Portfolio Construction under parameter uncertainty George Photiou Lincoln College University of Oxford A dissertation submitted in partial fulfilment for
More informationMuch of what appears here comes from ideas presented in the book:
Chapter 11 Robust statistical methods Much of what appears here comes from ideas presented in the book: Huber, Peter J. (1981), Robust statistics, John Wiley & Sons (New York; Chichester). There are many
More informationStatistics for Business and Economics
Statistics for Business and Economics Chapter 7 Estimation: Single Population Copyright 010 Pearson Education, Inc. Publishing as Prentice Hall Ch. 7-1 Confidence Intervals Contents of this chapter: Confidence
More information1 Bayesian Bias Correction Model
1 Bayesian Bias Correction Model Assuming that n iid samples {X 1,...,X n }, were collected from a normal population with mean µ and variance σ 2. The model likelihood has the form, P( X µ, σ 2, T n >
More informationTHE UNIVERSITY OF TEXAS AT AUSTIN Department of Information, Risk, and Operations Management
THE UNIVERSITY OF TEXAS AT AUSTIN Department of Information, Risk, and Operations Management BA 386T Tom Shively PROBABILITY CONCEPTS AND NORMAL DISTRIBUTIONS The fundamental idea underlying any statistical
More informationcontinuous rv Note for a legitimate pdf, we have f (x) 0 and f (x)dx = 1. For a continuous rv, P(X = c) = c f (x)dx = 0, hence
continuous rv Let X be a continuous rv. Then a probability distribution or probability density function (pdf) of X is a function f(x) such that for any two numbers a and b with a b, P(a X b) = b a f (x)dx.
More informationArbitrages and pricing of stock options
Arbitrages and pricing of stock options Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ November
More informationDecision theoretic estimation of the ratio of variances in a bivariate normal distribution 1
Decision theoretic estimation of the ratio of variances in a bivariate normal distribution 1 George Iliopoulos Department of Mathematics University of Patras 26500 Rio, Patras, Greece Abstract In this
More informationLogit Models for Binary Data
Chapter 3 Logit Models for Binary Data We now turn our attention to regression models for dichotomous data, including logistic regression and probit analysis These models are appropriate when the response
More informationRandom Variables Handout. Xavier Vilà
Random Variables Handout Xavier Vilà Course 2004-2005 1 Discrete Random Variables. 1.1 Introduction 1.1.1 Definition of Random Variable A random variable X is a function that maps each possible outcome
More informationSTAT 830 Convergence in Distribution
STAT 830 Convergence in Distribution Richard Lockhart Simon Fraser University STAT 830 Fall 2013 Richard Lockhart (Simon Fraser University) STAT 830 Convergence in Distribution STAT 830 Fall 2013 1 / 31
More informationRandom Variables and Probability Distributions
Chapter 3 Random Variables and Probability Distributions Chapter Three Random Variables and Probability Distributions 3. Introduction An event is defined as the possible outcome of an experiment. In engineering
More informationPoint Estimators. STATISTICS Lecture no. 10. Department of Econometrics FEM UO Brno office 69a, tel
STATISTICS Lecture no. 10 Department of Econometrics FEM UO Brno office 69a, tel. 973 442029 email:jiri.neubauer@unob.cz 8. 12. 2009 Introduction Suppose that we manufacture lightbulbs and we want to state
More informationDynamic Portfolio Execution Detailed Proofs
Dynamic Portfolio Execution Detailed Proofs Gerry Tsoukalas, Jiang Wang, Kay Giesecke March 16, 2014 1 Proofs Lemma 1 (Temporary Price Impact) A buy order of size x being executed against i s ask-side
More informationMATH 3200 Exam 3 Dr. Syring
. Suppose n eligible voters are polled (randomly sampled) from a population of size N. The poll asks voters whether they support or do not support increasing local taxes to fund public parks. Let M be
More informationBooth School of Business, University of Chicago Business 41202, Spring Quarter 2012, Mr. Ruey S. Tsay. Solutions to Midterm
Booth School of Business, University of Chicago Business 41202, Spring Quarter 2012, Mr. Ruey S. Tsay Solutions to Midterm Problem A: (34 pts) Answer briefly the following questions. Each question has
More informationECSE B Assignment 5 Solutions Fall (a) Using whichever of the Markov or the Chebyshev inequalities is applicable, estimate
ECSE 304-305B Assignment 5 Solutions Fall 2008 Question 5.1 A positive scalar random variable X with a density is such that EX = µ
More informationInformation, Interest Rates and Geometry
Information, Interest Rates and Geometry Dorje C. Brody Department of Mathematics, Imperial College London, London SW7 2AZ www.imperial.ac.uk/people/d.brody (Based on work in collaboration with Lane Hughston
More informationProblems from 9th edition of Probability and Statistical Inference by Hogg, Tanis and Zimmerman:
Math 224 Fall 207 Homework 5 Drew Armstrong Problems from 9th edition of Probability and Statistical Inference by Hogg, Tanis and Zimmerman: Section 3., Exercises 3, 0. Section 3.3, Exercises 2, 3, 0,.
More informationCharacterization of the Optimum
ECO 317 Economics of Uncertainty Fall Term 2009 Notes for lectures 5. Portfolio Allocation with One Riskless, One Risky Asset Characterization of the Optimum Consider a risk-averse, expected-utility-maximizing
More informationA Stochastic Reserving Today (Beyond Bootstrap)
A Stochastic Reserving Today (Beyond Bootstrap) Presented by Roger M. Hayne, PhD., FCAS, MAAA Casualty Loss Reserve Seminar 6-7 September 2012 Denver, CO CAS Antitrust Notice The Casualty Actuarial Society
More informationUnit 5: Sampling Distributions of Statistics
Unit 5: Sampling Distributions of Statistics Statistics 571: Statistical Methods Ramón V. León 6/12/2004 Unit 5 - Stat 571 - Ramon V. Leon 1 Definitions and Key Concepts A sample statistic used to estimate
More informationA Bayesian Control Chart for the Coecient of Variation in the Case of Pooled Samples
A Bayesian Control Chart for the Coecient of Variation in the Case of Pooled Samples R van Zyl a,, AJ van der Merwe b a PAREXEL International, Bloemfontein, South Africa b University of the Free State,
More informationUnit 5: Sampling Distributions of Statistics
Unit 5: Sampling Distributions of Statistics Statistics 571: Statistical Methods Ramón V. León 6/12/2004 Unit 5 - Stat 571 - Ramon V. Leon 1 Definitions and Key Concepts A sample statistic used to estimate
More informationStatistical and Computational Inverse Problems with Applications Part 5B: Electrical impedance tomography
Statistical and Computational Inverse Problems with Applications Part 5B: Electrical impedance tomography Aku Seppänen Inverse Problems Group Department of Applied Physics University of Eastern Finland
More informationStrategies for Improving the Efficiency of Monte-Carlo Methods
Strategies for Improving the Efficiency of Monte-Carlo Methods Paul J. Atzberger General comments or corrections should be sent to: paulatz@cims.nyu.edu Introduction The Monte-Carlo method is a useful
More informationStatistical Inference and Methods
Department of Mathematics Imperial College London d.stephens@imperial.ac.uk http://stats.ma.ic.ac.uk/ das01/ 14th February 2006 Part VII Session 7: Volatility Modelling Session 7: Volatility Modelling
More information1. You are given the following information about a stationary AR(2) model:
Fall 2003 Society of Actuaries **BEGINNING OF EXAMINATION** 1. You are given the following information about a stationary AR(2) model: (i) ρ 1 = 05. (ii) ρ 2 = 01. Determine φ 2. (A) 0.2 (B) 0.1 (C) 0.4
More informationPoint Estimation. Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage
6 Point Estimation Stat 4570/5570 Material from Devore s book (Ed 8), and Cengage Point Estimation Statistical inference: directed toward conclusions about one or more parameters. We will use the generic
More informationSTAT 509: Statistics for Engineers Dr. Dewei Wang. Copyright 2014 John Wiley & Sons, Inc. All rights reserved.
STAT 509: Statistics for Engineers Dr. Dewei Wang Applied Statistics and Probability for Engineers Sixth Edition Douglas C. Montgomery George C. Runger 7 Point CHAPTER OUTLINE 7-1 Point Estimation 7-2
More information2 Control variates. λe λti λe e λt i where R(t) = t Y 1 Y N(t) is the time from the last event to t. L t = e λr(t) e e λt(t) Exercises
96 ChapterVI. Variance Reduction Methods stochastic volatility ISExSoren5.9 Example.5 (compound poisson processes) Let X(t) = Y + + Y N(t) where {N(t)},Y, Y,... are independent, {N(t)} is Poisson(λ) with
More informationPractice Exam 1. Loss Amount Number of Losses
Practice Exam 1 1. You are given the following data on loss sizes: An ogive is used as a model for loss sizes. Determine the fitted median. Loss Amount Number of Losses 0 1000 5 1000 5000 4 5000 10000
More informationCourse information FN3142 Quantitative finance
Course information 015 16 FN314 Quantitative finance This course is aimed at students interested in obtaining a thorough grounding in market finance and related empirical methods. Prerequisite If taken
More informationSTA 114: Statistics. Notes 10. Conjugate Priors
STA 114: Statistics Notes 10. Conjugate Priors Conjugate family Once we get a /pmf ξ(θ x) by combining a model X f(x θ) with a /pmf ξ(θ) on θ Θ, a report can be made by summarizing the. It helps to have
More informationIntroduction Dickey-Fuller Test Option Pricing Bootstrapping. Simulation Methods. Chapter 13 of Chris Brook s Book.
Simulation Methods Chapter 13 of Chris Brook s Book Christopher Ting http://www.mysmu.edu/faculty/christophert/ Christopher Ting : christopherting@smu.edu.sg : 6828 0364 : LKCSB 5036 April 26, 2017 Christopher
More informationEconometric Methods for Valuation Analysis
Econometric Methods for Valuation Analysis Margarita Genius Dept of Economics M. Genius (Univ. of Crete) Econometric Methods for Valuation Analysis Cagliari, 2017 1 / 25 Outline We will consider econometric
More informationChapter 5 Univariate time-series analysis. () Chapter 5 Univariate time-series analysis 1 / 59
Chapter 5 Univariate time-series analysis () Chapter 5 Univariate time-series analysis 1 / 59 Time-Series Time-series is a sequence fx 1, x 2,..., x T g or fx t g, t = 1,..., T, where t is an index denoting
More informationThe Analytics of Information and Uncertainty Answers to Exercises and Excursions
The Analytics of Information and Uncertainty Answers to Exercises and Excursions Chapter 6: Information and Markets 6.1 The inter-related equilibria of prior and posterior markets Solution 6.1.1. The condition
More informationQualifying Exam Solutions: Theoretical Statistics
Qualifying Exam Solutions: Theoretical Statistics. (a) For the first sampling plan, the expectation of any statistic W (X, X,..., X n ) is a polynomial of θ of degree less than n +. Hence τ(θ) cannot have
More informationWeight Smoothing with Laplace Prior and Its Application in GLM Model
Weight Smoothing with Laplace Prior and Its Application in GLM Model Xi Xia 1 Michael Elliott 1,2 1 Department of Biostatistics, 2 Survey Methodology Program, University of Michigan National Cancer Institute
More informationModelling Returns: the CER and the CAPM
Modelling Returns: the CER and the CAPM Carlo Favero Favero () Modelling Returns: the CER and the CAPM 1 / 20 Econometric Modelling of Financial Returns Financial data are mostly observational data: they
More informationReview of the Topics for Midterm I
Review of the Topics for Midterm I STA 100 Lecture 9 I. Introduction The objective of statistics is to make inferences about a population based on information contained in a sample. A population is the
More informationMissing Data. EM Algorithm and Multiple Imputation. Aaron Molstad, Dootika Vats, Li Zhong. University of Minnesota School of Statistics
Missing Data EM Algorithm and Multiple Imputation Aaron Molstad, Dootika Vats, Li Zhong University of Minnesota School of Statistics December 4, 2013 Overview 1 EM Algorithm 2 Multiple Imputation Incomplete
More information