Hidden Markov Models. Selecting model parameters or training
|
|
|
- Sherman Robertson
- 6 years ago
- Views:
Transcription
1 idden Markov Models Selecting model parameters or training
2 idden Markov Models Motivation: The n'th observation in a chain of observations is influenced by a corresponding latent variable... Observations Markov Model idden Markov Model L L Latent values If the latent states are discrete and form a Markov chain, then it is a hidden Markov model (MM)
3 idden Markov Models Motivation: The n'th observation in a chain of observations is The jointbydistribution of observables X and influenced a corresponding latent variable... latent values Z: Observations Markov Model idden Markov Model L L Latent values If the latent states are discrete and form a Markov chain, then it is a hidden Markov model (MM)
4 idden Markov Models Emission probabilities Ф Motivation: The n'th observation Transition probabilities A and π in a chain of observations is The jointbydistribution of observables X and influenced a corresponding latent variable... latent values Z: Observations Markov Model idden Markov Model L L Latent values If the latent states are discrete and form a Markov chain, then it is a hidden Markov model (MM)
5 MMs as a generative model A MM generates a sequence of observables by moving from latent state to latent state according to the transition probabilities and emitting an observable (from a discrete set of observables, i.e. a finite alphabet) from each latent state visited according to the emission probabilities of the state... Model M: A run follows a sequence of states: L L And emits a sequence of symbols: For a MM that generates finite strings (e.g. a MM with an endstate), the language L = {X p(x) > 0} is regular...
6 What we know Introduced hidden Markov models (MMs) The forward- and backward-algorithms for determining the likelihood p(x) of a sequence of observations, and predicting the next observation in a sequence of observations. The Viterbi-algorithm for finding the most likely underlying explanation (sequence of latent states) of a sequence of observation ow to implement them using log-space and scaling. Today Training, or how to select model parameters (transition and emission probabilities) to reflect either a set of corresponding (X,Z)'s, or just a set of X's...
7 Selecting the right parameters Assume that (several) sequences of observations X={x1,...,xn} and corresponding latent states Z={z1,...,zn} are given... L L ow should we set the model parameters, i.e. transition A, π, and emission probabilities Ф, to make the given (X,Z)'s most likely?
8 Selecting the right parameters Assume that (several) sequences of observations X={x1,...,xn} and corresponding latent states Z={z1,...,zn} are given... L L ow should we set the model parameters, i.e. transition A, π, and emission probabilities Ф, to make the given (X,Z)'s most likely? Intuition: The parameters should reflect what we have seen...
9 Selecting the right transition probs L L Ajk is the probability of a transition from state j to state k, and πk is the probability of starting in state k... ow many times is the transition from state j to state k taken ow many times is a transition from state j to any state taken
10 Selecting the right transition probs L L Ajk is the probability of a transition from state j to state k, and πk is the probability of starting in state k... ow many times is the transition from state j to state k taken ow many times is a transition from state j to any state taken
11 Selecting the right emission probs L L If we assume discrete observations, then Φik is the probability of emitting symbol i from state k... ow many times is symbol i emitted from state k ow many times is a symbol emitted from state k
12 Selecting the right parameters Assume that (several) sequences of observations X={x1,...,xn} and corresponding latent states Z={z1,...,zn} are given... We simply count how many times each outcome of the multinomial variables (a transition or emission) is observed...
13 Selecting the right parameters Assume that (several) sequences of observations X={x1,...,xn} and corresponding latent states Z={z1,...,zn} are given... We simply count how many times each outcome of the multinomial variables (a transition or emission) is observed... This yield a maximum likelihood estimate (MLE) θ* of p(x,z θ), which is what we mathematically want...
14 Selecting the right parameters Assume that (several) sequences of observations X={x1,...,xn} and corresponding latent states Z={z1,...,zn} are given... We simply count how many times each outcome of the multinomial variables (a transition or emission) is observed... This yield a maximum likelihood estimate (MLE) θ* of p(x,z θ), which is what we mathematically want... Any problems? What if e.g. the transition from state j to k is not observed, then probability Ajk is set to 0. Practical solution: Assume that every transition and emission is seen once (pseudocount)...
15 Selecting the right parameters Assume that (several) sequences of observations X={x1,...,xn} and corresponding latent states Z={z1,...,zn} are given... We simply count how many times each outcome of the multinomial variables (a transition or emission) is observed... This yield a maximum likelihood estimate (MLE) θ* of p(x,z θ), which is what we mathematically want... Any problems? What if e.g. the transition from state j to k is not observed, then probability Ajk is set to 0. Practical solution: Assume that every transition and emission is seen once (pseudocount)...
16 Selecting the right parameters Assume that (several) sequences of observations X={x1,...,xn} and corresponding latent states Z={z1,...,zn} are given... We simply count how many times each outcome of the multinomial variables (a transition or emission) is observed... This yield a maximum likelihood estimate (MLE) θ* of p(x,z θ), which is what we mathematically want... Any problems? What if e.g. the transition from state j to k is not observed, then probability Ajk is set to 0. Practical solution: Assume that every transition and emission is seen once (pseudocount)...
17 Example Without pseudocounts: A = 1/2 AL = 1/2 AL = 1/2 ALL = 1/2 π = 1 πl = 0 p(sun ) = 1 p(rain ) = 0 p(sun L) = 1/2 p(rain L) = 1/2 L L
18 Example L L Without pseudocounts: With pseudocounts: A = 1/2 AL = 1/2 AL = 1/2 ALL = 1/2 π = 1 πl = 0 A = 2/4 AL = 2/4 AL = 2/4 ALL = 2/4 π = 2/3 πl = 1/3 p(sun ) = 1 p(rain ) = 0 p(sun L) = 1/2 p(rain L) = 1/2 p(sun ) = 4/5 p(rain ) = 1/5 p(sun L) = 2/4 p(rain L) = 2/4
19 Selecting the right parameters What if only (several) sequences of observations X={x1,...,xn} is given, i.e the corresponding latent states Z={z1,...,zn} are unknown? L L ow should we set the model parameters, i.e. transitions A, π, and emission probabilities Ф, to make the given X's most likely?
20 Selecting the right parameters What if only (several) sequences of observations X={x1,...,xn} is given, i.e the corresponding latent states Z={z1,...,zn} are unknown? L L ow should we set the model parameters, i.e. transitions A, π, and emission probabilities Ф, to make the given X's most likely? Maximize w.r.t. θ...
21 Selecting the right parameters What if only (several) sequences of observations X={x1,...,xn} is given, i.e the corresponding latent states Z={z1,...,zn} are unknown? L L Direct maximization of the likelihood (or log-likelihood) is hard... ow should we set the model parameters, i.e. transitions A, π, and emission probabilities Ф, to make the given X's most likely? Maximize w.r.t. θ...
22 Viterbi training A more practical thing to do is Viterbi Training: 1.Decide on some initial parameter θ0 2.Find the most likely sequence of states Z* explaining X using the the Viterbi Algorithm and the current parameters θi 3.Update parameters to θi+1 by counting (with pseudo counts) according to (X,Z*). 4.Repeat 2-3 until P(X,Z* θi) is satisfactory.
23 Viterbi training A more practical thing to do is Viterbi Training: 1.Decide on some initial parameter θ0 2.Find the most likely sequence of states Z* explaining X using the the Viterbi Algorithm and the current parameters θi 3.Update parameters to θi+1 by counting (with pseudo counts) according to (X,Z*). 4.Repeat 2-3 until P(X,Z* θi) is satisfactory. Finds a (local) maximum of: Not a MLE (because right-hand side isn't a likelihood), but works ok
24 Expectation Maximization E-Step: Define the Q-function: i.e. the expectation of the log-likelihood of the complete data (i.e. observations X and underlying states Z) as a function of θ M-Step: Maximize Q(θ, θold) w.r.t. θ When iterated, the likelihood p(x θ) converges to a (local) maximum
25 Maximizing the likelihood Direct maximization of the likelihood (or log-likelihood) is hard... Assume that we have valid set of parameters θold, and that we want to estimate a set θ which yields a better likelihood. We can write:
26 Maximizing the likelihood Direct maximization of the likelihood (or log-likelihood) is hard... Assume that we have valid set of parameters θold, and that we want to estimate a set θ which yields a better likelihood. We can write: This sums to 1...
27 Maximizing the likelihood Direct maximization of the likelihood (or log-likelihood) is hard... Assume that we have valid set of parameters θold, and that we want to estimate a set θ which yields a better likelihood. We can write: The expectation of the log-likelihood of the complete data (i.e. observations X and underlying states Z) as a function of θ
28 Maximizing the likelihood Direct maximization of the likelihood (or log-likelihood) is hard... Assume that we have valid set of parameters θold, and that we want to estimate a set θ which yields a better likelihood. We can write:
29 Maximizing the likelihood Assume that we have valid set of parameters θold, and that we want to estimate a set θ which yields a better likelihood. We have: The increase of the log-likelihood can thus be written as:
30 Maximizing the likelihood Assume that we have valid set of parameters θold, and that we want to estimate a set θ which yields a better likelihood. We have: The increase of the log-likelihood can thus be written as: The relative entropy of p(z X,θold) relative to p(z X,θ), i.e. 0
31 Maximizing the likelihood Assume that we have valid set of parameters θold, and that we want to estimate a set θ which yields a better likelihood. We have: The increase of the log-likelihood can thus be written as: By maximizing the expectation Q(θ, θold) w.r.t. θ, we increase the likelihood, hence name expectation maximization...
32 EM for MMs E-Step: Define the Q-function: i.e. the expectation of the log-likelihood of the complete data (i.e. observations X and underlying states Z) as a function of θ M-Step: Maximize Q(θ, θold) w.r.t. θ For MMs Q has a closed form and maximization can be performed explicitly. Iterate until no or little increase in likelihood is observed, or some maximum number of iterations is reached... When iterated, the likelihood p(x θ) converges to a (local) maximum
33 EM for MMs Init: Pick suitable parameters (transition and emission probabilities). Observe that if a parameter is initialized to zero, it remains zero... E-Step: 1) Run the forward- and backward-algorithms with the current choice of parameters (to get the params of Q-func). Stop?: 2) Compute the likelihood p(x θ), if sufficient (or another stopping criteria is meet) then stop. M-Step: 3) Compute new parameters using the values stored by the forward- and backward-algorithms. Repeat 1-3.
34 EM for MMs We want a closed form for
35 EM for MMs We want a closed form for
36 EM for MMs We want a closed form for Taking the log yields:
37 EM for MMs We want a closed form for Taking the log yields: Taking the expectation over all Z's yields Q(θ, θold), i.e:
38 EM for MMs E-Step: To calculate Q, we must compute the expectations E(z1k), E(znk), and E(zn-1,j,znk). Consider the probabilities: A K-vector where entry k is the prob γ(znk) of being in state k in the n'th step... A KxK-table where entry (j,k) is the prob ξ(zn-1,j, znk) of being in state j and k in the (n-1)'th and n'th step...
39 EM for MMs E-Step: To calculate Q, we must compute the expectations E(z1k), E(znk), and E(zn-1,j,znk). Consider the probabilities: A K-vector where entry k is the prob γ(znk) of being in state k in the n'th step... binary variables A KxK-table where entry (j,k) is the prob ξ(zn-1,j, znk) of being in state j and k in the (n-1)'th and n'th step... Fact: The expectation of a binary variable z is just p(z=1)...
40 EM for MMs To calculate Q (the E-step), we must compute the expectations E(z1), E(zk), and E(zn-1,j,znk). Consider the probabilities: A K-vector where entry k is the prob γ(znk) of being in state k in the n'th step... binary variables A KxK-table where entry (j,k) is the prob ξ(zn-1,j, znk) of being in state j and k in the (n-1)'th and n'th step... Fact: The expectation of a binary variable z is just p(z=1)...
41 EM for MMs M-Step: If we assume discrete observables xi, then maximizing the above w.r.t. θ, i.e. A, π, and Фk, yields:
42 EM for MMs M-Step: If we assume discrete observables xi, then maximizing the above w.r.t. θ, i.e. A, π, and Фk, yields: Expected number of transitions from state j to state k Expected number of transitions from state j to any state
43 EM for MMs M-Step: If we assume discrete observables xi, then maximizing the above w.r.t. θ, i.e. A, π, and Фk, yields: Expected number of times symbol i is emitted from state k Expected number of times a symbol is emitted from state k
44 EM for MMs M-Step: If we assume discrete observables xi, then maximizing the above w.r.t. θ, i.e. A, π, and Фk, yields:
45 EM for MMs M-Step: If we assume discrete observables xi, then maximizing the above w.r.t. θ, i.e. A, π, and Фk, yields: Compare this to the formulas when X and Z where given:
46 Computing γ and ζ Can be computed efficiently using the forward- and backward-algorithm
47 Computing the new parameters α(znk) or β(znk) k n
48 Computing the new parameters The old parameters α(znk) or β(znk) The new parameters k n
49 EM for MMs - Summary Init: Pick suitable parameters (transition and emission probabilities). Observe that if a parameter is initialized to zero, it remains zero... E-Step: 1) Run the forward- and backward-algorithms with the current choice of parameters (to get t.he params of Q-func). Stop?: 2) Compute the likelihood p(x θ), if sufficient (or another stopping criteria is meet) then stop. M-Step: 3) Compute new parameters using the values stored by the forward- and backward-algorithms. Repeat 1-3. Running time per iteration: O(K2N + KK + K2NK + KDN), where D is number of observable symbols By using memorization in 3), we can improve it to O(K2N + KDN)
50 Using the scaled values in EM Can be computed using the modified forward- and backward-algorithm
51 Using the scaled values in EM Error in book Can be computed using the modified forward- and backward-algorithm
52 Computing the new parameters c1... cn α^(znk) or β^(znk) k 1 n N
53 Summary Selecting parameters by counting to reflect a set of (X,Z)'s, i.e. if full information about observables and corresponding latent values is given. Selecting parameters by Viterbi Training or Expectation Maximization to reflect a set of X's, i.e. if only information about observables is given.
54 Summary Selecting parameters by counting to reflect a set of (X,Z)'s, i.e. if full information about observables and corresponding latent values is given. Selecting parameters by Viterbi Training or Expectation Maximization to reflect a set of X's, i.e. if only information about observables is given. ow to deal with multiple training sequences?
55 When multiple (X, Z)'s are given... Assume that (several) sequences of observations X={x1,...,xn} and corresponding latent states Z={z1,...,zn} are given just sum each nominator and denominator over all (X,Z)'s, i.e. we divide total counts...
56 When multiple X's are given... Assume that a set sequences of observations X={x1,...,xn} is given... just sum each nominator and denominator over all X's, i.e. we divide total expectation, and we must run the forward- and backward algorithms for each training sequence X...
Notes on the EM Algorithm Michael Collins, September 24th 2005
Notes on the EM Algorithm Michael Collins, September 24th 2005 1 Hidden Markov Models A hidden Markov model (N, Σ, Θ) consists of the following elements: N is a positive integer specifying the number of
Computer Vision Group Prof. Daniel Cremers. 7. Sequential Data
Group Prof. Daniel Cremers 7. Sequential Data Bayes Filter (Rep.) We can describe the overall process using a Dynamic Bayes Network: This incorporates the following Markov assumptions: (measurement) (state)!2
Handout 4: Deterministic Systems and the Shortest Path Problem
SEEM 3470: Dynamic Optimization and Applications 2013 14 Second Term Handout 4: Deterministic Systems and the Shortest Path Problem Instructor: Shiqian Ma January 27, 2014 Suggested Reading: Bertsekas
Lecture 17: More on Markov Decision Processes. Reinforcement learning
Lecture 17: More on Markov Decision Processes. Reinforcement learning Learning a model: maximum likelihood Learning a value function directly Monte Carlo Temporal-difference (TD) learning COMP-424, Lecture
The EM algorithm for HMMs
The EM algorithm for HMMs Michael Collins February 22, 2012 Maximum-Likelihood Estimation for Fully Observed Data (Recap from earlier) We have fully observed data, x i,1... x i,m, s i,1... s i,m for i
Exact Inference (9/30/13) 2 A brief review of Forward-Backward and EM for HMMs
STA561: Probabilistic machine learning Exact Inference (9/30/13) Lecturer: Barbara Engelhardt Scribes: Jiawei Liang, He Jiang, Brittany Cohen 1 Validation for Clustering If we have two centroids, η 1 and
6.231 DYNAMIC PROGRAMMING LECTURE 3 LECTURE OUTLINE
6.21 DYNAMIC PROGRAMMING LECTURE LECTURE OUTLINE Deterministic finite-state DP problems Backward shortest path algorithm Forward shortest path algorithm Shortest path examples Alternative shortest path
Hidden Markov Models. Slides by Carl Kingsford. Based on Chapter 11 of Jones & Pevzner, An Introduction to Bioinformatics Algorithms
Hidden Markov Models Slides by Carl Kingsford Based on Chapter 11 of Jones & Pevzner, An Introduction to Bioinformatics Algorithms Eukaryotic Genes & Exon Splicing Prokaryotic (bacterial) genes look like
Reinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration
Reinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration Piyush Rai CS5350/6350: Machine Learning November 29, 2011 Reinforcement Learning Supervised Learning: Uses explicit supervision
Reinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration
Reinforcement Learning (1): Discrete MDP, Value Iteration, Policy Iteration Piyush Rai CS5350/6350: Machine Learning November 29, 2011 Reinforcement Learning Supervised Learning: Uses explicit supervision
6.825 Homework 3: Solutions
6.825 Homework 3: Solutions 1 Easy EM You are given the network structure shown in Figure 1 and the data in the following table, with actual observed values for A, B, and C, and expected counts for D.
Algorithmic Trading using Reinforcement Learning augmented with Hidden Markov Model
Algorithmic Trading using Reinforcement Learning augmented with Hidden Markov Model Simerjot Kaur (sk3391) Stanford University Abstract This work presents a novel algorithmic trading system based on reinforcement
Heterogeneous Hidden Markov Models
Heterogeneous Hidden Markov Models José G. Dias 1, Jeroen K. Vermunt 2 and Sofia Ramos 3 1 Department of Quantitative methods, ISCTE Higher Institute of Social Sciences and Business Studies, Edifício ISCTE,
Methods and Models of Loss Reserving Based on Run Off Triangles: A Unifying Survey
Methods and Models of Loss Reserving Based on Run Off Triangles: A Unifying Survey By Klaus D Schmidt Lehrstuhl für Versicherungsmathematik Technische Universität Dresden Abstract The present paper provides
Unobserved Heterogeneity Revisited
Unobserved Heterogeneity Revisited Robert A. Miller Dynamic Discrete Choice March 2018 Miller (Dynamic Discrete Choice) cemmap 7 March 2018 1 / 24 Distributional Assumptions about the Unobserved Variables
Reasoning with Uncertainty
Reasoning with Uncertainty Markov Decision Models Manfred Huber 2015 1 Markov Decision Process Models Markov models represent the behavior of a random process, including its internal state and the externally
Making Complex Decisions
Ch. 17 p.1/29 Making Complex Decisions Chapter 17 Ch. 17 p.2/29 Outline Sequential decision problems Value iteration algorithm Policy iteration algorithm Ch. 17 p.3/29 A simple environment 3 +1 p=0.8 2
6. Genetics examples: Hardy-Weinberg Equilibrium
PBCB 206 (Fall 2006) Instructor: Fei Zou email: fzou@bios.unc.edu office: 3107D McGavran-Greenberg Hall Lecture 4 Topics for Lecture 4 1. Parametric models and estimating parameters from data 2. Method
COS 513: Gibbs Sampling
COS 513: Gibbs Sampling Matthew Salesi December 6, 2010 1 Overview Concluding the coverage of Markov chain Monte Carlo (MCMC) sampling methods, we look today at Gibbs sampling. Gibbs sampling is a simple
CSE 312 Winter Learning From Data: Maximum Likelihood Estimators (MLE)
CSE 312 Winter 2017 Learning From Data: Maximum Likelihood Estimators (MLE) 1 Parameter Estimation Given: independent samples x1, x2,..., xn from a parametric distribution f(x θ) Goal: estimate θ. Not
Sequential Decision Making
Sequential Decision Making Dynamic programming Christos Dimitrakakis Intelligent Autonomous Systems, IvI, University of Amsterdam, The Netherlands March 18, 2008 Introduction Some examples Dynamic programming
Reinforcement Learning and Simulation-Based Search
Reinforcement Learning and Simulation-Based Search David Silver Outline 1 Reinforcement Learning 2 3 Planning Under Uncertainty Reinforcement Learning Markov Decision Process Definition A Markov Decision
4 Reinforcement Learning Basic Algorithms
Learning in Complex Systems Spring 2011 Lecture Notes Nahum Shimkin 4 Reinforcement Learning Basic Algorithms 4.1 Introduction RL methods essentially deal with the solution of (optimal) control problems
Pakes (1986): Patents as Options: Some Estimates of the Value of Holding European Patent Stocks
Pakes (1986): Patents as Options: Some Estimates of the Value of Holding European Patent Stocks Spring 2009 Main question: How much are patents worth? Answering this question is important, because it helps
Learning From Data: MLE. Maximum Likelihood Estimators
Learning From Data: MLE Maximum Likelihood Estimators 1 Parameter Estimation Assuming sample x1, x2,..., xn is from a parametric distribution f(x θ), estimate θ. E.g.: Given sample HHTTTTTHTHTTTHH of (possibly
Regime Switching Volatility Calibration by the Baum-Welch Method
Regime Switching Volatility Calibration by the Baum-Welch Method Abstract Regime switching volatility models provide a tractable method of modelling stochastic volatility. Currently the most popular method
Markov Processes and Applications
Markov Processes and Applications Algorithms, Networks, Genome and Finance Etienne Pardoux Laboratoire d'analyse, Topologie, Probabilites Centre de Mathematiques et d'injormatique Universite de Provence,
Intro to Reinforcement Learning. Part 3: Core Theory
Intro to Reinforcement Learning Part 3: Core Theory Interactive Example: You are the algorithm! Finite Markov decision processes (finite MDPs) dynamics p p p Experience: S 0 A 0 R 1 S 1 A 1 R 2 S 2 A 2
6.231 DYNAMIC PROGRAMMING LECTURE 3 LECTURE OUTLINE
6.21 DYNAMIC PROGRAMMING LECTURE LECTURE OUTLINE Deterministic finite-state DP problems Backward shortest path algorithm Forward shortest path algorithm Shortest path examples Alternative shortest path
Introduction to Fall 2007 Artificial Intelligence Final Exam
NAME: SID#: Login: Sec: 1 CS 188 Introduction to Fall 2007 Artificial Intelligence Final Exam You have 180 minutes. The exam is closed book, closed notes except a two-page crib sheet, basic calculators
A potentially useful approach to model nonlinearities in time series is to assume different behavior (structural break) in different subsamples
1.3 Regime switching models A potentially useful approach to model nonlinearities in time series is to assume different behavior (structural break) in different subsamples (or regimes). If the dates, the
Budget Management In GSP (2018)
Budget Management In GSP (2018) Yahoo! March 18, 2018 Miguel March 18, 2018 1 / 26 Today s Presentation: Budget Management Strategies in Repeated auctions, Balseiro, Kim, and Mahdian, WWW2017 Learning
Probability Distributions: Discrete
Probability Distributions: Discrete Introduction to Data Science Algorithms Jordan Boyd-Graber and Michael Paul SEPTEMBER 27, 2016 Introduction to Data Science Algorithms Boyd-Graber and Paul Probability
Rapid computation of prices and deltas of nth to default swaps in the Li Model
Rapid computation of prices and deltas of nth to default swaps in the Li Model Mark Joshi, Dherminder Kainth QUARC RBS Group Risk Management Summary Basic description of an nth to default swap Introduction
Hidden Markov Models & Applications Using R
R User Group Singapore (RUGS) Hidden Markov Models & Applications Using R Truc Viet Joe Le What is a Model? A mathematical formalization of the relationships between variables: Independent variables (X)
Hidden Markov Model for High Frequency Data
Hidden Markov Model for High Frequency Data Department of Mathematics, Florida State University Joint Math Meeting, Baltimore, MD, January 15 What are HMMs? A Hidden Markov model (HMM) is a stochastic
Project exam for STK Computational statistics
Project exam for STK4051 - Computational statistics Fall 2017 Part 1 (of 2) This is the first part of the exam project set for STK4051/9051, fall semester 2017. It is made available on the course website
Markov Decision Processes: Making Decision in the Presence of Uncertainty. (some of) R&N R&N
Markov Decision Processes: Making Decision in the Presence of Uncertainty (some of) R&N 16.1-16.6 R&N 17.1-17.4 Different Aspects of Machine Learning Supervised learning Classification - concept learning
Identification and Estimation of Dynamic Games when Players Beliefs are not in Equilibrium
and of Dynamic Games when Players Beliefs are not in Equilibrium Victor Aguirregabiria and Arvind Magesan Presented by Hanqing Institute, Renmin University of China Outline General Views 1 General Views
A Hidden Markov Model Approach to Information-Based Trading: Theory and Applications
A Hidden Markov Model Approach to Information-Based Trading: Theory and Applications Online Supplementary Appendix Xiangkang Yin and Jing Zhao La Trobe University Corresponding author, Department of Finance,
Estimation of the Markov-switching GARCH model by a Monte Carlo EM algorithm
Estimation of the Markov-switching GARCH model by a Monte Carlo EM algorithm Maciej Augustyniak Fields Institute February 3, 0 Stylized facts of financial data GARCH Regime-switching MS-GARCH Agenda Available
Estimation of a Ramsay-Curve IRT Model using the Metropolis-Hastings Robbins-Monro Algorithm
1 / 34 Estimation of a Ramsay-Curve IRT Model using the Metropolis-Hastings Robbins-Monro Algorithm Scott Monroe & Li Cai IMPS 2012, Lincoln, Nebraska Outline 2 / 34 1 Introduction and Motivation 2 Review
ECON5160: The compulsory term paper
University of Oslo / Department of Economics / TS+NCF March 9, 2012 ECON5160: The compulsory term paper Formalities: This term paper is compulsory. This paper must be accepted in order to qualify for attending
Hidden Markov Models for Financial Market Predictions
Hidden Markov Models for Financial Market Predictions Department of Mathematics and Statistics Youngstown State University Central Spring Sectional Meeting, Michigan State University, March 15 1 Introduction
Machine Learning in Computer Vision Markov Random Fields Part II
Machine Learning in Computer Vision Markov Random Fields Part II Oren Freifeld Computer Science, Ben-Gurion University March 22, 2018 Mar 22, 2018 1 / 40 1 Some MRF Computations 2 Mar 22, 2018 2 / 40 Few
EE641 Digital Image Processing II: Purdue University VISE - October 29,
EE64 Digital Image Processing II: Purdue University VISE - October 9, 004 The EM Algorithm. Suffient Statistics and Exponential Distributions Let p(y θ) be a family of density functions parameterized by
Non-Deterministic Search
Non-Deterministic Search MDP s 1 Non-Deterministic Search How do you plan (search) when your actions might fail? In general case, how do you plan, when the actions have multiple possible outcomes? 2 Example:
Chapter 4: Asymptotic Properties of MLE (Part 3)
Chapter 4: Asymptotic Properties of MLE (Part 3) Daniel O. Scharfstein 09/30/13 1 / 1 Breakdown of Assumptions Non-Existence of the MLE Multiple Solutions to Maximization Problem Multiple Solutions to
Complex Decisions. Sequential Decision Making
Sequential Decision Making Outline Sequential decision problems Value iteration Policy iteration POMDPs (basic concepts) Slides partially based on the Book "Reinforcement Learning: an introduction" by
BCJR Algorithm. Veterbi Algorithm (revisted) Consider covolutional encoder with. And information sequences of length h = 5
Chapter 2 BCJR Algorithm Ammar Abh-Hhdrohss Islamic University -Gaza ١ Veterbi Algorithm (revisted) Consider covolutional encoder with And information sequences of length h = 5 The trellis diagram has
An Improved Saddlepoint Approximation Based on the Negative Binomial Distribution for the General Birth Process
Computational Statistics 17 (March 2002), 17 28. An Improved Saddlepoint Approximation Based on the Negative Binomial Distribution for the General Birth Process Gordon K. Smyth and Heather M. Podlich Department
A simple wealth model
Quantitative Macroeconomics Raül Santaeulàlia-Llopis, MOVE-UAB and Barcelona GSE Homework 5, due Thu Nov 1 I A simple wealth model Consider the sequential problem of a household that maximizes over streams
Optimal Switching Games in Emissions Trading
Emissions Trading Numerics Conclusion Optimal in Emissions Trading Mike Department of Statistics & Applied Probability University of California Santa Barbara Bachelier Congress, June 24, 2010 1 / 26 Emissions
Estimating a Dynamic Oligopolistic Game with Serially Correlated Unobserved Production Costs. SS223B-Empirical IO
Estimating a Dynamic Oligopolistic Game with Serially Correlated Unobserved Production Costs SS223B-Empirical IO Motivation There have been substantial recent developments in the empirical literature on
Optimal Stopping. Nick Hay (presentation follows Thomas Ferguson s Optimal Stopping and Applications) November 6, 2008
(presentation follows Thomas Ferguson s and Applications) November 6, 2008 1 / 35 Contents: Introduction Problems Markov Models Monotone Stopping Problems Summary 2 / 35 The Secretary problem You have
Back to estimators...
Back to estimators... So far, we have: Identified estimators for common parameters Discussed the sampling distributions of estimators Introduced ways to judge the goodness of an estimator (bias, MSE, etc.)
A start of Variational Methods for ERGM Ranran Wang, UW
A start of Variational Methods for ERGM Ranran Wang, UW MURI-UCI April 24, 2009 Outline A start of Variational Methods for ERGM [1] Introduction to ERGM Current methods of parameter estimation: MCMCMLE:
Reinforcement Learning. Slides based on those used in Berkeley's AI class taught by Dan Klein
Reinforcement Learning Slides based on those used in Berkeley's AI class taught by Dan Klein Reinforcement Learning Basic idea: Receive feedback in the form of rewards Agent s utility is defined by the
Chapter 7: Estimation Sections
Chapter 7: Estimation Sections 7.1 Statistical Inference Bayesian Methods: 7.2 Prior and Posterior Distributions 7.3 Conjugate Prior Distributions Frequentist Methods: 7.5 Maximum Likelihood Estimators
a 13 Notes on Hidden Markov Models Michael I. Jordan University of California at Berkeley Hidden Markov Models The model
Notes on Hidden Markov Models Michael I. Jordan University of California at Berkeley Hidden Markov Models This is a lightly edited version of a chapter in a book being written by Jordan. Since this is
Final exam solutions
EE365 Stochastic Control / MS&E251 Stochastic Decision Models Profs. S. Lall, S. Boyd June 5 6 or June 6 7, 2013 Final exam solutions This is a 24 hour take-home final. Please turn it in to one of the
CPSC 540: Machine Learning
CPSC 540: Machine Learning Monte Carlo Methods Mark Schmidt University of British Columbia Winter 2018 Last Time: Markov Chains We can use Markov chains for density estimation, p(x) = p(x 1 ) }{{} d p(x
Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay
Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 15 Adaptive Huffman Coding Part I Huffman code are optimal for a
Agricultural and Applied Economics 637 Applied Econometrics II
Agricultural and Applied Economics 637 Applied Econometrics II Assignment I Using Search Algorithms to Determine Optimal Parameter Values in Nonlinear Regression Models (Due: February 3, 2015) (Note: Make
Bayesian course - problem set 3 (lecture 4)
Bayesian course - problem set 3 (lecture 4) Ben Lambert November 14, 2016 1 Ticked off Imagine once again that you are investigating the occurrence of Lyme disease in the UK. This is a vector-borne disease
Maximum Likelihood Estimation
Maximum Likelihood Estimation The likelihood and log-likelihood functions are the basis for deriving estimators for parameters, given data. While the shapes of these two functions are different, they have
Gamma Distribution Fitting
Chapter 552 Gamma Distribution Fitting Introduction This module fits the gamma probability distributions to a complete or censored set of individual or grouped data values. It outputs various statistics
On Implementation of the Markov Chain Monte Carlo Stochastic Approximation Algorithm
On Implementation of the Markov Chain Monte Carlo Stochastic Approximation Algorithm Yihua Jiang, Peter Karcher and Yuedong Wang Abstract The Markov Chain Monte Carlo Stochastic Approximation Algorithm
CPSC 540: Machine Learning
CPSC 540: Machine Learning Monte Carlo Methods Mark Schmidt University of British Columbia Winter 2019 Last Time: Markov Chains We can use Markov chains for density estimation, d p(x) = p(x 1 ) p(x }{{}
Modeling and Estimation of
Modeling and of Financial and Actuarial Mathematics Christian Doppler Laboratory for Portfolio Risk Management Vienna University of Technology PRisMa 2008 29.09.2008 Outline 1 2 3 4 5 Credit ratings describe
Academic Research Review. Classifying Market Conditions Using Hidden Markov Model
Academic Research Review Classifying Market Conditions Using Hidden Markov Model INTRODUCTION Best known for their applications in speech recognition, Hidden Markov Models (HMMs) are able to discern and
Math-Stat-491-Fall2014-Notes-V
Math-Stat-491-Fall2014-Notes-V Hariharan Narayanan December 7, 2014 Martingales 1 Introduction Martingales were originally introduced into probability theory as a model for fair betting games. Essentially
sociology SO5032 Quantitative Research Methods Brendan Halpin, Sociology, University of Limerick Spring 2018 SO5032 Quantitative Research Methods
1 SO5032 Quantitative Research Methods Brendan Halpin, Sociology, University of Limerick Spring 2018 Lecture 10: Multinomial regression baseline category extension of binary What if we have multiple possible
Stochastic Approximation Algorithms and Applications
Harold J. Kushner G. George Yin Stochastic Approximation Algorithms and Applications With 24 Figures Springer Contents Preface and Introduction xiii 1 Introduction: Applications and Issues 1 1.0 Outline
Stochastic Manufacturing & Service Systems. Discrete-time Markov Chain
ISYE 33 B, Fall Week #7, September 9-October 3, Introduction Stochastic Manufacturing & Service Systems Xinchang Wang H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of
Maximum Likelihood Estimation Richard Williams, University of Notre Dame, https://www3.nd.edu/~rwilliam/ Last revised January 13, 2018
Maximum Likelihood Estimation Richard Williams, University of otre Dame, https://www3.nd.edu/~rwilliam/ Last revised January 3, 208 [This handout draws very heavily from Regression Models for Categorical
Maximum Likelihood Estimation Richard Williams, University of Notre Dame, https://www3.nd.edu/~rwilliam/ Last revised January 10, 2017
Maximum Likelihood Estimation Richard Williams, University of otre Dame, https://www3.nd.edu/~rwilliam/ Last revised January 0, 207 [This handout draws very heavily from Regression Models for Categorical
Extracting Information from the Markets: A Bayesian Approach
Extracting Information from the Markets: A Bayesian Approach Daniel Waggoner The Federal Reserve Bank of Atlanta Florida State University, February 29, 2008 Disclaimer: The views expressed are the author
Construction and behavior of Multinomial Markov random field models
Graduate Theses and Dissertations Iowa State University Capstones, Theses and Dissertations 2010 Construction and behavior of Multinomial Markov random field models Kim Mueller Iowa State University Follow
Hill Climbing on Speech Lattices: A New Rescoring Framework
Hill Climbing on Speech Lattices: A New Rescoring Framework Ariya Rastrow, Markus Dreyer, Abhinav Sethy, Sanjeev Khudanpur, Bhuvana Ramabhadran and Mark Dredze Motivation Availability of large amounts
A short note on parameter approximation for von Mises-Fisher distributions
Computational Statistics manuscript No. (will be inserted by the editor) A short note on parameter approximation for von Mises-Fisher distributions And a fast implementation of I s (x) Suvrit Sra Received:
Two hours. To be supplied by the Examinations Office: Mathematical Formula Tables and Statistical Tables THE UNIVERSITY OF MANCHESTER
Two hours MATH20802 To be supplied by the Examinations Office: Mathematical Formula Tables and Statistical Tables THE UNIVERSITY OF MANCHESTER STATISTICAL METHODS Answer any FOUR of the SIX questions.
Markov Decision Processes (MDPs) CS 486/686 Introduction to AI University of Waterloo
Markov Decision Processes (MDPs) CS 486/686 Introduction to AI University of Waterloo Outline Sequential Decision Processes Markov chains Highlight Markov property Discounted rewards Value iteration Markov
Advanced Numerical Methods
Advanced Numerical Methods Solution to Homework One Course instructor: Prof. Y.K. Kwok. When the asset pays continuous dividend yield at the rate q the expected rate of return of the asset is r q under
The Agent-Environment Interface Goals, Rewards, Returns The Markov Property The Markov Decision Process Value Functions Optimal Value Functions
The Agent-Environment Interface Goals, Rewards, Returns The Markov Property The Markov Decision Process Value Functions Optimal Value Functions Optimality and Approximation Finite MDP: {S, A, R, p, γ}
Econ 582 Nonlinear Regression
Econ 582 Nonlinear Regression Eric Zivot June 3, 2013 Nonlinear Regression In linear regression models = x 0 β (1 )( 1) + [ x ]=0 [ x = x] =x 0 β = [ x = x] [ x = x] x = β it is assumed that the regression
Chapter 3. Dynamic discrete games and auctions: an introduction
Chapter 3. Dynamic discrete games and auctions: an introduction Joan Llull Structural Micro. IDEA PhD Program I. Dynamic Discrete Games with Imperfect Information A. Motivating example: firm entry and
Estimating Mixed Logit Models with Large Choice Sets. Roger H. von Haefen, NC State & NBER Adam Domanski, NOAA July 2013
Estimating Mixed Logit Models with Large Choice Sets Roger H. von Haefen, NC State & NBER Adam Domanski, NOAA July 2013 Motivation Bayer et al. (JPE, 2007) Sorting modeling / housing choice 250,000 individuals
Log-linear Modeling Under Generalized Inverse Sampling Scheme
Log-linear Modeling Under Generalized Inverse Sampling Scheme Soumi Lahiri (1) and Sunil Dhar (2) (1) Department of Mathematical Sciences New Jersey Institute of Technology University Heights, Newark,
Likelihood Methods of Inference. Toss coin 6 times and get Heads twice.
Methods of Inference Toss coin 6 times and get Heads twice. p is probability of getting H. Probability of getting exactly 2 heads is 15p 2 (1 p) 4 This function of p, is likelihood function. Definition:
CS340 Machine learning Bayesian model selection
CS340 Machine learning Bayesian model selection Bayesian model selection Suppose we have several models, each with potentially different numbers of parameters. Example: M0 = constant, M1 = straight line,
Econometrics II Multinomial Choice Models
LV MNC MRM MNLC IIA Int Est Tests End Econometrics II Multinomial Choice Models Paul Kattuman Cambridge Judge Business School February 9, 2018 LV MNC MRM MNLC IIA Int Est Tests End LW LW2 LV LV3 Last Week:
TDT4171 Artificial Intelligence Methods
TDT47 Artificial Intelligence Methods Lecture 7 Making Complex Decisions Norwegian University of Science and Technology Helge Langseth IT-VEST 0 helgel@idi.ntnu.no TDT47 Artificial Intelligence Methods
Definition 4.1. In a stochastic process T is called a stopping time if you can tell when it happens.
102 OPTIMAL STOPPING TIME 4. Optimal Stopping Time 4.1. Definitions. On the first day I explained the basic problem using one example in the book. On the second day I explained how the solution to the
Lecture 2: Making Good Sequences of Decisions Given a Model of World. CS234: RL Emma Brunskill Winter 2018
Lecture 2: Making Good Sequences of Decisions Given a Model of World CS234: RL Emma Brunskill Winter 218 Human in the loop exoskeleton work from Steve Collins lab Class Structure Last Time: Introduction
Equity, Vacancy, and Time to Sale in Real Estate.
Title: Author: Address: E-Mail: Equity, Vacancy, and Time to Sale in Real Estate. Thomas W. Zuehlke Department of Economics Florida State University Tallahassee, Florida 32306 U.S.A. tzuehlke@mailer.fsu.edu
An Introduction to Bayesian Inference and MCMC Methods for Capture-Recapture
An Introduction to Bayesian Inference and MCMC Methods for Capture-Recapture Trinity River Restoration Program Workshop on Outmigration: Population Estimation October 6 8, 2009 An Introduction to Bayesian
CS 188: Artificial Intelligence. Outline
C 188: Artificial Intelligence Markov Decision Processes (MDPs) Pieter Abbeel UC Berkeley ome slides adapted from Dan Klein 1 Outline Markov Decision Processes (MDPs) Formalism Value iteration In essence
Chapter 4: Commonly Used Distributions. Statistics for Engineers and Scientists Fourth Edition William Navidi
Chapter 4: Commonly Used Distributions Statistics for Engineers and Scientists Fourth Edition William Navidi 2014 by Education. This is proprietary material solely for authorized instructor use. Not authorized
Multistage risk-averse asset allocation with transaction costs
Multistage risk-averse asset allocation with transaction costs 1 Introduction Václav Kozmík 1 Abstract. This paper deals with asset allocation problems formulated as multistage stochastic programming models.