Monte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMSN50)

Transcription

1 Monte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMSN50) Magnus Wiktorsson Centre for Mathematical Sciences Lund University, Sweden Lecture 6 Sequential Monte Carlo methods II February 1, 2018 M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (1) 1 / 27

2 Plan of today's lecture 1 Last time: Sequential MC problems 2 3 M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (2) 2 / 27

3 We are here 1 Last time: Sequential MC problems 2 3 M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (3) 3 / 27

4 Last time: Sequential MC problems In the sequential MC framework, we aim at sequentially estimating sequences (τ n ) n 0 of expectations τ n = E fn (φ(x 0:n )) = φ(x 0:n )f n (x 0:n ) dx 0:n ( ) X n over spaces X n of increasing dimension, where the densities (f n ) are known up to normalizing constants only, i.e., for every n 0, where c n is an unknown constant. f n (x 0:n ) = z n(x 0:n ) c n, M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (4) 4 / 27

5 Last time: Markov chains Some applications involved the notion of Markov chains: A Markov chain on X R d is a family of random variables (= stochastic process) (X k ) k 0 taking values in X such that P(X k+1 B X 0, X 1,..., X k ) = P(X k+1 B X k ). The density q of the distribution of X k+1 given X k = x k is called the transition density of (X k ). Consequently, P(X k+1 B X k = x k ) = q(x k+1 x k ) dx k+1. As a rst example we considered an AR(1) process: X 0 = 0, X k+1 = αx k + ɛ k+1, where α is a constant and (ɛ k ) are i.i.d. variables. M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (5) 5 / 27 B

6 Last time: Markov chains (cont.) The following theorem provides the joint density f n (x 0, x 1,..., x n ) of X 0, X 1,..., X n. Theorem Let (X k ) be Markov with X 0 χ. Then for n > 0, n 1 f n (x 0, x 1,..., x n ) = χ(x 0 ) q(x k+1 x k ). k=0 Corollary (The Chapman-Kolmogorov equation) Let (X k ) be Markov. Then for n > 1, f n (x n x 0 ) = ( n 1 k=0 q(x k+1 x k ) ) dx 1 dx n 1. M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (6) 6 / 27

7 Last time: Rare event analysis (REA) for Markov chains Let (X k ) be a Markov chain. Assume that we want to compute, for n = 0, 1, 2,... τ n = E(φ(X 0:n ) X 0:n B) = = B B f n (x 0:n ) φ(x 0:n ) P(X 0:n B) dx 0:n φ(x 0:n ) χ(x 0) n 1 k=0 q(x k+1 x k ) dx 0:n, P(X 0:n B) where B is a possibly rare event and P(X 0:n B) is generally unknown. We thus face a sequential MC problem ( ) with { z n (x 0:n ) χ(x 0 ) n 1 k=0 q(x k+1 x k ), c n P(X 0:n B). M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (7) 7 / 27

8 Last time: Estimation in general HMMs Graphically: Y k 1 Y k Y k+1 (Observations)... X k 1 X k X k+1... (Markov chain) Y k X k = x k p(y k x k ) X k+1 X k = x k q(x k+1 x k ) X 0 χ(x 0 ) (Observation density) (Transition density) (Initial distribution) M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (8) 8 / 27

9 Last time: Estimation in general HMMs In an HMM, the smoothing distribution f n (x 0:n y 0:n ) is the conditional distribution of a set X 0:n of hidden states given Y 0:n = y 0:n. Theorem (Smoothing distribution) where f n (x 0:n y 0:n ) = χ(x 0)p(y 0 x 0 ) n k=1 p(y k x k )q(x k x k 1 ), L n (y 0:n ) L n (y 0:n ) = density of the observations y 0:n n = χ(x 0 )p(y 0 x 0 ) p(y k x k )q(x k x k 1 ) dx 0 dx n. k=1 M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (9) 9 / 27

10 Last time: Estimation in general HMMs Assume that we want to compute, online for n = 0, 1, 2,..., τ n = E(φ(X 0:n ) Y 0:n = y 0:n ) = φ(x 0:n )f n (x 0:n y 0:n ) dx 0 dx n = φ(x 0:n ) χ(x 0)p(y 0 x 0 ) n k=1 p(y k x k )q(x k x k 1 ) dx 0 dx n, L n (y 0:n ) where L n (y 0:n ) (= obscene integral) is generally unknown. We thus face a sequential MC problem ( ) with { z n (x 0:n ) χ(x 0 )p(y 0 x 0 ) n k=1 p(y k x k )q(x k x k 1 ), c n L n (y 0:n ). M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (10) 10 / 27

11 We are here 1 Last time: Sequential MC problems 2 3 M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (11) 11 / 27

12 Conditional methods Say that we want to generate a random vector from a given bivariate density p(x, y). If we know how to draw from the conditional distribution p(y x) and the marginal p(x) this can be done naturally using the following scheme. draw Z 1 p(x) draw Z 2 p(y x = Z 1 ) return (Z 1, Z 2 ) M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (12) 12 / 27

13 Conditional methods This can be naturally extended to n-variate densities p(x 1,..., x n ): draw Z 1 p(x 1 ) draw Z 2 p(x 2 x 1 = Z 1 ) draw Z 3 p(x 3 x 1 = Z 1, x 2 = Z 2 ). draw Z n 1 p(x n 1 x 1 = Z 1, x 2 = Z 2,..., x n 2 = Z n 2 ) draw Z n p(x n x 1 = Z 1, x 2 = Z 2,..., x n 1 = Z n 1 ) return (Z 1,..., Z n ) Theorem The vector (Z 1,..., Z n ) has indeed n-variate density function p(x 1,..., x n ). M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (13) 13 / 27

14 We are here 1 Last time: Sequential MC problems 2 3 M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (14) 14 / 27

15 We are here 1 Last time: Sequential MC problems 2 3 M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (15) 15 / 27

16 It is natural to aim at solving the problem using usual self-normalized IS. However, the generated samples (Xi 0:n, ω n (Xi 0:n )) should be such that having (Xi 0:n, ω n (Xi 0:n )), the next sample (Xi 0:n+1, ω n+1 (Xi 0:n+1 )) is easily generated with a complexity that does not increase with n (online sampling). the approximation remains stable as n increases. We call each draw Xi 0:n = (Xi 0,..., Xn i ) a particle. Moreover, we denote importance weights by ωn i def = ω n (Xi 0:n ). M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (16) 16 / 27

17 We are here 1 Last time: Sequential MC problems 2 3 M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (17) 17 / 27

18 We proceed recursively. Assume that we have generated particles (X 0:n i ) from g n (x 0:n ) so that N i=1 ωn i N φ(x l=1 ωl i 0:n ) E fn (φ(x 0:n )), n where, as usual, ω i n = ω n (X 0:n i ) = z n (X 0:n i )/g n (X 0:n i ). Key trick: Choose an instrumental distribution satisfying g n+1 (x 0:n+1 ) = g n+1 (x n+1 x 0:n )g n+1 (x 0:n ) g n+1 (x 0:n+1 ) = g n+1 (x n+1 x 0:n )g n (x 0:n ). M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (18) 18 / 27

19 SIS (cont.) Last time: Sequential MC problems Now assume that we have drawn X 0:n g n (x 0:n ). Then, as g n+1 (x 0:n+1 ) = g n+1 (x n+1 x 0:n )g n+1 (x 0:n ) = g n+1 (x n+1 x 0:n )g n (x 0:n ), the conditional method allows us to generate a draw X 0:n+1 from g n+1 (x 0:n+1 ) using the following procedure: draw X n+1 g n+1 (x n+1 x 0:n = X 0:n ) let X 0:n+1 (X 0:n, X n+1 ) This can be repeated recursively, yielding online sampling from the sequence (g n ). M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (19) 19 / 27

20 SIS (cont.) Last time: Sequential MC problems Consequently, Xi 0:n+1 and ωn+1 i can be generated by keeping the previous Xi 0:n, simulating Xi n+1 g n+1 (x n+1 Xi 0:n ), setting Xi 0:n+1 = (Xi 0:n, Xi n+1 ), and computing ωn+1 i = z n+1(xi 0:n+1 ) g n+1 (Xi 0:n+1 ) z n+1 (Xi 0:n+1 ) = z n (Xi 0:n )g n+1 (X n+1 = z n+1 (X 0:n+1 i ) i Xi 0:n ) zn(x0:n i ) g n (X 0:n z n (X 0:n i )g n+1 (X n+1 i X 0:n i ) ωi n. i ) M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (20) 20 / 27

21 SIS (cont.) Last time: Sequential MC problems Voilà, the sample (Xi 0:n+1, ωn+1 i ) can now be used to approximate E fn+1 (φ(x 0:n+1 ))! So, by running the SIS algorithm, we have updated an approximation to an approximation N i=1 N i=1 ωn i N φ(x l=1 ωl i 0:n ) E fn (φ(x 0:n )) n ωn+1 i N φ(x l=1 ωl i 0:n+1 ) E fn+1 (φ(x 0:n+1 )) n+1 by only adding a component Xi n+1 to Xi 0:n weights. and sequentially updating the M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (21) 21 / 27

22 SIS: Pseudo code for i = 1 N do draw Xi 0 g 0 set ω0 i = z 0(Xi 0) g 0 (Xi 0) end for return (Xi 0, ωi 0 ) for k = 0, 1, 2,... do for i = 1 N do draw Xi k+1 g k+1 (x k+1 X 0:k set Xi 0:k+1 (Xi 0:k, Xi k+1 ) set ω i k+1 end for return (Xi 0:k+1, ωk+1 i ) end for i ) z k+1 (X 0:k+1 i ) z k (Xi 0:k )g k+1 (X k+1 i Xi 0:k ) ωi k M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (22) 22 / 27

23 Example: REA reconsidered We consider again the example of REA for Markov chains (X = R, X 0 = x 0 = a): τ n = E(φ(X 0:n ) a X l, l = 0,..., n) = φ(x 0:n ) (a, ) n n 1 k=1 q(x k+1 x k ) P(a X l, l) } {{ } =z n(x 0:n )/c n dx 1:n. Choose g k+1 (x k+1 x 0:k ) to be the conditional density of X k+1 given X k and X k+1 a: g k+1 (x k+1 x 0:k ) = {cf. HA1, Problem 1} = q(x k+1 x k ) a q(z x k) dz. M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (23) 23 / 27

24 Example: REA Last time: Sequential MC problems This implies that (recall that we have conditioned on X 0 = x 0 = a) g n (x 0:n ) = n 1 k=0 q(x k+1 x k ) a q(z x k) dz. In addition, the weights are updated according to ωk+1 i = z k+1 (Xi 0:k+1 ) z k (Xi 0:k )g k+1 (X k+1 = = k 1 l=0 q(xl+1 a i Xi 0:k k l=0 q(xl+1 i X l i ) ) ωi k i Xi l) q(xk+1 i Xi k) ωk i q(z Xk i ) dz q(z X k i ) dz ω i k. a M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (24) 24 / 27

25 Example: REA; Matlab implementation for AR(1) process with Gaussian noise % design of instrumental distribution: int 1 normcdf(a,alpha*x,sigma); trunk_td_rnd =... % use e.g. HA1, Problem 1, to simulate % the conditional transition density; % SIS: part = a*ones(n,1); % initialization of all particles in a w = ones(n,1); for k = 1:(n 1), % main loop part_mut = trunk_td_rnd(part); w = w.*int(part); part = part_mut; end c = mean(w); % estimated probability M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (25) 25 / 27

26 REA: Importance weight distribution Serious drawback of SIS: the importance weights degenerate! n = n = n = Importance weights (base 10 logarithm) M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (26) 26 / 27

27 What's next? Last time: Sequential MC problems Weight degeneration is a universal problem with the SIS method and is due to the fact that the particle weights are generated through subsequent multiplications. This drawback preventedduring several decadesthe SIS method from being practically useful. Next week we will discuss an elegant solution to this problem: SIS with resampling (SISR). M. Wiktorsson Monte Carlo and Empirical Methods for Stochastic Inference, L6 (27) 27 / 27