Asymptotic study of an inhomogeneous Markov jump process

Transcription

1 Asymptotic study of an inhomogeneous Markov jump process A Berry-Esseen theorem for the median algorithm? Claire Delplancke Work in progress with Sébastien Gadat and Laurent Miclo Journées IOPS 5-7 Juillet Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 0 / 18

2 Time-inhomogeneous Markov jump process Object of study: Markov process (Z t ) t 0, with instantaneous generator (L t ) t 0 acting on bounded functions f : R R as: L t [f ](z) = p t (z)(f (z + a t ) f (z)) + q t (z)(f (z a t ) f (z)), At time t 0, the process can make jumps of two types : z R, t 0. from z to z + a t, with rate p t (z), from z to z a t, with rate q t (z). Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 1 / 18

3 Asymptotic normality Set µ the normal distribution, If µ(x) = e x2 /2 2π. p t (z a t )µ(z a t ) = q t (z)µ(z), z R, t 0, the process (Z t ) t 0 is reversible with respect to µ. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 2 / 18

4 Asymptotic normality Set µ the normal distribution, If µ(x) = e x2 /2 2π. p t (z a t )µ(z a t ) = q t (z)µ(z), z R, t 0, the process (Z t ) t 0 is reversible with respect to µ. Now, if p t (z a t )µ(z a t ) q t (z)µ(z) 0, t + then one can think that the process is asymptotically normal, i.e. Goal: quantify this convergence, L(Z t ) µ. t + W (L(Z t ), µ)? Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 2 / 18

5 1 Motivation: link with an online median-finding algorithm 2 Homogeneous case 3 Non-homogeneous case 4 Elements of proof 5 Perspectives Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 2 / 18

6 1 Motivation: link with an online median-finding algorithm 2 Homogeneous case 3 Non-homogeneous case 4 Elements of proof 5 Perspectives Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 2 / 18

7 Median-finding algorithm Set (X i ) i N independent, identically distributed real random variables with distribution ν, and ( Y n+1 = Y n + γ n+1 1 Xn+1 >Y n 1 ), n 1, 2 with γ n = γ 1 n β, 0 < β 1. Assume that ν has density φ, and φ(q 1/2 ) > 0. Then, Y n n + q 1/2 a.s. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 3 / 18

8 CLT for the algorithm If 0 < β < 1, L ( ) Yn q 1/2 σ 2 γ µ, n + n with σ 2 = (8φ(q 1/2 )) 1. If β = 1 and φ(q 1/2 ) > γ 1 /2, then with σ 2 = (8(φ(q 1/2 ) γ 1 /2)) 1. L ( ) Yn q 1/2 σ 2 γ µ, n + n Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 4 / 18

9 Non-asymptotic bounds Robbins-Monro algorithm: where Y n+1 = Y n γ n+1 H(X n+1, Y n ), n 1, H(x, y) = (1 x>y 1/2), h(y) := E [H(X, y)] = F (y) 1/2, x, y R. Non-asymptotic bounds for the quadratic risk: Moulines et al.: h not Lipschitz without assumptions on the density φ. Cénac, Godichon: dimension 1. Concentration bounds, Frikha, Menozzi: h not Lipschitz. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 5 / 18

10 Non-asymptotic bounds Robbins-Monro algorithm: where Y n+1 = Y n γ n+1 H(X n+1, Y n ), n 1, H(x, y) = (1 x>y 1/2), h(y) := E [H(X, y)] = F (y) 1/2, x, y R. Non-asymptotic bounds for the quadratic risk: Moulines et al.: h not Lipschitz without assumptions on the density φ. Cénac, Godichon: dimension 1. Concentration bounds, Frikha, Menozzi: h not Lipschitz. Idea: quantification of the convergence ( ) Yn q 1/2 L σ 2 γ µ, n + n Berry-Esseen bound for the non-homogeneous Markov chain ((Y n q 1/2 )/ σ 2 γ n ) n N? Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 5 / 18

11 From the real model to the simplified model I Consider Ỹ n = Y n q 1/2 σ 2 γ n. Assumptions: q 1/2 = 0, γ n = γ 1 n β, with 0 < β < 1. σ 2 = 1, Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 6 / 18

12 From the real model to the simplified model I Consider Ỹ n = Y n q 1/2 σ 2 γ n. Assumptions: q 1/2 = 0, γ n = γ 1 n β, with 0 < β < 1. σ 2 = 1, Then, for all n 1, Ỹ n+1 = θ n+1 Ỹ n + ( γ n+1 1 Xn+1 >θ n+1 γn+1 Ỹ n 1 ), 2 γn θ n+1 := = 1 + β ( ) 1 γ n+1 2n + o. n First simplification: Z n+1 = Z n + ( γ n+1 1 Xn+1 > γ n+1 Z n 1 ). 2 Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 6 / 18

13 From the real model to the simplified model II The non-homogeneous Markov chain Z n+1 = Z n + ( γ n+1 1 Xn+1 > γ n+1 Z n 1 ) 2 has for transition operator Q γn [f ](z) = (1 F ( γ n z))f (z + γ n /2) + F ( γ n z)f (z γ n /2), where F is the cdf of the X i s. Second simplification: continuous-time process (Z t ) t 0, with instantaneous generator L ηt [f ](z) = 1 η 2 t ( (1 F (ηt z))(f (z + η t /2) f (z)) + F (η t z)(f (z η t /2) f (z)) ), Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 7 / 18

14 From the real model to the simplified model II The non-homogeneous Markov chain Z n+1 = Z n + ( γ n+1 1 Xn+1 > γ n+1 Z n 1 ) 2 has for transition operator Q γn [f ](z) = (1 F ( γ n z))f (z + γ n /2) + F ( γ n z)f (z γ n /2), where F is the cdf of the X i s. Second simplification: continuous-time process (Z t ) t 0, with instantaneous generator L ηt [f ](z) = 1 η 2 t ( (1 F (ηt z))(f (z + η t /2) f (z)) + F (η t z)(f (z η t /2) f (z)) ), and if γ n = γ 1 n β then η t = η 0 (t + 1) β/(2(1 β)). Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 7 / 18

15 Toy-model More generally, consider a continuous-time process (Z t ) t 0, with instantaneous generator L t [f ](z) = p t (z)(f (z + a t ) f (z)) + q t (z)(f (z a t ) f (z)), In the case relative to the median algorithm, p t (z) = 1 F (η tz) η 2 t z R, t 0., q t (z) = F (η tz) ηt 2, η t = η 0 (t + 1) β/(2(1 β)). Remark: assumption that the X i s are identically distributed can be weakened. (Cf Frikha 2014.) Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 8 / 18

16 1 Motivation: link with an online median-finding algorithm 2 Homogeneous case 3 Non-homogeneous case 4 Elements of proof 5 Perspectives Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 8 / 18

17 Homogeneous case First, consider an homogenous Markov jump process (Z t ) t 0 with generator L[f ](z) = p(z)(f (z + a) f (z)) + q(z)(f (z a) f (z)), z R. It is reversible with respect to µ if p(z a)µ(z a) = q(z)µ(z), z R. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 9 / 18

18 Homogeneous case First, consider an homogenous Markov jump process (Z t ) t 0 with generator L[f ](z) = p(z)(f (z + a) f (z)) + q(z)(f (z a) f (z)), z R. It is reversible with respect to µ if p(z a)µ(z a) = q(z)µ(z), z R. Key observation: If Z 0 = x, then the process (Z t ) t 0 lives on the grid Γ a,x = {x + an, n Z}. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 9 / 18

19 Homogeneous case First, consider an homogenous Markov jump process (Z t ) t 0 with generator L[f ](z) = p(z)(f (z + a) f (z)) + q(z)(f (z a) f (z)), z R. It is reversible with respect to µ if p(z a)µ(z a) = q(z)µ(z), z R. Key observation: If Z 0 = x, then the process (Z t ) t 0 lives on the grid Γ a,x = {x + an, n Z}. The process (Z t ) t 0 cannot converge towards µ. Conditionnally to stay in Γ a,x, the process behaves a birth-death process, with birth rate (p(x + an)) n Z and death rate (q(x + an)) n Z. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 9 / 18

20 On the conditional birth-death process The µ-reversibility p(z a)µ(z a) = q(z)µ(z), z R, implies the reversibility of the conditional birth-death process with respect to the probability distribution µ a,x := 1 µ(x + an)δ x+an. Z a,x n Z Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 10 / 18

21 On the conditional birth-death process The µ-reversibility p(z a)µ(z a) = q(z)µ(z), z R, implies the reversibility of the conditional birth-death process with respect to the probability distribution µ a,x := 1 µ(x + an)δ x+an. Z a,x n Z The conditional birth-death process has good ergodicity properties if κ > 0, z 0, p(z) κ ; z 0, q(z) κ. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 10 / 18

22 Result in the homogeneous, reversible case Homogeneous reversible case, D., Gadat, Miclo 2017 Assume that (µ-reversibility) p(z a)µ(z a) = q(z)µ(z), z R, (Assumption on rates) κ > 0, z 0, p(z) κ ; z 0, q(z) κ. Then, there exists c, C, D > 0 such that for all t 0, W (L(Z t ), µ) c ( exp ( Cκa 2 t ) + Da ), t 0, where W stands for the Wasserstein distance of order 1 defined as W (ν, µ) = sup { ν(f ) µ(f ), f Lip 1 }. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 11 / 18

23 Comments on the theorem We find that W (L(Z t ), µ) c ( exp ( Cκa 2 t ) + Da ), t 0. W (L(Z t ), µ) does not converge towards 0 as t +. The term in exp ( Cκa 2 t ) comes from the ergodicity of the conditional birth-death process. The term in Da comes from the error made by integrating with respect to µ a,x instead of µ. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 12 / 18

24 1 Motivation: link with an online median-finding algorithm 2 Homogeneous case 3 Non-homogeneous case 4 Elements of proof 5 Perspectives Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 12 / 18

25 Non-homogeneous case Back to the non-homogeneous dynamic: for z R and t 0, Assumption: L t [f ](z) = p t (z)(f (z + a t ) f (z)) + q t (z)(f (z a t ) f (z)). (Decreasing jump size) a t 0. t + the process may converge to µ, but is still weakly mixing. // lack of strong convexity for the objective function in the median algorithm. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 13 / 18

26 Non-homogeneous, reversible case Back to the non-homogeneous dynamic: for z R and t 0, L t [f ](z) = p t (z)(f (z + a t ) f (z)) + q t (z)(f (z a t ) f (z)). Conjecture for the non-homogeneous reversible case Assume that (Decreasing jump size) a t (µ-reversibility) 0. t + p t (z a t )µ(z a t ) = q t (z)µ(z), z R, t 0. (Assumption on rates) λ > 0, z 0, p t (z) λ a 2 t ; z 0, q t (z) λ at 2. Then, we conjecture the existence of c, C, D > 0 such that for all t 0, W (L(Z t ), µ) c (exp ( Cλt) + Da t ), t 0. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 14 / 18

27 From the reversible to the asymptotically normal case The µ-reversibility assumption, δ(t, z) := p t (z a t )µ(z a t ) q t (z)µ(z)= 0, z R, t 0, is replaced by a strong normality assumption, δ > 0, δ(t, + a t ) δ(t, ) µ( ) δa t, t 0. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 15 / 18

28 Non-homogeneous, asymptotically normal case Conjecture for the non-homogeneous, asymptotically normal case Assume that (Jump size) a t = a 0, t 0, r > 1. (t + 1) r (Strong normality assumption) δ > 0, δ(t, + a t ) δ(t, ) µ( ) δa t, t 0. (Assumption on rates) λ > 0, z 0, p t (z) λ a 2 t ; z 0, q t (z) λ at 2. Then, we conjecture the existence of c, C, D > 0 such that for all t T, ( W (L(Z t ), µ) c exp ( Cλt) + D ) t r 1, t 0. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 16 / 18

29 On the conjectures Proved up to a technical point, coming from the need to control the evolution of the grid Γ at,x as a t 0. t + Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 17 / 18

30 1 Motivation: link with an online median-finding algorithm 2 Homogeneous case 3 Non-homogeneous case 4 Elements of proof 5 Perspectives Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 17 / 18

31 1 Motivation: link with an online median-finding algorithm 2 Homogeneous case 3 Non-homogeneous case 4 Elements of proof 5 Perspectives Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 17 / 18

32 Perspectives Investigation of the technical point. Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 18 / 18

33 Perspectives Investigation of the technical point. Back to the median algorithm: corresponds to p t (z) = 1 F (η tz) η 2 t, q t (z) = F (η tz) ηt 2, η t = η 0 (t + 1) β/(2(1 β)). Assumption at = a 0 (t + 1) r with r > 1 corresponds to β > 2/3: OK. Assumption on the rates λ > 0, z 0, p t (z) λ a 2 t ; z 0, q t (z) λ a 2 t OK. Strong normality assumption δ > 0, δ(t, + a t ) δ(t, ) µ( ) δa t, t 0 Simple condition on the distribution of the X i s? Claire Delplancke (IMT) Asymptotic study of a jump process Journées IOPS 18 / 18