Stochastic Dynamical Systems and SDE s An Informal Introduction Olav Kallenberg Graduate Student Seminar, April 18, 2012 1 / 33
2 / 33
Simple recursion: Deterministic system, discrete time x n+1 = f (x n ), n = 0, 1,... More generally: x n+1 = f n (x 0, x 1,..., x n ) Reduction to the previous case: y n = (n, x 0, x 1,..., x n ) 3 / 33
Recursion with randomness (Markov chain) Recursion: X n+1 = f (X n, ξ n+1 ), n = 0, 1,... with ξ 1, ξ 2,... independent randomization variables More generally: X n+1 = f n (X 0,..., X n ; ξ 1,..., ξ n+1 ) Reduction to previous case: Y n = (n, X 0,..., X n ; ξ 1,..., ξ n ) 4 / 33
Recursion in terms of increments Recursion: X n = X n+1 X n = f (X n, ξ n+1 ) X n = g(x n, ξ n+1 ) Summation: n 1 X n = X 0 + g(x k, ξ k+1 ) k=0 5 / 33
Space-homogeneous case (random walk) Assume g(x, u) is independent of x : X n = g(ξ n+1 ) = η n+1 Here η 1, η 2,... are i.i.d. random variables. Distribution: ν(b) = P{η B} = P{g(ξ) B} Summation: X n = X 0 + η 1 + + η n (Return to general case...) 6 / 33
7 / 33
Transition kernel, Markov property Conditional distribution: µ(x, B) = P[ X n+1 B X n = x ] = P{f (x, ξ) B} Conditioning on (X 0,..., X n ) gives same result! Conditional independence: (X 0,..., X n 1 ) (X n+1, X n+2,... ) X n In words: (past) (future) (present) 8 / 33
9 / 33
Dynamical system, continuous time Simple recursion: x t+h = f h (x t ), t, h 0 Iteration: x t+h+k = f k (x t+h ) = f k f h (x t ), t, h, k 0 Semigroup property: f s+t = f s f t, s, t 0 10 / 33
11 / 33
Differential equation Increments: Now let h 0: x t+h x t h In differential form: = f h(x t ) f 0 (x t ) h dx t dt = b(x t), t 0 dx t = b(x t ) dt = f h f 0 h (x t ) 12 / 33
Dynamical system with randomness (Markov process) Recursion: Transition kernel: X t+h = f h (X t, ξ t+h t ), t, h 0 µ h (x, B) = P[ X t+h B X t = x ] = P{f h (x, ξ) B} Markov property: {X s, s < t} {X u, u > t} Semigroup property: X t µ s+t = µ s µ t, s, t 0 Generator: µ h µ 0 h A 13 / 33
Space-homogeneous case (Lévy process) Stationary, independent increments: Increment distributions: Semigroup property: Paths may have jumps... X t+h X t = g h (ξ t+h t ), t, h 0 ν h (B) = P{g h (ξ) B}, h 0 ν s+t = ν s ν t, s, t 0 14 / 33
15 / 33
16 / 33
17 / 33
Continuous paths (Brownian motion) By central limit theorem and semigroup property: X t N(b t, σ 2 t), t 0 for some constants: b drift σ diffusion rate Now take b = 0 and σ = 1 (standardize): B t N(0, t), t 0 Then in general: X t = b t + σb t, t 0 In differential form: dx t = b dt + σdb t 18 / 33
19 / 33
Stochastic differential equation, diffusion processes Now let b and σ depend on location: dx t = b(x t ) dt + σ(x t ) db t In integrated form: t t X t = X 0 + b(x s ) ds + σ(x s ) db s 0 0 The first integral is elementary. The second is not, since B is: nowhere differentiable has unbounded variation has extremely irregular paths How to make sense of this? 20 / 33
Itô stochastic integrals and SDE s We can define the process: (Y B) t = t 0 Y s db s, t 0 in a weak probabilistic sense, provided that: Y is non-anticipating (depends only on the past) t 0 Y 2 s ds <, t 0 Then we can prove existence and uniqueness of solutions to: or more generally: dx t = b(x t ) dt + σ(x t ) db t dx t = b(t, X ) dt + σ(t, X ) db t for suitable functions b and σ. 21 / 33
Itô s formula, stochastic calculus Semimartingale decomposition: X t = X 0 + t 0 σ s (X ) db s + t M t martingale part (centered process) A t compensator (drift component) 0 b s (X ) ds = X 0 + M t + A t Itô s formula (transformation of semimartingales): f (X t ) = f (X 0 ) + t t 0 f (X s ) dx s + 1 2 where [X ] t denotes the quadratic variation of X t. 0 f (X s ) d[x ] s In ordinary calculus, [X ] t 0, and the last term vanishes. 22 / 33
Reductions to Brownian motion 1. Diffusion continuous martingale: X t diffusion M t = f (X t ) a continuous martingale for a suitable function f 2. Continuous martingale Brownian motion: M t continuous martingale B t = M τ t a Brownan motion for a suitable process τ t Similarly, any point process can be time-changed into a Poisson process. Thus, Brownian motion and Poisson processes emerge as the basic building blocks of stochastic processes. 23 / 33
24 / 33
25 / 33
Connection: Brownian motion has generator 1 2. 26 / 33 Probability and potential theory Diffusion (or heat) equation u = 1 u, or: 2 Fundamental solution: 2 u t = 2 u x1 2 + 2 u x2 2 + 2 u x3 2 u(x, t) = (2πt) 3/2 exp( x 2 /2t) This is also the probability density of B t N(0, t). The PDE describes the average motion of a huge number of particles. Brownian motion describes the individual motion of each particle.
27 / 33
Some early history Bachelier (1900 01) random walk, Brownian motion Markov (1906) Markov property, Markov chains Wiener (1923) existence of Brownian motion Bernstein (1927 37) martingale property Kolmogorov (1933 35) conditioning, Markov processes Lévy (1934 48) Brownian paths, Lévy processes Feller (1936 54) Markov semigroups and generators Doob (1940 53) modern martingale theory Itô (1942 51) stochastic integration and SDE s Dynkin (1955 56) modern Markov process theory 28 / 33
Glossary of probability terms stochastic model involving randomness stochastic process randomly evolving function probability theory study of stochastic processes Markov chain recursion involving randomness random walk space-homogeneous random recursion Markov process stochastic dynamical system diffusion continuous Markov process Brownian motion space-homogeneous diffusion 29 / 33
Glossary of probability terms (continued) Lévy process space-homogeneous Markov process Poisson process independent-increment point process Itô integral stochastic integral w.r.t. Brownian motion SDE stochastic differential equation martingale process centered to have drift zero potential theory PDE-theory involving the Laplacian semigroup functions f t satisfying f s+t = f s f t 30 / 33
31 / 33
Basic graduate course in probability Discrete time (fall): elements of measure theory random variables and processes conditioning, independence, 0 1 laws strong limits, law of large numbers weak limits, central limit theorem martingales Markov property and chains Poisson and related processes stationary processes and ergodic theory 32 / 33
Basic graduate course in probability (continued) Continuous time (spring): random walk, Brownian motion Skorohod embedding, weak convergence Markov processes and semigroups Itô integrals and calculus stochastic differential equations continuous-time martingales change of time, space, measure 33 / 33