MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.65/15.070J Fall 013 Lecture 11 10/9/013 Martingales and stopping times II Content. 1. Second stopping theorem.. Doob-Kolmogorov inequality. 3. Applications of stopping theorems to hitting times of a Brownian motion. 1 Second stopping theorem In the previous lecture we established no gambling scheme can produce a positive gain in expectation if there is a limit on the number of rounds the game is played. We now establish a similar result assuming that there is a limit on the amount of wealth possessed by the player. Theorem 1. Suppose X n is a supermartingale that is uniformly bounded. That is X n M a.s., for some M. Suppose τ is a stopping time. Then E[X τ ] E[X 0 ]. If, in addition X n is a martingale, then E[X τ ] = E[X 0 ]. The gambling interpretation of this theorem is as follows: suppose we tried to use the double the stakes algorithm, which we know guarantees winning a dollar, when there are no restrictions. But now suppose that there is a limit on how negative we can go (for example the amount of debt allowed). Say this limit is M. Consider a modified process Y n = X n τ. Then from our description M Y n 1 < M. Also we remember from the previous lecture that Y n is a supermartingale. Thus it is a bounded supermartingale. Theorem 1 then tells us that E[Y τ ] = E[X τ ] E[X 0 ], namely the scheme does not work anymore. (Remember that without the restriction, the scheme produces wealth X τ = 1 with probability one. In particular, E[X τ ] = 1 > X 0 = 0). Proof of Theorem 1.. Observe, that E[ X τ ] M <. Consider Y n = X n τ. Then Y n converges to X τ a.s. as n. Since Y n M a.s., then 1
using the Bounded Convergence Theorem, lim n E[Y n ] = E[X τ ]. On the other hand, we established in Corollary 1 in the previous lecture, that Y n is a supermartingale. Therefore E[Y n ] E[Y 0 ] = E[X 0 ]. Combining, we obtain E[X τ ] E[X 0 ]. Doob-Kolmogorov inequality We now establish a technical but a very useful result, which is an analogue of Markov/Chebyshev s bounds on random variables. Theorem (Doob-Kolmogorov inequality). Suppose X n is a non-negative submartingale adapted to {F n } and E > 0. Then for every n N E[X ] n P( max X n E). E If X n is a martingale, then the non-negativity condition can be dropped. The convenience of this result is that we can bound the worst case deviation of a submartingale using its value at the end of time interval. Proof. Using Jensen s inequality we established that if X n is a martingale then X n is a submartingale. Since X n is non-negative, the second part follows from the first. To establish the first part, consider the events A = {max X m E} and B m = {max 1 i m1 X i E, X m > E}. Namely, A is the event that the submartingale never exceeds E and B m is the event that it does so at time m for the first time. We have Ω = A B m and the events A, B m are mutually exclusive. Then Note E[X ] = E[X 1{A}] + E[X 1{B m }] E[X 1{B m }]. n n n n E[X 1{B m }] = E[(X n X m + X m ) 1{B m }] n = E[(X n X m ) 1{B m }] + E[(X n X m )X m 1{B m }] + E[X 1{B m }] m The first of the summands is non-negative. The last is at least E P(B m ), since on the event B m we have X m > E. We now analyze the second term and
here we use the tower property: E[(X n X m )X m 1{B m }] = E[E[(X n X m )X m 1{B m } F m ]] = E[X m 1{B m }E[(X n X m ) F m ]] 0, where the second equality follows since 1{B m } F m and the last inequality follows since X n is a submartingale and X m 1{B m } E > 0 on ω B m and = 0 on ω / B m. We conclude that E[X ] E P(B m ) = E P( m B m ) = E P( max X m > E). n This concludes the proof. Corollary 1. Suppose X n is a martingale and p 1. Then for every E > 0 E[ X n p ] P( max X n E). Proof. The proof of the general case is more complicated, but when p we almost immediately obtain the result. Using conditional Jensen s inequality we know that X n is a submartingale, as is a convex function. It is also nonis convex increasing when p and x 0. Recall from p negative. Function x p the previous lecture that this implies X n is also a submartingale. Applying Theorem we obtain P( max X n E) = P( max X n E p p p E ) E[ X n p ] E p. Analogue of this corollary holds in continuous time when the process is continuous. We just state this result without proving it. Theorem 3. Suppose {X t } t R+ is a martingale which has a.s. continuous sample paths. Then for every p 1, T > 0, E > 0, E[ X T p ] P( sup X t E). 0 t T E p 3
3 Applications to hitting times of a Brownian motion We now use the martingale theory and optional stopping theorems to derive some properties of hitting times of a Brownian motion. Our setting is either a standard Brownian motion B(t) or a Brownian motion with drift B µ (t) = µt + σb(t). In both cases the starting value is assumed 0. We fix a < 0 < b and ask the question: what is the probability that B µ (t) hits a before b? For simplicity we use B instead of B µ, but mention that we talk about Brownian motion with drift. We define T a = inf{t : B µ (t) = a}, T b = inf{t : B µ (t) = b}, T ab = min(t a, T b ). In Lecture 6, Problem 1 we established that when B is standard, lim sup t B(t) = a.s. Thus T b < a.s. By symmetry, T a < a.s. Now we ask the question: what is the probability P(T ab = T a )? We will use the optional stopping theorems established before. The only issue is that we are now dealing with continuous time processes. The derivations of stopping theorems require more details (for example defining predictable sequences is trickier). We skip the details and just assume that optional stopping theorems apply in our case as well. The case of the standard Brownian motion is the simplest. Theorem 4. Let T a, T b, T ab be defined with respect to the standard Brownian motion B(t). Then b P(T ab = T a ) =. a + b Proof. Recall that B is a martingale. Observe that T ab defines a stopping time: the event {T ab t} B t (stopping T ab t is determined completely by the path of the Brownian motion up to time t). Therefore by Corollary 1 in the previous lecture, Y t B(t T ab ) is also a martingale. Note that it is a bounded martingale, since its absolute value is at most max( a, b ). Theorem 1 applied to Y t then implies that E[Y Tab ] = E[B(T ab )] = E[Y 0 ] = E[B(0)] = 0. On the other hand, when T ab = T a, we have B(T ab ) = B(T a ) = a and, conversely, when T ab = T b, we have B(T ab ) = B(T b ) = b. Therefore E[B(T ab )] = ap(t ab = T a ) + bp(t ab = T b ) = a P(T ab = T a ) + b P(T ab = T b ) Since P(T ab = T a )+P(T ab = T b ) = 1, then, combining with the fact E[B(T ab )] we obtain b a P(T ab = T a ) =, P(T ab = T b ) =. a + b a + b 4
We now consider the more difficult case, when the drift of the Brownian motion µ = 0. Specifically, assume µ < 0. Recall, that in this case lim t B(t) = a.s., so T ab T a < a.s. Again we want to compute P(T ab = T a ). 1 We fix drift µ < 0, variance σ > 0 and consider q(β) = µβ + σ β. Proposition 1. For every β, the process V (t) = e βb(t)q(β)t is a martingale. Proof. We first need to check that E[ V (t) ] <. We leave it as an exercise. We have for every 0 s < t β(b(t)b(s)) q(β)(ts) E[V (t) B s ] = E[e e e βb(s)q(β)s B s ] β(b(t)b(s)) q(β)(ts) βb(s)q(β)s = E[e ]e e q(β)(ts) E[e β(b(t)b(s)) ]V (s). = e where the second equality follows from the ind. increments property of the Brownian motion, and from the fact E[e βb(s)q(β)s B βb(s)q(β)s s ] = e. Since d B(t) B(s) = N(µ(t s), σ (t s)), then E[e β(b(t)b(s)) ] is the Laplace transform of this normal r.v. which is βµ(ts)+ 1 σ β (ts) q(β)(ts) e = e. Combining, we obtain that E[V (t) B s ] = V (s). Therefore V (t) is a martingale. Now that we know that V (t) is a martingale, we can try to apply the optional stopping theorem. For that we need to have a stopping time, and we will use T ab. We also need conditions for which the expected value at the stopping time is equal to the expected value at time zero. We use the following observation. Suppose β is such that q(β) 0. Then 0 V (t T ab ) e βb a.s. Indeed, the left side of the inequality follows trivially from non-negativity of V. For the righthand side, observe that for t T β a βb ) βb ab we have V (t) max(e, e = e, and the assertion follows. Thus V is a.s. a bounded martingale. Again we use Theorem 1 to conclude that We now set β = µ/σ. Then q(β) = 0. Note that E[V (T ab )] = V (0) = 1. (1) V (T µa ab )1{T ab = T a } = e σ 1{T ab = T a } V (T µb ab )1{T ab = T b } = e σ 1{T ab = T b } 5
The previous identity gives µa µb σ σ 1 = E[V (T ab )] = e P(T ab = T a ) + e P(T ab = T b ). From this and using µ < 0, we recover P(T ab = T b ) = µa µ a σ σ 1 e 1 e =. µb µa µ b µ a e σ e σ e σ e σ Compared with the driftless case, the probability of hitting b first is exponentially tilted. Now let us take a. The events A a = {T ab = T a } are monotone: A t a A a ' for a < a < 0. Therefore µ a [ ] 1 e σ P( µ b a<0 A a ) = lim P(A a ) = lim 1 = 1 e σ. µ b µ a a a e σ e σ But what is the event a<0 A a? Since Brownian motion has continuous paths, this event is exactly the event that the Brownian motion never hits the positive level b. That is the event sup t 0 B(t) < b. We conclude that when the drift µ of the Brownian motion is negative µ b σ P(sup B(t) b) = e. t 0 Recall, from Lecture 6, that we already established this fact directly from the properties of the Brownian motion the supremum of a Brownian motion with a negative drift has an exponential distribution with parameter µ /σ. 4 Additional reading materials Durrett [1] Chapter 4. Grimmett and Stirzaker [] Section 7.8. References [1] R. Durrett, Probability: theory and examples, Duxbury Press, second edition, 1996. [] G. R. Grimmett and D. R. Stirzaker, Probability and random processes, Oxford University Press, 005. 6
MIT OpenCourseWare http://ocw.mit.edu 15.070J / 6.65J Advanced Stochastic Processes Fall 013 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.