Markov Processes and Applications

Transcription

1 Markov Processes and Applications Algorithms, Networks, Genome and Finance Etienne Pardoux Laboratoire d'analyse, Topologie, Probabilites Centre de Mathematiques et d'injormatique Universite de Provence, Marseille, France. WILEY A John Wiley and Sons, Ltd., Publication This work is in the Wiley-Dunod Series co-published between Dunod and John Wiley & Sons, Ltd.

2 Contents Preface xi 1 Simulations and the Monte Carlo method Description of the method Convergence theorems Simulation of random variables Variance reduction techniques Exercises 13 2 Markov chains Definitions and elementary properties Examples Random walk in E = iß Bienayme-Galton-Watson process A discrete time queue Strong Markov property Recurrent and transient states The irreducible and recurrent case The aperiodic case Reversible Markov chain Rate of convergence to equilibrium The reversible finite state case The general case Statistics of Markov chains Exercises 43 3 Stochastic algorithms Markov chain Monte Carlo An application The Ising model Bayesian analysis of images Heated chains Simulation of the invariant probability Perfect simulation 65

3 CONTENTS Coupling from the past Rate of convergence towards the invariant probability Simulated annealing Exercises 75 4 Markov chains and the genome Reading DNA CpG islands Detection of the genes in a prokaryotic genome The i.i.d. model The Markov model Application to CpG islands Search for genes in a prokaryotic genome Statistics of Markov chains Mk Phased Markov chains Locally homogeneous Markov chains Hidden Markov models Computation of the likelihood The Viterbi algorithm Parameter estimation Hidden semi-markov model Limitations of the hidden Markov model What is a semi-markov chain? The hidden semi-markov model The semi-markov Viterbi algorithm Search for genes in a prokaryotic genome Alignment of two sequences The Needleman-Wunsch algorithm Hidden Markov model alignment algorithm A posteriori probability distribution of the alignment A posteriori probability of a given match A multiple alignment algorithm Exercises Control and filtering of Markov chains Deterministic optimal control Control of Markov chains Ill 5.3 Linear quadratic optimal control Ill 5.4 Filtering of Markov chains The Kalman-Bucy filter Motivation Solution of the filtering problem Linear-quadratic control with partial observation Exercises 121

4 CONTENTS 6 The Poisson process Point processes and counting processes The Poisson process The Markov property Large time behaviour Exercises Jump Markov processes General facts Infinitesimal generator The strong Markov property Embedded Markov chain Recurrent and transient states The irreducible recurrent case Reversibility Markov models of evolution and phylogeny Models of evolution Likelihood methods in phylogeny The Bayesian approach to phylogeny Application to discretized partial differential equations Simulated annealing Exercises Queues and networks M/M/l queue M/M/l/K queue M/M/s queue M/M/s/s queue Repair shop Queues in series M/G/oo queue M/G/l queue An embedded chain The positive recurrent case Open Jackson network Closed Jackson network Telephone network Kelly networks Single queue Multi-class network Exercises Introduction to mathematical finance Fundamental concepts Option 206 vii

5 viii CONTENTS Arbitrage Viable and complete markets European options in the discrete model The model Admissible strategy Martingales Viable and complete market Call and put pricing The Black-Scholes formula The Black-Scholes model and formula Introduction to stochastic calculus Stochastic differential equations The Feynman-Kac formula The Black-Scholes partial differential equation The Black-Scholes formula (2) Generalization of the Black-Scholes model The Black-Scholes formula (3) Girsanov's theorem Markov property and partial differential equation Contingent claim on several underlying stocks Viability and completeness Remarks on effective computation Historical and implicit volatility American options in the discrete model Snell envelope Doob's decomposition Snell envelope and Markov chain Back to American options American and European options American options and Markov model American options in the Black-Scholes model Interest rate and bonds Future interest rate Future interest rate and bonds Option based on a bond An interest rate model Exercises Solutions to selected exercises Chapter Chapter Chapter Chapter Chapter Chapter 6 279

6 CONTENTS 10.7 Chapter Chapter Chapter Reference 295 Index 297 ix Notations The following notations will be used throughout this book. IN = {0, 1, 2,...} stands for the set of positive integers, including 0. IN* = {1, 2,...} stands for the set of positive integers, 0 excluded.