Overview. Transformation method Rejection method. Monte Carlo vs ordinary methods. 1 Random numbers. 2 Monte Carlo integration.

Transcription

1 Overview 1 Random numbers Transformation method Rejection method 2 Monte Carlo integration Monte Carlo vs ordinary methods 3 Summary

2 Transformation method Suppose X has probability distribution p X (x), take Y = f (X ). The probability of finding X in (x, x + dx) must be the same as the probability of finding Y in (y, y + dy), (where dy = f (x + dx) f (x)). p y (y)dy = p x (x)dx

3 Transformation method Suppose X has probability distribution p X (x), take Y = f (X ). The probability of finding X in (x, x + dx) must be the same as the probability of finding Y in (y, y + dy), (where dy = f (x + dx) f (x)). p y (y)dy = p x (x)dx Since p x (x), p Y (y) are both positive, this gives us p Y (y) f (x)dx = p x (x) dx = p Y (y) = p x(x) f (x) with f (x) = df (x) dx.

4 Obtaining a specific distribution We want a certain p y (y). What is y = f (x) if x is uniform? Inverting this gives us 1 f (x) = dx dy = x = = p(y) = dx = p(y)dy y y(x) = P 1 (x) We can find the transformation function if p(z)dz P(y) 1 we can integrate our distribution cumulative distribution P(y) 2 we can invert the cumulative distribution function analytically Very useful for scaling and shifting distributions

5 Shifting and scaling The transformation method is useful for shifting and scaling distributions: Y = X /a + b = p Y (y) = ap X (x) = ap X ( a(y b) ) Example X is gaussian with average 0 and variance 1. We want Y to be gaussian with average µ and variance σ 2, p Y (y) = 1 σ /2σ2 e (y µ)2 2π

6 Shifting and scaling The transformation method is useful for shifting and scaling distributions: Y = X /a + b = p Y (y) = ap X (x) = ap X ( a(y b) ) Example X is gaussian with average 0 and variance 1. We want Y to be gaussian with average µ and variance σ 2, We achieve this by Y = σx + µ p Y (y) = 1 σ /2σ2 e (y µ)2 2π

7 Shifting and scaling The transformation method is useful for shifting and scaling distributions: Y = X /a + b = p Y (y) = ap X (x) = ap X ( a(y b) ) Example X is gaussian with average 0 and variance 1. We want Y to be gaussian with average µ and variance σ 2, We achieve this by Y = σx + µ We can understand this intuitively: p Y (y) = 1 σ /2σ2 e (y µ)2 2π Multiplying by σ stretches or squeezes the distribution Adding µ shifts everything to the left or right

8 Gaussian random numbers We want a gaussian distribution P(y) = 1 2π e y 2 /2 But we have no closed expression for P(y)dy we certainly have no closed expression for the inverse! [Python s scipy package has functions erf and erfinv providing numerical approximations (in the scipy.special module).] A way around We can integrate two gaussian distributions, P(y 1, y 2 ) = 1 2π e (y 2 1 +y 2 2 )/2 = P(y 1 )P(y 2 )

9 Gaussian random numbers Transformation of two or more probability distributions: (x 1, x 2,...) P Y (y 1, y 2,...) = P X (x 1, x 2,...) (y 1, y 2,...) If we take X 1, X 2 uniform on 0, 1 and Y 1 = 2 ln X 1 cos(2πx 2 ) Y 2 = 2 ln X 1 sin(2πx 2 ) then Y 1, Y 2 are gaussian Note Python s numpy package has an inbuilt gaussian rng np.random.randn use the method above if you program in C/C++/Fortran/Java Avoid calls to ln, sin, cos,... as much as possible: they are slow!

10 Rejection method What if we cannot integrate or invert?

11 Rejection method What if we cannot integrate or invert? The shaded blue area is the prob of finding X between x and x + dx

12 Rejection method What if we cannot integrate or invert? The shaded blue area is the prob of finding X between x and x + dx Can we find a way of picking (X, Y ) uniformly distributed under the blue curve?

13 Rejection method What if we cannot integrate or invert? The shaded blue area is the prob of finding X between x and x + dx Can we find a way of picking (X, Y ) uniformly distributed under the blue curve? Assume we can do this for f (x). The ratio of areas is p(x)/f (x)

14 Rejection method What if we cannot integrate or invert? The shaded blue area is the prob of finding X between x and x + dx Can we find a way of picking (X, Y ) uniformly distributed under the blue curve? Assume we can do this for f (x). The ratio of areas is p(x)/f (x) Idea: Select points under the f (x) curve, reject those in the red shaded area

15 Rejection method Implementation 1 Pick uniform Z 0, A ; A = f (x)dx 2 Find X = F 1 (Z) 3 Pick random uniform Y 0, 1 4 If Y < p(x )/f (X ) then accept X as your random number else reject X and try again

16 Rejection method Implementation 1 Pick uniform Z 0, A ; A = f (x)dx 2 Find X = F 1 (Z) 3 Pick random uniform Y 0, 1 4 If Y < p(x )/f (X ) then accept X as your random number else reject X and try again Note: If p(x) is normalised, f (x) is not normalised. A is the total area under the curve, so f (x)/a is normalised F (x) = x f (ξ)dξ is the cumulative probability or the area under the curve up to x. Computing x = F 1 (z) is the transformation method for f (x) or f (x)/a. If you already have a generator for f (x)/a you can apply it directly.

17 Rejection method Simplest version: f (x) = const = sup p(x) just choose a uniform random number X x min, x max

18 Rejection method Simplest version: f (x) = const = sup p(x) just choose a uniform random number X x min, x max will not work when X is unbounded can have very high rejection rate for peaked distributions

19 Rejection method Simplest version: f (x) = const = sup p(x) just choose a uniform random number X x min, x max will not work when X is unbounded can have very high rejection rate for peaked distributions In physics we often have smallish perturbations on gaussian or exponential Example Generate a pseudo-random number with the distribution p(x) e x 1 + x 2, x > 0, assuming we already have a generator for the exponential distribution. Algorithm: 1 Generate an exponentially distributed number z. 2 Generate a standard uniform deviate u. 3 If u < 1/(1 + z 2 ), set x = z, otherwise go back to 1. 4 Repeat this to generate as many numbers x as you require.

20 Monte Carlo methods Monte Carlo is a town on the Mediterranean famous for its casinos

21 Monte Carlo methods Monte Carlo is a town on the Mediterranean famous for its casinos Monte Carlo methods are numerical methods based on random numbers

22 Monte Carlo methods Monte Carlo is a town on the Mediterranean famous for its casinos Monte Carlo methods are numerical methods based on random numbers Throw the dice, spin the roulette wheel, rake in the cash

23 Monte Carlo methods Monte Carlo is a town on the Mediterranean famous for its casinos Monte Carlo methods are numerical methods based on random numbers Throw the dice, spin the roulette wheel, rake in the cash Three main types Direct Monte Carlo Monte Carlo integration Markov chain Monte Carlo

24 Monte Carlo methods Monte Carlo is a town on the Mediterranean famous for its casinos Monte Carlo methods are numerical methods based on random numbers Throw the dice, spin the roulette wheel, rake in the cash Three main types Direct Monte Carlo model complicated or unknown processes by random numbers Stochastic dynamics, eg Brownian motion or traffic modelling Monte Carlo integration Markov chain Monte Carlo

25 Monte Carlo methods Monte Carlo is a town on the Mediterranean famous for its casinos Monte Carlo methods are numerical methods based on random numbers Throw the dice, spin the roulette wheel, rake in the cash Three main types Direct Monte Carlo model complicated or unknown processes by random numbers Stochastic dynamics, eg Brownian motion or traffic modelling Monte Carlo integration calculate integrals using random numbers especially useful in many dimensions Markov chain Monte Carlo

26 Monte Carlo methods Monte Carlo is a town on the Mediterranean famous for its casinos Monte Carlo methods are numerical methods based on random numbers Throw the dice, spin the roulette wheel, rake in the cash Three main types Direct Monte Carlo model complicated or unknown processes by random numbers Stochastic dynamics, eg Brownian motion or traffic modelling Monte Carlo integration calculate integrals using random numbers especially useful in many dimensions Markov chain Monte Carlo generate statistical distributions using random walks a stalwart of many-particle physics

27 Monte Carlo integration We want to integrate a function f on interval [a, b]: I = b a f (x)dx Standard numerical integration (previous semester): divide [a, b] into N evenly spaced points x i evaluate f (x) of those points I = b a N N w i f (x i ) i=0

28 Monte Carlo integration We want to integrate a function f on interval [a, b]: I = b a f (x)dx Standard numerical integration (previous semester): divide [a, b] into N evenly spaced points x i evaluate f (x) of those points I = b a N N w i f (x i ) i=0 Monte Carlo integration Pick the points x i randomly!

29 But why?? It works! Take X to be uniformly distributed on [a, b] : p X (x) = 1/(b a)

30 But why?? It works! Take X to be uniformly distributed on [a, b] : p X (x) = 1/(b a) Then the expectation value of f (X ) is f p(ξ)f (ξ)dξ = 1 b f (ξ)dξ = b a a I b a

31 But why?? It works! Take X to be uniformly distributed on [a, b] : p X (x) = 1/(b a) Then the expectation value of f (X ) is Advantages f p(ξ)f (ξ)dξ = 1 b f (ξ)dξ = b a a I b a It works in arbitrary numbers of dimensions: fdv = V f It works for complicated boundaries

32 Computing π π is the area of the unit circle x 2 + y 2 < 1 π 4 is the area of the quadrant x 2 + y 2 < 1; x, y 0, 1

33 Computing π π is the area of the unit circle x 2 + y 2 < 1 π 4 is the area of the quadrant x 2 + y 2 < 1; x, y 0, 1 This can be written as a 2-dimensional integral: π 4 = = Θ ( 1 (x 2 + y 2 ) ) dxdy Θ ( 1 (x 2 + y 2 ) ) dxdy/ dxdy

34 Computing π π is the area of the unit circle x 2 + y 2 < 1 π 4 is the area of the quadrant x 2 + y 2 < 1; x, y 0, 1 This can be written as a 2-dimensional integral: π 4 = = Θ ( 1 (x 2 + y 2 ) ) dxdy Θ ( 1 (x 2 + y 2 ) ) dxdy/ dxdy Procedure 1 Generate N pairs of random numbers (x, y) 2 Add 1 each time x 2 + y 2 < 1 3 Divide by N to get the average 4 Multiply by 4, and you have π!

35 Error estimates The variance of a stochastic variable X is σ 2 X var X (X X ) 2 = X 2 X 2

36 Error estimates The variance of a stochastic variable X is σ 2 X var X (X X ) 2 = X 2 X 2 For our MC integral we get (b a ) 2 b a 2 σ 2 = f i f i N N i i (b [ a)2 1 = fi f i f j N f 2] N N N = = (b ( a)2 f N N (b ( a)2 f 2 + N i i j fi fj N f 2) i j N(N 1) f 2 N f 2) = N (b a)2 ( f 2 f 2) N In going from the second to the third line we have used that f i and f j are independent and uncorrelated.

37 Advantages in many dimensions In general: σ 2 = V 2 N ( f 2 f 2) the error decreases as 1 N

38 Advantages in many dimensions In general: σ 2 = V 2 N ( f 2 f 2) the error decreases as 1 N Standard numerical integration gives errors powers of the grid spacing, err δ 2 1 N, δ3 1 N 3,... Adding more points gives much faster improvement in accuracy than MC Not good for Monte Carlo in one dimension, but...

39 Advantages in many dimensions In general: σ 2 = V 2 N ( f 2 f 2) the error decreases as 1 N Standard numerical integration gives errors powers of the grid spacing, err δ 2 1 N, δ3 1 N 3,... Adding more points gives much faster improvement in accuracy than MC Not good for Monte Carlo in one dimension, but... In d dimensions N ( L δ )d so err N k/d The Monte Carlo error is still err N 1/2

40 Summary Random number distributions Transformation method when desired distribution integrable and invertible useful for scaling and shifting distributions

41 Summary Random number distributions Transformation method when desired distribution integrable and invertible useful for scaling and shifting distributions Rejection method works for any distribution

42 Summary Random number distributions Transformation method when desired distribution integrable and invertible useful for scaling and shifting distributions Rejection method works for any distribution Monte Carlo integration Integrate functions by randomly sampling points Errors decrease as 1/ N Superior for high-dimensional integrals