Monte Carlo Methods for Uncertainty Quantification

Transcription

1 Monte Carlo Methods for Uncertainty Quantification Abdul-Lateef Haji-Ali Based on slides by: Mike Giles Mathematical Institute, University of Oxford Contemporary Numerical Techniques Haji-Ali (Oxford) Monte Carlo methods 1 / 27

2 Lecture outline Lecture 4: PDE applications PDEs with uncertainty examples multilevel Monte Carlo Extensions Haji-Ali (Oxford) Monte Carlo methods 2 / 27

3 PDEs with Uncertainty Looking at the history of numerical methods for PDEs, the first steps were about improving the modelling: 1D 2D 3D steady unsteady laminar flow turbulence modelling large eddy simulation direct Navier-Stokes simple geometries (e.g. a wing) complex geometries (e.g. an aircraft in landing configuration) adding new features such as combustion, coupling to structural / thermal analyses, etc.... and then engineering switched from analysis to design. Haji-Ali (Oxford) Monte Carlo methods 3 / 27

4 PDEs with Uncertainty The big move now is towards handling uncertainty: uncertainty in modelling parameters uncertainty in geometry uncertainty in initial conditions uncertainty in spatially-varying material properties inclusion of stochastic source terms Engineering wants to move to robust design taking into account the effects of uncertainty. Other areas want to move into Bayesian inference, starting with an a priori distribution for the uncertainty, and then using data to derive an improved a posteriori distribution. Haji-Ali (Oxford) Monte Carlo methods 4 / 27

5 PDEs with Uncertainty Range of applications: parabolic, elliptic, hyperbolic, numerical homogenization, reduced basis approximation. Examples: Long-term climate modelling: Lots of sources of uncertainty including the effects of aerosols, clouds, carbon cycle, ocean circulation ( Short-range weather prediction Considerable uncertainty in the initial data due to limited measurements Haji-Ali (Oxford) Monte Carlo methods 5 / 27

6 PDEs with Uncertainty Engineering analysis Perhaps the biggest uncertainty is geometric due to manufacturing tolerances Nuclear waste repository and oil reservoir modelling Considerable uncertainty about porosity of rock Astronomy Random spatial/temporal variations in air density disturb correlation in signals received by different antennas Finance Stochastic forcing due to market behaviour Haji-Ali (Oxford) Monte Carlo methods 6 / 27

7 PDEs with Uncertainty In the past, Monte Carlo simulation has been viewed as impractical due to its expense, and so people have used other methods: stochastic collocation polynomial chaos Because of Multilevel Monte Carlo, this is changing and there are now several research groups using MLMC for PDE applications The approach is very simple, in principle: use a sequence of grids of increasing resolution in space (and time) as with SDEs, determine the optimal allocation of computational effort on the different levels the savings can be much greater because the cost goes up more rapidly with level (higher dimensionality). Haji-Ali (Oxford) Monte Carlo methods 7 / 27

8 PDEs with Uncertainty: Example Elliptic PDE coming from Darcy s law: ( ) κ(x) p = 0 where the permeability κ(x) is uncertain, and log κ(x) is often modelled as being Normally distributed with a spatial covariance such as R(log κ(x 1 ), log κ(x 2 )) = σ 2 exp( x 1 x 2 /λ) Modelling of oil reservoirs and groundwater contamination in nuclear waste repositories (Giles, Scheichl, Cliffe, 2011). Haji-Ali (Oxford) Monte Carlo methods 8 / 27

9 Stochastic Field A typical realisation of κ for λ = 0.01, σ = 1. Haji-Ali (Oxford) Monte Carlo methods 9 / 27

10 Stochastic Field Samples of log κ are provided by a Karhunen-Loève expansion: log κ(x, ω) = θn ξ n (ω) f n (x), n=0 where θ n, f n are eigenvalues / eigenfunctions of the correlation function: R(x, y) f n (y) dy = θ n f n (x) and ξ n (ω) are standard Normal random variables. Numerical experiments truncate the expansion. (Latest 2D/3D work uses an efficient FFT construction based on a circulate embedding.) Haji-Ali (Oxford) Monte Carlo methods 10 / 27

11 Stochastic Field Decay of 1D eigenvalues λ=0.01 λ=0.1 λ=1 eigenvalue n When λ = 1, can use a low-dimensional polynomial chaos approach, but it s impractical for smaller λ. Haji-Ali (Oxford) Monte Carlo methods 11 / 27

12 PDEs with Uncertainty: Results Boundary conditions for unit square [0, 1] 2 : fixed pressure: p(0, x 2 )=1, p(1, x 2 )=0 Neumann b.c.: p/ x 2 (x 1, 0)= p/ x 2 (x 1, 1)=0 Output quantity mass flux: Correlation length: λ = 0.2 k p x 1 dx 2 Coarsest grid: h = 1/8 (comparable to λ) Finest grid: h = 1/128 Karhunen-Loève truncation: m KL = 4000 Cost taken to be proportional to number of nodes. h 2 l Haji-Ali (Oxford) Monte Carlo methods 12 / 27

13 2D Results log 2 variance 4 6 log 2 mean P l P l P l 1 10 P l P l P l level l level l V[ P l P l 1 ] h 2 l E[ P l P l 1 ] h 2 l Haji-Ali (Oxford) Monte Carlo methods 13 / 27

14 MLMC General Theorem If there exist independent estimators Ŷl based on N l Monte Carlo samples, each costing C l, and positive constants α, β, γ, c 1, c 2, c 3 such that α 1 2 min(β, γ) and i) E[ P l P] c 1 2 α l E[ P 0 ], l = 0 ii) E[Ŷl] = E[ P l P l 1 ], l > 0 iii) V[Ŷl] c 2 N 1 l 2 β l iv) E[C l ] c 3 2 γ l Haji-Ali (Oxford) Monte Carlo methods 14 / 27

15 MLMC General Theorem then there exists a positive constant c 4 such that for any ε<1 there exist L and N l for which the multilevel estimator Ŷ = L Ŷ l, l=0 [ (Ŷ ) ] 2 has a mean-square-error with bound E E[P] < ε 2 with a computational cost C with bound c 4 ε 2, β > γ, C c 4 ε 2 (log ε) 2, β = γ, c 4 ε 2 (γ β)/α, 0 < β < γ. Haji-Ali (Oxford) Monte Carlo methods 15 / 27

16 2D Results ε= ε=0.001 ε=0.002 ε=0.005 ε= Std MC MLMC N l ε 2 Cost level l accuracy ε Haji-Ali (Oxford) Monte Carlo methods 16 / 27

17 Complexity analysis Relating things back to the MLMC theorem: E[ P l P] 2 2l = α = 2 V l 2 2l = β = 2 C l 2 dl = γ = d (dimension of PDE) To achieve r.m.s. accuracy ε requires finest level grid spacing h ε 1/2 and hence we get the following complexity: dim MC MLMC 1 ε 2.5 ε 2 2 ε 3 ε 2 (log ε) 2 3 ε 3.5 ε 2.5 Haji-Ali (Oxford) Monte Carlo methods 17 / 27

18 Non-geometric multilevel Almost all applications of multilevel in the literature so far use a geometric sequence of levels, refining the timestep (or the spatial discretisation for PDEs) by a constant factor when going from level l to level l + 1. Coming from a multigrid background, this is very natural, but it is NOT a requirement of the multilevel Monte Carlo approach. All MLMC needs is a sequence of levels with increasing accuracy increasing cost increasingly small difference between outputs on successive levels Haji-Ali (Oxford) Monte Carlo methods 18 / 27

19 Applying MLMC to your problems First, identify the approximation parameter(s) in the problem to build a hierarchy of approximations. We also need to make sure that we can to sample correlated approximations. Try to find estimators that increase the variance convergence rate, β, as much as possible to get the optimal complexity. The MLMC/MIMC theory depends on a set of assumptions. Checking these assumptions numerically is straightforward but proving them can be challenging. Haji-Ali (Oxford) Monte Carlo methods 19 / 27

20 Multilevel Monte Carlo: Summary Let l P = Pl P l 1 and 0 = P 0 and observe that E[P] = Then, the MLMC estimator E[ l P] l=0 L 1 N l N l l=0 m=1 L E[ l P]. l=0 l P(l,m) l with optimal choice of L and N l has complexity γ β 2 max(0, O (ε α ) ), ( when γ β and O ε 2 log ε 2) otherwise. Haji-Ali (Oxford) Monte Carlo methods 20 / 27

21 Multilevel Monte Carlo for high-dimensional problems The problem is that the cost usually grows exponentially as the number of discretization parameters, d, increases, i.e., the cost is O ( 2 dγl). On the other hand, the weak error, O ( 2 αl), and variance, O ( 2 βl), do not decrease with increasing number of discretization parameters. Leading to a complexity that suffers from the curse of dimensionality dγ β 2 max(0, O (ε α ) ). Can we leverage some mixed-regularity between these discretization parameters to increase the convergence rate of the weak error and the variance? Haji-Ali (Oxford) Monte Carlo methods 21 / 27

22 Multi-Index Monte Carlo Haji-Ali (Oxford) Monte Carlo methods 22 / 27

23 Multi-Index Monte Carlo Haji-Ali (Oxford) Monte Carlo methods 22 / 27

24 Multi-Index Monte Carlo Let l = (l i ) d i=1 Nd be a multi-index for d discretization parameters and denote by P l the corresponding approximation. Then, define the i th -dimension difference operator { Pl if l i = 0, i,l P = P l P l ei otherwise Observe, again, that E[g] = E[ l P] E[ l P], l N d d and l P = i,l P. for some I N d. Then, estimate the expectations with independent Monte Carlo samplers to yield the MIMC estimator 1 N l N l l I m=1 l I l P(m). i=1 Haji-Ali (Oxford) Monte Carlo methods 23 / 27

25 Multi-Index Monte Carlo Assuming that the cost is O ( 2 γ l ), and ( E[ l g] = O 2 α l ), ( V[ l g] = O 2 β l ), l 2 Decreasing ε where l = l 1 + l l d. Then, there is an optimal choice of N l and I (total degree or simplex) so that the complexity of MIMC is l 1 O (ε 2 max(0, γ β α ) log ε d 1 ), ( when γ β and O ε 2 log ε 2d+max(0,d 3)) otherwise. Haji-Ali (Oxford) Monte Carlo methods 24 / 27

26 Other MLMC extensions Unbiased MLMC: Based on the observation E[P] = E[ l P] = E[ l P/pl ], l=0 where, in the RHS, l is a random variable with P[l = i] = p i, chosen optimally. Useful when β > γ to simplify the analysis of some composite methods but has the same complexity as MLMC. Multilevel Richardson-Romberg extrapolation (ML2R): Applying Richardson extrapolation to the weak error estimate, find optimal w l in E[ P L L ] = w l E[ l P] l=0 to have a smaller number of required levels, L. ML2R is useful when β γ and improves the computational complexity. Haji-Ali (Oxford) Monte Carlo methods 25 / 27

27 Other hierarchical samplers The same multilevel/multi-index ideas (and extensions) can be easily applied (and laboriously analysed) to other samplers such as: Stochastic Collocation and Quasi-Monte Carlo: When we have higher mixed regularity between the stochastic variables. Leads to better complexities. Random L 2 projection: To produce surrogate models of the quantities of interest instead of computing their expectations. Useful for regression and Machine Learning. Markov Chain Monte Carlo or Sequential Monte Carlo: for inverse problems where the expectation is with respect to a posterior distribution depending on a set of observed data points. Haji-Ali (Oxford) Monte Carlo methods 26 / 27

28 Final comments Uncertainty Quantification is a hot topic, with its own conferences and journals Monte Carlo methods are a powerful approach to handle uncertainty in a number of different settings Multilevel Monte Carlo greatly reduces the cost in a lot of settings, particularly when dealing with PDEs for more details, see (Giles, 2015) in Acta Numerica. Haji-Ali (Oxford) Monte Carlo methods 27 / 27