18-660: Numerical Methods for Engineering Design and Optimization

Transcription

1 18-660: Numerical Methods for Engineering Design and Optimization Xin Li Department of ECE Carnegie Mellon Universit Pittsburgh, PA Slide 1

2 Overview Monte Carlo Analsis Latin hpercube sampling Importance sampling Slide 2

3 Latin Hpercube Sampling (LHS) A great number of samples are tpicall required in traditional Monte Carlo to achieve good accurac Various techniques eist to improve Monte Carlo accurac Controlling sampling points is the ke Latin hpercube sampling is a widel-used method to generate controlled random samples The basic idea is to make sampling point distribution close to probabilit densit function (PDF) M. Mcka, R. Beckman and W. Conover, A comparison of three methods for selecting values of input variables in the analsis of output from a computer code, Technometrics, vol. 21, no. 2, pp , Ma Slide 3

4 Latin Hpercube Sampling (LHS) One dimensional Latin hpercube sampling Evenl partition CDF into N regions Randoml pick up one sampling point in each region One random sample in one region 1 CDF Evenl partitioned regions 0 Avoid the probabilit that all sampling points come from the same local region Slide 4

5 Latin Hpercube Sampling (LHS) Two dimensional Latin hpercube sampling 1 and 2 must be independent Generate one-dimensional LHS samples for 1 Generate one-dimensional LHS samples for 2 Randoml combine the LHS samples to two-dimensional pairs 1 Probabilit of Probabilit of 1 One sample in each row and each column Sampling is random in each grid Higher-dimensional LHS samples can be similarl generated Slide 5

6 Latin Hpercube Sampling (LHS) Matlab code for LHS sampling of independent standard Normal distributions data = rand(nsample,nvar); for i = 1:NVar inde = randperm(nsample); prob = (inde'-data(:,i))/nsample; data(:,i) = sqrt(2)*erfinv(2*prob-1); end; NVar: NSample: data: # of random variables # of samples LHS sampling points Slide 6

7 Latin Hpercube Sampling (LHS) Compare Monte Carlo accurac for a simple eample ~ N(0,1) (standard Normal distribution) Repeatedl estimate the mean value b Monte Carlo analsis Accurac is improved b increasing sampling # Estimated Mean Value Estimated Mean Value # of Samples Random sampling Monte Carlo is not deterministic # of Samples Latin hpercube sampling Slide 7

8 Even with Latin hpercube sampling, Monte Carlo analsis requires a HUGE number of sampling points Eample: rare event estimation ~ N( 0,1) Standard Normal distribution Estimate P( 5 ) =??? The theoretical answer for P( -5) is equal to ~100M sampling points are required if we attempt to estimate this probabilit b random sampling or LHS Slide 8

9 Ke idea: Do not generate random samples from (t) Instead, find a good distorted (t) to improve Monte Carlo sampling accurac Eample: if ~ N(0,1), what is P( -5)? Intuitivel, if we draw sampling points based on (t), more samples will fall into the gre area (t) PDF (t) How do we calculate P( -5) when sampling (t)? P( -5) 0 t Slide 9

10 Assume that we want to estimate the following epected value E [ f ( ) ] f Eample: if we want to estimate P( -5), then + = dt f ( ) = 1 0 ( 5) ( > 5) E 5 + [ f ( ) ] 1 dt + 0 dt = P( 5) = 5 Slide 10

11 Estimate E[f()] where ~ (t) b importance sampling E [ f ( ) ] f dt = f + + = dt E [ f ( ) ] f + = + = dt = g dt E[ g( ) ] Slide 11

12 Estimate E[f()] where ~ (t) b traditional sampling Step 1: draw M random samples {t 1,t 2,...,t M } based on (t) Step 2: calculate f m = f(t m ) at each sampling point m = 1,2,...,M Step 3: calculate E[f] (f 1 +f f M )/M Estimate E[f()] where ~ (t) b importance sampling Step 1: draw M random samples {t 1,t 2,...,t M } based on (t) Step 2: calculate g m = f(t m ) (t m )/ (t m ) at each sampling point m = 1,2,...,M Step 3: calculate E[f] (g 1 +g g M )/M How do we decide the optimal (t) to achieve minimal Monte Carlo analsis error? Slide 12

13 Determine optimal (t) for importance sampling E [ f ] µ = f ( t ) Estimator f 1 M ( tm ) ( t ) The accurac of an estimator can be quantitativel measured b its variance To improve Monte Carlo analsis accurac, we should minimize VAR[μ f ] M m= 1 m [ ] Error ~ VAR µ f m Slide 13

14 Determine optimal (t) for importance sampling E [ f ] µ = f ( t ) m= 1 We achieve the minimal VAR[μ f ] = 0 if f = k f 1 M (Constant) f = (Optimal PDF) k M m ( tm ) ( t ) m f(t m ) (t m )/ (t m ) is equal to the same constant for all sampling points Slide 14

15 How do we decide the value k? = f k K cannot be arbitraril selected (t) must be a valid PDF that satisfies the following condition + dt = + f + k = f = k dt dt E[ f ] =1 Finding the optimal (t) requires to know E[f], i.e., the answer of our Monte Carlo analsis!!! Slide 15

16 In practice, such an optimal (t) cannot be easil applied Instead, we tpicall look for a sub-optimal solution that satisfies the following constraints Eas to construct we do not have to know k = E[f] Eas to sample not all random distributions can be easil sampled b a random number generator Minimal estimator variance the sub-optimal (t) is close to the optimal case as much as possible Slide 16

17 Finding the right (t) is nontrivial for practical problems No magic equation eists in general Engineering approach is based on heuristics Sometimes require a lot of human eperience and numerical optimization The criterion to choose (t) is also application-dependent Slide 17

18 Eample: if ~ N(0,1), what is P( -5)? Shifted Normal PDF (t) Uniform P( -5) 0 Several possible choices for (t) t Slide 18

19 Summar Monte Carlo analsis Latin hpercube sampling Importance sampling Slide 19