The Values of Information and Solution in Stochastic Programming

Transcription

1 The Values of Information and Solution in Stochastic Programming John R. Birge The University of Chicago Booth School of Business JRBirge ICSP, Bergamo, July

2 Themes The values of information and of solution provide measure of maximum to invest in: Increased for resolving uncertainty Increased effort for modeling and optimizing i i Both goals can guide computational methods that t may combine optimization i and resolving uncertainty Can unify the concepts by viewing i optimization as resolving uncertainty over optimal decisions JRBirge ICSP, Bergamo, July

3 Outline Basics of EVPI and VSS Relevance in dynamic stochastic optimization i i Using measures to guide computation Unifying views with optimization as uncertainty resolution Results for dynamic Gaussian problems Extensions to Gaussian mixtures JRBirge ICSP, Bergamo, July

4 Dynamic Model Discrete Form (Stochastic Model): E[z(x*,ξ) ( ]= minimize E Σ T p [ t=1 f t t( (x t,x t+1,ξ) ] s.t. x t X t, x t nonanticipative ξ ~ P (distribution) EVPI: E[z(x*,ξ) ]- E[min x z(x,ξ) ] VSS: E[z(x(Eξ),ξ) ( (E ) ) ]- E[z(x*,ξ) ( * ) ] where x(eξ) argmin Σ t=1 T f t (x t,x t+1, Eξ) John R. Birge INFORMS, Pittsburgh, November

5 Example: Routing Problem Serve clients at A, B, C, and D from depot with vehicle of capacity 10. Demand=2 at A, B, C; Demand=1 or 7 at C B2 B, C1or7 C, A, 2 D, 2 Depot JRBirge SP, Winter

6 Routing Observations B2 B, A, 2 C1or7 C, D, 2 Depot Expected cost of expected value solution (ABCD) depends on direction (14 or 13.5) (E[z(x(Eξ),ξ) ]=14 or 13.5 Best recourse solution (CBAD, RP= E[z(x*,ξ) ( * ) ] =12.5) is not optimal for any fixed demand Note: For high h reliability, return to depot. ) JRBirge SP, Winter

7 Values of Information Definition: z(x,ξ) =c T x + Q(x,ξ) st s.t. Ax=b b, x 0 Wait-and-See: (EVWPI): WS=E ξ [min x z(x,ξ)]=12. Recourse Problem: min x E ξ [z(x,ξ)]=12.5. JRBirge SP, Winter

8 EVPI and VSS EVPI=RP-WS=0.5 for Routing. Mean Value Problem: EV=min x z(x,e(ξ))=10 for Routing. Expectation of EV problem solution: EEV=E ξ [z(x EV,ξ)]=14 or 13.5 for Routing. Value of the Stochastic Solution: VSS=EEV-RP=1.5 or 1 for Routing (note: 2 or 3 EVPI) JRBirge SP, Winter

9 EVPI=RP-WS 0 VSS=EEV-RP 0 Relationships EVPI EEV-EV VSS EEV EV But, examples: EVPI=0, VSS 0; VSS=0, EVPI 0 Note: If EEV>>WS, then potential for improvement. JRBirge SP, Winter

10 Computational Implication At any branch of a decision tree, can compare the value of expectation over a given policy (e.g., EEV) and the expectation for a clairvoyant policy (e.g., WS) A large difference indicates a value from: Resolving the uncertainty t or Improving the solution JRBirge ICSP, Bergamo, July

11 EVPI Processes and Computation From MAHD+ (1981, 1988, 1999+) Consider the EVPI at each node: η t (ω t ) with marginal process ρ t (ω t ) For a given sample tree, estimate η t (ω t ) Use importance sampling over distribution on value More samples where η t high (fewer where low) JRBirge ICSP, Bergamo, July

12 Computational Goals Additional sampling effort yields: Better resolution of uncertainty Better computation But, while correlated, not necessarily the same Ideally, sampling should improve both at the same time Strategy: put sampling and optimization together JRBirge ICSP, Bergamo, July

13 Optimization via Sampling Problem: find x to Pincus (1968): enough to sample from: min x f(x)subject to g(x) 0 π κ κ( (x) exp{ κ(f(x))} with constraints (e.g., Geman/Geman (1985)) : π κ,λ( (x) exp { κ (f(x) + λ κ g(x))} Extension: Augment with latent variables (λ, ω) thenwecanwrite π κ,λ (x) exp{ κ(f(x) { + λ κ g(x))} } = e ax E ω {e x2 2 ω }e bx E λ {e x2 2 λ } where the parameters (a, b) depend on (κ, λ κ ) and the functional forms of (f(x),g(x)). JRBirge ICSP, Bergamo, July

14 Basic Optimization via Simulation Pincus results: Model: min F(x) Create a distribution Pλ( λ (x) exp(-λ F(x)) ()) Then: lim * λ E Pλ (x) = x JRBirge ICSP, Bergamo, July

15 Markov Chain Simulation From x t to x t+1 Sample over exp(-λ F(μ x t +(1-μ) x t+1 ) ) Choose directions i to obtain mixing i in the region JRBirge ICSP, Bergamo, July

16 Example Rosenbrock function: φ(x =(x 1,x 2 )) = (1 x 1 ) (x 2 x 2 1) 2, which h has a global l minimum i at x = (1, 1). Specification for f: x t = {u if v e κ(φ(u)/φ(xt 1)) ; x t 1 1 otherwise,, } where u U([ 4, 4] 2 ), a uniform distribution over [ 4, 4] [ 4, 4], and v U([0, 1]), a standard uniform draw. Note: corresponds to the Metropolis-Hastings method of Markov Chain Monte Carlo using the distribution proportional to e κφ(x) for the acceptance criterion. JRBirge ICSP, Bergamo, July

17 Rosenbrock Graphs κ= JRBirge MIT ORC, March

18 Particle Samples and Optimization Suppose N particles are used Consider a 2-stage problem f(x,ξ) = c(x) + Q(x,y,ξ) Distribution ib ti on x, y i, ξ: ξ P(x,y 1, y N,ξ) Π i=1n (c(x)+q(x,y i,ξ i )) Result: E P (x) x * JRBirge ICSP, Bergamo, July

19 Implementation Propagation forward on y s: q( ξ j x 1 ) {c(x)+q(ξ ξ j, y j, x)} p( ξ j ) where y j =argmin(c(x) + Q( ξ j, y j, x)) p( x ξξ 1,y 1,, ξ N y N ) j=1n {c(x) + Q(ξ j, y j, x) }p( ξ j ) The particles concentrate the distribution on x* as in the deterministic optimization. JRBirge ICSP, Bergamo, July

20 Example From B/Louveaux, Chapter 1 (Farmer): JRBirge ICSP, Bergamo, July

21 Dynamic Models General Idea: Create a distribution over decision variables and random variables Use MCMC to sample with sufficient number of particles or annealing parameter marginal mean on optimum Extension for dynamics: Keep constant sample particle numbers Convergence in fixed number of particles per stage? JRBirge ICSP, Bergamo, July

22 Example: Linear-Quadratic Gaussian All distributions are available in closed form x t =Ax t-1 +B t u t-1 +v t All normal distributions with risk-sensitive objective: -log E(exp(-(θ/2)(x 2T Qx 2 +u 1T Ru 1 ))) where u 1 is control lin the first period Note: Equivalent to robust formulation Now, all can be found in closed form JRBirge ICSP, Bergamo, July

23 Mean of u: m u 1 m = L u 1 1(θ) x 1, Results for LQG where L 1 is the result from the Kalman filter Variance of u: V u (N) where (N) 0 as N V u 1 m (θ) u * u as θ where u 1* is the two-stage risk-neutral solution. JRBirge ICSP, Bergamo, July

24 Extensions to T Stages Can derive S θ t recursively to show that: p(x t-1 t-1 t x,u )=K exp(-(1/2) (1/2) ( s=1 t-1 (x tt Q s x s +u st R s u s )+x tt S t θ x t ) + (x t -Ax t-1 -Bu t-1 ) T V t-1 (x t -Ax t-1 -Bu t-1 )) For particle methods, j=1 N: Each x tj drawn consistently from this distribution JRBirge ICSP, Bergamo, July

25 w t1,x t 1 u 1 t w t2, x t 2 u 2 t. w tn, x N t u N t General Method Structure Propogate x t+1 1 x 2 t+1... x t+1 N Re-sample: w t1, x 1 t u 1 t w t2, x 2 t u 2. u t w tn, x t N N N u N t JRBirge ICSP, Bergamo, July

26 General Result If the propagation step is unbiased, then E(u * 1 ) u 1 The unbiased result is possible in LQG from the recursive structure. Harder to obtain in more general systems (without additional i future sampling) JRBirge ICSP, Bergamo, July

27 Two-Stage: Compare Direct MC Low Risk Aversion High Risk Aversion JRBirge ICSP, Bergamo, July

28 Three Stage Example JRBirge ICSP, Bergamo, July

29 Generalizing Objectives Direct: risk-sensitive exponential utility: exp(-θ Q(x,y,ξ)) - Q quadratic Objective: e -θ y - d E θ λ [exp((-θ 2 /2λ j )(y j d) 2 )] where λ ~ Exp(2) Also, use: E[ e -θ y - d ] 1 + θ y-d + O(1) JRBirge ICSP, Bergamo, July

30 Further Generalizations ations Use objective approximations that preserve normal forms Extend LQG-like results to this general framework Find other frameworks where unbiased future samples are available Obtain sensitivity results on bias from imperfect future sampling Use optimization i to pick ML instead JRBirge ICSP, Bergamo, July

31 Conclusions EVPI and VSS indicate the values in resolving uncertainty and optimizing Can use importance sampling based on EVPI to guide samples for optimization (but may have some lost efficiency) Combining i optimization i and sampling results in consistent importance sampling for uncertainty resolution and optimization i Convergence possible for symmetric problems (LQG) and maybe others JRBirge ICSP, Bergamo, July

32 Thank you! Questions? JRBirge ICSP, Bergamo, July