Stochastic Dual Dynamic integer Programming

Size: px

Start display at page:

Download "Stochastic Dual Dynamic integer Programming"

Peregrine Philip Horton
5 years ago
Views:

1 Stochastic Dual Dynamic integer Programming Shabbir Ahmed Georgia Tech Jikai Zou Andy Sun

2 Multistage IP Canonical deterministic formulation ( X T ) f t (x t,y t ):(x t 1,x t,y t ) 2 X t 8 t x t min x,y t=1 = state variables; = local/stage variables y t Linear objective and constraints A t x t 1 + B t x t + C t y t b t Bounded (mixed integer) variables

3 Multistage Stochastic IP Stochastic Data: { t =(f t,x t )} T t=1 Dynamics: t #... (x t 1,y t 1 )! (x t,y t )... Formulation: n min f 1 (x 1,y 1 )+E (x 1,y 1 )2X 1 h h + E n min f 2 (x 2,y 2 )+ (x 2,y 2 )2X 2 (x 1 ) min (x T,y T )2X T (x T 1 ) n oioio f T (x T,y T )

4 SIP Applications Energy Natural resources Healthcare Logis4cs Telecommunica4ons Finance 7 of 21 applica4ons are SIP

5 Trifecta of Complexities Integrality Dynamics Mul5stage IP Integer Programming Mul5stage LP MSIP Sta5c Stochas5c IP Mul5stage Stochas5c LP Sta5c Stochas5c LP Uncertainty

6 Scenario Tree Dynamic uncertainty modeling: Scenario tree Explicit construction/discretization Monte Carlo Sampling Markov Chain (implicit)

7 Extensive Form min x n,y n ( X p n f n (x n,y n ): (x a(n),x n,y n ) 2 X n 8 n 2 T n2t Explicit decision variables and constraints for each node Very large scale but deterministic Decomposition methods (MSLP): Nested Benders (Birge`85) Progressive Hedging (Rockafellar and Wets `91) )

8 Dynamic Programming Formulation: min x 1,y 1 f 1 (x 1,y 1 )+Q 1 (x 1 ): (x a(1),x 1,y 1 ) 2 X 1 Expected Cost-to-go (ECTG) function: Q n (x n ):= X q nm Q m (x n ) m2c(n) Cost-to-go function: Q m (x n )= min x m,y m {f m (x m,y m )+Q m (x m ): (x n,x m,y m ) 2 X m }

9 Dynamic Programming Formulation: min x 1,y 1 f 1 (x 1,y 1 )+Q 1 (x 1 ): (x a(1),x 1,y 1 ) 2 X 1 Expected Cost-to-go (ECTG) function: Q n (x n ):= X q nm Q m (x n ) m2c(n) Cost-to-go function: Polyhedral in LP setting Q m (x n )= min x m,y m {f m (x m,y m )+Q m (x m ): (x n,x m,y m ) 2 X m }

10 Stage-wise Independence N t Stage t has independent realizations Recombining scenario tree One expected cost-to-go function per stage Q n ( ) Q t ( ) 8 n 2 S t Q t ( ) implicitly encodes solution (policy): action under j-th realization in stage t given previous action x t 1 is (x t,y t ) = argmin x,y {f j,t (x, y)+q t (x) :(x t 1,x,y) 2 X j,t }

11 Stochastic Dual Dynamic Programming

12 Illustra5on of SDDP Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

13 Iter 1: Forward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

14 Iter 1: Forward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

15 Iter 1: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) Benders Cut x 3 x 2 x 1

16 Iter 1: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

17 Iter 1: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

18 Iter 2: Forward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

19 Iter 2: Forward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

20 Iter 2: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

21 Iter 2: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

22 Iter 2: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

23 Iter 3: Forward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

24 Iter 3: Forward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

25 Iter 3: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

26 Iter 3: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

27 Iter 3: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

28 Iter 4: Forward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

29 Iter 4: Forward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

30 Iter 4: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

31 Iter 4: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

32 Iter 4: Backward pass Q 2 (x 2 ) Q 3 (x 3 ) Q 1 (x 1 ) x 3 x 2 x 1

33 SDDP Bounds Stochastic algorithm Forward pass generates candidate solutions along sample paths, average of these objective values provides a statistical upper bound Optimal value of stage 1 problem (with lower approximation of ECTGF) provides a deterministic lower bound

34 SDDP Convergence Theorem: If sampling is done with replacement and basic dual optimal solutions are used to construct Benders cuts, then w.p. 1 SDDP converges in a finite number of iterations to an optimal policy for a multistage stochastic LP. Structure of LP value function structure is crucial Chen & Powell (1999) PhilpoL & Guan (2008) Shapiro (2011) Girardeau et al. (2014)

MSIP with Binary State Variables Integer variables destroy convexity of ECTG function Stage-wise decomposition relies on approximating ECTG

35 MSIP with Binary State Variables Integer variables destroy convexity of ECTG function Stage-wise decomposition relies on approximating ECTG functions by cuts i.e. convex polyhedral functions A function of binary variables can always be represented as a convex polyhedral function

36 Binary State Variables min x n,y n s.t. X p n f n (x n,y n ) n2t (x a(n),x n,y n ) 2 X n 8 n 2 T x n 2 {0, 1} d 8 n 2 T

37 Binary State Variables min x n,y n s.t. X p n f n (x n,y n ) n2t (x a(n),x n,y n ) 2 X n 8 n 2 T x n 2 {0, 1} d 8 n 2 T Theorem: [Under complete continuous recourse] A MSIP with d general state variables can be approximated to " accuracy by a MSIP with k binary state variables and p! d k = O d log " Proof: Sensitivity theorems for mixed integer linear programming (Cook et al. (1986))

38 Local Convexification (P ): min x n,y n s.t. X p n f n (x n,y n ) n2t (x a(n),x n,y n ) 2 X n 8 n 2 T x n 2 {0, 1} d 8 n 2 T

39 Local Convexification (P ): min x n,y n s.t. X p n f n (x n,y n ) n2t (x a(n),x n,y n ) 2 X n 8 n 2 T (Q) : x n 2 {0, 1} d 8 n 2 T X min p n f n (x n,y n ) x n,y n,z n s.t. n2t (z n,x n,y n ) 2 X n 8 n 2 T z n 2 [0, 1] d 8 n 2 T z n = x a(n) 8 n 2 T x n 2 {0, 1} d 8 n 2 T

40 Local Convexification (P ): min x n,y n s.t. X p n f n (x n,y n ) n2t (x a(n),x n,y n ) 2 X n 8 n 2 T x n 2 {0, 1} d 8 n 2 T (Q) : min x n,y n,z n s.t. X p n f n (x n,y n ) n2t (z n,x n,y n ) 2 X 0 n 8 n 2 T z n = x a(n) 8 n 2 T x n 2 {0, 1} d 8 n 2 T

41 Local Convexification (P ): min x n,y n s.t. X p n f n (x n,y n ) n2t (x a(n),x n,y n ) 2 X n 8 n 2 T (Q 0 ): min x n,y n,z n s.t. x n 2 {0, 1} d 8 n 2 T X p n f n (x n,y n ) n2t (z n,x n,y n ) 2 conv(x 0 n) 8 n 2 T z n = x a(n) 8 n 2 T x n 2 {0, 1} d 8 n 2 T Theorem: (P) and (Q ) are equivalent.

42 Binary State variables and a reformulation Node (forward) problem: Q j,t (x t 1 ) := min x,y,z s.t. f j,t (x, y)+q t (x) (z,x,y) 2 X j,t z = x t 1 z 2 [0, 1] d x 2 {0, 1} d j =1 Expected Cost-to-go function: Q t (x) = 1 NX Q j,t+1 (x) N j=1 t 1 t t +1 j =. j = N

43 ECTG Function Approximation Node (forward) problem: Q i (x j,t t 1) := min x,y,z s.t. f j,t (x, y)+ i t (x) (z,x,y) 2 X 0 j,t z = x t 1 Approx. Expected Cost-to-go function: i t(x) = min s.t. 1 N NX j=1 v`j,t+1 +( `j,t+1) > x 8` =1,...,i 1

44 ECTG Function Approximation Node (forward) problem: Q i (x j,t t 1) := min x,y,z s.t. f j,t (x, y)+ i t (x) (z,x,y) 2 X 0 j,t z = x t 1 Approx. Expected Cost-to-go function: i t(x) = min s.t. 1 N NX j=1 v`j,t+1 +( `j,t+1) > x 8` =1,...,i 1 Cut coefficients obtained from (relaxations of) node problems in stage t+1 in previous iterations

45 Cut Conditions and Convergence Valid: Q j,t (x) v i j,t +( i j,t) > x 8 x 2 {0, 1} d Tight: Q i+1 j,t (xi t 1) =v i j,t +( i j,t) > x i t 1 Finite: Possible cut coefficients finite Theorem: If sampling is done with replacement and cuts are valid, tight and finite, then w.p. 1 SDDiP converges in a finite number of iterations to an optimal policy for MSIP with binary state variables.

46 Convergence Proof Consider extensive tree structure since solutions are path dependent A solution/policy {x i n} n2t generated at iteration i is i specified by the approximate ECTG functions { t( )} T t=1 x i n 2 argminx,y s.t. i f n (x, y)+ tn (x) (x i a(n),x,y) 2 X n, x 2 {0, 1} d Lemma 1: If is optimal. i t n (x i n)=q tn (x i n) 8 n 2 T Proof: Backward induction of DP equations then {x i n} n2t

47 Convergence Proof (contd.) Let K =sup{i : {x i n} n2t is not optimal} Two types of iterations: (a) At least one approx ECTG function changes (b) Approx ECTG functions do not change so policy does not change K = K a + K b K = K a + P x Kx b Finite cuts mean finitely many approximations of ECTG K a < +1 Finite solution set means the summation is finite

48 Convergence Proof (contd.) Lemma 2: Proof: Pr[K x b < +1] =1 If solution is not optimal then by Lemma 1 there is a (last) node where approximation is loose By Borel-Cantelli lemma, w.p.1 forward pass will hit this node within finite iterations, then a tight cut is generated, and we have new approximation of ECTGF Thus 1 = Pr[Kb x < +1] apple Pr[K = Ka + P x Kx b < +1] Q.E.D

49 Relaxations and Cuts Benders Cut 1 N NX j=1 h i Q LP j,t+1 (xi t)+( j,t+1 ) > (x x i t) Solve LP relaxation Use LP dual solutions Valid and finite (using basic dual solutions) Not tight

50 Relaxations and Cuts Integer Cut (L vt+1 i+1 ) x xi t + vt+1 i+1 where v i+1 t+1 = 1 N NX j=1 Q i+1 j,t+1 (xi t) Laporte and Louveaux (1993) Solve IP Valid, tight and finite

51 Improved LL Cuts Angulo, A. and Dey. Improving the integer L-shaped Method, IJOC, Gustavo Angulo Santanu Dey

52 Relaxations and Cuts Lagrangian Cut NX Lj,t+1 ( j,t+1)+( j,t+1) > x t 1 N j=1 where L j,t+1 ( ) = Solve Lagrangian Dual min x,y,z s.t. j,t+1 2 argmax f j,t+1 (x, y)+ Valid and finite (using basic dual solutions) i+1 t+1 (x) > z (x, y, z) 2 X, z 2 [0, 1] d L j,t+1( )+ > x i t

53 Lagrangian Cut Theorem Lagrangian cut is tight. Primal characterization (convexification) due Lagrangian dual Local copies of state variables Binary nature of state variables (facial property) Recursive application

54 Lagrangian Cut Q(x) Q(x) = min y 1 + y 2 s.t. 2y 1 + y 2 3x 0 apple y 1 apple 2 0 apple y 2 apple 3 y 1 2 Z 1 x

55 Lagrangian Cut Cerisola et al. (2009) Thome et. al. (2013) Q(x) Q(x) = min y 1 + y 2 s.t. 2y 1 + y 2 3x 0 apple y 1 apple 2 0 apple y 2 apple 3 y 1 2 Z 1.5x 1 x

56 Lagrangian Cut Q(x) Q(x) = min y 1 + y 2 s.t. 2y 1 + y 2 3z z = x 0 apple y 1 apple 2 0 apple y 2 apple 3 0 apple z apple 1 y 1 2 Z 1 x

57 Lagrangian Cut Q(x) Q(x) = min y 1 + y 2 s.t. 2y 1 + y 2 3z z = x 0 apple y 1 apple 2 0 apple y 2 apple 3 0 apple z apple 1 y 1 2 Z 1 1+3x x

58 Lagrangian Cut Q(x) Q(x) = min y 1 + y 2 s.t. 2y 1 + y 2 3z z = x 0apple y 1 apple 2 0apple y 2 apple 3 z0 2 apple{0, z apple1} 1 y 1 2 Z 2x 1 x

59 Relaxations and Cuts Strengthened Bender s Cut 1 N NX j=1 Lj,t+1 ( LP j,t+1)+( LP j,t+1) > x t Solve LP relaxation Solve Lagrangian relaxation with LP dual values Valid and finite Not tight Dominates Benders cut Incomparable to Lagrangian cut

60 Computations: Test Case Generation Expansion Planning min s.t. P T t=1 (a> t x t + b > t y t ) A t y t apple P t =1 x 8 t e > y t = d t 8 t x t 2 Z n +, y t 2 R n + 8 t Uncertain demand and price Sampled scenarios Data from Jin et al. (2011) Six types of generators

61 Computations Small case: 10 Stages 3 nodes per stage (3^9 scenarios) Extensive form: >600K integer variables, >200K constraints CPLEX: after 2 hrs, gap = 7% Comparison of Cuts in SDDiP: Benders (B) Integer (I) Lagrangian (L) Strengthened Benders (SB)

62 Cut Comparison (Time) time FW I L B+I B+L S+I S+L S+I+L method

63 Cut Comparison (Gap) 0.6 gap 0.4 FW I L B+I B+L S+I S+L S+I+L method

64 Cut Comparison (Convergence)

65 Scalability Extensive form has > 11 trillion variables!

66 Summary SDDiP: Extension of SDDP to MSIP with binary state vars Exploit binary state variables and a reformulation with local copies of state variables Computations indicate Strengthened Benders + Integer Cuts work best Recent Applications: Hydroscheduling (M. Hjelmeland and A. Helseth) Unit commitment (J. Zou and X. Sun)

References G. Angulo, S. Ahmed, S.S. Dey, V. Kaibel. Forbidden vertices, Mathematics of Operations Research, vol.40, pp.350-360, 2015. G. Angulo, S. Ahmed, S.S. Dey. Improving the integer L-shaped method, INFORMS Journal on Computing, vol.

67 References G. Angulo, S. Ahmed, S.S. Dey, V. Kaibel. Forbidden vertices, Mathematics of Operations Research, vol.40, pp , G. Angulo, S. Ahmed, S.S. Dey. Improving the integer L-shaped method, INFORMS Journal on Computing, vol. 28, pp , M.N. Hjelmeland, J. Zou, A. Helseth, S. Ahmed. Nonconvex medium-term hydropower scheduling by Stochastic Dual Dynamic integer Programming, optimization-online.org, J. Zou, S. Ahmed and X. Sun. Multistage stochastic unit commitment using Stochastic Dual Dynamic Integer Programming, optimization-online.org, J. Zou, S. Ahmed, X. Sun. Stochastic Dual Dynamic Integer Programming, optimization-online.org, 2016.

Robust Dual Dynamic Programming

1 / 18 Robust Dual Dynamic Programming Angelos Georghiou, Angelos Tsoukalas, Wolfram Wiesemann American University of Beirut Olayan School of Business 31 May 217 2 / 18 Inspired by SDDP Stochastic optimization