Approximations of Stochastic Programs. Scenario Tree Reduction and Construction

Transcription

1 Approximations of Stochastic Programs. Scenario Tree Reduction and Construction W. Römisch Humboldt-University Berlin Institute of Mathematics Berlin, Germany (J. Dupačová, N. Gröwe-Kuska, H. Heitsch) GAMS Workshop, Heidelberg, Sept. 1-3,

2 1 Introduction Let {ξ t } T t=1 be a discrete-time stochastic data process defined on some probability space (Ω, F, P ) and with ξ 1 deterministic. The stochastic decision x t at period t is assumed to depend only on (ξ 1,..., ξ t ) (nonanticipativity). Typical financial and production planning model: min{ie[ T c t (ξ t, x t )] : x t X t, x t nonanticipative, t=1 A tt (ξ t )x t + A t,t 1 (ξ t )x t 1 g t (ξ t )} Alternative for the minimization of expected costs: Minimizing some risk measure IF of the stochastic cost process {c t (ξ t, x t )} T t=1 (risk management). First step of its numerical solution: Approximation of {ξ t } T t=1 by finitely many scenarios with certain probabilities. Nonanticipativity leads to a scenario tree structure of the approximation. 2

3 2 Data process approximation by scenario trees The data process ξ = {ξ t } T t=1 is approximated by a process forming a scenario tree which is based on a finite set N of nodes. n = 1 n n N T N + (n) t = 1 t 1 t(n) T Scenario tree with t 1 = 2, T = 5, N = 23 and 11 leaves The root node n = 1 stands for period t = 1. Every other node n has a unique predecessor n and a set N + (n) of successors. Let path(n) be the set {1,..., n, n} of nodes from the root to node n, t(n) := path(n) and N T := {n N : N + (n) = } the set of leaves. A scenario corresponds to path(n) for some n N T. With the given scenario probabilities {π n } n NT, we define recursively node probabilities π n := n + N + (n) π n +, n N. 3

4 3 Generation of scenario trees (i) Development of a stochastic model for the data process ξ (parametric [e.g. time series model], nonparametric [e.g. resampling]) Load Hours Scenarios for the weekly electrical load and generation of simulation scenarios; (ii) Construction of a scenario tree out of the stochastic model or of the simulation scenarios; (iii) optional scenario tree reduction. 4

5 Approaches for (ii): (1) Barycentric scenario trees (conditional expectations w.r.t. a decomposition of the support into simplices) (Frauendorfer 96,...); (2) Fitting of trees with prescribed structure to given moments (Hoyland/Wallace 01, Hoyland/Kaut/Wallace 03); (3) Conditional sampling by (Quasi) Monte Carlo methods (QMC means low discrepancy sequences) (Morton 03, Koivu/Pennanen 02, 03); (4) Clustering methods for bundling scenarios (Philpott/Craddock/Waterer 00); (5) Scenario tree construction based on optimal approximations w.r.t. certain probability metrics (Pflug 01, Hochreiter/Pflug 02, Gröwe-Kuska/Heitsch/Römisch 03). Recent reference: Kaut/Wallace 03 5

6 Example: (Hochreiter/Pflug 02) Let P denote the uniform distribution on [ 3, 3] and P be the distribution of Z := c 1 Z 1 + c 2 Z 2, where Z 1 is discrete with two equally probable scenarios 1 and 1, Z 2 is standard normal, i.e., Z 2 N(0, 1), and c 1 and c 2 are normalizing constants (c 1 := 4 3 5, c 3 2 := 1 5 ). Then the first four (central) moments coincide 9 IR ξi P (dξ) = IR ξi P (dξ) = 0, 1, 0,, i = 1, 2, 3, 4. 5 However, the densities of P and P have the following form and, thus, are quite different. 6

7 4 Distances of probability distributions Let P denote the probability distribution of the stochastic process {ξ t } T t=1 with ξ t in IR r, i.e., P has support Ξ IR rt = IR s. The Kantorovich functional or transportation metric takes the form µ c (P, Q) := inf{ c(ξ, ξ)η(dξ, d ξ) : π 1 η = P, π 2 η = Q}, Ξ Ξ where c : Ξ Ξ IR is a certain cost function and the minimum ist taken w.r.t. all probability measures η on Ξ Ξ having (fixed) marginals P and Q. Example: c p (ξ, ξ) := max{1, ξ ξ 0 p 1, ξ ξ 0 p 1 } ξ ξ (p 1, ξ 0 Ξ fixed) We consider the following convex stochastic program min{ f 0 (x, ξ)p (dξ) : x X} Ξ with a normal convex integrand f 0 and denote by v(p ) := inf f 0 (x, ξ)p (dξ) and S(P ) := arg min x X Ξ x X its optimal value and solution set, respectively. Ξ f 0 (x, ξ)p (dξ) 7

8 We choose c such that the property f 0 (x, ξ) f 0 (x, ξ) L( x )c(ξ, ξ), ξ, ξ Ξ, x X, holds with some function L( ) depending on x. This means that c plays the role of a continuity modulus of the function f 0 (x, ) from Ξ to IR (for each x X). Typically, f 0 is continuous and piecewise polynomial. Theorem: (Stability) Under weak conditions on the stochastic program the optimal values are Lipschitz continuous w.r.t. µ c, i.e., v(p ) v(q) ˆLµ c (P, Q), and the solution sets are upper semicontinuous. In particular, if S(P ) = { x} any element of the approximate solution set S(Q) is close to x if µ c (P, Q) is small. (Rachev/Römisch 02, Römisch 03) 8

9 Choice of p 1 in c = c p : two-stage with random right-hand side: p = 1. general two-stage with fixed recourse: p = 2. multi-stage with random right-hand side: p = 1. general multi-stage with T stages: p = T. ( present conjecture valid under appropriate assumptions on the dependence structure; not valid for mixed-integer models; in that case f 0 is piecewise continuous!) Approach: Select a probability metric a function c : Ξ Ξ IR such that the underlying stochastic optimization model is stable w.r.t. µ c. Given P and a tolerance ε > 0, determine a scenario tree such that its probability distribution P tr has the property µ c (P, P tr ) ε. 9

10 Distances of discrete distributions P : scenarios ξ i with probabilities p i, i = 1,..., N, Q: scenarios ξ j with probabilities q j, j = 1,..., M. Then N µ c (P, Q) = sup{ p i u i + i=1 M q j v j : u i + v j c(ξ i, ξ j ) i, j} j=1 = inf{ i,j η ij c(ξ i, ξ j ) : η ij 0, j η ij = p i, i η ij = q j } (optimal value of linear transportation problems) (a) Distances of distributions can be computed by solving specific linear programs. (b) The principle of optimal scenario generation can be formulated as a best approximation problem with respect to µ c. However, it is nonconvex and difficult to solve. (c) The best approximation problem simplifies considerably if the scenarios are taken from a specified finite set. 10

11 5 Scenario Reduction We consider discrete distributions P with scenarios ξ i and probabilities p i, i = 1,..., N, and Q having a subset of scenarios ξ j, j J {1,..., N}, of P, but different probabilities q j, j J. Optimal reduction of a given scenario set J: The best approximation of P with respect to µ c by such a distribution Q exists and is denoted by Q. It has the distance D J = µ c (P, Q) = p i min c(ξ i, ξ j ) j J i J and the probabilities q j = p j + p i, j J, where J j := i J j {i J : j = j(i)} and j(i) arg min c(ξ i, ξ j ), i j J J, i.e., the optimal redistribution consists in adding the deleted scenario weight to that of some of the closest scenarios. However, finding the optimal scenario set with a fixed number n of scenarios is a combinatorial optimization problem. 11

12 6 Fast reduction heuristics Starting point (n = N 1): min p l min c(ξ l, ξ j ) l {1,...,N} j l Algorithm 1: (Simultaneous backward reduction) Step [0]: Step [i]: Step [N-n+1]: Sorting of {c(ξ j, ξ k ) : j}, k, J [0] :=. l i arg min l J [i 1] k J [i 1] {l} J [i] := J [i 1] {l i }. Optimal redistribution. p k min c(ξ k, ξ j ). j J [i 1] {l} 12

13 Starting point (n = 1): min u {1,...,N} k=1 N p k c(ξ k, ξ u ) Algorithm 2: (Fast forward selection) Step [0]: Compute c(ξ k, ξ u ), k, u = 1,..., N, Step [i]: Step [n+1]: J [0] := {1,..., N}. u i arg min u J [i 1] J [i] := J [i 1] \ {u i }. k J [i 1] \{u} Optimal redistribution. p k min c(ξ k, ξ j ), j J [i 1] \{u} 13

14 1000 Original load scenario tree Reduced load scenario tree / backward Reduced load scenario tree / forward

15 Binary test scenario tree Let a binary scenario tree have N := 2 T 1 scenarios ξ i = (ξi 1,..., ξt i ), i = 1,..., N, with equal probabilities p i = 1 N, i = 1,..., N, and ξ1 1 =... = ξn 1 as its root node. Such a scenario tree is called regular if, for each t {1,..., T }, δ1 t := δ t and δ2 t := δ t with δ t IR + and ξ t i = t δi τ τ (t {1,..., T }) τ=1 where to each index i = 1,..., N there corresponds a T - tupel of indices (i 1,..., i T ) {1, 2} T. Proposition: Let a regular binary scenario tree with N = 2 T 1 scenarios and T 4 be given. Let t 0 arg min 2 t T δ t, t 0 T 2 and max{δ t 0+1, δ t 0+2 } 2δ t 0. Then it holds for each n IN with N 4 n < N: D opt n := min{d J : #J = N n} = N n N 2δt 0. Here, c is defined by c(ξ, ξ) := ξ ξ (ξ, ξ Ξ). 15

16 Example: (regular binary scenario tree) T = 11, N = 2 10 = 1024, (δ 1,..., δ 11 ) = (0, 0.5, 0.6, 0.7, 0.9, 1.1, 1.3, 1.6, 1.9, 2.3, 2.7), D opt n = N n N for each N 4 n < N Relative accuracy: µ rel c (P, Q) := µ c(p, Q) µ c (P, δ ξl ) µ c (P, δ ξl ) = min{d J : #J = N 1} and c(, ) :=. 16

17 Number Backward of Simultaneous Fast Minimal n of Scenario Sets Backward Forward Distance Scenarios ζc rel Time ζc rel Time ζc rel Time % 2 s % 96 s % 2 s % % 2 s % 96 s % 2 s * % 2 s % 96 s % 2 s * % 2 s % 96 s % 3 s * % 2 s % 96 s % 3 s * % 2 s % 95 s % 4 s * % 2 s % 95 s % 7 s * % 2 s % 94 s % 10 s * % 2 s % 93 s % 15 s * % 2 s % 89 s % 27 s * % 2 s % 85 s % 38 s * % 2 s % 81 s % 48 s * % 2 s % 76 s % 56 s * % 2 s % 75 s % 58 s % % 2 s % 74 s % 60 s % % 2 s % 72 s % 61 s % % 2 s % 71 s % 63 s % % 2 s % 70 s % 64 s % % 2 s % 64 s % 71 s % % 2 s % 57 s % 76 s % % 2 s % 51 s % 81 s % % 2 s 9.67 % 45 s 9.63 % 85 s 9.63 % % 2 s 7.79 % 33 s 7.79 % 91 s 7.79 % % 2 s 5.95 % 22 s 5.95 % 95 s 5.95 % % 2 s 4.12 % 12 s 4.12 % 97 s 4.12 % Computational results for the binary scenario tree 17

18 7 Constructing scenario trees from data scenarios Let a fan of data scenarios ξ i = (ξ1, i..., ξt i ) with probabilities π i, i = 1,..., N, be given, i.e., all scenarios coincide at the starting point t = 1, i.e., ξ1 1 =... = ξ1 N =: ξ1. Hence, it has the form t = 1 may be regarded as the root node of the scenario tree consisting of N scenarios (leaves). Now, P is the (discrete) probability distribution of ξ. Let c be adapted to the underlying stochastic program containing P. We describe an algorithm that may produce, for each ε > 0, a scenario tree with distribution P ε, root node ξ 1, less nodes than P and µ c (P, P ε ) < ε. 18

19 Recursive reduction algorithm: Let ε t > 0, t = 1,..., T, be given such that T t=1 ε t ε, set t := T, I T +1 := {1,..., N}, πt i +1 := πi and P T +1 := P. For t = T,..., 2: Step t: Determine an index set I t I t+1 such that µ ct (P t, P t+1 ) < ε t, where {ξ i } i It is the support of P t and c t is defined by c t (ξ, ξ) := c((ξ 1,..., ξ t, 0,..., 0), ( ξ 1,..., ξ t, 0,..., 0)); (scenario reduction w.r.t. the time horizon [1, t]) Step 1: Determine a probability measure P ε such that its marginal distributions P ε Π 1 t are δ ξ 1 for t = 1 and P ε Π 1 t = πtδ i ξ i and π t t i := πt+1 i + π j t+1, i I t j J t,i where J t,i := {j I t+1 \ I t : i t (j) = i}, i t (j) arg min c t (ξ j, ξ i )} are the index sets according to the redistribution i I t rule. 19

20 Blue: compute c-distances of scenarios; delete the green scenario & add its weight to the red one 20

21 Application: ξ is the bivariate weekly data process having the components a) electrical load, b) hourly electricity spot prices (at EEX). Data scenarios are obtained from a stochastic model calibrated to the historical load data of a (small) German power utility and historical price data of the European Energy Exchange (EEX) at Leipzig. We choose N = 50, T = 7, ε = 0.05, ε t = ε T, and arrive at a tree with 4608 nodes (instead of 8400 nodes of the original fan). t hours I t

22 400 Scenario tree for the electrical load Load Hours 100 Scenario tree for hourly spot prices Spot price Hours 22

23 8 GAMS/SCENRED GAMS/SCENRED introduced to GAMS Distribution 20.6 (May 2002) SCENRED is a collection of C++ routines for the optimal reduction of scenarios or scenario trees GAMS/SCENRED provides the link from GAMS programs to the scenario reduction algorithms. The reduced problems can then be solved by a deterministic optimization algorithm provided by GAMS. SCENRED contains three reduction algorithms: - FAST BACKWARD method - Mix of FAST BACKWARD/FORWARD methods - Mix of FAST BACKWARD/BACKWARD methods Automatic selection (best expected performance w.r.t. running time) Details: