Stochastic Optimization with cvxpy EE364b Project Final Report

Transcription

1 Stochastic Optimization with cvpy EE364b Project Final Report Alnur Ali June 5, Introduction A stochastic program is a conve optimization problem that includes random variables, and has the form minimize E f 0 (, ω) (1) subject to E f i (, ω) 0, i = 1,..., m, where R n is the optimization variable, ω R p is a random variable, and the f i : R n R p R, i = 0,..., m are conve functions of for each value of ω. In recent work, we implemented into cvpy a (basic) framework for specifying and solving stochastic programs; we refer to this framework as cvstoc. cvstoc works by replacing the epectations in (1), which we assume are difficult to compute in general, with their sample average approimations (SAAs), i.e., N E f i (, ω) (1/N) f i (, ω j ), (2) where the ω j are (Monte Carlo) samples of ω (N is fied by the user). In this project, we investigate the quality of the SAAs induced by (2), for both unconstrained and constrained problems (by adding new diagnostic functionality to cvstoc): we implement (into cvstoc 1 ) simple estimators of the optimality gap between a candidate solution (e.g., of a SAA) and the true solution of (1) these estimators, in turn, rely on estimators (which we also implement) of (a) the true objective value at a candidate and (b) the true optimal value we evaluate all our functionality on (unconstrained, constrained, and chance constrained) stochastic programs we document cvstoc 2. j=1 1 See for code. 2 See ali supp.pdf enclosed with the final project submission. 1

2 2 Lower bounds for unconstrained problems First, we consider the unconstrained problem minimize E f 0 (, ω), (3) the optimal value of which we denote as p. Computing a lower bound on p can be useful, e.g., when checking if the objective value f 0 () of a candidate solution (e.g., of a SAA) is near p. 2.1 Mean field approimation One way to compute a lower bound on p is as follows. Assume f 0 is conve in ω for each value of ; then, by Jensen s inequality, we have [Boy15, slide 5] E f 0 (, ω) f 0 (, E ω) = inf E f 0(, ω) inf f 0(, E ω) = p ˆp mf := inf f 0(, E ω). ˆp mf is referred to as the mean field approimation. In words, replace ω in (1) with its epectation, and then minimize w.r.t. ; the optimal value is a lower bound on p. 2.2 Averaging sample average approimations We can compute another lower bound on p as follows. Let minimize (1/N) N f 0(, ω i ), (4) be a SAA for (3), where the ω i are samples, and ˆp N denotes the optimal value of (4); then, by the discussion below (5.22) in [SDR14], we have that p E ˆp N. Unfortunately, the epectation on the r.h.s. here can be hard to calculate, so we obtain an approimate lower bound by averaging M times (M is fied by the user) instead. In symbols, let ˆp (1) N,..., ˆp (M) N be the optimal values of M (different) SAAs; then, we compute ˆp M,N := (1/M) M ˆp (i) N p. 2.3 A simple estimator of the optimality gap We estimate the optimality gap := f 0 () p at a candidate solution by where ˆ M,N := ˆf 0,N () ˆp M,N, ˆf 0,N () := (1/N) N f 0 (, ω i ) (i.e., average f 0 N times). ( ˆ M,N and ˆp M,N are biased estimators of and p, respectively [SDR14, chap. 5].) 2

3 2.4 Results In this section, we evaluate the estimators ˆp mf, ˆp M,N, and ˆ M,N on a stochastic least squares problem minimize E A b 2 2, (5) where the entries of A R m n are N (1, 1), and the entries of b R m are N ( 1, 1); in this case, (5) has the analytical solution T E A T A 2 E b T A + E b T b, assuming we know the second moments of (A, b). Fig. 1 presents the results 3. In the upper left, we plot the lower bounds ˆp mf (green) and ˆp M,N (blue) on the true optimal value of (5), p (red), vs. the number of Monte Carlo samples N (the number of averaged SAAs M is fied to 10 here): ˆp mf is quite loose, but ˆp M,N gets near p after M 10, N 100. In the upper right, we plot the estimator ˆf 0,N () (blue) of the true objective value E f 0 () (red) at a (arbitrarily chosen) candidate solution (of a SAA) vs. the number of Monte Carlo samples N: ˆf0,N () converges rather rapidly to f 0 () (as epected). In the lower left, we plot the estimator ˆ M,N (blue) of the true optimality gap (red) vs. the number of averaged SAAs M (the number of Monte Carlo samples N is fied to 100 here): ˆ M,N gets near after M 10, N 100 (consistent with the upper left plot). In the lower right, we plot the error of the gap estimator ˆ M,N vs. the number of Monte Carlo samples N and the number of averaged SAAs M: we see that the number of samples affects the error more than the number of SAAs (from the tilt of the plane), but costs more (in terms of runtime). Confidence intervals In Fig. 1, we plot 95% (t-based) confidence intervals 4 in light blue: in the upper left plot, the intervals do not contain the population parameter (i.e., p ) until N 100 samples, although in the upper right plot, the intervals are generally reliable. 3 Lower bounds for constrained problems Net, we consider the constrained problem (1); by weak duality, we have that, for any λ 0, { } { m M m } p inf E f 0 (, ω) + λ i E f i (, ω) (1/M) inf ˆf (j) 0,N () + (j) λ i ˆf i,n () := ˆp M,N. In words, form the Lagrangian for (1), replace epectations with their SAAs, fi λ 0, and then minimize w.r.t. ; averaging the infimums yields a (approimate) lower bound on p. 5 We can then use this ˆp M,N just as we did in the unconstrained setting. 3 We also recreated all the plots in Fig. 1 by varying the number of Monte Carlo samples (while holding the number of SAAs fied): the results are similar, and therefore omitted due to space constraints. 4 We also implemented (percentile) bootstrap confidence intervals [Was11, page 111] for the midway report, but omit those here to keep the presentation uncluttered. 5 It is also possible to upper bound p using a saddle point interpretation of the Lagrangian [SDR14, chap. 5], but these bounds are often trivial (i.e., + ), so we do not pursue them here. 3 j=1

4 Figure 1: Upper left: various lower bounds on the true optimal value (p, red) vs. the number of Monte Carlo samples. Upper right: the estimator (blue) of the true objective value (red) vs. the number of Monte Carlo samples. Lower left: the estimator (blue) of the true optimality gap (red) vs. the number of averaged SAAs. Lower right: the error of the gap estimator (see tet for definition) vs. the number of Monte Carlo samples and the number of averaged SAAs. 95% (t-based) confidence intervals in light blue. 3.1 Quadratically constrained quadratic programs In this section, we evaluate our estimators on a stochastic quadratically constrained quadratic program minimize E T P 0 + q0 T + r 0 subject to E T P 1 + q1 T + r 1 0, where P i := A T i A i with the entries of A i N (0, 1), and q i and r i are fied, for i = 1, 2. The top row of Fig. 2 presents the results. In the upper left, we plot ˆp M,N (blue) vs. the number of Monte Carlo samples: ˆp M,N gets near and its confidence interval contains 4

5 p after N 100 samples. The upper center and right plots are just as in Fig. 1, and have similar interpretations, e.g., our estimators get near their population parameters after M 100, N 100. Additional eperiments (omitted due to space constraints) indicate that varying λ does not matter much. Figure 2: Upper left: ˆp M,N (blue) vs. the number of Monte Carlo samples. Upper center: ˆ M,N (blue) vs. the number of Monte Carlo samples. Upper right: the error of the gap estimator vs. the number of Monte Carlo samples and the number of averaged SAAs. Lower left (see Sec. 3.2): the wealth obtained (higher is better) by solving a portfolio optimization problem with various analytic and approimate risk constraints vs. the number of Monte Carlo samples. Lower right: the value of the approimate risk constraint vs. the number of Monte Carlo samples. 95% (t-based) confidence intervals in light blue. 3.2 Chance constraints A chance constraint is a constraint of the form Prob (f(, ω) 0) η, where f is conve in for each value of ω, and η is typically a large probability (e.g., 0.95). Chance constraints are typically nonconve; thus, the conve approimation E (f(, ω)/α + 1) +, (6) 5

6 where α R is a new variable, is often employed instead (e.g., by cvstoc). It is difficult in general to construct estimators of the optimality gap for chance constrained problems [SDR14, chap. 5]; thus, in this section, we directly investigate the quality of a SAA to the conve approimation (6) by studying the value-at-risk (VaR) constrained portfolio optimization problem minimize E p T subject to Prob(p T 0) 1 η, 1 T = 1, 0, (7) where the vector of returns p N ( p, Σ). In this case, the VaR constraint in (7) has the analytical form p T Φ 1 (η) Σ 1/2 2, (8) where Φ 1 ( ) is the inverse standard normal cdf, while its conve approimation induced by (6), which is equivalent to a conditional value-at-risk (CVaR) constraint, can be epressed as p T ep ( (Φ 1 (η)) 2 /2 ) ( ) / 2π(1 η) Σ 1/2 2. (9) The bottom row of Fig. 2 presents the results. In the lower left, we plot the optimal value of (7) with the analytic VaR constraint (i.e., (8), denoted as p (VaR), and shown in green), (7) with the analytic CVaR constraint (i.e., (9), denoted p (CVaR), in red), and (7) with a SAA to (6) (denoted ˆp N, in blue) vs. the number of Monte Carlo samples: ˆp N gets near p (CVaR) after N 100, and is (indeed) more conservative than p (VaR). In the lower right, we plot the approimate probability Prob( T p 0) of the constraint failing: the probability approaches our specified tolerance of 1 η = 0.05 after N API overview Lastly, to situate our code, we describe the (updated) cvpy package hierarchy below. Essentially, a user may now create either an UnconstrDiagnostics or ConstrDiagnostics object, pass in a cvpy Problem, N, and M, and receive back ˆp mf, ˆp M,N, and/or ˆ M,N : cvpy atoms epressions constraints. stochastic (i.e., cvstoc) RandomVariable RandomVariableFactory epectation() prob partial optimize() {Unconstr, Constr}Diagnostics est optval est cost est optgap 6

7 References [Boy15] Stephen Boyd. Chance constrained optimization. ee364a/lectures/stoch_prog.pdf, January [SDR14] Aleander Shapiro, Darinka Dentcheva, and Andrzej Ruszczyński. Lectures on stochastic programming: modeling and theory, volume 16. SIAM, [Was11] Larry Wasserman. All of statistics. Springer Science & Business Media,