What's New in Mathematical Optimisation from NAG

Transcription

1 What's New in Mathematical Optimisation from NAG Jan Fiala, Benjamin Marteau

2 Nonlinear programming: active set versus interior point methods Overview Sequential quadratic programming Interior point methods Illustration on a few examples Mixed integer nonlinear optimisation Semide nite programming Sample applications in nance Coming next Large-scale linear programming Derivative free solver for calibration Working with customers 2

3 Nonlinear programming: active set versus interior point methods Overview Sequential quadratic programming Interior point methods Illustration on a few examples Mixed integer nonlinear optimisation Semide nite programming Sample applications in nance Coming next Large-scale linear programming Derivative free solver for calibration Working with customers 3

4 Nonlinear optimisation Problems of the form: min f (x) x Rn hk (x) = 0, k = 1...me gk (x) 0, k = 1...mi Two di erent approaches: Sequential quadratic programming: Active set method based on Gill et al., Stanford University Interior point method based on Wächter, Biegler, Carnegie Mellon University 4

5 Formalisation of the problem Karush-Kuhn-Tucker (KKT) optimality conditions: Stationarity condition f (x) + me X λk hk (x) + k=1 mi X µk gk (x) = 0 k=1 Primal feasibility condition h(x) = 0 g(x) 0 Dual feasibility condition k {1,..., mi }, µk 0 Complementarity condition k {1,..., mi }, µk gk (x) = 0 5

6 Two approches to tackle these equations The Complementarity condition is problematic due to its combinatorial nature. Two distincts strategy: An SQP solver guesses which constraints are binding An IPM perturbs the equation 6

7 Nonlinear programming: active set versus interior point methods Overview Sequential quadratic programming Interior point methods Illustration on a few examples Mixed integer nonlinear optimisation Semide nite programming Sample applications in nance Coming next Large-scale linear programming Derivative free solver for calibration Working with customers 7

8 Sequential quadratic programming De nition An inequality constraint (g(x) k is said to be active at x if it is binding = 0). SQP methods iteratively build the set of active constraints by solving quadratic programs: Initialisation Choose a rst estimate of the solution quadratic model of the objective around x0 x0. Build a and take a rst guess of the set of active constraints Iteration k Solve the quadratic program warm started by the active set estimation Update xk+1 and the set of active constraints Build a new quadratic model around xk+1 8

9 A few characteristics of SQP methods Perform lots of inexpensive iterations Work on the null space of the constraints The more active constraints there are, the cheaper the iterations are As a consequence, SQP methods scale very well to large NLP problems with a high number of constraints. 9

10 Nonlinear programming: active set versus interior point methods Overview Sequential quadratic programming Interior point methods Illustration on a few examples Mixed integer nonlinear optimisation Semide nite programming Sample applications in nance Coming next Large-scale linear programming Derivative free solver for calibration Working with customers 10

11 Interior point methods If one tries to solve the KKT system directly, the complementarity condition turns out to be problematic. Therefore, a IPM iteration can be: Relax the complementarity condition (µg(x ) =ν with ν > 0) Perform one Newton iteration towards the solution of the relaxed KKT system Update the current solution estimate and the relaxation parameter ν Interior point methods aim at nding a sequence of points converging to the solution that satisfy the constraints strictly. 11

12 A few characteristics of Interior Point methods Perform a few expensive iterations In the absence of constraints, behave as a Newton method As a consequence, Interior Point methods scale very well to large NLP problems with a small number of constraints. 12

13 Illustration on a few highly constrained problems Problems were selected from the CUTER test set. Name Number Number e04vh (SQP) e04st (IPM) of vars of constrs time (s) time (s) READING NCVXQP MADSSCHJ MINC44 13

14 Illustration on a few weakly constrained problems Problems selected from the CUTER test set. Name Number Number e04vh (SQP) e04st (IPM) of vars of constrs time (s) time (s) JIMACK OSORIO TABLE OBSTCLBL

15 Illustration on a few weakly constrained problems Problems selected from the CUTER test set. Name Number Number e04vh (SQP) e04st (IPM) of vars of constrs time (s) time (s) JIMACK OSORIO TABLE OBSTCLBL The number of constraints is not the only factor... 14

16 Other characteristics IPM (e04st) advantages E cient on unconstrained or loosely constrained problems Can exploit 2nd derivatives E cient also for quadratic problems Better use of multi-core architecture New and simpler interface SQP (e04vh) advantages E cient on highly constrained problems Can capitalize on good initial point Stay feasible with respect to the linear constraints throughout the optimization Usually better results on pathological problems Usually requires less function evaluations Infeasibility detection Allows warm starting 15

17 Nonlinear programming: active set versus interior point methods Overview Sequential quadratic programming Interior point methods Illustration on a few examples Mixed integer nonlinear optimisation Semide nite programming Sample applications in nance Coming next Large-scale linear programming Derivative free solver for calibration Working with customers 16

18 Mixed integer nonlinear optimisation Problems of the form: min x Rn,y Zm f (x, y) l c(x, y) u x: continuous variables y: integer variables SQP with branch-and-cut techniques Ordinal variables Does not require the model evaluation on fractional values of integer variables 17

19 Some characteristics It might be necessary to use integral variables in an optimization model, for example: Cardinality constraints Decision logic between variables (e.g. constraints only present if a certain variable is nonzero) Variables can only take values inside a predecided set... Included in NAG, Mark 25 as h02da. Based on Schittkowski et al., University of Bayreuth. 18

20 Nonlinear programming: active set versus interior point methods Overview Sequential quadratic programming Interior point methods Illustration on a few examples Mixed integer nonlinear optimisation Semide nite programming Sample applications in nance Coming next Large-scale linear programming Derivative free solver for calibration Working with customers 19

21 Semide nite Programming (SDP) Linear Programming (LP) well-known, well-researched convex (local global) strong theoretical properties but only linear 20

22 Semide nite Programming (SDP) Linear Programming (LP) well-known, well-researched convex (local global) strong theoretical properties but only linear Extensions: NLP but some nice properties lost (e.g., convexity, duality theory) 20

23 Semide nite Programming (SDP) Linear Programming (LP) well-known, well-researched convex (local global) strong theoretical properties but only linear Extensions: NLP but some nice properties lost (e.g., convexity, duality theory) SDP retain the theory, change geometry add matrix inequality, symmetric matrix is positive semide nite (all eignevalues are nonnegative) highly nonlinear notation: A(x) 0 20

24 Semide nite Programming (SDP) formulation LP min x Rn subject to ct x lb Bx ub lx x ux 21

25 Semide nite Programming (SDP) formulation LP SDP ct x min x Rn subject to lb Bx ub lx x ux A(x) = A0 + n X x i Ai 0 i=1 Ai given symmetric matrices A(x) is linear in x, with special choice LMI = linear matrix inequality A(x) can be a matrix variable X 21

26 Semide nite Programming (SDP) formulation LP SDP BMI-SDP minn x R subject to 1 ct x + xt Hx 2 lb Bx ub lx x ux A(x) = A0 + n X i=1 x i Ai + n X xi xj Qij 0 i,j=1 further (quadratic) extension BMI = bilinear matrix inequalities unique to NAG, included in Mark 26 as e04sv in collaboration with Ko vara at al., University of Birmingham 21

27 Semide nite Programming (SDP) Applications? SDP = special tool It's there when you need it! 22

28 Semide nite Programming (SDP) Applications? SDP = special tool It's there when you need it! very powerful concept matrix constraints might not appear naturally reformulations, relaxations structural optimization, chemical engineering, combinatorial optimization, statistics, control and system theory, polynomial optimization,... 22

29 Semide nite Programming (SDP) Applications? SDP = special tool It's there when you need it! very powerful concept matrix constraints might not appear naturally reformulations, relaxations structural optimization, chemical engineering, combinatorial optimization, statistics, control and system theory, polynomial optimization,... spark interest Warning: I am not a quant! 22

30 Nonlinear programming: active set versus interior point methods Overview Sequential quadratic programming Interior point methods Illustration on a few examples Mixed integer nonlinear optimisation Semide nite programming Sample applications in nance Coming next Large-scale linear programming Derivative free solver for calibration Working with customers 23

31 SDP Applications in Finance positive semide nite requirement appears directly construction of a correlation/covariance matrix nearest correlation matrix (with constraints) robust (worst-case) portfolio optimization calibration of volatility structure for Libor market swaption 24

32 SDP Applications in Finance positive semide nite requirement appears directly construction of a correlation/covariance matrix nearest correlation matrix (with constraints) robust (worst-case) portfolio optimization calibration of volatility structure for Libor market swaption eigenvalue optimization (min/max eigenvalue/singular value, matrix condition number, nuclear norm as heuristic for rank minimization,...) risk-management: limit Γ of your portfolio 24

33 SDP Applications in Finance positive semide nite requirement appears directly construction of a correlation/covariance matrix nearest correlation matrix (with constraints) robust (worst-case) portfolio optimization calibration of volatility structure for Libor market swaption eigenvalue optimization (min/max eigenvalue/singular value, matrix condition number, nuclear norm as heuristic for rank minimization,...) risk-management: limit Γ of your portfolio relaxations many relaxations of (NP-hard) combinatorial problems asian option pricing bounds(?) 24

34 SDP Applications in Finance positive semide nite requirement appears directly construction of a correlation/covariance matrix nearest correlation matrix (with constraints) robust (worst-case) portfolio optimization calibration of volatility structure for Libor market swaption eigenvalue optimization (min/max eigenvalue/singular value, matrix condition number, nuclear norm as heuristic for rank minimization,...) risk-management: limit Γ of your portfolio relaxations many relaxations of (NP-hard) combinatorial problems asian option pricing bounds(?) reformulations polynomial nonnegativity matrix inequality (e.g., interpolation by nonnegative splines) Lyapunov stability of ODE in nance? 24

35 Nearest Correlation Matrix (with Constraints) min X subject to n X (Xij Hij )2 i,j=1 Xii = 1, i = 1,..., n X 0 correlation matrix = symmetric positive semide nite matrix with unit diagonal H approximate correlation matrix X new (true) correlation matrix closest to H in Frobenius norm 25

36 Nearest Correlation Matrix (with Constraints) min X subject to n X (Xij Hij )2 i,j=1 Xii = 1, i = 1,..., n X 0 correlation matrix = symmetric positive semide nite matrix with unit diagonal H approximate correlation matrix X new (true) correlation matrix closest to H in Frobenius norm do not use SDP on vanilla NCM due to algorithm complexity; special solvers in G02 are preferrable 25

37 Nearest Correlation Matrix (with Constraints) n X min X (Xij Hij )2 i,j=1 subject to Xii = 1, i = 1,..., n X 0 Possible new constraints: x elements: Xij = Hij element-wise bounds: for some i, j lij Xij uij smallest eigenvalue constraint: X λmin I, where λmin given λmax I X λmin I, λmax κλmin, λmin, λmax are new variables limit condition number: where κ is given and 25

38 Nearest Correlation Matrix (with Constraints) n X min X (Xij Hij )2 i,j=1 subject to Xii = 1, i = 1,..., n X 0 Possible di erent objective: weight elements: P Wij (Xij Hij )2 P V arα : λzα2 wt DXDw + (Xij Hij )2 (dii = σi ), w asset allocation, λ weighting factor consider portfolio D deviations 25

39 Nearest Correlation Matrix (with Constraints) min X subject to n X (Xij Hij )2 i,j=1 Xii = 1, i = 1,..., n X 0 Full control over the formulation! 25

40 Robust Portfolio Optimization mean-variance analysis often very sensitive to the data are nominal µ (expected returns) and Σ (covariance) correct? robust EF = limit sensitivity of the results by incorporating uncertainity model on parameters choose solution in the worst-case scenario (see Boyd '07) 26

41 Robust Portfolio Optimization mean-variance analysis often very sensitive to the data are nominal µ (expected returns) and Σ (covariance) correct? robust EF = limit sensitivity of the results by incorporating uncertainity model on parameters choose solution in the worst-case scenario (see Boyd '07) min (µ r 1 + λ)t Σ 1 (µ r 1 + λ) subject to Fµ 0 µi µ i α1 µ i, T T i = 1,..., n T 1 µ 1 µ α2 1 µ Σij Σ ij β1 Σ ij, i, j = 1,..., n Σ Σ F β2 Σ F Σ 0 λ 0 26

42 Calibration of volatility structure How to extract correlation information from market option prices? assume LIBOR market model with covariance structure swap weights X and Ω = wwt under some assumptions, swaption prices are given by Black-Scholes formula with volatility parameter Task: calibrate X σ = Tr(ΩX) to observed swaption market prices: nd X subject to Tr(ΩX) =σ X 0 where σ are observed swaption implied vols 27

43 Calibration of volatility structure cont. Correlation X in the previous feasibility problem not unique, can choose objective: e min or max price of some other option: min/max Tr(ΩX) norm of X: minkxk smoothness: mink Xk robustness via Bid/Ask spread: max t rank of s.t. X σ Bid + t Tr(ΩX) σ Ask t as a heuristic via nuclear norm of X 28

44 Risk-management: How to construct positive Γ portfolio? Π of derivatives/exotics on underlying Si : Π = F (S1,..., Sn ) managed usual Delta hedging: Π/ S = 0 assume existing portfolio Π must be risk but Delta hedging only works for very small movements in the underlyings, for larger would like to keep positive (or small) Π S Γ as S T SΠ2 S + to construct positive Γ: buy xi yi of underlying Si dπ = units of vanilla option pi on Si and 29

45 Risk-management: How to construct positive Γ portfolio? Π of derivatives/exotics on underlying Si : Π = F (S1,..., Sn ) managed usual Delta hedging: Π/ S = 0 assume existing portfolio Π must be risk but Delta hedging only works for very small movements in the Γ as S T SΠ2 S + to construct positive Γ: buy xi units of vanilla option pi on Si and yi of underlying Si X minx,y xi pi (Si ) + yi Si 2F 2 pi subject to + diag xi 2 0 S 2 Si pi F + xi + yi = 0, i = 1,..., n Si Si underlyings, for larger would like to keep positive (or small) dπ = Π S 29

46 Nonlinear programming: active set versus interior point methods Overview Sequential quadratic programming Interior point methods Illustration on a few examples Mixed integer nonlinear optimisation Semide nite programming Sample applications in nance Coming next Large-scale linear programming Derivative free solver for calibration Working with customers 30

47 Coming next new LP solver NAG = Amazon of optimization (be a one-stop-shop for all you need in optimization) Constant evolution of the library based on our roadmap customers' requests latest research & collaborations... ongoing hard work 31

48 Coming next new LP solver NAG = Amazon of optimization (be a one-stop-shop for all you need in optimization) Constant evolution of the library based on our roadmap customers' requests latest research & collaborations... ongoing hard work New LP solver new solver for large-scale LP problems based on interior point method (IPM) lling the missing gap signi cant speed-up 31

49 Coming next DFO for calibration Standard data- tting (calibration) problem given oberved data parameters Task: nd [ti, yi ]; model f ( ; x) depending on model x x to t the data as close as possible, typically in least square sense: minx P (yi f (ti ; xi ))2 32

50 Coming next DFO for calibration Standard data- tting (calibration) problem given oberved data parameters Task: nd [ti, yi ]; model f ( ; x) depending on model x x to t the data as close as possible, typically in least square sense: minx P (yi f (ti ; xi ))2 Additional requirements small number of parameters (< 100) black-box model, no derivatives available possibly expensive and/or inaccurate function evaluations typically reasonable starting point, small improvement su cient nite di erences shouldn't be used! 32

51 Coming next DFO for calibration Standard data- tting (calibration) problem given oberved data parameters Task: nd [ti, yi ]; model f ( ; x) depending on model x x to t the data as close as possible, typically in least square sense: minx P (yi f (ti ; xi ))2 Additional requirements small number of parameters (< 100) black-box model, no derivatives available possibly expensive and/or inaccurate function evaluations typically reasonable starting point, small improvement su cient nite di erences shouldn't be used! New Derivative free optimization (DFO) solver exploiting the problem structure (the only of its kind!) 32

52 Nonlinear programming: active set versus interior point methods Overview Sequential quadratic programming Interior point methods Illustration on a few examples Mixed integer nonlinear optimisation Semide nite programming Sample applications in nance Coming next Large-scale linear programming Derivative free solver for calibration Working with customers 33

53 Working with customers Sometimes solution out of the box is not su cient! Is it possible to speed up the solver? Does the model t the solver? Can a special problem structure be exploited? 34

54 Working with customers Sometimes solution out of the box is not su cient! Is it possible to speed up the solver? Does the model t the solver? Can a special problem structure be exploited? NAG Mathematical Optimization Consultancy ready to help! choice and tuning of the solver adjustments with the model bespoke solver development 34

55 Examples of optimisation projects Energy & Commodities Trading Co. The client's model was demonstrating unusual behaviour - signi cant memory footprint and slow convergence. Analysis of the model showed that a more suitable equivalent reformulation is available. When the model was adjusted, the solver performed as expected. Financial Services Software Vendor extended site visit of a client allowed us to discuss client's problem in detail and helped to identify a weak point which was causing convergence issues and x. Financial Brokerage Co. The client wanted a class of problems to be solved within the prescribed time limit. After the initial assessment of the problem, a possible solution was identi ed using recent research from Stanford university. A bespoke solution was delivered during a short consulting engagement. The new solver drastically improved the performance so that even bigger problems could be considered by the client. 35