Interior-Point Algorithm for CLP II. yyye

Transcription

1 Conic Linear Optimization and Appl. Lecture Note #10 1 Interior-Point Algorithm for CLP II Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. yyye

2 Conic Linear Optimization and Appl. Lecture Note #10 2 Initialization Combining the primal and dual into a single linear feasibility problem, then applying LP algorithms to find a feasible point of the problem. Theoretically, this approach can retain the currently best complexity result. The big M method, i.e., add one or more artificial column(s) and/or row(s) and a huge penalty parameter M to force solutions to become feasible during the algorithm. Phase I-then-Phase II method, i.e., first try to find a feasible point (and possibly one for the dual problem), and then start to look for an optimal solution if the problem is feasible and bounded. Combined Phase I-Phase II method, i.e., approach feasibility and optimality simultaneously. To our knowledge, the best complexity of this approach is O(n log(r/ɛ)).

3 Conic Linear Optimization and Appl. Lecture Note #10 3 Homogeneous and Self-Dual LP Algorithm It solves the linear programming problem without any regularity assumption concerning the existence of optimal, feasible, or interior feasible solutions, while it retains the currently best complexity result It can start at any positive primal-dual pair, feasible or infeasible, near the central ray of the positive orthant (cone), and it does not use any big M penalty parameter or lower bound. Each iteration solves a system of linear equations whose dimension is almost the same as that solved in the standard (primal-dual) interior-point algorithms. If the LP problem has a solution, the algorithm generates a sequence that approaches feasibility and optimality simultaneously; if the problem is infeasible or unbounded, the algorithm will produce an infeasibility certificate for at least one of the primal and dual problems.

4 Conic Linear Optimization and Appl. Lecture Note #10 4 Primal-Dual Alternative Systems A pair of LP has two alternatives (Solvable) Ax b = 0 A T y + c 0, b T y c T x =0, y free, x 0 or (Infeasible) Ax = 0 A T y 0, b T y c T x > 0, y free, x 0

5 Conic Linear Optimization and Appl. Lecture Note #10 5 An Integrated Homogeneous System The two alternative systems can be homogenized as one: where the two alternatives are (HLP) Ax bτ = 0 A T y + cτ = s 0, b T y c T x = κ 0, y free, (x; τ) 0 (Solvable) : (τ > 0,κ =0) or (Infeasible) : (τ =0,κ > 0)

6 Conic Linear Optimization and Appl. Lecture Note #10 6 The Homogeneous System is Self-Dual (HLP) Ax bτ = 0, (y ) A T y + cτ = s 0, (x ) b T y c T x = κ 0, (τ ) y free, (x; τ) 0 (HLD) Ax bτ = 0, A T y cτ 0, b T y + c T x 0, y free, (x ; τ ) 0 Theorem 1 System (HLP) is feasible (e.g. all zeros) and any feasible solution (y, x, τ, s, κ) is self-complementary: x T s + τκ =0. Furthermore, it has a strictly self-complementary feasible solution x + s τ + κ > 0,

7 Conic Linear Optimization and Appl. Lecture Note #10 7 Let s Find Such a (non-trivial) Feasible Solution Given x 0 = e > 0, s 0 = e > 0, and y 0 = 0, we formulate min θ s.t. Ax bτ + bθ = 0, A T y +cτ cθ 0, b T y c T x + zθ 0, y free, x 0, τ 0, θ free, where b = b Ae, c = c e, z = c T e +1. But it may just give us the all-zero solution.

8 Conic Linear Optimization and Appl. Lecture Note #10 8 A HSLP linear program Let s try to add one more constraint to prevent the all-zero solution (HSLP) min (n +1)θ s.t. Ax bτ + bθ = 0, A T y +cτ cθ 0, b T y c T x + zθ 0, b T y + c T x zτ = (n +1), y free, x 0, τ 0, θ free. Note that the constraints of (HSLP) form a skew-symmetric system and the objective coefficient vector is the negative of the right-hand-side vector, so that it remains a self-dual linear program. (y = 0, x = e, τ=1,θ=1)is a strictly feasible point for (HSLP).

9 Conic Linear Optimization and Appl. Lecture Note #10 9 Denote by F h the set of all points (y, x,τ,θ,s,κ) that are feasible for (HSLP). Denote by Fh 0 the set of interior feasible points with (x,τ,s,κ) > 0 in F h.by combining the constraints, we can derive the last (equality) constraint as e T x + e T s + τ + κ (n +1)θ =(n +1), which serves indeed as a normalizing constraint for (HSLP) to prevent the all-zero solution.

10 Conic Linear Optimization and Appl. Lecture Note #10 10 Theorem 2 Consider problems (HSLP). i) (HSLP) has a strictly feasible point y = 0, x = e > 0, τ =1, θ =1, s = e > 0, κ =1. ii) (HSLP) has an optimal solution and its optimal solution set is bounded iii) The optimal value of (HSLP) is zero, and (y, x,τ,θ,s,κ) F h implies that (n +1)θ = x T s + τκ. iv) There is an optimal solution (y, x,τ,θ =0, s,κ ) F h such that x + s > 0, τ + κ which we call a strictly self-complementary solution.

11 Conic Linear Optimization and Appl. Lecture Note #10 11 Theorem 3 Let (y, x,τ,θ =0, s,κ ) be a strictly self complementary solution for (HSLP). i) (LP) has a solution (feasible and bounded) if and only if τ > 0. In this case, x /τ is an optimal solution for (LP) and (y /τ, s /τ ) is an optimal solution for (LD). ii) (LP) has no solution if and only if κ > 0. In this case, x /κ or s /κ or both are certificates for proving infeasibility: ifc T x < 0 then (LD) is infeasible; if b T y < 0 then (LP) is infeasible; and if both c T x < 0 and b T y < 0 then both (LP) and (LD) are infeasible.

12 Conic Linear Optimization and Appl. Lecture Note #10 12 Theorem 4 i) For any μ>0, there is a unique (y, x,τ,θ,s,κ) in F 0 h of (HSLP) such that Xs = μe. τκ ii) Let (d y, d x,d τ,d θ, d s,d κ ) be in the null space of the constraint matrix of (HSLP) after adding surplus variables s and κ, i.e., Ad x bd τ + bd θ = 0, A T d y +cd τ cd θ d s = 0, b T d y c T d x + zd θ d κ = 0, b T d y + c T d x zd τ = 0. (1) (d x ) T d s + d τ d κ =0.

13 Conic Linear Optimization and Appl. Lecture Note #10 13 Endogenous Potential Function and Central Path ψ n+ρ (x, s,τ,κ):=(n+1+ρ)log(x T s+τκ) and C = (y, x,τ,θ,s,κ) F0 h : Xs τκ n log(x j s j ) log(τκ), j=1 = xt s + τκ n +1 e. Obviously, the initial interior feasible point proposed in Theorem 2 is on the path with μ =1or (x 0 ) T s 0 + τ 0 κ 0 = n +1.

14 Conic Linear Optimization and Appl. Lecture Note #10 14 Solving (HSLP) Consider solving the following system of linear equations for (d y, d x,d τ,d θ, d s,d κ ) that satisfies (1) and Xd s + Sd x τ k d κ + κ k d τ = n +1 n +1+ρ μe Xs τκ.

15 Conic Linear Optimization and Appl. Lecture Note #10 15 Theorem 5 The O( n log((x 0 ) T s 0 /ɛ)) interior-point algorithm, coupled with a termination technique described above, generates a strictly self-complementary solution for (HSLP) in O( n(log(c(a, b, c)) + log n)) iterations and O(n 3 (log(c(a, b, c)) + log n)) operations, where c(a, b, c) is a positive number depending on the data (A, b, c). If (LP) and (LD) have integer data with bit length L, then by the construction, the data of (HSLP) remains integral and its length is O(L). Moreover, c(a, b, c) 2 L. Thus, the algorithm terminates in O( nl) iterations and O(n 3 L) operations.

16 Conic Linear Optimization and Appl. Lecture Note #10 16 Example Consider the example where A = ( ), b =1, and c = ( ). Then, y =2, x =(0, 2, 1) T, τ =0, θ =0, s =(2, 0, 0) T, κ =1 could be a strictly self-complementary solution generated for (HSDP) with c T x =1> 0, by =2> 0. Thus (y, s ) demonstrates the infeasibility of (LP), but x doesn t show the infeasibility of (LD). Of course, if the algorithm generates instead x =(0, 1, 2) T, then we get demonstrated infeasibility of both.

17 Conic Linear Optimization and Appl. Lecture Note #10 17 Primal-Dual SDP Alternative Systems A pair of SDP has two alternatives under mild conditions (Solvable) AX b = 0 A T y + C 0, b T y C X =0, y free, X 0 or (Infeasible) AX = 0 A T y 0, b T y C X > 0, y free, X 0

18 Conic Linear Optimization and Appl. Lecture Note #10 18 An Integrated Homogeneous System The two alternative systems can be homogenized as one: (HSDP) AX bτ = 0 A T y + Cτ = s 0, b T y C X = κ 0, y free, X 0, τ 0, where the three alternatives are (Solvable) : (Infeasible) : (τ>0,κ=0) (τ =0,κ>0) (All others) : (τ = κ =0).

19 Conic Linear Optimization and Appl. Lecture Note #10 19 The Homogeneous System is Self-Dual (HSDP) AX bτ = 0, (y ) A T y + Cτ = S 0, (X ) b T y C X = κ 0, (τ ) y free, X 0, τ 0. (HSDD) AX bτ = 0, A T y Xτ 0, b T y + C X 0, y free, X 0, τ 0,

20 Conic Linear Optimization and Appl. Lecture Note #10 20 Theorem 6 System (HSDP) is feasible (e.g. all zeros) and any feasible solution (y,x,τ,s,κ) is self-complementary: X S + τκ =0 or X 0 0 T τ S 0 0 T κ = 0. Furthermore, it has a max-rank complementary feasible solution, that is, is maximal. rank X 0 0 T τ + rank S 0 0 T κ

21 Conic Linear Optimization and Appl. Lecture Note #10 21 Let s Find Such a Feasible Solution Given X 0 = I 0, S 0 = I 0, and y 0 = 0, we formulate min θ s.t. AX bτ + bθ = 0, A T y +Cτ Cθ 0, b T y C X + zθ 0, y free, X 0, τ 0, θ free, where b = b A I, C = C I, z = C I +1. But it may just give us the all-zero solution.

22 Conic Linear Optimization and Appl. Lecture Note #10 22 A HS SDP program Let s try to add one more constraint to prevent the all-zero solution (HSSDP) min (n +1)θ s.t. AX bτ + bθ = 0, A T y +Cτ Cθ = S 0, b T y C X + zθ = κ 0, b T y + C X zτ = (n +1), y free, X 0, τ 0, θ free. Note that the constraints of (HSSDP) form a skew-symmetric system and the objective coeffcient vector is the negative of the right-hand-side vector, so that it remains a self-dual linear program.

23 Conic Linear Optimization and Appl. Lecture Note #10 23 By combining the constraints, we can derive the last (equality) constraint as I X + I S + τ + κ (n +1)θ =(n +1), which serves indeed as a normalizing constraint for (HSSDP) to prevent the all-zero solution.

24 Conic Linear Optimization and Appl. Lecture Note #10 24 Theorem 7 Consider problems (HSDP). i) (HSSDP) has a strictly feasible point y = 0, X = I 0, τ =1, θ =1, S = I 0, κ =1. ii) (HSSDP) has an optimal solution and its optimal solution set is bounded. iii) The optimal value of (HSSDP) is zero, and (y,x,τ,θ,s,κ) F h implies that (n +1)θ = X S + τκ. iv) There is an optimal solution (y,x,τ,θ =0,S,κ ) F h such that rank X 0 + rank S 0 0 T τ 0 T κ is maximal.

25 Conic Linear Optimization and Appl. Lecture Note #10 25 Theorem 8 Let (y,x,τ,θ =0,S,κ ) be a max-rank solution for (HSSDP). i) (SDP) and (SDD) are (Solvable) if and only if τ > 0. In this case, X /τ is an optimal solution for (SDP) and (y /τ,s /τ ) is an optimal solution for (SDD). ii) (SDP) or (SDD) is (Infeasible) if and only if κ > 0. In this case, X /κ or S /κ or both are certificates for proving infeasibility: ifc X < 0 then (SDD) is infeasible; if b T y < 0 then (SDP) is infeasible; and if both C X < 0 and b T y < 0 then both (SDP) and (SDD) are infeasible. iii) (SDP) and/or (SDD) are in all other cases if and only if τ = κ =0.

26 Conic Linear Optimization and Appl. Lecture Note #10 26 Software Implementation SEDUMI: SDPT3: mattohkc/sdpt3.html/ DSDP: hhttp://www-unix.mcs.anl.gov/dsdp/ MOSEK: CVX: boyd/cvx/