2007 : Exploiting Problem Structure in the MOSEK simplex optimizers (seattle 2007)

Exploiting Problem Structure in the
MOSEK Simplex Optimizers.
Bo Jensen ∗
MOSEK ApS,
Fruebjergvej 3, Box 16, 2100 Copenhagen,
Denmark.
Email: bo.jensen@mosek.com

INFORMS Annual Meeting Seattle Nov. 5, 2007
∗
http://www.mosek.com

Introduction

2 / 27

What is MOSEK

Introduction
I A software package for solving large-scale optimization
Topics

The simplex
problems.
optimizers I Solves linear, conic, and nonlinear convex problems.
Exploiting problem
dependent structure
I Has mixed-integer capabilities.
in the simplex
optimizer, example
I Stand-alone as well as embedded.
one: I Used to solve problems with up to millions of constraints
Exploiting problem
dependent structure and variables.
in the simplex
optimizer, example I Version 1 released in 1999.
two:
I Version 5 released July 2007.
Conclusions

3 / 27

Topics

Introduction
I The problem:
Topics

The simplex
(P ) min cT x
optimizers st Ax = b,
Exploiting problem
dependent structure
x ≥ 0.
in the simplex
optimizer, example
one: I The linear optimizers.
Exploiting problem
dependent structure N Interior-point optimizer (Not main focus in this talk).
in the simplex
optimizer, example N Simplex optimizer.
two:

Conclusions I Talk aim : Demonstrate by example the importantness of
using model structure in the simplex optimizer.

4 / 27

The simplex optimizers

5 / 27

What makes a good simplex optimizer ?

Introduction
I Exploit sparsity (i.e. LU and FTRAN and BTRAN
The simplex
optimizers routines).
What makes a good
simplex optimizer ?
I Exploit problem dependent structure.
MOSEK I Choose right path (i.e. good pricing strategy).
simplex-overview

Exploiting problem
I Long steps (i.e. avoid degeneracy).
dependent structure I Numerical stability (i.e. reliable and consistent results).
in the simplex
optimizer, example I Fast hotstarts (i.e. MIP and other hotstart applications).
one:
Exploiting problem I Other tricks.
dependent structure
in the simplex
optimizer, example
two:

Conclusions

6 / 27

MOSEK simplex-overview

Introduction
I Primal and dual simplex optimizer.
The simplex
optimizers
What makes a good
N Efﬁcient cold start and warm start.
simplex optimizer ? N Crashes an initial basis.
MOSEK
simplex-overview N Multiple pricing options:
Exploiting problem
dependent structure I Full (Dantzig).
in the simplex
optimizer, example I Partial.
one:
I Approximate/exact steepest edge.
Exploiting problem
dependent structure I Hybrid.
in the simplex
optimizer, example
two: N Degeneration handling.
Conclusions
I Revised simplex algorithm + many enhancements.
I Many enhancements still possible!.

7 / 27

Exploiting problem dependent structure in
the simplex optimizer, example one:

8 / 27

Problem specifications ex : neos2 [HM:07]

Introduction
Constraints :
The simplex
optimizers
I 132569 constraints.
Exploiting problem
dependent structure I 552596 non-zeroes in constraint matrix.
in the simplex
optimizer, example I Only one constraint has a non-zero lower bound.
one:
Problem
I All upper bounds are infinity.
specifications
Problem structure
Solution times from
different solvers
Observations Variables :
First improvement-
general I 1560 variables.
Second
improvement- I Only 77 variables has non-zero a cost.
Primal
simplex I All lower bounds are zero.
Exploiting problem I All upper bounds are infinity.
dependent structure
in the simplex
optimizer, example
two:

Conclusions

9 / 27

Problem structure

Introduction
The simplex
optimizers

Exploiting problem
dependent structure
in the simplex
optimizer, example
one:
Problem
speciﬁcations
Problem structure
Solution times from
different solvers
Observations
First improvement-
general
Second
improvement-
Primal
simplex

Exploiting problem
dependent structure
in the simplex
optimizer, example
two:

Conclusions

10 / 27

Solution times from different solvers [HM:07]

Introduction
Non Commercial :
The simplex
optimizers
Problem Rows Cols Clp Qsopt Soplex GLPK
Exploiting problem
dependent structure neos1 131582 1892 113 987 497 261
in the simplex
optimizer, example
neos2 132569 1560 814 1634 746 > 30000
one: neos3 512209 6624 31471 ? ? f
Problem
speciﬁcations
Problem structure Commercial :
Solution times from
different solvers
Observations
Problem Cplex (D) Cplex (P) Mosek 5 (D) Mosek 5 (P)
First improvement- neos1 18 930 39 8
general
Second neos2 31 532 75 13
improvement-
Primal
neos3 6350 4046 7008 59
simplex

Exploiting problem Warning : The numerical results displayed is only use to illustrate a speciﬁc simplex issue and should not be read
dependent structure as a benchmark, please see http://plato.la.asu.edu/bench.html or http://www.mosek.com/index.php?id=103
in the simplex for a benchmark on a larger problem set.
optimizer, example
two:

Conclusions Why are the primal simplex so fast on these problems ?

11 / 27

Observations

Introduction
The simplex
optimizers
I High tall problem i.e. has many more constraints than
Exploiting problem variables.
dependent structure
in the simplex
I Few non-zero costs in both primal and dual
optimizer, example
one:
formulation, potentially degeneracy problems.
Problem
speciﬁcations
I Dual simplex does quite well, but it is hard to explain
Problem structure neos3 times i.e. a very similar problem as neos1-2.
Solution times from
different solvers
Observations
First improvement-
general
Second
improvement-
Primal
simplex

Exploiting problem
dependent structure
in the simplex
optimizer, example
two:

Conclusions

12 / 27

First improvement-general

Introduction
Solve dual formulation instead :
The simplex
optimizers
I Maximization.
Exploiting problem
dependent structure I 1560 constraints and 132569 variables.
in the simplex
optimizer, example I Smaller basis dimension.
one:
Problem I Only one non-zero cost.
speciﬁcations
Problem structure
Solution times from
different solvers
Observations
First improvement-
general
Second
improvement-
Primal
simplex

Exploiting problem
dependent structure
in the simplex
optimizer, example
two:

Conclusions

13 / 27

Second improvement-Primal simplex

Introduction
Exploit the objective is deﬁned by the level of one variable :
The simplex
optimizers 0 ≤ xo ≤ inf where co < 0.
Exploiting problem
dependent structure
in the simplex
min co xo
optimizer, example
one:
st Ax = b,
Problem
speciﬁcations
x ≥ 0.
Problem structure
Solution times from
different solvers
Observations Idea :
First improvement-
general I High objective level in primal start solution ( even if it
Second
improvement- should be infeasible ).
Primal
simplex I Temporary impose a fake lower bound 0 < lo on xo .
Exploiting problem I If the new problem is primal infeasible lower lo .
dependent structure
in the simplex
optimizer, example
two:

Conclusions

14 / 27

Second improvement-Primal simplex-(continued)

Introduction
Advantages :
The simplex
optimizers
I Higher objective when feasibility is achieved.
Exploiting problem
dependent structure I Normally fast reoptimization from one adjustment to an-
in the simplex
optimizer, example other.
one:
Problem
I Avoids getting xo into the basis before late in the process.
specifications
I Less degenerated.
Problem structure
Solution times from
different solvers
Observations
First improvement- Disadvantages :
general
Second I Primal feasibility is achieved later.
improvement-
Primal
simplex N Skip the process if phase one takes a ”long time”.
Exploiting problem
dependent structure I If xo does not have an finite upper bound, how should we
in the simplex
optimizer, example choose lo ?
two:

Conclusions
N Use presolve techniques to find bounds on xo .
N Use other information to bound xo i.e dual feasible so-
lutions.
15 / 27
N Skip the process if bounds on xo are not ”tight”.


Introduction
Advantages :
The simplex
optimizers
I Higher objective when feasibility is achieved.
Exploiting problem
dependent structure I Normally fast reoptimization from one adjustment to an-
in the simplex
optimizer, example other.
one:
Problem
I Avoids getting xo into the basis before late in the process.
specifications
I Less degenerated.
Problem structure
Solution times from
different solvers Disadvantages :
Observations
First improvement- I Primal feasibility is achieved later.
general
Second
improvement-
N Skip the process if phase one takes a ”long time”.
Primal
simplex I If xo does not have an finite upper bound, how should we
Exploiting problem
dependent structure
choose lo ?
in the simplex
optimizer, example N Use presolve techniques to find bounds on xo .
two:
N Use other information to bound xo i.e dual feasible so-
Conclusions
lutions.
N Skip the process if bounds on xo are not ”tight”.
15 / 27


Introduction
Solution times primal simplex :
The simplex
optimizers

Exploiting problem
dependent structure Problem Without trick With trick
in the simplex
optimizer, example
Iter Time Iter Time
one: neos1 5339310 1377 12064 3
Problem
speciﬁcations neos2 8878560 2240 8792 3
Problem structure
Solution times from
neos3 939365 1969 22010 23
different solvers
Observations
First improvement-
general Run on a 2.33GHz Xeon Quad Core with 8GB RAM.
Second
improvement- Huge speed up !
Primal
simplex

Exploiting problem
dependent structure
in the simplex
optimizer, example
two:

Conclusions

16 / 27

Exploiting problem dependent structure in
the simplex optimizer, example two:

17 / 27

Problem speciﬁcations ex : rail4284 [BEAS:87]

Introduction
Set covering problem :
The simplex
optimizers

Exploiting problem
(P ) min cT x
dependent structure
in the simplex
st Ax ≥ 1,
optimizer, example
one:
x ≥ 0.
Exploiting problem
dependent structure I 4284 constraints.
in the simplex
optimizer, example I 1092610 variables.
two:
Problem I 12372358 non-zeroes in constraint matrix (all ones).
speciﬁcations
Observations based
on primal simplex
Primal restricted
pricing

Conclusions

18 / 27

Observations based on primal simplex

Introduction
The simplex
optimizers
I Problem is long and slim i.e. has many more variables
Exploiting problem than constraints (scheduling problems).
dependent structure
in the simplex
I Duals tend to be unstable i.e. changing a lot within few
optimizer, example
one:
iterations.
Exploiting problem I Partial pricing does not work well.
dependent structure
in the simplex I Updated row is expensive (excludes steepest-edge (SE)).
optimizer, example
two:
Problem
speciﬁcations What can we do ?
Observations based
on primal simplex
Primal restricted
pricing

Conclusions

19 / 27

Primal restricted pricing

Introduction
The simplex
Idea :
optimizers

Exploiting problem 1. Choose a subset X SP of dual infeasible columns from (P ).
dependent structure
in the simplex
optimizer, example
one:
2. If X SP = ∅ then stop, the solution is optimal.
Exploiting problem 3. Form a subproblem (SP ) with columns in X SP and
dependent structure
in the simplex columns in the basis B.
optimizer, example
two: 4. Optimize (SP ) with primal simplex.
Problem
speciﬁcations 5. Reduce (SP ) to contain only columns in B.
Observations based
on primal simplex
6. Goto 1.
Primal restricted
pricing

Conclusions

Yes, this is just a special case of partial pricing !

20 / 27

Restricted pricing (continued)

Introduction
Advantages :
The simplex
optimizers
I Updated row for the subproblems is less expensive.
Exploiting problem
dependent structure I SE,ASE and Devex pricing is now an option.
in the simplex
optimizer, example
one:
Exploiting problem
dependent structure Disadvantages :
in the simplex
optimizer, example I Induces some overhead.
two:
Problem I On some problems there can be a small slow down.
speciﬁcations
Observations based
on primal simplex
Primal restricted
pricing

Conclusions

21 / 27

Restricted pricing (continued)

Introduction
Mosek 5 WOR Mosek 5
The simplex
optimizers Problem Rows Cols Iter Time Iter Time
Exploiting problem rail582 582 55515 71905 9.11 6560 2.24
dependent structure
in the simplex rail2586 2586 920683 1748004 1605 73675 154.27
optimizer, example
one:
rail4284 4286 1092610 ? > 6000 111363 420.41
Exploiting problem
dependent structure
in the simplex
optimizer, example Run on a 2.33GHz Xeon Quad Core with 8GB RAM.
two:
Problem Improvements :
speciﬁcations
Observations based I Fewer iterations.
on primal simplex
Primal restricted I Faster solve.
pricing

Conclusions
See http://www.mosek.com/index.php?id=103 for
benchmarks between Mosek 4 and 5.

22 / 27

Conclusions

23 / 27

Conclusions

Introduction
I Exploiting problem dependent structure can speed up
The simplex
optimizers solution times dramatically.
Exploiting problem I Tricky sparsity and degeneracy issues often comes into
dependent structure
in the simplex play.
optimizer, example
one:
Exploiting problem
dependent structure
in the simplex
optimizer, example
two:

Conclusions
Conclusions
Difﬁculties
References
Want to know more
about Mosek ?

24 / 27

Difﬁculties

Introduction
I The modeler often has valuable model information which
The simplex
optimizers the optimizer has not.
Exploiting problem I The modeler often has limited knowledge about
dependent structure
in the simplex advanced simplex features.
optimizer, example
one: I Hard to auto-detect and generalize.
Exploiting problem
dependent structure
in the simplex
optimizer, example
two:

Conclusions
Conclusions
Difﬁculties
References
Want to know more
about Mosek ?

25 / 27

References

Introduction
The simplex
[HM:07] H.Mittelmann http://plato.la.asu.edu/bench.html
optimizers

Exploiting problem [BEAS:87] J.E.Beasley ”An algorithm for set covering problems”
dependent structure
in the simplex European Journal of Operational Research 31 (1987) 85-93.
optimizer, example
one:
Exploiting problem
dependent structure
in the simplex
optimizer, example
two:

Conclusions
Conclusions
Difﬁculties
References
Want to know more
about Mosek ?

26 / 27

Want to know more about Mosek ?

Introduction
The simplex
Session Information :
optimizers Wednesday Nov 07, 10:00 - 11:30.
Exploiting problem
dependent structure
Title :
in the simplex
optimizer, example
Joint Session ICS/OS: Recent Development in Software for
one: Linear and Convex Optimization Problems.
Exploiting problem
dependent structure Chair :
in the simplex
optimizer, example Erling Andersen,CEO, MOSEK ApS, Fruebjergvej 3 Box 16,
two:
Copenhagen O 2100, Denmark, e.d.andersen@mosek.com.
Conclusions
Conclusions
Difﬁculties
References
Want to know more
about Mosek ?

27 / 27

2007 : Exploiting Problem Structure in the MOSEK simplex optimizers (seattle 2007)

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie 2007 : Exploiting Problem Structure in the MOSEK simplex optimizers (seattle 2007)

Ähnlich wie 2007 : Exploiting Problem Structure in the MOSEK simplex optimizers (seattle 2007) (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

2007 : Exploiting Problem Structure in the MOSEK simplex optimizers (seattle 2007)