1. Mixed Integer
Programming:
Analyzing 12 Years of Progress

2. © 2013 IBM Corporation
Background
 2001: Manfred Padberg’s 60th birthday
– Bixby et al., “Mixed-Integer Programming: A Progress Report”, in: Grötschel (ed.) The
Sharpest Cut: The Impact of Manfred Padberg and His Work, MPS-SIAM Series on
Optimization, pp.309-325, SIAM, Philadelphia (2004)
• Analysis of the relative contributions of the key ingredients of Branch-and-Cut
Algorithms for solving MIPs
 2013: Martin Grötschel’s 65th birthday
– T. Achterberg and R.W., “Mixed Integer Programming: Analyzing 12 Years of
Progress”, in: Jünger and Reinelt (eds.) Facets of Combinatorial Optimization,
Festschrift for Martin Grötschel, pp.449-481, Springer, Berlin-Heidelberg (2013)
• What stayed?
• What changed?
• Why?
2

3. © 2013 IBM Corporation
Agenda
 Methodology
– How to Benchmark
– How to Measure Importance of Features
 Analysis
– Presolving
– Cuts
– Heuristics
– Parallelism
 Summary
3

4. © 2013 IBM Corporation
Benchmarking
 Run competing algorithms on set of problem instances
– Measure and compare runtime, timeouts
– Use geometric mean for aggregation
 Performance Variability
– Seemingly performance neutral changes (random seed, platform, permutation of
variables, …) have drastic impact on solution time
– Has been observed for a long time, e.g.
• Emilie Danna: Performance variability in mixed integer programming,
Presentation at Workshop on Mixed Integer Programming 2008
– Friend or foe for Solvers?
• Tuesday Oct 08, 08:00 - 09:30, Andrea Lodi:
Performance Variability in Mixed-integer Programming
• Wednesday Oct 09, 15:30 - 17:00, Andrea Tramontani:
Concurrent root cut loops to exploit random performance variability
– Definitely foe for Benchmarking
4

5. © 2013 IBM Corporation
A benchmarking myth
 Compare Solver S against reference Solver R
– Solver S claimed to be faster than R on “hard problems”
 Model Set M
– Solution times tR(m), m M, for S and R are in (0sec, 100sec]
 Classify Models in to hard models H and easy models E
– m  H, if tR(m) > 80sec
– m  E, otherwise
 Computational confirmation of speedup:
– 1.8x faster on hard models
– 0.8x slower on easy models
5
1.80
0.80
0.00
1.00Speedup

6. © 2013 IBM Corporation
A benchmarking myth
 Compare Solver S against reference Solver R
– Solver S claimed to be faster than R on “hard problems”
 Model Set M
– Solution times tR(m), m M, for S and R are in (0sec, 100sec]
 Classify Models in to hard models H and easy models E
– m  H, if tR(m) > 80sec
– m  E, otherwise
 Computational confirmation of speedup:
– 1.8x faster on hard models
– 0.8x slower on easy models
 Times for S and R uniform random numbers:
– <tR(E)> = 40 <tR(H)> = 90
– <tS(E)> = 50 <tS(H)> = 50
– Speedup: 4/5 9/5
6
1.80
0.80
0.00
1.00Speedup

7. © 2013 IBM Corporation
A benchmarking myth
 Compare Solver S against reference Solver R
– Solver S claimed to be faster than R on “hard problems”
 Model Set M
– Solution times tR(m), m M, for S and R are in (0sec, 100sec]
 Classify Models in to hard models H and easy models E
– m  H, if tR(m) > 80sec
– m  E, otherwise
 Computational confirmation of speedup:
– 1.8x faster on hard models
– 0.8x slower on easy models
 Times for S and R uniform random numbers:
– <tR(E)> = 40 <tR(H)> = 90
– <tS(E)> = 50 <tS(H)> = 50
– Speedup: 4/5 9/5
7
1.80
0.80
0.00
1.00Speedup

8. © 2013 IBM Corporation
Avoiding the bias
 The problem is real:
2 different random seeds
for CPLEX 12.5
 Problem comes from using times
from one solver to define subsets
of problems
8
biased subsets # of problems geomean
All 3159 1.00
[0,10k] 3082 1.00
[1,10k] 1848 0.99
[10,10k] 1074 0.95
[100,10k] 552 0.87
[1k,10k] 207 0.76

9. © 2013 IBM Corporation
Avoiding the bias
 The problem is real:
2 different random seeds
for CPLEX 12.5
 Problem comes from using times
from one solver to define subsets
of problems
 Solution
–Use (max) times from all solvers
to define subsets of problems
9
biased subsets # of problems geomean
All 3159 1.00
[0,10k] 3082 1.00
[1,10k] 1848 0.99
[10,10k] 1074 0.95
[100,10k] 552 0.87
[1k,10k] 207 0.76
unbiased subsets # of problems geomean
All 3159 1.00
[0,10k] 3082 1.00
[1,10k] 1879 1.00
[10,10k] 1121 1.01
[100,10k] 604 1.01
[1k,10k] 238 1.08

10. © 2013 IBM Corporation
Avoiding the bias
 The problem is real:
2 different random seeds
for CPLEX 12.5
 Problem comes from using times
from one solver to define subsets
of problems
 Solution
–Use (max) times from all solvers
to define subsets of problems
 Note
–250 models can not measure
performance difference of less
than 10%
–Will use [10,10k] bracket
10
biased subsets # of problems geomean
All 3159 1.00
[0,10k] 3082 1.00
[1,10k] 1848 0.99
[10,10k] 1074 0.95
[100,10k] 552 0.87
[1k,10k] 207 0.76
unbiased subsets # of problems geomean
All 3159 1.00
[0,10k] 3082 1.00
[1,10k] 1879 1.00
[10,10k] 1121 1.01
[100,10k] 604 1.01
[1k,10k] 238 1.08

11. © 2013 IBM Corporation
Measuring Impact
 MIP is a bag of tricks
–Presolving
–Cutting planes
–Branching
–Heuristics
–...
 How important is each trick?
Compare runs with feature turned on and off
–Solution time degradation
(geometric mean)
–# of solved models
• Essential or just speedup?
–Number of affected models
• General or problem specific?
11
Bixby et al. 2001
Feature Degradation
No cuts 53.7x
No presolve 10.8x
Trivial branching 2.9x
No heuristics 1.4x

12. © 2013 IBM Corporation
Component Impact CPLEX 12.5 Summary
Benchmarking setup
• 1769 models
• 12 core Intel Xenon 2.66 GHz
• Unbiased: At least one of all the
test runs took at least 10sec
99% 82% 91% 26%93%91% 46%83% 65%% affected
12

13. © 2013 IBM Corporation
Component Impact CPLEX 12.5 Summary
Fundamental Features
• Lots of models unsolvable
without
• Apply to most models
99% 82% 91% 26%93%91% 46%83% 65%% affected
13

14. © 2013 IBM Corporation
Component Impact CPLEX 12.5 Summary
Important Features
• Many models unsolvable
without
• Apply to most models
99% 82% 91% 26%93%91% 46%83% 65%% affected
14

15. © 2013 IBM Corporation
Component Impact CPLEX 12.5 Summary
Parallelism is not important
• turning off == 12x fewer cycles
i.e. just a tighter time limit
• Hardware cannot defeat
combinatorial explosion
99% 82% 91% 26%93%91% 46%83% 65%% affected
15

16. © 2013 IBM Corporation
Component Impact CPLEX 12.5 Summary
Special Features
• Few models unsolvable
without
• Apply to few models
99% 82% 91% 26%93%91% 46%83% 65%% affected
16

17. © 2013 IBM Corporation
Component Impact CPLEX 12.5 Summary
• Only 106 models
• Biased towards Cuts due to
selection of “hard” models
for CPLEX 5.0
• Impact of other components
matches
99% 82% 91% 26%93%91% 46%83% 65%% affected
17

18. © 2013 IBM Corporation
Component Impact CPLEX 12.5 Summary
99% 82% 91% 26%93%91% 46%83% 65%% affected
18

19. © 2013 IBM Corporation
Component Impact CPLEX 12.5 – Presolve
19

20. © 2013 IBM Corporation
Component Impact CPLEX 12.5 Summary
99% 82% 91% 26%93%91% 46%83% 65%% affected
20

21. © 2013 IBM Corporation
Component Impact CPLEX 12.5 – Cutting Planes
21

22. © 2013 IBM Corporation
Component Impact CPLEX 12.5 – Cutting Planes
2.52 1.40 1.19 1.22 1.041.021.83 1.02
Bixby et al. 2001
22

23. © 2013 IBM Corporation
Component Impact CPLEX 12.5 Summary
99% 82% 91% 26%93%91% 46%83% 65%% affected
23

24. © 2013 IBM Corporation
Component Impact CPLEX 12.5 – Primal Heuristics
24

25. © 2013 IBM Corporation
Component Impact CPLEX 12.5 Summary
99% 82% 91% 26%93%91% 46%83% 65%% affected
25

26. © 2013 IBM Corporation
Parallelism in CPLEX
Types of Parallelism
–Opportunistic Parallelism
–Deterministic Parallelism
• Identical runs produce same solution path and results
• Use deterministic locks
• Based on deterministic time
• Implemented via counting memory accesses
Parallel Tasks
–Root node parallelism
• Concurrent LP solve
• Heuristics concurrent to cutting phase
• Parallel Cut loop
–Parallel Processing of B&C Tree
26

27. © 2013 IBM Corporation
The Cost of Determinism
27

28. © 2013 IBM Corporation
0
200
400
600
800
1000
1200
1400
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
numberoftimeouts
0
50
100
150
200
250
totalspeedup
 10 sec
 100 sec
 1000 sec
Date: 28 September 2013
Testset: 3147 models (1792 in  10sec, 1554 in  100sec, 1384 in  1000sec)
Machine: Intel X5650 @ 2.67GHz, 24 GB RAM, 12 threads (deterministic since CPLEX 11.0)
Timelimit: 10,000 sec

29. © 2013 IBM Corporation
Related presentations
 Recent Developments in CPLEX Monday, 08:00 - 09:30 Tobias Achterberg
 Non-convex Quadratic Programming in CPLEX Tuesday, 11:00 - 12:30 Christian Bliek and
Pierre Bonami
 Tutorial: Performance Variability in Mixed-integer Programming Tuesday, 8:00 – 9:30
Andrea Lodi and Andrea Tramontani
 Software Tutorial: Expert Tips and Tricks for Using CPLEX Optimization Studio Tuesday
Oct 08, 08:00 - 09:30
 Lift-and-Project Cuts in CPLEX 12.5.1 Wednesday, 13:30 - 15:00 Andrea Tramontani
 Concurrent root cut loops to exploit random performance variability Wednesday, 15:30
- 17:00 A. Tramontani, M. Fischetti, A. Lodi, D. Salvignin, M. Monaci
 Interesting Use Cases for the CPLEX Remote Object Wednesday, 15:30 - 17:00 Lazlo
Ladanyi and Daniel Junglas
29

30. © 2013 IBM Corporation30

31. © 2013 IBM Corporation
Component Impact CPLEX 12.5 Summary
31

32. © 2013 IBM Corporation
Component Impact CPLEX 12.5 – Branching
32