1. Comparison-Based Complexity
of Multi-Objective Optimization
Paper by O. Teytaud
Presented by M. Schoenauer
TAO, Inria, Lri, UMR CNRS 8623,
Université Paris-Sud

2. Outline
Multiobjective optimization
Complexity upper bounds
Complexity lower bounds
Discussion

3. Evolutionary multi-objective
optimization
Generate a population
Evaluate fitness values
Select the « best » (various rules are possible
here)
Generate offspring
Go back to the Evaluation step.

4. Model of fitness functions ?
No assumption ?
Unrealistically pessimistic results
Unreadable lower bounds
Let's do as in:
==> quadratic
convex objective
functions

5. Outline
Multiobjective optimization
Complexity upper bounds
Complexity lower bounds
Discussion

6. Complexity upper bounds
Each objective function = a sphere
Below just a short overview of algorithms;
==> the real algorithms are a bit more
tricky
For finding the whole Pareto front :
Optimize each objective separately
The PF is the convex hull
For finding a single point :
Optimize any single objective

7. Finding one point of the Pareto Set
d objective functions
In dimension N
One point at distance at most e of the PF
cost=O( (N+1-d) log (1/e) )
Proof : M log(1/e) in monoobjective
optimization where M is the codimension
of the set of optima (Gelly Teytaud, 2006)

8. Finding the whole Pareto Set
d objective functions
In dimension N
One point at distance at most e of the PF for
the Hausdorff metric
cost=O( (Nd) log (1/e) )
Proof : d times the monoobjective case.

9. Outline
Multiobjective optimization
Complexity upper bounds
Complexity lower bounds
Proof technique (monoobjective)
The MO case

10. We want to know how many iterations we need for reaching precision 
in an evolutionary algorithm.
Key observation: (most) evolutionary algorithms are comparison-based
Let's consider (for simplicity) a deterministic selection-based non-elitist
algorithm
First idea: how many different branches we have in a run ?
We select  points among 
Therefore, at most K = ! / ( ! (  -  )!) different branches
Second idea: how many different answers should we able to give ?
Use packing numbers: at least N() different possible answers
Conclusion: the number n of iterations should verify
Kn ≥ N (  )
Frédéric Lemoine MIG 11/07/2008 10

11. We want to know how many iterations we need for reaching precision 
in an evolutionary algorithm.
Key observation: (most) evolutionary algorithms are comparison-based
Let's consider (for simplicity) a deterministic selection-based non-elitist
algorithm
First idea: how many different branches we have in a run ?
We select  points among 
Therefore, at most K = ! / ( ! (  -  )!) different branches
Second idea: how many different answers should we able to give ?
Use packing numbers: at least N() different possible answers
Conclusion: the number n of iterations should verify
Kn ≥ N (  )
Frédéric Lemoine MIG 11/07/2008 11

12. We want to know how many iterations we need for reaching precision 
in an evolutionary algorithm.
Key observation: (most) evolutionary algorithms are comparison-based
Let's consider (for simplicity) a deterministic selection-based non-elitist
algorithm
First idea: how many different branches we have in a run ?
We select  points among 
Therefore, at most K = ! / ( ! (  -  )!) different branches
Second idea: how many different answers should we able to give ?
Use packing numbers: at least N() different possible answers
Conclusion: the number n of iterations should verify
Kn ≥ N (  )
Frédéric Lemoine MIG 11/07/2008 12

13. We want to know how many iterations we need for reaching precision 
in an evolutionary algorithm.
Key observation: (most) evolutionary algorithms are comparison-based
Let's consider (for simplicity) a deterministic selection-based non-elitist
algorithm
First idea: how many different branches we have in a run ?
We select  points among 
Therefore, at most K = ! / ( ! (  -  )!) different branches
Second idea: how many different answers should we able to give ?
Use packing numbers: at least N() different possible answers
Conclusion: the number n of iterations should verify
Kn ≥ N (  )
Frédéric Lemoine MIG 11/07/2008 13

14. We want to know how many iterations we need for reaching precision 
in an evolutionary algorithm.
Key observation: (most) evolutionary algorithms are comparison-based
Let's consider (for simplicity) a deterministic selection-based non-elitist
algorithm
First idea: how many different branches we have in a run ?
We select  points among 
Therefore, at most K = ! / ( ! (  -  )!) different branches
Second idea: how many different answers should we able to give ?
Use packing numbers: at least N() different possible answers
Conclusion: the number n of iterations should verify
Kn ≥ N (  )
Frédéric Lemoine MIG 11/07/2008 14

15. We want to know how many iterations we need for reaching precision 
in an evolutionary algorithm.
Key observation: (most) evolutionary algorithms are comparison-based
Let's consider (for simplicity) a deterministic selection-based non-elitist
algorithm
First idea: how many different branches we have in a run ?
We select  points among 
Therefore, at most K = ! / ( ! (  -  )!) different branches
Second idea: how many different answers should we able to give ?
Use packing numbers: at least N() different possible answers
Conclusion: the number n of iterations should verify
Kn ≥ N (  )
Frédéric Lemoine MIG 11/07/2008 15

16. We want to know how many iterations we need for reaching precision 
in an evolutionary algorithm.
Key observation: (most) evolutionary algorithms are comparison-based
Let's consider (for simplicity) a deterministic selection-based non-elitist
algorithm
First idea: how many different branches we have in a run ?
We select  points among 
Therefore, at most K = ! / ( ! (  -  )!) different branches
Second idea: how many different answers should we able to give ?
Use packing numbers: at least N() different possible answers
Conclusion: the number n of iterations should verify
Kn ≥ N (  )
Frédéric Lemoine MIG 11/07/2008 16

17. We want to know how many iterations we need for reaching precision 
in an evolutionary algorithm.
Key observation: (most) evolutionary algorithms are comparison-based
Let's consider (for simplicity) a deterministic selection-based non-elitist
algorithm
First idea: how many different branches we have in a run ?
We select  points among 
Therefore, at most K = ! / ( ! (  -  )!) different branches
Second idea: how many different answers should we able to give ?
Use packing numbers: at least N() different possible answers
Conclusion: the number n of iterations should verify
Kn ≥ N (  )
Frédéric Lemoine MIG 11/07/2008 17

18. Outline
Multiobjective optimization
Complexity upper bounds
Complexity lower bounds
Proof technique (monoobjective)
The MO case

19. How to apply this in MOO ?
Covering numbers can be computed also for
Hausdorff distance ==>
Plus a little bit of boring maths
Leads to bounds as expected
Nd log(1/e) for the whole Pareto set (Hausdorff)
(N+1-d) log(1/e) for pointwise convergence (distance to
one point of the Pareto set)
N=dimension, d=nb of objectives

20. Results in multiobjective cases
The proof method is not new
(Fournier & Teytaud, Algorithmica 2010)
Its application to MOO is new :
Tight bounds
But no result in case of use of surrogate models
(as for corresponding results in the
monoobjective case) ; in fact, the problem
becomes unrealistically easy with surrogate
models...

21. Outline
Multiobjective optimization
Complexity upper bounds
Complexity lower bounds
Discussion

22. Sorry for not being here
==> really impossible
Discussion ==> all email questions welcome
Tight bounds thanks to a realistic model
Combining previous papers
Complexity bounds Relevant model
Maybe an extension : using VC-dimension
This paper did it
(single objective)