The document provides an overview of linear programming and its usage in approximation algorithms for NP-hard optimization problems. It discusses linear programming formulations, the complexity classes P and NP, approximation algorithms, and two case studies on the minimum weight vertex cover problem and the MAXSAT problem. Randomized rounding techniques are used to generate approximation algorithms for these problems from their linear programming relaxations.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
1. 1
Linear Programming and its Usage in
Approximation Algorithms for NP Hard
Optimization Problems
M. Reza Rahimi,
November 2005.
2. 2
Outline
• Introduction
• Linear Programming Overview
• NP Complexity Class
• Approximation Algorithms
• Case Study1: Minimum Weight Vertex Cover
• Case Study2: MAXSAT Problem
• Conclusion
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
3. 3
Introduction
• One of the most challenging problem in complexity
theory is problem P vs. NP.
NP
• Research on this open problem leads researchers to
think of new methods and different approaches.
• For example PCP, IP, approximation algorithms,...
• For some problems it is proved that there does not
exist any suitable approximation unless P=NP.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
4. 4
• So it seems that research on approximation
algorithms may lead us to good results about P vs.
NP.
• General Method for Approximation Algorithms of
NP Hard Optimization is Greedy Method.
• But we must search for Suitable Framework for
studying NP problems.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
5. 5
• Linear Programming was this brake through
method.
• In this talk I focus on LP and its usage in NP
Hard Optimization Problems.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
6. 6
Linear Programming Overview
• General formulation of linear programming is
presented as follow:
Maximize C1X1 + C2X2 + … + CdXd
Subject to A1,1X1 + … + A1,dXd ≤ b1
A2,1X1 + … + A2,dXd ≤ b2
…
An,1X1 + … + An,dXd ≤ bn
All calculation and Numbers are on Real Numbers.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
7. 7
• There are several Polynomial Algorithms for this
problem such as Ellipsoid and Interior point
method.
method
• I neglect talking about them.
• For these algorithms please refer to optimization
references.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
8. 8
NP Complexity Class
• NP Complexity Class is only related to Decision
Problem.
• For example:
SAT = {< Φ >: Φ is a satisfying assignment}.
• So for any optimization problem we consider its
related decision problem.
• This will give us intuition about its hardness.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
9. 9
• We know the following definition about NP:
L ∈ NP ⇔ ∃V(.,.) ∈ P, ∃P(.), ∀x ∈ Σ∗ ,
1. x ∈ L ⇒ ∃y, y ≤ P( x ) and V(x, y) accepts.
2. x ∉ L ⇒ ∀y, y ≤ P( x ) and V(x, y) rejects.
• We can look at this process like this:
Y
Find Certificate Y y V(X,Y)
X x
X
x
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
10. 10
• Example:
SAT = {< Φ >: Φ is a satisfying assignment}.
SAT ∈ NP Because :
1) if Φ 0 ∈ SAT ⇒ We are given True assignment
, and could check it in polynomial time.
2)if Φ 0 ∉ SAT ⇒ Then there is no true assignment.
• There is one another important concept in
complexity which is Reduction.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
11. 11
Definition :
L ≤ P L* ⇔ ∃f (Polynomial Time Function) ∀x ∈ L ⇔ f ( x) ∈ L* .
f
L L*
f
L L
L*
Fig1: Graphical Diagram of Reduction Concept
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
12. 12
Example :
SAT = {< Φ >| Φ is satisfiable assignment}.
3 - CNF = {< Φ >| Φ is 3 - cnf and satisfiable}.
we show that :
SAT ≤ P 3 − CNF .
(3 − CNF is in this form for example :
(x1 ∨ x 2 ∨ x 3 ) ∧ ( x1 ∨ x 2 ∨ x 3 ) ∧ (x1 ∨ x 2 ∨ x 3 )
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
13. 13
Φ ≡ (¬x1 → x2 ) ↔ ( x3 ∧ x2 )
<->
y1 y2
-> ^
-x1 x2 x3 x4
Φ ≡ ( y1 ↔ (¬x1 → x2 )) ∧ (( x3 ∧ x4 ) ↔ y2 ) ∧ ( y1 ↔ y2 )
Now each paranthesis can be converted into 3 - OR form
using truth table.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
14. 14
Example :
INTEGER − PROGRAMMING = {< Am×n , bm×1 >| Am×n and bm×1 are integer matrices,
∃X n×1 ∈ Ζ n , Am×n X n×1 ≤ bm×1}.
Obviously INTEGER − PROGRAMMING ∈ NP.
We will show that :
3 − CNF ≤ P INTEGER − PROGRAMMING.
Proof :
¬X 1 ∨ X 2 ∨ X 3 ↔ (1 − X 1 ) + X 2 + X 3
¬X 1 ∨ X 2 ∨ X 3 ≡ 1 ↔ (1 − X 1 ) + X 2 + X 3 ≥ 1
X i ∈ Ζ, 0 ≤ X i ≤ 1
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
15. 15
• We conclude this section by the following theorem:
Every language in NP is polynomial time
reducible to SAT language.
• Language like SAT, INTEGER-PROGRAMMING,
Hamiltonian Cycle, and 3-CNF are also the same
and they belong to NP-Complete set.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
16. 16
Approximation Algorithms
• The following describes the connection of NP
optimization Complete problem and its related
optimization problem.
Decision Version :
TSP = {< Gn×n , b >| G n×n is complete non negative weighted graph and there
exists Hamiltonian Cycle such that its cost is less than or
equal b}.
Optimization Version :
OP − TSP :
we have complete non negative weighted graph G n×n , find the Hamiltonian Cycle
which its cost is minimun.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
17. 17
• Bound of Approximation:
• Suppose that problem P has an optimum solution with cost C-
op. Algorithm X solves problem P
with bound ρ(n) if it finds feasible solution with cost C such
as:
max {c/c-op, c-op/c} ≤ ρ(n).
• Note that with this definition we always have ρ(n) ≥1.
• In the following I try to show that sometimes, finding good
approximation for optimization problems is very hard.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
18. 18
• Theorem:
It does not exist any polynomial time approximation
algorithm with Polynomial Bound for OPT-TSP unless
P=NP.
• Proof:
we prove that if such an algorithm exists then we can Solve
Hamiltonian Cycle in Polynomial Time.
For each graph G with n vertex convert
It into weighted complete graph G1 as G has Hamiltonian Cycle
the following procedure: Iff
4) Assign 1 to each of its edge. OPT-TSP(G1)=n
5) Assign nρ(n) to the other edges.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
19. 19
• I think that it’s now time to study Randomized
Rounding technique for solving optimization problems.
• I explore it with two examples.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
20. 20
Case Study1: Minimum Weight Vertex
Cover
• Now we study randomized rounding method in
approximation algorithms with an example.
• Definition:
• Vertex Cover: In undirected graph G=(V,E), Set A of
vertices is said vertex cover if
for every (u,v) ε E then u ε A or v ε A .
• Minimum Weight Vertex Cover: In undirected graph
G=(V,E) which each vertex has w(v) as positive weight Set
A of vertices is said minimum weight vertex cover if for
every (u,v) ε E then u ε A or v ε A and w(A) is the least.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
21. 21
Fig2: Vertex Cover
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
22. 22
• It is proved that decision version of this
optimization problem is NP Hard.
• At the first step we model it as integer
programming.
x : V → {0,1}
if v ∈ Minimum Weight Vertex Cover Set ⇒ x(v) = 1.
else x(v) = 0.
Integer Programming Method :
min ∑ w(v)x(v)
v∈V
∀u , v ∈ V x(v) + x (u ) ≥ 1
0 ≤ x (v ) ≤ 1
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
23. 23
• As it was known it is believed that there is no
polynomial time algorithm for integer programming.
• We must think about another method.
• We investigate its related linear programming.
x : V → [0,1]
Linear Programming Method :
min ∑ w(v)x(v)
v∈V
∀u , v ∈ V x(v) + x(u ) ≥ 1
0 ≤ x (v ) ≤ 1
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
24. 24
• The Last method is called Relaxation to linear
programming.
programming
• It’s now time to state the approximation
algorithm.
Approximation Min Weight Vertex Cover(G, w)
1) C ← Φ
2) Compute x an optimal solution to linear programming.
3) for each v ∈ V
4) do if x (v) ≥ 0.5 / * Rounding Method * /
5) then C ← C ∪ {v}
6) Return C.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
25. 25
Proof :
C :: The soulution of algorithm.
C* :: The real optimal set of integer programming.
Z* :: The optimal value of the linear programming.
Obviously we have Z* ≤ w(C* ).
Obviously C is vertex cover.
Z* = ∑ w(v) x(v) ≥ ∑ w(v) x(v)
v∈V v∈V , x ( v ) ≥ 0.5
≥ ∑ w(v)0.5 =∑ w(v)0.5 = 0.5w(C ).
v∈C
v∈V , x ( v ) ≥ 0.5
0.5w(C) ≤ Z* ≤ w(C* ) ⇒ { ρ = 2}
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
26. 26
• Hastad proved that there is no Polynomial time algorithm for
vertex cover that achieves an approximation better that 1.16
unless P=NP (1997).
• Now we consider another example.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
27. 27
Case Study2: MAXSAT Problem
MAXSAT :
Given K - CNF Formula with each Clause C j has weight Wj ≥ 0.
Find assignment to variables that maximizes :
∑w
C j is true
j
Example :
Φ = (x1 + x 2 + x 3 + x 4 )( x1 + x 2 + x 3 + x 4 )(x1 + x 2 + x 3 + x 4 ).
C1 = (x1 + x 2 + x 3 + x 4 ) → w 1
C 2 = ( x1 + x 2 + x 3 + x 4 ) → w 2
C3 = (x1 + x 2 + x 3 + x 4 ) → w 3
Max ∑w
C j is true
j
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
28. 28
• We state the following randomized algorithm for this
problem. (Johnson 1974).
1) for variables x1 ,...x n randomly assign
to 0 or 1 with probability 0.5.
2) return ∑w
C J istrue
j
Claim :
1 −1
The preceding algorithm has [(1- k
)] approximation bound.
2
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
29. 29
Proof :
Define the following random varibles :
I j = 1 if clause C j satisfiable if not 0.
∑ w =∑ w I
C j is true
j
Cj
j j
1
p(I j = 1 ) = (1 - ).
2k
1
p(I j = 0 ) = ( ).
2k
1
E{ ∑ w j} =E{∑ w jI j} =∑ w jE{I j} =(1-
C j is true Cj Cj
)∑ w j .
2k C j
1 −1
ρ = [(1 - k
)]
2
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
30. 30
• We can extend Johnson algorithm as follow:
1) for variables x1 ,...x n randomly and independently assign
to 1 or 0 with probability p i and 1 - p i
2) return ∑w
C J istrue
j
Claim :
E{ ∑ w } = ∑ w (1 − ∏ (1 − p )∏ p )
C j is true
j
Cj
j
if xi
i i
if xi
The proof is just the same.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
31. 31
• Now we model it with IP and relax it.
Integer Programming Model for MAXSAT :
Max ∑ w jz j
Relaxation
Cj
∀C j ∑ y + ∑ (1- y ) ≥ z
if x i
i i j
if x i
0 ≤ yi ≤ 1
0 ≤ zj ≤1 yi , z j ∈ Ζ
Relaxation for MAXSAT :
Max ∑ w jz j
Cj
∀C j ∑ y + ∑ (1- y ) ≥ z
if x i
i
if x i
i j
0 ≤ yi ≤ 1
0 ≤ zj ≤1 yi , z j ∈ ℜ
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
32. 32
Algorithm
1) Solve the LP and find vector (y* , z * ).
2) use extended Johnson algorithm with p i = y*i .
3) return
Lemma :
for any feasible solution (y* , z* ) to LP and for any clause C j with k Literals,
we have
1
1 - ∏ (1- y i )∏ (y i ) ≥ (1 − (1 − ) k ) z j
if x i if x i k
Claim :
The above algorithm has (1- 1/e) -1 approximation ratio.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
33. 33
Proof :
∑ w {1 − ∏ (1 − p )∏ ( p )} =
Cj
j
xi
i i
xi
1
∑ w j {1 − ∏ (1 − yi )∏ ( yi )} ≥ (1 − (1 − ) k )∑ w j z j =
* * *
Cj xi xi k Cj
1 k * 1 k *
(1 − (1 − ) ) Z LP ≥ (1 − (1 − ) ) Z IP
k k
[ρ = (1 − 1 / e) −1 ]
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
35. 35
• Algorithm
2. Compute the value from Johnson
algorithm (w1).
3. Compute the value from LP Based
algorithm (w2).
4. Return MAX(w1,w2).
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
36. 36
Conclusion
• In this talk the randomized rounding method is explored
with two example.
• This new method opens new insight into approximation
algorithms and complexity theory.
The End
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems