A major challenge in the field of random graphs is constructing fast algorithms for solving a variety of combinatorial optimization problems, such as finding largest independent set of a graph or finding a satisfying assignment in random instances of K-SAT problem. Most of the algorithms that have been successfully analyzed in the past are so-called local algorithms which rely on making decisions based on local information. In this talk we will discuss fundamental barrier on the power of local algorithms to solve such problems, despite the conjectures put forward in the past. In particular, we refute a conjecture regarding the power of local algorithms to find nearly largest independent sets in random regular graphs. Similarly, we show that a broad class of local algorithms, including the so-called Belief Propagation and Survey Propagation algorithms, cannot find satisfying assignments in random NAE-K-SAT problem above a certain asymptotic threshold, below which even simple algorithms succeed with high probability. Our negative results exploit fascinating geometry of feasible solutions of random constraint satisfaction problems, which was first predicted by physicists heuristically and now confirmed by rigorous methods. According to this picture, the solution space exhibits a clustering property whereby the feasible solutions tend to cluster according to the underlying Hamming distance. We show that success of local algorithms would imply violation of such a clustering property thus leading to a contradiction. Joint work with Madhu Sudan (Microsoft Research).

1. Limits of local algorithms for random graphsLimits of local algorithms for random graphs
David Gamarnik
Joint work with
Madhu Sudan (Microsoft Research, New England)
Yandex workshop on extremal combinatorics
June 24, 2014

2. Local Algorithms

3. Computer Science.
Applications: hardware, fault-tolerant models of computation,
sensor networks.
Lynch (book) [1996]
Nguyen & Onak [2008]
Rubinfeld, Tamir, Vardi, Xie [2011]
Suomela (survey) [2011]
Wireless communications
Applications: communication protocols with low overhead
Shah [2008]
Shin & Shah [2012]
Who is interested in local algorithms?

4. Network Science. Economic team modeling
Applications: models of social interaction, structure of social networks
Rusmevichientong, Van Roy [2003]
Judd, Kearns & Vorobeychik [2010]
Borgs, Brautbar, Chayes, Khanna & Lucier [2012]
Signal Processing. Machine learning.
Applications: image processing, coding theory, wireless communication,
gossip algorithms.
Wainwright & Jordan [2008], Mezard & Montanari [2009], Shah [2008]
Shin & Shah [2012]
Physics.
Applications: Spin glass theory
Mezard & Montanari [2009]
Who is interested in local algorithms?

5. Mathematics/Combinatorics.
Applications: graph limits.
Lovasz (book) [2012]
Hatami, Lovasz & Szegedy [2012]
Elek & Lippner [2010]
Lyons & Nazarov [2011]
Czoka & Lippner [2012]
Aldous [2012].
Who is interested in local algorithms?

6. Random n-node d-regular graph
I. Local algorithms for random graphs : i.i.d.
factors
locally regular tree like

7. Local algorithms: i.i.d. factors
Hatami-Lovasz-Szegedy [2012] framework
Generate i.i.d. U[0,1] weights
Apply some local rule f: decorated tree ! {0,1} for every node
f
0 or 1

8. Conjecture. Hatami, Lovasz & Szegedy [2012] There exists a local rule f
which produces a nearly largest independent set in a random d-regular graph
Example. f =1 iff the weight of the node is larger than the weights of all
of its neighbors
IN OUT
far from optimal !
Local algorithms: i.i.d. factors

9. Algorithms for independent sets in random graphs
Frieze & Luczak [1992]
Best known algorithm: Greedy
Hatami-Lovasz-Szegedy Conjecture: there exists f such that

10. Theorem. [G & Sudan ] Hatami-Lovasz-Szegedy conjecture is not valid: No
local rule f can produce an independent set larger than factor of
the optimal, for large enough d.
Theorem. [Rahman & Virag 2014] No local rule f can produce an
independent set larger than factor of the optimal, for large enough d.
Notes:
1. Rahman & Virag’s result is the best possible: factor ½ can be achieved by
local algorithms, Lauer & Wormald [2007], G & Goldberg [2010]
2. Proof technique. Spin Glass theory: geometry of large independent sets
Main result

11. Geometry of solutions: clustering phenomena
Coja-Oghlan & Efthymiou [2010], G & Sudan [2012] Every two ‘’large’’
independent sets either have a very ‘’small’’ or a very ‘’large’’ intersection:

12. Geometry of solutions: clustering phenomena
Coja-Oghlan & Efthymiou [2010], G & Sudan [2012] Every two ‘’large’’
independent sets either have a very ‘’small’’ or a very ‘’large’’ intersection:

13. Proof sketch: argue by contradiction. Suppose rule f exists
?
(a) Generate two i.i.d. sequences

14. Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences

15. Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences

16. Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences

17. Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences

18. Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences

19. Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences

20. Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences

21. Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Intersection size belongs to
the ‘’non-existent interval’’

22. Theory is great not when it is correct but when
it is
interesting… Murray Davis, 1 97 1

23. II. Sequential local algorithms for random NAE-
K-SAT problem
Not-All-Equal-K-Satisfiability formula on n boolean variables and m clauses.
Each clause contains K (i.e. K=3)
Assignment
is satisfiable if every clause is satisfied by at least one variable, and is not satisfied
by at least some other variable
Formula is satisfiable if there exists at least one satisfying assignment.
NP-Complete.

24. Sequential local algorithms for random NAE-K-SAT
problem
Random NAE-K-SAT problem
Generate m clauses independently uniformly at random from the space of all
possible clauses (with replacement). d=m/n – clause density.
Theorem. [Coja-Oglan and Panagiotou 2014] The threshold for probability that the
formula is satisfiable
Best algorithm Unit Clause succeeds only when
Achlioptas et al. 2001
K gap!

25. Sequential local algorithms for the random NAE-K-SAT
problem
Sequential r-local algorithms
1.Fix a “local rule” which maps formula © with variable x to probability
value.
2.For i=1,2,…,n apply rule ¿ to variables xi and depth r sub-formulas ©i rooted at xi : fix xi to
be 1 with probability ¿(©i, xi), and 0 with probability 1- ¿(©i, xi)
Notes:
Belief Propagation (BP) guided decimation and Survey Propagation (SP) guided
decimation algorithms are sequential local algorithms when the number of iterations
is bounded by a constant independent of the number of variables
Unit Clause algorithm is a sequential local algorithm

26. Belief Propagation and Survey Propagation algorithms – message passing type
algorithms proposed by statistical physicists. Work remarkably well for low
values of K (K=3,4,5).
Mezard, Parisi & Zecchina [2003]
Krzakala, Montanari, Ricci-Tersenghi, Semerjian, Zdeborova [2007]
Mezard & Montanari (book) [2009]

27. Theorem. G & Sudan [2013] Every sequential local algorithm fails to find a
satisfying assignment when
simple algorithm
(Unit Clause) exist
no sequential local
algorithms exist
no solutions exist
Sequential local algorithms cannot bridge the 1/K gap

28. Proof Technique
(a) Clustering property for random NAE-K-SAT
(b) Interpolation between m randomly generated solutions
(c) Decisions for variables are localized
UNSAT

29. Proof Technique
(a) Clustering property for random NAE-K-SAT
Theorem. Let
With probability approaching unity as n increases, there does not exist m
satisfying assignments such that all pairwise Hamming
distances are
Proof: first moment method.

30. Some further thoughts
• The approach should apply to other problems with “symmetry”, for example
coloring of graphs. But applying the approach to random K-SAT is
problematic.
• In practice Survey Propagation algorithm is run for many iterations.
Challenge: establish similar result when there is no bound on the number of
iterations. Coja-Oghlan [2010] – true for Belief Propagation for K-SAT.
• Establish limits for the performance of local algorithms for random constraint
satisfaction problems using the most general (Computer Science) definition
of local algorithms.

31. Mick can’t get satisfaction above
(from Sequential Local Algorithms)
Chvatal & Reed [1998]
’’Mick gets some (the odds are on his side) [satisfiability]’’
Parting thoughts …

32. Thank you