Design

1. Constraint-based problem solving
 Model problem
 specify in terms of constraints on acceptable solutions
 define variables (denotations) and domains
 define constraints in some language
 Solve model
 define search space / choose algorithm
– incremental assignment / backtracking search
– complete assignments / stochastic search
 design/choose heuristics
 Verify and analyze solution

2.  Model problem
 specify in terms of constraints on acceptable solutions
 define variables (denotations) and domains
 define constraints in some language
 Solve model
 define search space / choose algorithm
– incremental assignment / backtracking search
– complete assignments / stochastic search
 design/choose heuristics
 Verify and analyze solution
Constraint-based problem solving
Constraint
Satisfaction
Problem

3. Constraint satisfaction problem
 A CSP is defined by
 a set of variables
 a domain of values for each variable
 a set of constraints between variables
 A solution is
 an assignment of a value to each variable that
satisfies the constraints

4. Example: Crossword puzzles

5. CSP, binary CSP, SAT, 3-SAT, ILP, ...
 Model and solve in one of these languages
 Model in one language, translate into
another to solve
Options

6. Article of faith
“Constraints arise naturally in most areas of human
endeavor. They are the natural medium of expression for
formalizing regularities that underlie the computational
and physical worlds and their mathematical abstractions.”
P. Van Hentenryck and V. Saraswat
Constraint Programming: Strategic Directions

7. Reducibility
NP-Complete
3-SAT
ILP
CSP binary CSP
(0,1)-ILP

8. CSP, binary CSP, SAT, 3-SAT, ILP, ...
 Model and solve in one of these languages
 Model in one language, translate into
another to solve
Options

9. Importance of the model
“In integer programming, formulating a ‘good’ model is of
crucial importance to solving the model.”
G. L. Nemhauser and L. A. Wolsey
Handbook in OR & MS, 1989
“Same for constraint programming.”
Folk Wisdom, CP practitioners

10. Measures for comparing models
 How easy is it to
–write down,
–understand,
–modify, debug,
–communicate?
 How computationally difficult is it to solve?

11. Constraint systems/languages
System Language
ILOG Solver C++ library
Cosytec CHIP C++ library
Eclipse Logic programming
Oz Functional programming

12. Abstractions
 Capture commonly occurring constraints
 special propagation algorithms
 e.g. alldifferent,
cardinality,
cumulative, ...
 User-defined constraints
 full power of host language

13. CSP versus ILP
 OO databases versus Relational databases
 C++ versus C
 C++ versus Fortran
 C++ versus Assembly language

14. Computational difficulty?
 What is a good model depends on algorithm
 Choice of variables defines search space
 Choice of constraints defines
 how search space can be reduced
 how search can be guided

15. Improving model efficiency
Given a model:
 Add/remove variables, values, constraints
(keep the denotation of the variables)
 Use/translate to a different representation
(change the denotation of the variables)

16. Adding redundant constraints
 Improve computational efficiency of model
by adding “right” constraints
 symmetries removed
 dead-ends encountered earlier in search process
 Three methods:
 add hand-crafted constraints during modeling
 apply a consistency algorithm before solving
 learn constraints while solving

17. Examples:
 Adding hand-crafted constraints:
 crossword puzzles
 Applying a consistency algorithm:
 dual representation

18. Adding redundant variables
 Variables that are abstractions of other variables
 e.g, decision variables
Suppose x has domain {1,…,10}
Add Boolean variable to represent decisions
(x < 5), (x  5)
 Variables that represent constraints

19. Translations between models
 Improve computational efficiency by
completely changing the model
 change denotation of variables
– e.g, convert from non-binary to binary
 aggregate variables
– e.g, timetabling multiple sections of a course
 Not much as has been done on either the
theory or the practice side

20. Conversion to binary
 Any CSP can be converted into one with only
binary constraints
 Two techniques known
 dual graph method (Dechter & Pearl, 1989)
 hidden variable method (Peirce, 1933; Dechter 1990)
 Translations are polynomial if constraints are
represented extensionally

21. (x1  x2  x6)  (x1  x3  x4) 
(x4  x5  x6)  (x2  x5  x6)
 Non-binary CSP:
 Boolean variables: x1, …, x6
 constraints: one for each clause
3-SAT example
C1(x1, x2, x6) = {(0,0,1), (0,1,0), (0,1,1),
(1,0,0), (1,0,1), (1,1,0), (1,1,1)}
C2(x1, x3, x4) = ...

22. Dual graph representation
y1 = (x1, x2, x6) y4 = (x2, x5, x6)
y2 = (x1, x3, x4) y3 = (x4, x5, x6)
x2, x6
x5, x6
x4
x6
x1

23. Hidden variable representation
x1 x2 x3 x4 x5 x6
h1
h2 h3
h4

24.  Non-binary constraint:
 Augmented constraint:
 Binary constraints:
Hidden variable representation
C1(x1, x2, x6) = {(0,0,1), (0,1,0), (0,1,1),
(1,0,0), (1,0,1), (1,1,0), (1,1,1)}
C1(x1, x2, x6, h1) = {(0,0,1,0), (0,1,0,1), (0,1,1,2),
(1,0,0,3), (1,0,1,4), (1,1,0,5), (1,1,1,6)}
R1(x1, h1) = {(0,0), (0,1), (0,2), (1,3), (1,4), (1,5), (1,6)}
R2(x2, h1) = {(0,0), (1,1), (1,2), (0,3), (0,4), (1,5), (1,6)}
R3(x6, h1) = {(1,0), (0,1), (1,2), (0,3), (1,4), (0,5), (1,6)}

25. x1
x2
x3
h1
h2
h3
Dual exponentially worse
FC on non-binary: O(n)
consistency checks
FC on dual:
O(2n) consistency checks
x1, x1  x2, x1  x2  x3, ..., x1  …  xn-1  xn

26. x1
x2
x3
h1
h2
h3
Dual exponentially better
FC on non-binary:
O(n2n) consistency checks
FC on dual:
O(n2) consistency checks
x1  …  xn-1, x1  …  xn-2  xn, …, x2  …  xn

27.  Worst case:
 Best case:
Bounds on performance
FC(non-binary)  FC+(hidden)
dk
FC+(hidden)  FC (non-binary)
dn

28. Crossword puzzles
1 2 3
6
4
7
5
8
10
9
20
11
22
12
21
13
17
14
18
16
15
23
19
a
aardvark
aback
abacus
abaft
abalone
abandon
...
Mona Lisa
monarch
monarchy
monarda
...
zymurgy
zyrian
zythum

29. Crossword puzzles
puzzle orig. cc’s dual cc’s n m
1 52 292 4 3
2 138 3,267 6 8
3 185 11,404 8 14
4 208 10,481 12 14
5 12,509 18,686 10 19
6 12,636 19,876 14 21
7 556,660 28,962 14 23

30. Crossword puzzles
puzzle orig. cc’s dual cc’s n m
8 5,780,710 27,656 20 30
9 * 47,167 26 41
10 * 224,258 18 35
11 * 29,777 24 40
12 * 729,125 18 38
13 28,446,460 50,281 28 52
14 * 72,307 34 65

31. Model of random non-binary CSP
 n variables
 each with domain size d
 m constraints
 each with k variables, chosen at random
 each with t tuples, chosen at random

32. Order of magnitude curves
(number
of
constraints)
/
(number
of
variables)
(number of tuples in constraints) / (maximum tuples)
n = 20
d = 2
k = 3
n = 20
d = 10
k = 3
n = 20
d = 10
k = 5
n = 20
d = 2
k = 5

33. Random 3-SAT
clauses/
variables
cost
ratio
0.25 0.9
0.5 1.5
1 2.5
2 4.2
4 31
8 33
16 38

34.  Given start state, goal state, and actions:
determine a plan (a sequence of actions)
Box1
Project: Planning
Box2
Box2

35. Contrasts
 CP philosophy/methodology
 emphasis on modeling, domain knowledge
 general purpose search algorithm
– backtracking with constraint propagation
 Successful
 e.g., can solve practical scheduling problems

36. Contrasts
 Planning philosophy/methodology
 emphasis on minimal model
– just representation of actions
 special-purpose search algorithms
 Not as successful
 has not solved many practical problems

37. Tradeoffs
 Robust CSP model needed for each new domain
 can require much intellectual effort
 Less work needs to be done on algorithms
 many general purpose constraint solvers available

38. CSP model of planning
 State-based model
 model each state by a collection of variables
 constraints enforce valid transitions between states
 Example: logistics world
 variable for each package, truck, plane
 domains of packages: all locations, trucks, planes
 domains of trucks, planes: all locations

39. CSP model: constraints (I)
 Action constraints
 model the effects of actions
 patterned after explanation closure axioms
 State constraints
 variables within a state must be consistent

40. Improving model efficiency
 Can add/remove/aggregate/decompose
 variables
 domain values
 constraints
 Here:
 added hidden variables
 added redundant & symmetry-breaking constraints

41. CSP model: constraints (II)
 Symmetric values constraints
 break symmetries on values variables can be
assigned
 Action choice constraints
 break symmetries on equivalent permutations
of actions

42. CSP model: constraints (III)
 Domain constraints
 restrictions on original domains of variables
 Capacity constraints
 bounds on resources
 Distance constraints
 bounds on steps needed for a variable to change
from one value to another

43. Solving the CSP model
 Given an instance of a planning problem
 generate a model with one step in it
 instantiate variables in the initial and goal states
 search for a solution (GAC+CBJ)
 repeat, incrementing number of steps,
until plan is found
 Properties:
 forwards, backwards, or middle out planner
 sound, complete, guaranteed to terminate

44. Experiments
 Five test domains from AIPS’98
 Five planners:
 CPlan
 Blackbox (Kautz & Selman)
 HSP (Bonet & Geffner)
 IPP (Koehler & Nebel)
 TLPlan (Bacchus & Kabanza)
 Setup:
 machines: 400MHz Pentium II’s
 resources: 1 hour CPU time, 256 Mb memory

45. Gripper problems
CPlan BBox HSP IPP
1 0.0 0.1 0.0 0.0
2 0.0 5.7 0.1 0.4
3 0.1 0.1 7.9
4 0.2 0.2 100.4
5 0.3 0.3
6 0.5 0.4
20/20 2/20 20/20 4/20

46. Logistics problems
CPlan BBox HSP IPP
1 0.1 1.5 0.6
2 0.1 4.3 552.2
3 0.9
4 0.2
5 1.7 148.0 1.2 2.5
6 18.9
28/35 8/35 5/35 10/35

47. Mystery problems
CPlan BBox HSP IPP
1 0.0 0.1 0.1 0.1
2 0.0 4.2 7.1 11.4
3 0.0 0.4 0.4 0.9
4 0.0 1.2 0.4
5 0.0 11.4 7.6
6 3.1 134.0
30/30 19/30 16/30 24/30

48. MysteryPrime problems
CPlan BBox HSP IPP
1 0.1 0.6 0.1 2.3
2 0.1 4.4 13.9 23.9
3 0.2 0.6 0.5 4.4
4 0.2 0.6 0.6 2.5
5 0.2 7.2
6 2.3 201.8
35/35 18/35 25/35 21/35

49. Grid problems
CPlan BBox HSP IPP
1 0.7 8.1 1.1 3.0
2 33.4 4.3 9.4
3
4 1773.3 57.0
5

50. Related work
 Planning as a CSP
 satisfiability (Kautz & Selman)
 ILP (e.g., Bockmayr & Dimopoulos)
 Adding declarative knowledge to improve efficiency
 hand-coded (e.g, Kautz & Selman, Bacchus & Kabanza)
 automatically derived (e.g., Fox & Long, Nebel et al.)

51. Project: Resource-constrained
instruction scheduling (I)
 Modern architectures (VLIW) allow
instruction level parallelism
 multiple functional units
 Compiler to generate code that takes
advantage of parallelism
 resource-constrained scheduling task
a: j  i + h
b: k i + g
c: l  j + 1
1: a, b
2: c

52. Project: Resource-constrained
instruction scheduling (II)
 Current methods:
 heuristic
 recent: integer linear programming approach
 Project:
 investigate constraint programming approach
 GNU compiler, MERCED chip

53. Project: Theoretical analysis of
alternative models
 Improve computational efficiency by
completely changing the model
 change denotation of variables
– e.g, convert from non-binary to binary
 aggregate variables
– e.g, timetabling multiple sections of a course
 Not much as has been done on either the
theory or the practice side

54. Conclusions
 Importance of the model
 ease of modeling is important
 form of model is important
 Constraint programming advantages
 succinctness, declarativeness of models
 flexibility in specifying model
 on some classes of problems: speed

55. Conclusions
 Advantages shared with CSP-like approaches
 expressiveness of modeling language
 declarativeness of models
 independence of model and solving algorithm
 Advantages over other CSP-like approaches
 succinctness of models
 robustness: scales well, not as brittle
 speed

56. Future Work
 Approximate planning
 heuristic CSP models
 solve same CSP models using local search
 Alternative CSP models
 action-based models vs state-based models
 finding the right model can be key to solving
difficult combinatorial problems