Tutorial given during the 2011 Paris Game AI Conference on Automated Planning applied to Games

1. The Next Generation of
Game Planners
The "Everything You (N)Ever Wanted to Know" Tour
Luke Dicken
Strathclyde AI and Games Research Group
University of Strathclyde

2. Controversy!
2

3. Controversy!
• “...STRIPS-style goal oriented action planning has turned out
to be a dead end.”
2

4. Controversy!
• “...STRIPS-style goal oriented action planning has turned out
to be a dead end.”
• “Academia has long discarded such planners in favor of
hierarchical ones...”
2

5. Controversy!
• “...STRIPS-style goal oriented action planning has turned out
to be a dead end.”
• “Academia has long discarded such planners in favor of
hierarchical ones...”
Alex, “This Year in Game AI”
(Jan ’11)
2

6. Controversy!
• “...STRIPS-style goal oriented action planning has turned out
to be a dead end.”
• “Academia has long discarded such planners in favor of
hierarchical ones...”
Alex, “This Year in Game AI”
(Jan ’11)
• This session will drill into what Automated Planning is and
why (some) parts of it are still relevant for Game AI
2

7. What is Automated
Planning?
3

8. What is Automated
Planning?
• “Strong” AI
3

9. What is Automated
Planning?
• “Strong” AI
• Finds action sequences - Plan
3

10. What is Automated
Planning?
• “Strong” AI
• Finds action sequences - Plan
• Over 40 years of research
3

11. What is Automated
Planning?
• “Strong” AI
• Finds action sequences - Plan
• Over 40 years of research
• Planning Domain Description Language (PDDL) - 1998
3

12. How Does it Work?
4

13. How Does it Work?
1. Description of actions possible
4

14. How Does it Work?
1. Description of actions possible
2. Complete description of initial state of the world
4

15. How Does it Work?
1. Description of actions possible
2. Complete description of initial state of the world
3. Definition of goals that need to be achieved
4

16. Planning
S0
5

17. Planning
S0
S1 S2 S3
5

18. Planning
S0
S1
5

19. Planning
S0
S1
S4 S5 S6
5

20. Planning
S0
S1
S6
5

21. Planning
S0
S1
S6
And so on, until goal reached.
5

22. GOAP
6

23. GOAP
6

24. GOAP
6

25. GOAP
6

26. GOAP
6

27. GOAP
6

28. Issues with GOAP
7

29. Issues with GOAP
• Issue 1 : Lack of directorial control.
7

30. Issues with GOAP
• Issue 1 : Lack of directorial control.
‣ When NPCs get smart enough to realise standing next
to exploding barrels is hazardous, cinematic experience is
diminished.
7

31. Issues with GOAP
• Issue 1 : Lack of directorial control.
‣ When NPCs get smart enough to realise standing next
to exploding barrels is hazardous, cinematic experience is
diminished.
• Issue 2 : Computational Complexity
7

32. Issues with GOAP
• Issue 1 : Lack of directorial control.
‣ When NPCs get smart enough to realise standing next
to exploding barrels is hazardous, cinematic experience is
diminished.
• Issue 2 : Computational Complexity
‣ GOAP is derived directly from STRIPS. NP-Hard search
problems in the general case.
7

33. Issues with GOAP
8

34. Issues with GOAP
• Issue 1 - either a “strong” AI approach is suitable to your
design or it isn’t. Places it often will be include sandbox
environments and companion AI.
8

35. Issues with GOAP
• Issue 1 - either a “strong” AI approach is suitable to your
design or it isn’t. Places it often will be include sandbox
environments and companion AI.
• Issue 2 is what will be the focus of the rest of the session -
how have planning systems improved since STRIPS/GOAP?
8

36. Complexity Reduction
9

37. Complexity Reduction
• If you can reduce complexity of the problem, it
becomes easier to solve...
9

38. Complexity Reduction
• If you can reduce complexity of the problem, it
becomes easier to solve...
• Either less depth to the problem or less breadth.
9

39. Landmark Analysis
10

40. Landmark Analysis
Initial State
10

41. Landmark Analysis
Initial State
10

42. Landmark Analysis
Initial State
Goal Found
10

43. Landmark Analysis
Initial State
Goal Found
10

44. Landmark Analysis
Initial State
Landmark 1
Goal Found
10

45. Landmark Analysis
Initial State
Landmark 1
Goal Found
10

46. Landmark Analysis
Initial State
Landmark 1
Landmark 2
Goal Found
10

47. Landmark Analysis
Initial State
Landmark 1
Landmark 2
Goal Found
10

48. Landmark Analysis
Initial State
Landmark 1
Landmark 2
Goal Found
10

49. Abstraction
A
B C D
E F
11

50. Abstraction
A
B C D
E F
11

51. Abstraction
11

52. Horizon Management
A
B C D
E F
12

53. Horizon Management
12

54. Horizon Management
E
12

55. Horizon Management
B
E
12

56. Horizon Management
B C
E
12

57. Horizon Management
B C D
E
12

58. Horizon Management
B C D
E F
12

59. Heuristics
13

60. Heuristics
• Since GOAP came out, major advances in heuristics
13

61. Heuristics
• Since GOAP came out, major advances in heuristics
• Most significant :
13

62. Heuristics
• Since GOAP came out, major advances in heuristics
• Most significant :
‣ Relaxed Plan Graph
13

63. Heuristics
• Since GOAP came out, major advances in heuristics
• Most significant :
‣ Relaxed Plan Graph
‣ Landmark Heuristic
13

64. Hierarchical Task Network
14

65. Hierarchical Task Network
Kill Enemy
14

66. Hierarchical Task Network
Kill Enemy
Approach Shoot
Face Enemy
Enemy Enemy
14

67. Hierarchical Task Network
Kill Enemy
Approach Shoot
Face Enemy
Enemy Enemy
Leave Cover Navigate
14

68. Hierarchical Task Network
Kill Enemy
Approach Shoot
Face Enemy
Enemy Enemy
Leave Cover Navigate ...and so on
14

69. Hierarchical Task Network
Kill Enemy
Approach Shoot
Face Enemy
Enemy Enemy
Leave Cover Navigate ...and so on
Until executable actions reached.
14

70. Optimality
15

71. Optimality
• Optimality is a big issue for academic vs industry
15

72. Optimality
• Optimality is a big issue for academic vs industry
• Academics
15

73. Optimality
• Optimality is a big issue for academic vs industry
• Academics
‣ Aim is optimal - shortest, most efficient, least cost
15

74. Optimality
• Optimality is a big issue for academic vs industry
• Academics
‣ Aim is optimal - shortest, most efficient, least cost
• Industry
15

75. Optimality
• Optimality is a big issue for academic vs industry
• Academics
‣ Aim is optimal - shortest, most efficient, least cost
• Industry
‣ Aim is entertaining - believable, beatable, pseudo-smart
15

76. Optimality
• Optimality is a big issue for academic vs industry
• Academics
‣ Aim is optimal - shortest, most efficient, least cost
• Industry
‣ Aim is entertaining - believable, beatable, pseudo-smart
• How can we bridge this disconnect?
15

77. Metrics
16

78. Metrics
• Plan Metrics allow you to define optimal on your
terms.
16

79. Metrics
• Plan Metrics allow you to define optimal on your
terms.
• Not a total solution, adds extra compute time.
16

80. 17

81. But what happens after planning?
17

82. Plan Execution
18

83. Plan Execution
• Planning is not the same as doing something
18

84. Plan Execution
• Planning is not the same as doing something
• Big question is: “what happens next?”
18

85. Plan Execution
• Planning is not the same as doing something
• Big question is: “what happens next?”
‣ Especially considering that the traditional assumptions of
planning make doing things with plans “challenging”!
18

86. Execute Blind
19

87. Execute Blind
Execute
Plan Plan
Start End
19

88. Execute Blind
19

89. Execute Blind
19

90. Execute Blind
19

91. Execute/Replan
20

92. Execute/Replan
Execute
Plan
Start
20

93. Execute/Replan
? ??
?
Execute
Plan
Start
20

94. Execute/Replan
? ??
?
Execute
Plan
Start
Replan
20

95. Execute/Replan
? ??
?
Execute Execute
Plan
Start
Replan
20

96. Execute/Replan
? ??
?
Execute Execute
Plan Goal
Start Reached
Replan
20

97. Execution Monitoring
21

98. Execution Monitoring
21

99. Integrated Influence
22

100. Integrated Influence
22

101. Integrated Influence
22

102. Integrated Influence
22

103. Integrated Influence
22

104. Integrated Influence
22

105. Integrated Influence
22

106. Integrated Influence
22

107. Summary
23

108. Summary
• GOAP is not the extent of planning
23

109. Summary
• GOAP is not the extent of planning
• We’ve come a long way in the 40 years since
STRIPS was invented.
23

110. Summary
• GOAP is not the extent of planning
• We’ve come a long way in the 40 years since
STRIPS was invented.
• Planning is still mostly focused on the “big”
problems.
23

111. Summary
• GOAP is not the extent of planning
• We’ve come a long way in the 40 years since
STRIPS was invented.
• Planning is still mostly focused on the “big”
problems.
• There is work in planning of relevance.
23

112. Contact
• Email - luke@cis.strath.ac.uk
• Website - http://saig.cis.strath.ac.uk
• Twitter - @LukeD
24

113. References
• Landmarks
‣ “On the Extraction, Ordering and Usage of Landmarks in Planning” Porteous et al, ECP ’01
• Abstraction
‣ “Applying Clustering Techniques to Reduce Complexity in Automated Planning Domains” Dicken &
Levine, IDEAL ’10
• Relaxed Plan Graph
‣ “The FF Planning System: Fast plan Generation Through Heuristic Search” Hoffman, JAIR Vol. 14
• Landmark Heuristic
‣ “The LAMA Planner Using Landmark Counting in Heuristic Search” Richter & Westphal, IPC ’08
• HTNs
‣ “SHOP2 : An HTN Planning System” Nau et al, JAIR Vol. 20
• Execute/Replan
‣ “FF-Replan: A baseline for probabilistic planning” Yoon et al, ICAPS ’07
• Execution Monitoring
‣ “T-REX: A Deliberative System for AUV Control” McGann et al, PPERWS ’07
25