Talk at Ludicious 2019 about AI in video games and how to use AI to design video games

1. S C I E N C E * PA S S I O N * T E C H N O L O G Y
WHY AI IS SHAPING OUR GAMES
D R . J O H A N N A P I R K E R , T U G R A Z , A U S T R I A
L U D I C I O U S 2 0 1 9

2. JOHANNA PIRKER
▸ Computer Scientist & Software Engineering @Graz University of Technology
▸ Virtual Realities (Maroon) @Massachusetts Institute of Technology
▸ Research & Edu at Institute for Interactive Systems & Data Science, @TUGraz
▸ Virtual Realities & Worlds
▸ HCI, E-Learning, UX, Data Analysis (SNA)
▸ Games Education (for CS) &
Research, Design, Development & Analysis
▸ GUR Consulting @BetterChaos
▸ Website: www.jpirker.com
@JOEYPRINK

4. AI MIRACLES..

5. “MAKING COMPUTERS ACT LIKE THEY DO IN THE MOVIES.”

6. 1. THE CAPABILITY OF A MACHINE TO IMITATE
INTELLIGENT HUMAN BEHAVIOR.
2. A BRANCH OF COMPUTER SCIENCE DEALING WITH THE
SIMULATION OF INTELLIGENT BEHAVIOR IN COMPUTERS.
Merriam-Webster defines artificial intelligence this way.

7. “REAL” AI
▸ 1. learn over time in response to changes in its
environments
▸ (e.g. Netflix recommendations but not Twitter
black lists)
▸ 2. what it learns should be interesting enough that it
would take humans some effort to learn
▸ (Turing test)

9. AI IN GAMES
▸ … generate responsive, adaptive, & intelligent behaviour
▸ uses path finding, decision trees, data mining, PCG, …
▸ usually do not facilitate computer learning
▸ -> predetermined & limited set of responses to a limited set of inputs
▸ ILLUSION OF INTELLIGENCE
▸ good gameplay without environment restrictions
▸ learn & use from “real AI” strategies
▸ Learning Tamagotchi

10. ▸ decision trees (scripting)
▸ -> AI stupidity, predictive behaviour, loss of immersion
▸ pathfinding
▸ (Half Life, “Crouch Cover”)
▸ NPC behaviour in Doom
▸ NPCs fighting NPCs
AI IN GAMES - ISSUES

11. PLAY
GAMES.
CONTRIBUTE
CONTENT.
DESIGN
GAMES.
UNDERSTAND
PLAYERS.

12. I. PLAY GAMES.

13. AI TO PLAY GAMES
ROBOCUP

14. AI TO PLAY GAMES
CHESS - IBM DEEP BLUE VS. GARRY KASPAROV (1997)
"I could feel — I could smell — a new kind of intelligence across the table,"

15. AI TO PLAY GAMES
JEOPARDY! - IBM WATSON VS. KEN JENNINGS (2011)
"I could feel — I could smell — a new kind of intelligence across the table,"

16. AI TO PLAY GAMES
GO - GOOGLE ALPHAGO (DEEPMIND) VS. LEE SEDOL (2016)

17. AI TO PLAY GAMES
DEEPMIND VS. STARCRAFT II (2019)

18. AI TO PLAY GAMES
http://gameaibook.org/book.pdf

19. ▸ Chess Two-player adversarial, deterministic, fully observable,
branching factor ~35, ~70 turns
▸ Go Two-player adversarial, deterministic, fully observable, branching
factor ~350, ~150 turns
▸ Frogger (Atari 2600) 1 player, deterministic, fully observable, bf 6,
hundreds of ticks
▸ Halo 1.5 player, deterministic, partially observable, bf ???, tens of
thousands of ticks
▸ Starcraft 2-4 players, stochastic, partially observable, bf > a million,
tens of thousands of ticks
▸ Togelius
AI TO PLAY GAMES

20. AI TO PLAY GAMES
TRAIN AI HOW TO PLAY SNAKE (DEEP REINFORCEMENT LEARNING)
On the left, the agent was not trained and had no clues on what to do whatsoever. The game on the right
refers to the game after 100 iterations (about 5 minutes). The highest score was 83 points, after 200
iterations.
https://github.com/maurock/snake-ga

21. AI TO PLAY GAMES
TRAIN AI HOW TO PLAY STARCRAFT
‣ A Machine Learning API developed by Blizzard that gives researchers and developers hooks into the game.
‣ A dataset of half a million anonymised game replays,.
‣ An open source version of DeepMind’s toolset, PySC2
‣ A series of simple RL mini-games to test the performance of agents on specific tasks.
https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/

22. AI TO PLAY GAMES
WHY USE AI TO PLAY GAMES?
▸ Playing to win vs playing for experience
▸ For experience: human-like, fun, predictable…?
▸ Playing in the player role vs playing in a non-player role
http://gameaibook.org/book.pdf

23. METHODS
▸ Planning-Based
▸ Uninformed search (e.g. BFS),Informed search (e.g. A*),
Evolutionary algorithms
▸ Reinforcement learning (training time)
▸ TD-learning / approximate dynamic programming,
Evolutionary algorithms
▸ Supervised learning (requires play traces to learn from)
▸ Neural nets, k-nearest neighbors etc
▸ Random (requires nothing)
AI TO PLAY GAMES
▸ Togelius

24. II. CONTRIBUTE CONTENT.

25. CONTRIBUTE CONTENT
PROCEDURAL CONTENT GENERATION

26. CONTRIBUTE CONTENT
PROCEDURAL CONTENT GENERATION

27. CONTRIBUTE CONTENT
PROCEDURAL CONTENT GENERATION

28. CONTRIBUTE CONTENT
PROCEDURAL CONTENT GENERATION
• Artistic aspects
• Corner-cases
• Lack of complete control
• Depends on the content
• Client-side calculations?
• Replayable content?
• Cheap
• Lots of content
• Dynamic Reaction on player
• Reduce burden of artist
• Save memory
• Large worlds
• Replayable content
• http://pcg.wikidot.com/category-pcg-algorithms

29. METHODS
▸ Search-Based Methods
▸ Solver-Based Methods
▸ Grammar-Based Methods
▸ Cellular Automata
▸ Noise and Fractals
▸ Machine Learning
CONTRIBUTE CONTENT

30. GENERATE CONTENT FOR…
▸ Environments (Random Maps, Random Dungeons)
▸ Generative Art and models
▸ Textures
▸ Music
▸ Story
▸ Gameplay
CONTRIBUTE CONTENT

31. III. UNDERSTAND PLAYERS

32. PLAYER MODELING
PLAYER MODELING
▸ … detection, prediction and expression of human player
characteristics that are manifested through cognitive,
affective and behavioral patterns while playing games
▸ can be used to dynamically adjust the gameplay (dynamic
difficult adjustment)

33. BEHAVIOURAL PROFILING

34. B A R T L E ’ S G A M E R T Y P E S
http://www.gamerdna.com/quizzes/bartle-test-of-gamer-psychology

35. Story
Story Enjoyer
Party Player
Killer
Online Hero
Allrounder
0% 20% 40% 60% 80% 100%
Story Enjoyer
Party Player
Killer
Online Hero
Allrounder
Time spent
Story
Campaign
Arena
Online MulAplayer
Local MulAplayer
P L AY E R H A B I T ( P L AY E R F I N G E R P R I N T )

36. P L AY E R P R O F I L E S I N F O R Z A
• What Drives People: Creating Engagement Profiles of
Players from Game Log Data
• 120 mio race entries from 1.2 mil players
•
Harpstead, E., Zimmermann, T., Nagapan, N., Guajardo, J. J., Cooper, R., Solberg, T., & Greenawalt, D. (2015, October). What Drives People: Creating Engagement Profiles of Players from Game Log Data. In Proceedings of the
2015 Annual Symposium on Computer-Human Interaction in Play (pp. 369-379). ACM.

37. F L O W ( M I H A LY C S I K S Z E N T M I H A LY I )

38. HOW PLAYSTYLES EVOLVE:
PROGRESSION ANALYSIS AND
PROFILING IN JUST CAUSE 2
https://link.springer.com/chapter/10.1007/978-3-319-46100-7_8

39. F E AT U R E S
• Spatio-temporal navigation
• combat performance
• progression through the main storyline
• side quests..
• Agency missions (+ reach specific level of Chaos)
• subset of features based on the core mechanics
• -> does not impact the analytical framework
• -> impacts the kinds of conclusions that can be derived

40. P L AY E R P R O G R E S S I O N A L O N G T H E
M I S S I O N S

41. R E S U LT S
• How can we describe player behaviour of the
different player profiles?

42. P L AY E R B E H AV I O U R A L O N G T H E
S T O RY L I N E
jpirker.com/jc2/aaSankey.html

43. G O A L S
• Improve our understanding of the different player
behaviours and factors to improve engagement
• Find issues to avoid drop-outs
• Provide tools for game designers to (visually)
analyse the game and improve the understanding
of players
• Find game design flaws early and automatically

44. IV. DESIGN GAMES

45. AI AS A PART OF
GAME DESIGN!!!!

46. AI TO DESIGN GAMES
ROLES OF AI IN GAMES
▸ AI in the foreground of games - Foregrounding AI
▸ create gameplay based around thinking about how agents
work
▸ Designing games that use AI techniques in a new way as a
core of their gameplay
https://medium.com/@mtrc/tombs-of-tomeria-7c2e800a6511
Mike Treanor, Alexander Zook, Mirjam P Eladhari, Julian Togelius, Gillian Smith, Michael Cook, Tommy Thompson, Brian
Magerko, John Levine and Adam Smith: AI-Based Game Design Patterns. Computational Creativity and Games Workshop,
2015.

47. AI-BASED GAME DESIGN
▸ Game design strategies/rules described when AI still
“young” and most games are designed to not need AI
▸ Game designers often claim that AI won’t make games
better
▸ Our goal: show where AI can be used, show alternative
routes
▸ we need to design new games from scratch based on
new design principles
Mike Treanor, Alexander Zook, Mirjam P Eladhari, Julian Togelius, Gillian Smith, Michael Cook, Tommy Thompson, Brian
Magerko, John Levine and Adam Smith: AI-Based Game Design Patterns. Computational Creativity and Games Workshop,
2015.
AI TO DESIGN GAMES

48. AI GAME DESIGN PATTERNS
Mike Treanor, Alexander Zook, Mirjam P Eladhari, Julian Togelius, Gillian Smith, Michael Cook, Tommy Thompson, Brian
Magerko, John Levine and Adam Smith: AI-Based Game Design Patterns. Computational Creativity and Games Workshop,
2015.
AI TO DESIGN GAMES

49. AI DESIGN PATTERNS
1 AI IS VISUALIZED
▸ Pattern: Provide a visual representation of the underlying AI state, making gameplay revolve around
explicit manipulation of the AI state.
▸ Example: Third Eye Crime is a stealth game that illustrates this pattern by visualizing the guard AI position
tracking and estimation system. Gameplay involves avoiding guards or throwing distractions to manipulate
the guards’ predictions of player location. The direct visualization of AI state allows a designer to build a
game around manipulating, understanding, and mentally modeling how the AI state changes.

50. 2 AI AS ROLE-MODEL
▸ Pattern: Provide one or more AI agents for the player to behave similarly to.
▸ Example: Spy Party is a game where one player is a spy at a party populated by FSM agents and the
opposing player is a sniper watching the party with a single shot to kill the spy. Gameplay for the
spy centers on the player attempting to act similarly to the party agents while discreetly performing
tasks in the environment like planting a bug or reading a code from a book.
AI DESIGN PATTERNS

51. 3 AI AS TRAINEE
▸ Pattern: Have player actions train an AI agent to perform tasks central to gameplay.
▸ Example: Black & White is a god game where the player trains a creature to act as
an autonomous assistant in spatial regions where the player cannot take direct
action. The creature learns sets of behaviors through a reward signal based on a
needs model; the creature also takes direct feedback through player action (e.g.,
slapping or petting the creature after it takes actions).
AI DESIGN PATTERNS

52. 4 AI IS EDITABLE
▸ Pattern: Have the player directly change elements of an AI agent that is central to gameplay.
▸ Example: Galactic Arms Race is a space shooter where how the player uses different weapons evolves an underlying neural
network representation to change weapon firing behavior. Base gameplay revolves around finding a set of firing behaviors that
together enable a player to succeed at destroying opposition (another example of the AI as Trainee pattern). One gameplay mode
allows the player to explicitly manipulate the network weights on weapons, allowing more precise control over the firing patterns
of the evolved weapons. This control enables players to more finely explore the space of parameterizations, leading to an indirect
way to understand the processes of the AI system.
Erin J. Hastings, Ratan K. Guha, and Kenneth O. Stanley (2009)
Automatic Content Generation in the Galactic Arms Race Video Game
In: IEEE Transactions on Computational Intelligence and AI in Games, volume 1, number 4, pages 245-263, New York: IEEE Press, 2009. (Manuscript 19 pages)
AI DESIGN PATTERNS

53. 5 AI IS GUIDED
▸ Pattern: The player assists a simple or brittle AI agent that is threatened with self-destruction.
▸ Example: The Sims addressed the problem of “human-like” agents in a social world by making
gameplay revolve around the player addressing the needs of simple agents. AI agents have a set of
needs and desires they attempt to pursue while players intervene to provide for the needs of the
agents through food, shelter, work, socialization, and eventually more grand life aspirations. By having
players care for the AI, players come to (at least indirectly) model some of the processes used by the AI.
AI DESIGN PATTERNS

54. 8 AI AS VILLAIN
▸ Pattern: Require players to complete a task or overcome an AI opponent where the AI is aiming to create an
experience (e.g., tension or excitement) rather than defeat the player.
▸ Example: Alien: Isolation is a first-person survival horror game where the opposing alien was designed to harass
the player without using an optimal strategy that would always kill the player directly. The enemy alien spends
the game hunting the player, displaying behaviors of seeking the player’s location (a weak version of AI is
Visualized), and gradually learning from tactics the player uses repeatedly (an oppositional application of AI as
Trainee). By having players continually reason on what the alien has learned and where it will go the player is
forced to consider the state of the AI and (after repeated play) the processes involved in the AI learning.
AI DESIGN PATTERNS

55. AI TO CREATE
GAMES!!!!

58. AI TO DESIGN GAMES /
COMPUTATIONAL
CREATIVITY

59. RESOURCES
▸ IEEE Computational Intelligence and Games (CIG)
▸ AAAI Artificial Intelligence in Interactive Digital
Entertainment (AIIDE)
▸ Foundations of Digital Games (FDG)
▸ IEEE Transactions on Games (ToG)
▸ Yannakakis and Togelius: Artificial Intelligence and Games
www.gameaibook.org

60. THANK YOU FOR YOUR
ATTENTION.
JOHANNA PIRKER, JPIRKER@MIT.EDU, @JOEYPRINK
Further information:
jpirker.com
This is how others play your game!