Raheel Yawar, Game Programmer | Flying Sheep Studios, Germany In-game Content Generation using Machine Learning Adapting gameplay difficulty and content based on player profiles is vital for better retention. The Free-to-play Dreamwork’s TrollhuntersRPG is taken as an example for an in-depth look at how Machine Learning algorithms can be employed to discover player profiles and present profile relevant content. The discussion includes implementation details on how these methods can be crafted and adapted for other game genres or use cases and how they can positively impact player retention. Primarily the talk deals with a special breed of Machine Learning methods called Recommender Systems that can be used to take the load off the designers and learn from players about their skill level and preferences and use that information to tailor in-game content. Presented by the Serious Play Conference seriousplayconf.com at Orlando, University of Central Florida, UCF, July 24-26, 2019

1. In-game Content Generation using
Machine Learning
Raheel Yawar
Game Developer, Flying Sheep Studios

2. Who am I?
● Raheel Yawar
○ Game Developer @ Flying Sheep Studios
○ Mobile and HTML5 game development
○ MSc Media Informatics
○ ML/ AI Enthusiast
● @raheelyawar

3. Flying Sheep Studios
● Founded in 2014 in Cologne Germany
● Over 150 cross-platform HTML5 games
● Worked with over 50 brands
● Team of 17 people
○ 47% women
○ 35% internationals

4. Recommender

5. What are Recommender Systems?

6. What is this about?
● Motivation
● Action RPG Introduction
● Machine Learning Methods
● Pros and Cons
● Results and Findings
Note: This talk is beginner friendly

7. Motivation
Doom, id Software

8. Player Tailored Content
The Walking Dead, Telltale Games
Mass Effect, BioWare

9. Flow

10. The Game
● Single Player Hack n’ Slash Dungeon
Crawler Role Playing Game
● Free-2-Play
● Cross-platform HTML5
● Created using Three.js and Phaser
● Gameplay spans over
○ 30 levels
○ 9 story quests
○ Virtually infinite side-quests
http://www.toggo.de/spiele/trolljaeger/abenteuer-in-den-trollhoehlen-4774.htm

12. Quest Types
● Collection
● Combat
● Rescue
● Collection (Unique)
● Combat (Boss)
Collection Combat
Rescue Boss

13. The Quest Givers

14. Enemies

15. Environments

16. Procedural Content Generation
● Input: Seed and Difficulty Value
● Output: Quest and Level

17. PCG – Level Generation

18. The System
Recommender ServerGame Client
Player Statistics
Quests

19. The Recommender System
● Basic Client-Server architecture
● Built using Golang and Python
● MariaDB used for storage

20. Tracking Data - Quest
1. Quest environment
2. Quest Giver (NPC)
3. Difficulty Value (1 to 30+)
4. Quest Phases
5. Player Skill Build (Level, Strength, Agility, Damage)
6. Weapon Accuracy
7. Completion Time
8. Health Lost
9. Quest completion (Retries, Abandonment)

21. Quest Feedback
● Explicit Feedback
○ Ratings
● Implicit Feedback
○ Tracking
Did you enjoy
playing the last
quest?

22. Recommender System Types
● Procedural Content Generation
● Pre-generated 90,000 quests
● Content-Based/ Rule-Based Filtering
● Collaborative Filtering
○ Neighbourhood Filtering
○ Matrix Factorization
○ Tensor Factorization

23. The Content-Based Method
● Weapon Accuracy
● Health Lost
● Failure Rate
● Quest Features
○ Environment
○ Quest Giver
○ Phase Types

24. The Content-Based Method
Global Average Accuracy
40%
Player Average Accuracy
25%

25. The Content-Based Method
Piecewise Function of Average Accuracy

26. The Content-Based Method

27. The Content-Based Method
● Easy to formulate
● Small runtime
● Need to be tailor made
● Iterations needed
● Can miss out on information

28. Collaborative Filtering

29. Collaborative Filtering
Users/ Movie Lego Movie Expendables Notebook
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
User/ Movie Ratings - Utility Matrix

30. Collaborative Filtering
Users/ Movie Lego Movie Expendables Notebook
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Utility Matrix
Cosine Similarity of User 1 and User 3

31. Collaborative Filtering
Users/ Quest Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Utility Matrix

32. Collaborative Filtering
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Utility Matrices

33. Collaborative Filtering
● Level (25%)
● Strength (10%)
● Agility (10%)
● Damage (10%)
● Health Lost (25%)
● Accuracy (15%)
● Time Duration (5%)
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Utility Matrices

34. Collaborative Filtering
● Level (25%)
● Strength (10%)
● Agility (10%)
● Damage (10%)
● Health Lost (25%)
● Accuracy (15%)
● Time Duration (5%)

35. Collaborative Filtering - Neighbourhood
Construct
Utility
Matrices
Compute
Cosine
Similarity
Weight
Cosine
Similarity
Measure
Pick
Users
with
Highest
Similarity
Choose
Quests

36. Collaborative Filtering – Matrix Factorization
Construct
Utility
Matrices
SVD
Compute
Cosine
Similarity
Weight
Cosine
Similarity
Measure
Pick
Users
with
Highest
Similarity
Choose
Quests

37. Collaborative Filtering - Tensor Factorization
Construct
Utility
Matrices
Dual
DEDICOM
Compute
Cosine
Similarity
Weight
Cosine
Similarity
Measure
Pick Users
with
Highest
Similarity
Choose
Quests

38. Collaborative Filtering – Learnings
● Applicable over a variety of scenarios
● Can detect latent factors
● Low maintenance
● Cold start problem
● Resource intensive
Level (25%)
Strength (10%)
Agility (10%)
Damage (10%)
Health Lost (25%)
Accuracy (15%)
Time Duration (5%)
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -
Users/ Movie Quest 1 Quest 2 Quest 3
User 1 5 5 1
User 2 3 1 5
User 3 5 - 1
User 4 3 1 -

39. Test Approach
● Approach
○ Round Robin Assignment
○ Bucket Testing with PCG group
● 25,686 players

40. Results – Quest Failure & Abandonment Rates
Quest Failure Rate Quest Abandonment Rate

41. Results – Retention Rate
Baseline
(PCG)
Content Based
(CB)
Neighbourhoo
d Oriented
(NO)
Matrix
Factorization
(MF)
Tensor
Factorization
(RTDD)
Day 1 8.21 8.64 7.79 8.92 8.93
Day 2 5.62 6.62 5.90 6.49 6.78
Day 3 3.96 4.64 4.40 4.55 4.74
Day 4 2.88 3.49 2.89 3.30 3.48
Day 5 2.02 2.44 1.82 1.94 2.30
Day 6 1.19 1.55 1.11 1.25 1.51
Day 7 0.25 0.54 0.61 0.47 0.72
Average 3.45 3.99 3.50 3.85 4.07

42. Summary
● Need for better tailored content
● ML/ AI algorithms can be used to:
○ Better identify player profiles
○ Tailor procedurally generated content
○ Matrix/ Tensor based approaches outperform rule-based approaches

43. References
● Matrix and Tensor Factorization Based
Game Content Recommender Systems:
A Bottom-Up Architecture and a
Comparative Online Evaluation, AIIDE
2018
● Clip Art: https://flaticon.com

44. Thank You
● Raheel Yawar
○ Game Developer
○ Flying Sheep Studios
● Website: http://raheelyawar.com/
● Twitter: @raheelyawar