Alexander Andelkovic. Comaqa Spring 2018. Using Artificial Intelligence to Test the Candy Crush Saga Game.
•Download as PPTX, PDF•
2 likes•285 views
This document discusses using artificial intelligence to test levels in the Candy Crush Saga game. It describes King.com, the developer of Candy Crush, and their use of QA teams. It then covers using AI techniques like Monte Carlo tree search and neuroevolution to have automated bots play levels and provide feedback. This could help level designers, reduce human testing workload, and assist data scientists. Challenges include maintaining the bots and integrating them into King's development process.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Alexander Andelkovic. Comaqa Spring 2018. Using Artificial Intelligence to Test the Candy Crush Saga Game.
Similar to Alexander Andelkovic. Comaqa Spring 2018. Using Artificial Intelligence to Test the Candy Crush Saga Game. (20)
More from COMAQA.BY
More from COMAQA.BY (20)
Recently uploaded
Recently uploaded (20)
Alexander Andelkovic. Comaqa Spring 2018. Using Artificial Intelligence to Test the Candy Crush Saga Game.
- 1. Using Artificial Intelligence to Test the Candy Crush Saga Game
- 2. © King.com Ltd 2017 About us King/Midasplayer AB • Founded 2003 • +200 Games • 2000 employees • 13 Studios(Stockholm, London, Barcelona, Seattle…) • 290 million MAU(Q4 2017) • Acquired by Activision/Blizzard(2016-02-23) Page 2
- 3. © King.com Ltd 2017 Products Franchise Games Page 3
- 4. © King.com Ltd 2017 QA at King King QA roles • ATL – Agile Testing Lead • QAAnalyst – Exploratory tester • QRT – Quick Regression Team • TAE – Test Automation Engineer • DS – Data Scientist • Developer – (Frontend/Backend) Page 4
- 5. © King.com Ltd 2017 QA Problem More than 2000 levels in Candy Page 9
- 6. © King.com Ltd 2017 Introduction History of AI Page 6
- 7. © King.com Ltd 2017 Introduction So, what is a bot exactly? Page 7
- 8. © King.com Ltd 2017 Introduction Application that performs an automated task Page 8
- 9. © King.com Ltd 2017 Introduction AI-bot – why? Page 9 • Level designers • QualityAssistance (QA) • Game-domain research
- 10. © King.com Ltd 2017 AI-bot How would an AI-bot actually think(heuristic, simulation)? Page 9
- 11. © King.com Ltd 2017 Heuristic Heuristic is good, maintenance is bad! Page 11 • Having a function that ranks moves – heuristic – could help a lot • Each simulation much richer • Has to be generic • Increasing complexity with more features • Can we create the heuristic automatically?
- 12. © King.com Ltd 2017 MCTS Simulation – Monte Carlo Tree Search Page 12
- 13. © King.com Ltd 2017 MCTS Page 13 Simulate example
- 14. © King.com Ltd 2017 MCTS Page 14 Win Loss Win Is this reliable?
- 15. © King.com Ltd 2017 MCTS Page 15 36% 28% 19% Repeat simulations – Monte-Carlo simulations
- 16. © King.com Ltd 2017 MCTS Page 16 36% 28% 19% Gradually we build a tree => Monte-Carlo Tree Search 40% 32%
- 17. © King.com Ltd 2017 MCTS Search tree for level 13 in Candy Page 17
- 18. © King.com Ltd 2017 MCTS Available actions Page 18
- 19. © King.com Ltd 2017 MCTS Future states Page 19
- 20. © King.com Ltd 2017 MCTS Page 20 After 299 simulations
- 21. © King.com Ltd 2017 MCTS demo Page 21 MCTS demo
- 22. © King.com Ltd 2017 MCTS - Impact What do we have? Page 22 • Generic method • Works for all levels • Works for other games
- 23. © King.com Ltd 2017 MCTS - summary MCTS summary Page 23 • Closer to human success rate • Easy to tune with more simulation per decision • Time consuming, one level takes ~ 5-10 mins • How can we improve?
- 24. © King.com Ltd 2017 NEAT NEAT – NeuroEvolution of Augmented Topologies! Page 24 • Artificial Neural Network(ANN) • Reinforced learning(action, learn, evolve) • Guided playout
- 25. © King.com Ltd 2017 NEAT Play Candy Crush! Page 25 4 1 0 0
- 26. © King.com Ltd 2017 NEAT NEAT – Steps Page 26 • Get training data - Predict score (hours) • Create random bots • Compete (hours) • Create new bots • Choose the best bot - ANN • Play Candy choosing moves in playout with ANN
- 27. © King.com Ltd 2017 NEAT The jungle law in Candy Page 27 Focus on blockers! Look at jellies! Special candies are tasty!
- 28. © King.com Ltd 2017 NEAT Making babies Page 28 Mom Dad
- 29. © King.com Ltd 2017 NEAT Page 29 Newborn Mom Dad
- 30. © King.com Ltd 2017 NEAT Product of evolution Page 30
- 31. © King.com Ltd 2017 NEAT demo Page 31 NEAT training • Red=Fittest Bot • Yellow=Avg Bot fitness • Green=# Bot’s • Blue=Fittest Bot ANN nodes
- 32. © King.com Ltd 2017 NEAT demo Page 32 NEAT training demo
- 33. © King.com Ltd 2017 Results Guiding the playout helps Page 33
- 34. © King.com Ltd 2017 NEAT NEAT – Performance Page 34 • Simulation time, game engine needs to replay each game state • Traversing time, game engine does not support jump to state • More GPU’s for bot training(5-6 hours > on commit)
- 35. © King.com Ltd 2017 Content QA Content QA – No bot Page 35
- 36. © King.com Ltd 2017 Content QA Content QA – With bot Page 36
- 37. © King.com Ltd 2017 Benefits Bot benefits Page 37 • Level development: level difficulty, less tweaks • Quality assistance: crash testing, performance testing, regression testing • Data scientists: Game domain knowledge, fun levels, game balancing
- 38. © King.com Ltd 2017 Challenges Bot challenges Page 38 • Ownership(Knowledge, Resources, Bot team) • Integration(Headless mode, C++/Lua/JS, Hackday/Sprint, Bot Api) • Maintenance(Bot Training, Infrastructure, Extend)
- 39. © King.com Ltd 2017 Links Crushing Candy Crush: Predicting Human Success Rate in a Mobile Game using Monte-Carlo Tree Search http://kth.diva-portal.org/smash/get/diva2:1093469/FULLTEXT01.pdf Predicting Game Level Difficulty Using Deep Neural Networks http://kth.diva-portal.org/smash/get/diva2:1154062/FULLTEXT01.pdf Simulating Human Game Play for Level Difficulty Estimation with Convolutional Neural Networks http://kth.diva-portal.org/smash/get/diva2:1149021/FULLTEXT01.pdf Page 39 Links
- 40. © King.com Ltd 2017 Q&A alexander.andelkovic@king.c om Page 40 Questions?
- 41. © King.com Ltd 2017 Thank you! Page 41
Editor's Notes
- Simple mobile games, can they be a challenge to AI? Single-player game with simple mechanics? There is only one level in chess and Go and no randomness. This is a BIG difference.
- Talk about the great success of AI in games, Kasparov 97 and Sedol 2016
- Spend some time here, explaining how it improves the work for level designers etc.
- The traditional AI in games, like in chess, builds a function which can rank the moves – a heuristic function – based on the relevant features of the game. We have done something like that for many of our games. Very fast and pretty good ... to begin with High maintenance, no transfer of method between games – new heuristic for each game. How would it choose between the possible actions? Should it try to think ahead, predict the future?
- We would like to have some heuristic but the levels are always changing. Get feedback about automatic heuristic construction.
- So, we tried simulation
- Here are three of the possible moves, how can we figure out which move is the best?
- Let‘s simulate each action. One simulation each move very unreliable. How should the playout be? Random at the moment. => repeat! Not too fast here, make sure everybody follows and understand how we play until the end while still thinking – no action taken yet.
- After repeated simulations we start to get some kind of an estimate for the expected value of each of these three actions.
- Current position is in the middle – the root of the tree. The bot is still thinking and has used 100 simulations. The smartness of MCTS comes from how we manage our resources – greed vs. exploration
- The circle shows the possible actions in the current position
- Further down the tree we see possible future states and future actions. Both trees from the same level but with different values for the parameter controlling greed. One is ”flat” the other more ”deep”.
- Colored tree, more simulations, red = loss, green = win
- The solid line shows the mean/average. The gray line is where we want to be
- Game engine code is not optimized for this. Easily scalable though. How can we improve? Talk about different methods available – get feedback from audience and steer it towards random playout. Strength and speed, one can often be exchanged for he other.
- We tried NEAT and will continue experimenting with automatically constructing heuristic.
- We want a bot that can weigh the importance of the game features, e.g. Number of candies in a combination, striped candy, number of jellies and number of blockers as shown here. But we are lazy and don‘t want to do anything ourself.
- Let’s create a lot of bots, and we are still lazy so we just create them randomly and let them play.
- Exemple for one child Mom uses interaction between candies/special candies Dad focuses on blockers
- Child do both Emphasize that this is how you generate the new generation Slights changes between newborns A couple won’t always produce the same child Different couples
- Complex pattern = More interactivity between features Straightforward = all features are independent A more complex pattern allows deeper thinking.
- The dotted line is the mean and the solid line is the mean with ANN. Remember, we want to be close to gray. Talk about that the heuristic can save us time – we can get good enough with more simulations
- We tried NEAT and will continue experimenting with automatically constructing heuristic.
- We tried NEAT and will continue experimenting with automatically constructing heuristic.
- We tried NEAT and will continue experimenting with automatically constructing heuristic.