How games are driving advances in AI research- Unite Copenhagen 2019
•
2 likes•143 views
Many recent advances in deep reinforcement and artificial intelligence learning have stemmed from video games. In this session, we'll explore a brief history of this relationship, looking specifically at how Unity is pushing the boundaries of AI research with the Obstacle Tower Challenge. We'll also show how Unity is leveraging cutting-edge research to solve gaming's biggest challenges with the Unity Machine Learning Agents Toolkit, one of the most popular open-source toolkits for deep learning. Speaker: Matthew Crosby - Imperial College London
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to How games are driving advances in AI research- Unite Copenhagen 2019
Similar to How games are driving advances in AI research- Unite Copenhagen 2019 (20)
More from Unity Technologies
More from Unity Technologies (20)
Recently uploaded
Recently uploaded (20)
How games are driving advances in AI research- Unite Copenhagen 2019
- 2. Driving AI Research Using Games Jeffrey Shih Lead Product Manager, AI Unity Technologies @shihzy Dr. Matt Crosby Postdoctoral Researcher Leverhulme Centre for the Future of Intelligence Imperial College London @MaCroPhilosophy Benjamin Beyret Researcher Leverhulme Centre for the Future of Intelligence Imperial College London @BenBeyret
- 3. Video games have been used to drive artificial intelligence research 3
- 4. Video games have been used to drive artificial intelligence research (for a long time) 4
- 5. Video games 5 A few examples
- 6. Video games 6 A few examples ● Atari 2600 Games - DeepMind, OpenAI
- 7. Video games 7 A few examples ● Atari 2600 Games - DeepMind, OpenAI ● Doom - VizDoom from Poznan University
- 8. Video games 8 A few examples ● Atari 2600 Games - DeepMind, OpenAI ● Doom - VizDoom from Poznan University ● Quake 3 - DeepMind
- 9. Video games 9 A few examples ● Atari 2600 Games - DeepMind, OpenAI ● Doom - VizDoom from Poznan University ● Quake 3 - DeepMind ● Minecraft - Microsoft Project Malmo
- 10. Video games 10 A few examples ● Atari 2600 Games - DeepMind, OpenAI ● Doom - VizDoom from Poznan University ● Quake 3 - DeepMind ● Minecraft - Microsoft Project Malmo ● Starcraft 2 - DeepMind / Blizzard
- 11. Video games 11 A few examples ● Atari 2600 Games - DeepMind, OpenAI ● Doom - VizDoom from Poznan University ● Quake 3 - DeepMind ● Minecraft - Microsoft Project Malmo ● Starcraft 2 - DeepMind / Blizzard ● Dota 2 - OpenAI Five
- 12. Video games 12 Some novel approaches in AI
- 13. Video games 13 Some novel approaches in AI ● Deep Q-Network (DQN) paper published by DeepMind
- 14. Video games 14 Some novel approaches in AI ● Deep Q-Network (DQN) paper published by DeepMind ● Proximal Policy Optimization (PPO) paper published by OpenAI
- 15. Video games 15 Some novel approaches in AI ● Deep Q-Network (DQN) paper published by DeepMind ● Proximal Policy Optimization (PPO) paper published by OpenAI ● New AI systems and approaches to beating top players in StarCraft II and DOTA 2
- 19. Demis Hassabis - co-founder and CEO of DeepMind As a former video game designer myself, I couldn’t be more excited to be collaborating with Unity, creating virtual environments for developing and testing the kind of smart, flexible algorithms we need to tackle real-world problems
- 20. AI-Based Challenges Using Unity The Animal-AI Olympics
- 21. History of the Obstacle Tower Challenge — February 18, 2019: Launched 1st qualifying round on AICrowd with Google Cloud Platform as a co-sponsor. Over $100K in prizes. — Qualifying round participation: 2000+ entries from 350+ teams — May 15, 2019: 2nd round launched — August 7, 2019: Winners announced and Obstacle Tower open-sourced
- 22. Obstacle Tower Environment Research and benchmarks areas — Vision – High-fidelity 3D visuals – Realtime lighting/shadows — Control – Platforming puzzles — Planning – Complex floor layouts — Generalization – Procedural Floors, Rooms, and Visuals 22
- 23. Different Floors and Themes 23
- 24. Obstacle Tower Environment Procedurally generated design — Each episode a new tower – Each tower filled with 25 floors – Each floor filled with rooms – Each room filled with obstacles and puzzles! 24
- 25. Learnings... Winner: Alex Nicol — Classifier for object identification — Imitation Learning — PPO (Proximal Policy Optimization) for fine-tuning behavior 2nd: Gianni & Miha — PPO with modifications — Sampling algorithm — 10 Billion steps sampled 3rd: Songbin Choi — Standard PPO — Human Play experience added
- 26. The Animal-AI Olympics Download it today github.com/beyretb/AnimalAI-Olympics 26 animalaiolympics.com
- 28. Animal Cognition Tasks 28 Can we build AI systems to do this?
- 29. Animal Cognition Tasks 29 Can we build AI systems to do this? — No! — It’s way too hard.
- 30. Animal Cognition Tasks 30 Can we build AI systems to do this? — No! — It’s way too hard. Can we build a research pathway?
- 31. Animal Cognition Tasks 31 Can we build AI systems to do this? — No! — It’s way too hard. Can we build a research pathway? — Yes! — Using Unity - ML-Agents
- 32. Robot Corvids 32
- 33. Robot Corvids 33 Detect food is inside the containers
- 34. Robot Corvids 34 Detect food is inside the containers Detect Containers have open tops
- 35. Robot Corvids 35 Detect food is inside the containers Detect objects that could help Detect Containers have open tops
- 36. Robot Corvids 36 Detect food is inside the containers Successfully grip object Detect objects that could help Detect Containers have open tops
- 37. Robot Corvids 37 Detect food is inside the containers Successfully grip object Notice different substances Detect objects that could help Detect Containers have open tops
- 38. Robot Corvids 38 Not to mention anything required to actually solve the problem! Detect Containers have open tops Detect food is inside the containers Successfully grip object Notice different substances Detect objects that could help
- 39. Design Principles 39 — Simple tasks — Simple affordances — Abstract from animals
- 41. The Animal-AI Environment: Possibilities 41 — Goal is to get food — From simple building blocks can build complex structures/experiments.
- 43. Use example: Maze Navigation 43 Test animals’ navigation skills Claude Shannon
- 44. Use example: Maze Navigation 44 Claude Shannon
- 45. Use example: Maze Navigation 45 Claude Shannon
- 46. Use example: Maze Navigation 46 Claude Shannon
- 47. Use example: Maze Navigation 47 Claude Shannon Experimentation design
- 48. Use example: Maze Navigation 48 Step 1: design training
- 49. Use example: Maze Navigation 49 Step 1: design training
- 50. Use example: Maze Navigation 50 Step 1: design training
- 51. Use example: Maze Navigation 51 Step 1: design training
- 52. Use example: Maze Navigation 52 Step 1: design training
- 53. Use example: Maze Navigation 53 Step 1: design training
- 54. Use example: Maze Navigation 54 Step 1: design training
- 55. Use example: Maze Navigation 55 Step 1: design training
- 56. Use example: Maze Navigation 56 Step 1: design training
- 57. Use example: Maze Navigation 57 Step 1: design training
- 58. Use example: Maze Navigation In practice AGENT ENVIRONMENT
- 59. Use example: Maze Navigation In practice AGENT ENVIRONMENT LESSON
- 60. Use example: Maze Navigation In practice AGENT ENVIRONMENT ACTION LESSON
- 61. Use example: Maze Navigation 61 In practice AGENT ENVIRONMENT ACTION OBSERVATIONS REWARD LESSON
- 62. Use example: Maze Navigation 62 In practice AGENT ENVIRONMENT ACTION OBSERVATIONS REWARD LESSON
- 63. Use example: Maze Navigation 63 In practice AGENT ENVIRONMENT ACTION OBSERVATIONS REWARD LESSON
- 64. Use example: Maze Navigation 64 In practice AGENT ENVIRONMENT ACTION OBSERVATIONS REWARD LESSON
- 65. Use example: Maze Navigation 65 In practice AGENT ENVIRONMENT ACTION OBSERVATIONS REWARD LESSON
- 66. Use example: Maze Navigation 66 Step 3: Enjoy
- 67. 10 Categories 300 tests 67 1. Food 2. Preferences 3. Obstacles 4. Avoidance 5. Spatial Reasoning 6. Generalisation 7. Internal Models 8. Object Permanence 9. Advanced Preferences 10. Causal Reasoning
- 68. Competition Results 68 — 50+ teams — 25,000+ correct solutions — One month left to go!
- 72. Conclusions 72 — A gaming-style environment allows us to build an arena with simulated physics and visual inputs. — Many tests used on animals in the real world can be translated to our environment. — Creates a research framework for creating AIs with animal-like skills. — A crucial step towards Artificial General Intelligence.
- 73. Learn more! Additional Information (Point Camera) Chat with us, demos and more! Visit us at the Games AI Kiosk on the Expo Floor! https://tinyurl.com/unite-airesearch