SlideShare ist ein Scribd-Unternehmen logo
1 von 20
Downloaden Sie, um offline zu lesen
Designing and Evolving an
Unreal Tournament 2004
Expert Bot
A.M. Mora, F. Aisa, R. Caballero, P. García-Sánchez, J.J. Merelo, P.A. Castillo, R. Lara-Cabrera
International Work-Conference on Artificial Neural Networks
INDEX
• Unreal (game, environment)
• Pogamut tool
• Unreal Expert Bot (objectives, features)
• Evolutionary Algorithms
• Unreal Expert Bot Evolution (description,
approaches, results)
• E-BOT vs GE-BOT (results, demo)
• Conclusions
Environment description
UNREAL
Unreal is a first person shooter (FPS)first person shooter (FPS).
Famous due to the excelent AI of the enemiesexcelent AI of the enemies (bots), which makes it an
amazing multiplayer game. Unreal Tournament series is widely extended.
It offers an editor (UnrealEd)editor (UnrealEd) which lets us change almost anything in
the game even the behaviour of the bots. It uses the language
UnrealScriptUnrealScript.
General description
POGAMUT
A java middlewarejava middleware for Unreal Tournament series games and
Defcon games.
The architecture is as follows:
It is possible to interact with the game from a java program,
getting higher independencehigher independence (avoiding Unrealscript restrictions)
and increasing the possibilitiesincreasing the possibilities (java libraries).
On the contrary, the structures, classes, functions andstructures, classes, functions and
workflows defined in the Unreal engine, cannot be accessedworkflows defined in the Unreal engine, cannot be accessed, nor
used.
Objectives
UNREAL EXPERT BOT
• Create an autonomous agent for playing Unreal
Tournament deathmatch championship.
• Considering the constraints of this competition:
- 1 vs 1 matches
- Small arenas
- Weapons are not respawned
- Some forbidden items (U-Damage, for instance)
- 15 minutes per match instead of a number of frags (kills)
• Human-like behaviour is desired.
• Modelling Expert player knowledge (and tricks).
- High control in timing (items respawn time)
- Deep knowledge about weapons and their advantages and
disadvantages
- Deep knowledge about items
Features
UNREAL EXPERT BOT
• Defined by means of a Finite State Machine based AIState Machine based AI with two
state levels.
• Translated into a set of rules which determine its behaviour.
• Database which models the bot’s memory, since it is uploaded
with data about locations of items and weapons in the map.
Bot performance
UNREAL EXPERT BOT
• Expert Bot (E-Bot) outperformed the standard bots in the game
(considering the number of frags), even in the maximum
difficulty level.
• This difficulty level is quite hard for a medium level
player.
• E-Bot is hard to beat for humans, even for the expert.
• Medium level players usually lose against it.
Evolutionary Algorithms
EXPERT BOT EVOLUTION
by Johann Dréo
i -> initial
population
f -> evaluation
function (fitness)
? -> stop condition
Se -> selection
Cr -> crossover
Mu -> mutation
Re -> replacement
Evolutionary Process in Unreal game
EXPERT BOT EVOLUTION
GE-BOT
Expert Bot based in a
Genetic Algorithm
Evolutionary
process
population
FITNESS EVALUATION
• Analyze Expert bot’s FSM
• Identify parameters
• Optimize them
Expert
Bot’s
AI
Approaches
EXPERT BOT EVOLUTION
• Generic Fitness
Just considers frags/deads
and damage produced/received
• Generational scheme
• 4-elitism
• Complex Fitness
- considers frags/deads
- damage produced/received
- time using the best or more
versatile weapons: Lightning
Gun and Shock Rifle
- getting the best items: Shield
and Super Shield
• Stationary scheme
• Chromosome 143
• Uniform Crossover
• Random mutation
• 4 Random individuals
• Chromosome 26
Approach 1: Chromosome 143 - Generic Fitness
EXPERT BOT EVOLUTION
• Generic Fitness
Just considers frags/deads
and damage produced/received
• Generational scheme
• 4-elitism
• Chromosome 143
• Uniform Crossover
• Random mutation
• 4 Random individuals
Approach 1. Results
EXPERT BOT EVOLUTION
• 30 generations
• 30 individuals
• 1 evaluation (left)
• 3 evaluations (right)
in order to avoid the
noisy nature of the
fitness function
• 15 minutes per
evaluation
• 10 days per run (left)
• One month (right)
• Lightly improvement
tendency
• Too many oscillations,
i.e. noise
• 143 genes are too much
EXPERT BOT EVOLUTION
• Generic Fitness
Just considers frags/deads
and damage produced/received
• Generational scheme
• 4-elitism
• Uniform Crossover
• Random mutation
• 4 Random individuals
• Chromosome 26
Approach 2: Chromosome 26 - Generic Fitness
Approach 2. Results
EXPERT BOT EVOLUTION
• 50 generations
• 30 individuals
• 5 minutes per
evaluation
• Results of 2 different
runs
• 5 days per run
• Again lightly
improvement tendency
• Too much noise
• Too much diversity
EXPERT BOT EVOLUTION
• Complex Fitness
- considers frags/deads
- damage produced/received
- time using the best or more
versatile weapons: Lightning
Gun and Shock Rifle
- getting the best items: Shield
and Super Shield
• Stationary scheme
• Uniform Crossover
• Random mutation
• 4 Random individuals
• Chromosome 26
Approach 3: Chromosome 26 - Complex Fitness
Approach 3. Results
EXPERT BOT EVOLUTION
• 40 generations
• 30 individuals
• 5 minutes per
evaluation
• Stationary scheme to
increase the
exploitation factor
• Results of 2 different
runs
• 5 days per run
• Quite good fitness
tendency
• Noise still remains,
but in a lower factor
Numerical results
E-BOT vs GE-BOT
• Expert Bot (E-Bot) and the best Genetic Expert Bots (GE-BOT)
have been fighting in four battles (in two maps).
• The average results of these matches are:
• The approach with 143 genes per chromosome is defeated
• GE-Bot with 26 genes outperforms E-Bot.
• The approach with the complex fitness function gets the best
results. Due to its lower noisy factor, and the higher
exploitation component.
DEMO
E-BOT vs GE-BOT
http://www.youtube.com/watch?v=ktcXHZ-nAfw
CONCLUSIONS
• We have designed a human-like Expert Bot (E-Bot) which
outperforms the standard Unreal Tournament 2K4 bots in the
hardest difficulty.
• It is also a hard rival against human players.
• We have tested three different approaches for improving this
bot by means of Genetic Algorithms.
• Too long chromosomes population performs worse than small
length one.
• These algorithms are affected by a high noisy factor regarding
the generic (and easier) fitness function.
• We have defined a complex fitness function which performs
better, with a softer noisy effect.
• The bots obtained after evolution outperform the E-Bot.
END
THE
Questions?!?!
Contact: amorag@geneura.ugr.es
Source Code: https://github.com/franaisa/ExpertAgent

Weitere ähnliche Inhalte

Ähnlich wie Designing and Evolving an Unreal Tournament 2004 Expert Bot

infoShare 2013: Adam Frańczak - Techniki optymalizacyjne w UDK/ UE3
infoShare 2013: Adam Frańczak - Techniki optymalizacyjne w UDK/ UE3infoShare 2013: Adam Frańczak - Techniki optymalizacyjne w UDK/ UE3
infoShare 2013: Adam Frańczak - Techniki optymalizacyjne w UDK/ UE3Infoshare
 
Requirement Engineering process on The Outer Worlds game
Requirement Engineering process on The Outer Worlds gameRequirement Engineering process on The Outer Worlds game
Requirement Engineering process on The Outer Worlds gameAmna Khalil
 
Dmytro Panchenko "Cracking Kaggle: Human Protein Atlas"
Dmytro Panchenko "Cracking Kaggle: Human Protein Atlas"Dmytro Panchenko "Cracking Kaggle: Human Protein Atlas"
Dmytro Panchenko "Cracking Kaggle: Human Protein Atlas"Lviv Startup Club
 
How to Design Effective Learning Games: Sharon Boller and Karl Kapp
How to Design Effective Learning Games: Sharon Boller and Karl KappHow to Design Effective Learning Games: Sharon Boller and Karl Kapp
How to Design Effective Learning Games: Sharon Boller and Karl KappSharon Boller
 
Games User Research is for Game Design!
Games User Research is for Game Design!Games User Research is for Game Design!
Games User Research is for Game Design!Marina Kobayashi
 
Ndss 2016 game_bot_final_no_video
Ndss 2016 game_bot_final_no_videoNdss 2016 game_bot_final_no_video
Ndss 2016 game_bot_final_no_videoEun-Jo Lee
 
Killzone Shadow Fall: Threading the Entity Update on PS4
Killzone Shadow Fall: Threading the Entity Update on PS4Killzone Shadow Fall: Threading the Entity Update on PS4
Killzone Shadow Fall: Threading the Entity Update on PS4jrouwe
 
AlphaZero and beyond: Polygames
AlphaZero and beyond: PolygamesAlphaZero and beyond: Polygames
AlphaZero and beyond: PolygamesOlivier Teytaud
 
Ropossum: A Game That Generates Itself
Ropossum: A Game That Generates ItselfRopossum: A Game That Generates Itself
Ropossum: A Game That Generates ItselfMohammad Shaker
 
The Power and Peril of PCG
The Power and Peril of PCGThe Power and Peril of PCG
The Power and Peril of PCGGillian Smith
 
weekly AI tech talk #85 ml-agents Enabling Learned Behaviors with Reinforceme...
weekly AI tech talk #85 ml-agents Enabling Learned Behaviors with Reinforceme...weekly AI tech talk #85 ml-agents Enabling Learned Behaviors with Reinforceme...
weekly AI tech talk #85 ml-agents Enabling Learned Behaviors with Reinforceme...Bill Liu
 
Forts and Fights Scaling Performance on Unreal Engine*
Forts and Fights Scaling Performance on Unreal Engine*Forts and Fights Scaling Performance on Unreal Engine*
Forts and Fights Scaling Performance on Unreal Engine*Intel® Software
 
Next Gen Asteroids
Next Gen AsteroidsNext Gen Asteroids
Next Gen AsteroidsJustin Young
 
Evolving Custom Communication Protocols
Evolving Custom Communication ProtocolsEvolving Custom Communication Protocols
Evolving Custom Communication ProtocolsWesley Faler
 
Evolutionary Multi-Agent Systems for RTS Games
Evolutionary Multi-Agent Systems for RTS GamesEvolutionary Multi-Agent Systems for RTS Games
Evolutionary Multi-Agent Systems for RTS GamesAdrián Palacios Corella
 
98 374 Lesson 04-slides
98 374 Lesson 04-slides98 374 Lesson 04-slides
98 374 Lesson 04-slidesTracie King
 
Making VR Games and Experiences in UE4
Making VR Games and Experiences in UE4Making VR Games and Experiences in UE4
Making VR Games and Experiences in UE4Unreal Engine
 

Ähnlich wie Designing and Evolving an Unreal Tournament 2004 Expert Bot (20)

infoShare 2013: Adam Frańczak - Techniki optymalizacyjne w UDK/ UE3
infoShare 2013: Adam Frańczak - Techniki optymalizacyjne w UDK/ UE3infoShare 2013: Adam Frańczak - Techniki optymalizacyjne w UDK/ UE3
infoShare 2013: Adam Frańczak - Techniki optymalizacyjne w UDK/ UE3
 
Endless runner
Endless runnerEndless runner
Endless runner
 
Requirement Engineering process on The Outer Worlds game
Requirement Engineering process on The Outer Worlds gameRequirement Engineering process on The Outer Worlds game
Requirement Engineering process on The Outer Worlds game
 
God Of War : post mortem
God Of War : post mortemGod Of War : post mortem
God Of War : post mortem
 
Dmytro Panchenko "Cracking Kaggle: Human Protein Atlas"
Dmytro Panchenko "Cracking Kaggle: Human Protein Atlas"Dmytro Panchenko "Cracking Kaggle: Human Protein Atlas"
Dmytro Panchenko "Cracking Kaggle: Human Protein Atlas"
 
How to Design Effective Learning Games: Sharon Boller and Karl Kapp
How to Design Effective Learning Games: Sharon Boller and Karl KappHow to Design Effective Learning Games: Sharon Boller and Karl Kapp
How to Design Effective Learning Games: Sharon Boller and Karl Kapp
 
Games User Research is for Game Design!
Games User Research is for Game Design!Games User Research is for Game Design!
Games User Research is for Game Design!
 
Ndss 2016 game_bot_final_no_video
Ndss 2016 game_bot_final_no_videoNdss 2016 game_bot_final_no_video
Ndss 2016 game_bot_final_no_video
 
Killzone Shadow Fall: Threading the Entity Update on PS4
Killzone Shadow Fall: Threading the Entity Update on PS4Killzone Shadow Fall: Threading the Entity Update on PS4
Killzone Shadow Fall: Threading the Entity Update on PS4
 
AlphaZero and beyond: Polygames
AlphaZero and beyond: PolygamesAlphaZero and beyond: Polygames
AlphaZero and beyond: Polygames
 
Ropossum: A Game That Generates Itself
Ropossum: A Game That Generates ItselfRopossum: A Game That Generates Itself
Ropossum: A Game That Generates Itself
 
The Power and Peril of PCG
The Power and Peril of PCGThe Power and Peril of PCG
The Power and Peril of PCG
 
weekly AI tech talk #85 ml-agents Enabling Learned Behaviors with Reinforceme...
weekly AI tech talk #85 ml-agents Enabling Learned Behaviors with Reinforceme...weekly AI tech talk #85 ml-agents Enabling Learned Behaviors with Reinforceme...
weekly AI tech talk #85 ml-agents Enabling Learned Behaviors with Reinforceme...
 
Forts and Fights Scaling Performance on Unreal Engine*
Forts and Fights Scaling Performance on Unreal Engine*Forts and Fights Scaling Performance on Unreal Engine*
Forts and Fights Scaling Performance on Unreal Engine*
 
Next Gen Asteroids
Next Gen AsteroidsNext Gen Asteroids
Next Gen Asteroids
 
Evolving Custom Communication Protocols
Evolving Custom Communication ProtocolsEvolving Custom Communication Protocols
Evolving Custom Communication Protocols
 
Evolutionary Multi-Agent Systems for RTS Games
Evolutionary Multi-Agent Systems for RTS GamesEvolutionary Multi-Agent Systems for RTS Games
Evolutionary Multi-Agent Systems for RTS Games
 
01 - Introduction to Game Mechanics
01 - Introduction to Game Mechanics01 - Introduction to Game Mechanics
01 - Introduction to Game Mechanics
 
98 374 Lesson 04-slides
98 374 Lesson 04-slides98 374 Lesson 04-slides
98 374 Lesson 04-slides
 
Making VR Games and Experiences in UE4
Making VR Games and Experiences in UE4Making VR Games and Experiences in UE4
Making VR Games and Experiences in UE4
 

Kürzlich hochgeladen

OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxYounusS2
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Introduction to Quantum Computing
Introduction to Quantum ComputingIntroduction to Quantum Computing
Introduction to Quantum ComputingGDSC PJATK
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceMartin Humpolec
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfAnna Loughnan Colquhoun
 

Kürzlich hochgeladen (20)

OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptx
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Introduction to Quantum Computing
Introduction to Quantum ComputingIntroduction to Quantum Computing
Introduction to Quantum Computing
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdf
 

Designing and Evolving an Unreal Tournament 2004 Expert Bot

  • 1. Designing and Evolving an Unreal Tournament 2004 Expert Bot A.M. Mora, F. Aisa, R. Caballero, P. García-Sánchez, J.J. Merelo, P.A. Castillo, R. Lara-Cabrera International Work-Conference on Artificial Neural Networks
  • 2. INDEX • Unreal (game, environment) • Pogamut tool • Unreal Expert Bot (objectives, features) • Evolutionary Algorithms • Unreal Expert Bot Evolution (description, approaches, results) • E-BOT vs GE-BOT (results, demo) • Conclusions
  • 3. Environment description UNREAL Unreal is a first person shooter (FPS)first person shooter (FPS). Famous due to the excelent AI of the enemiesexcelent AI of the enemies (bots), which makes it an amazing multiplayer game. Unreal Tournament series is widely extended. It offers an editor (UnrealEd)editor (UnrealEd) which lets us change almost anything in the game even the behaviour of the bots. It uses the language UnrealScriptUnrealScript.
  • 4. General description POGAMUT A java middlewarejava middleware for Unreal Tournament series games and Defcon games. The architecture is as follows: It is possible to interact with the game from a java program, getting higher independencehigher independence (avoiding Unrealscript restrictions) and increasing the possibilitiesincreasing the possibilities (java libraries). On the contrary, the structures, classes, functions andstructures, classes, functions and workflows defined in the Unreal engine, cannot be accessedworkflows defined in the Unreal engine, cannot be accessed, nor used.
  • 5. Objectives UNREAL EXPERT BOT • Create an autonomous agent for playing Unreal Tournament deathmatch championship. • Considering the constraints of this competition: - 1 vs 1 matches - Small arenas - Weapons are not respawned - Some forbidden items (U-Damage, for instance) - 15 minutes per match instead of a number of frags (kills) • Human-like behaviour is desired. • Modelling Expert player knowledge (and tricks). - High control in timing (items respawn time) - Deep knowledge about weapons and their advantages and disadvantages - Deep knowledge about items
  • 6. Features UNREAL EXPERT BOT • Defined by means of a Finite State Machine based AIState Machine based AI with two state levels. • Translated into a set of rules which determine its behaviour. • Database which models the bot’s memory, since it is uploaded with data about locations of items and weapons in the map.
  • 7. Bot performance UNREAL EXPERT BOT • Expert Bot (E-Bot) outperformed the standard bots in the game (considering the number of frags), even in the maximum difficulty level. • This difficulty level is quite hard for a medium level player. • E-Bot is hard to beat for humans, even for the expert. • Medium level players usually lose against it.
  • 8. Evolutionary Algorithms EXPERT BOT EVOLUTION by Johann Dréo i -> initial population f -> evaluation function (fitness) ? -> stop condition Se -> selection Cr -> crossover Mu -> mutation Re -> replacement
  • 9. Evolutionary Process in Unreal game EXPERT BOT EVOLUTION GE-BOT Expert Bot based in a Genetic Algorithm Evolutionary process population FITNESS EVALUATION • Analyze Expert bot’s FSM • Identify parameters • Optimize them Expert Bot’s AI
  • 10. Approaches EXPERT BOT EVOLUTION • Generic Fitness Just considers frags/deads and damage produced/received • Generational scheme • 4-elitism • Complex Fitness - considers frags/deads - damage produced/received - time using the best or more versatile weapons: Lightning Gun and Shock Rifle - getting the best items: Shield and Super Shield • Stationary scheme • Chromosome 143 • Uniform Crossover • Random mutation • 4 Random individuals • Chromosome 26
  • 11. Approach 1: Chromosome 143 - Generic Fitness EXPERT BOT EVOLUTION • Generic Fitness Just considers frags/deads and damage produced/received • Generational scheme • 4-elitism • Chromosome 143 • Uniform Crossover • Random mutation • 4 Random individuals
  • 12. Approach 1. Results EXPERT BOT EVOLUTION • 30 generations • 30 individuals • 1 evaluation (left) • 3 evaluations (right) in order to avoid the noisy nature of the fitness function • 15 minutes per evaluation • 10 days per run (left) • One month (right) • Lightly improvement tendency • Too many oscillations, i.e. noise • 143 genes are too much
  • 13. EXPERT BOT EVOLUTION • Generic Fitness Just considers frags/deads and damage produced/received • Generational scheme • 4-elitism • Uniform Crossover • Random mutation • 4 Random individuals • Chromosome 26 Approach 2: Chromosome 26 - Generic Fitness
  • 14. Approach 2. Results EXPERT BOT EVOLUTION • 50 generations • 30 individuals • 5 minutes per evaluation • Results of 2 different runs • 5 days per run • Again lightly improvement tendency • Too much noise • Too much diversity
  • 15. EXPERT BOT EVOLUTION • Complex Fitness - considers frags/deads - damage produced/received - time using the best or more versatile weapons: Lightning Gun and Shock Rifle - getting the best items: Shield and Super Shield • Stationary scheme • Uniform Crossover • Random mutation • 4 Random individuals • Chromosome 26 Approach 3: Chromosome 26 - Complex Fitness
  • 16. Approach 3. Results EXPERT BOT EVOLUTION • 40 generations • 30 individuals • 5 minutes per evaluation • Stationary scheme to increase the exploitation factor • Results of 2 different runs • 5 days per run • Quite good fitness tendency • Noise still remains, but in a lower factor
  • 17. Numerical results E-BOT vs GE-BOT • Expert Bot (E-Bot) and the best Genetic Expert Bots (GE-BOT) have been fighting in four battles (in two maps). • The average results of these matches are: • The approach with 143 genes per chromosome is defeated • GE-Bot with 26 genes outperforms E-Bot. • The approach with the complex fitness function gets the best results. Due to its lower noisy factor, and the higher exploitation component.
  • 19. CONCLUSIONS • We have designed a human-like Expert Bot (E-Bot) which outperforms the standard Unreal Tournament 2K4 bots in the hardest difficulty. • It is also a hard rival against human players. • We have tested three different approaches for improving this bot by means of Genetic Algorithms. • Too long chromosomes population performs worse than small length one. • These algorithms are affected by a high noisy factor regarding the generic (and easier) fitness function. • We have defined a complex fitness function which performs better, with a softer noisy effect. • The bots obtained after evolution outperform the E-Bot.
  • 20. END THE Questions?!?! Contact: amorag@geneura.ugr.es Source Code: https://github.com/franaisa/ExpertAgent

Hinweis der Redaktion

  1. Las imágenes de la izquierda corresponden a distintos juegos: Unreal (derecha), Unreal Tournament (izquierda) y Unreal Tournament 2004 (la que sobresale) Las de la derecha corresponden al editor (Unreal ED), que viene con cada juego y que deja abierto casi todo el motor (el Unreal Engine), para cambiar casi lo que quieras. Aunque hay muchas clases restringidas, claro. A la derecha se puede ver el árbol de clases, del que puede heredar al crear una nueva. ;) Sirve también para modificar personajes o escenarios (gráficamente). ;)
  2. Para hacer los bots se usará Pogamut. Gamebots fue la primera librería que apareció para comunicarse con el motor de Unreal Tournament 2004. Está definida en el mismo entorno (como un plugin) Pogamut se basa en ella y utiliza Gavialib para la comunicación. Se programa en Java con la librería de Pogamut.
  3. Al usar Pogamut, no se puede hacer uso del motor de IA de Unreal (basado también en estados), por lo que se ha creado una FSM o IA desde cero. Describir lo que es una FSM si quieres. El bot estará en un estado primario y cuando ocurran eventos se pasará a uno secundario (dentro de ese primario). Ejemplo: el bot está en Attack, pero recibe daño y en lugar de pasar a otro estado de retirada, busca salud, pero sin dejar de atacar. ;) Los datos se meten en la BD cuando el bot los ve en el mapa, no antes, como lo haría un humano. Aquí tienes una muestra sencilla de diagrama de flujo en el que se muestra cómo se elige el estado principal. ;)
  4. Estos resultados se obtuvieron después de publicar el artículo. En él hay menos experimentos (2 partidas sólo). Se hicieron experimentos contra humanos, aunque un poco ‘informales’. :D
  5. Esto te lo sabes. ;D Habría que decir que, una vez se tiene el bot experto funcionando, se va a evolucionar para mejorarlo.
  6. Partiendo de los estados definidos en la FSM y las reglas que éstos modelan, se eligen los parámetros de los que dependen, se ‘extraen’ como cromosomas/individuos del genético, se hacen una población de individuos aleatorios (o pseudoaleatorios) y se evolucionan. Cada vez que se quiere evaluar un individuo, se usan esos parámetros como valores dentro de las reglas del bot experto y se pone a jugar contra otro experto (con los valores por defecto). Tras un rato, se valora lo que ha hecho con la función de fitness, que considera las muertes, así como el daño hecho y recibido.
  7. Cr 143  considera decenas de valores relativos al uso de armas (ponderaciones de cual es mejor en cada posible situación) Cr 26  sustituye todo eso por un valor que pondera la elección de cada arma (se consideran 9 armas diferentes) Valores Health  decidir cuándo está bien y cuándo mal Distance  distancia al enemigo (cerca, media, lejos) Risk  riesgo que está dispuesto a correr (salud a arriesgar en un estado agresivo antes de cambiar a uno defensivo, por ejemplo) Time  tiempo que puede pasar buscando al enemigo en un estado Items  ponderación del uso de cada item Ambos algoritmos son iguales, sólo cambia el tipo de cromosoma. Mira los detalles de cada operador en el artículo, si quieres.
  8. Estos son los resultados de una de las ejecuciones. Las 3 evaluaciones se hacen para intentar ‘paliar’ el ruido de este tipo de problemas en los que una partida puede salir muy bien y otra muy mal para el mismo bot. ;) Cromosomas muy grandes y/o demasiada diversidad => malos resultados: demasiado ruido y muchas oscilaciones en la función de fitness
  9. Estos son los resultados de dos ejecuciones. Más individuos, menos tiempo de evaluación. Todavía demasiada diversidad => malos resultados: demasiado ruido y muchas oscilaciones en la función de fitness
  10. Stationary scheme para aumentar la presión selectiva => la explotación
  11. Estos son los resultados de dos ejecuciones. Resultados aceptables en cuanto a la tendencia del fitness. Pero aún con mucho ruido. Habría que hacer varias evaluaciones por individuo.
  12. Estos resultados se obtuvieron después de publicar el artículo. En él hay menos experimentos (2 partidas sólo). Se hicieron experimentos contra humanos, aunque un poco ‘informales’. :D