2010 Simulated Car Racing Championship @ GECCO-2010

The 2010 Simulated Car Racing
Championship @ GECCO-2010
Daniele Loiacono, Luigi Cardamone,
Martin V. Butz, and Pier Luca Lanzi

2

2010 Simulated Car Racing Championship
9 races during 3 conferences
ACM GECCO-2010, Portland, OR (USA), July 7-
IEEE WCCI-2010, Barcelona (Spain), July 18-23
IEEE CIG-2010, Copenhagen (Denmark), August 18–21

Develop a driver for TORCS
(hand-coded, learned, evolved, …)

Drivers will be awarded based on their score
in each conference competition

At the end, the team with highest overall score
wins the championship

2010 Simulated Car Racing Championship @ GECCO-2010

3

What is the structure of a race?

Three stages: warm up, qualifiers, actual race

During warm-up, each driver can explore the
track and learn something useful

During qualifiers, each driver races alone against
the clock (the best 8 drivers move to the race)

During the race all the drivers race together


Motivations

 Proposing a relevant game-based competition
more representative of commercial games AI
more similar to a real-world problem
 Proposing a funny and exciting competition
you can see and play with the entries of this competition
human players can interact with AI
a lot of programmed AI available for comparison
 Proposing a challenging competition
not designed with Machine Learning in mind
computationally expensive
real-time
dealing with a lot of practical issues


What‘s new?

If everything seems under control, you're not going fast enough
— Mario Andretti
 Warm-up stage
Before qualifying stage, competitors have 100000 game
ticks to race on the track
Allows track learning and optimization of parameters
 Noisy sensors
Track sensors and opponent sensors are affected by a
Gaussian noise (standard deviation equal to 10% of the
readings)
 Extended sensor model
Focus sensors
Z position and speed
Direction of track sensors fully customizable
Added clutch control and focus command


The Open Racing Car Simulator

 TORCS is a state of the art open source simulator written in C++

 Main features
Sophisticated dynamics
Provided with several
cars, tracks, and
controllers
Active community of
users and developers
Easy to develop your
own controller

 OS Support
Linux: binaries and building from sources
Windows: binaries and ―limited‖ bulding from sources support
OSX: legacy binaries and no building from sources support 


The Open Racing Car Simulator
& the Competition Software

TORCS TORCS
PATCH

BOT BOT BOT SBOT SBOT SBOT

 The competition server
UDP UDP UDP
 Separates the bots from TORCS
 Build a well-defined sensor model
 Works in real-time
BOT BOT BOT


Sensors and actuators

 Rangefinders for edges on the track and opponents
 Speed, RPM, fuel, damage, angle with track, distance race,
position on track, etc.

 Four effectors: steering wheel [-1,+1], gas pedal [0, +1],
Brake pedal [0,+1], Gearbox {-1,0,1,2,3,4,5,6}

The competitors

 Four entries submitted to this first leg
AUTOPIA, Madrid and Granada
J. Muñoz, Carlos III University of Madrid
S.Pohl, J. Quadflieg and T. Delbrügger, TU Dortmund
Joseph Alton, University of Birmingham

 Two more entries from the 2009 championship
COBOSTAR
(T. Lönneker & M.V. Butz, University of Würzburg)
POLIMI (Cardamone, Politecnico di Milano)


AUTOPIA

Industrial Computer Science Department.
Centro de Automática y Robótica
Consejo Superior de Investigaciones Científicas
Madrid, Spain
Contact:E. Onieva (enrique.onieva@car.upm-csic.es)

Architecture Schema

 Three basic modules for gear, steering and speed control

 Steady stage genetic algorithm to compute the best weights
to combine parameters for steering and target speed control

 Opponents module
Acts on steering and brake signal to overtake opponents
and avoid collisions

 Learning Module in Warm-up Stage
Factors over the target speed in certain track segments


Opponents Management (Steering)

 If O(-20,-10,10,20) detects an opponent with lateral
displacement less than 20 meters, move the steering
trackwidth/100 through the direction with more free
distance.
 If O(-40,-30,-20,-10) detects an opponent with lateral
displacement less than 20 meters, move the steering
trackwidth/50 through the right.
 If O(-70,-60,-50)<15 meters, move the steering
trackwidth/50 through the right.
 If O(-140,-130,-120,-110,-100,-90,-80)<15meters , move
the steering trackwidth/50 through the right.
 Mirroring actions for positive sensors

 O(X)  Opponent Measure of Sensor Oriented at X

 Steering Modification limited to 0.35

Learning Module (Warm-Up)

 Running normally in warm-up stage.

 Maintain a vector with as many real values as the track
length in meters.

 Vector initialized to 1.0

 If the vehicle goes out of the track or suffers damage then
multiply vector positions from 100 meters before the current
position by 0.9.


Mr Racer

Susanna Pohl, Jan Quadflieg and Tim Delbrügger
TU Dortmund


Mr. Racer 2009-2010

 Mr Racer 2009
Good classifier which identifies six situations
Acceleration/brake learned offline using an EA
Model of the track learned online
Simple heuristic to use the model: override the learned
behaviour on straights and in full speed corners to drive
flat out
 Mr Racer 2010
Save the model after warmup, use it during
qualifying and the race
Use the model to derive a plan consisting of
target speeds and a racing line
Optimize the parameter set of the planning module,
left to be done for the next round


Mr. Racer - Classification

Angle based measure mapped to six situations (straight, fast
corner, slow corner, etc)


Mr. Racer 2010 – The catch

 Noise completely breaks our classifier

 Without a descent classifier we can‗t learn the track

 Without a trackmodel we can‗t drive 

 Workaround for GECCO-2010:
Classify the whole track as being straight

 New classifier for the next leg at WCCI-2010


Jorge Muñoz Fuentes

Department of Computer Science
Carlos III University of Madrid


Jorge Muñoz

 Build a model of the track during the warm-up stage.

 Two neural networks to predict the trajectory using the track
model. Two neural networks to predict the target speed
given the model of the track and the current car position

 The four neural networks are trained with backpropagation
using data retrieved from a human player.

 The controller tries to imitate a human player.

 A scripted policy is used to follow the trajectory (steering
value), set the speed (accelerate and brake values), set the
clutch and the gear


Jorge Muñoz

 Other optimizations performed during
the warm-up and used in the race:
The car remember where it goes out of the car or drives
far form the trajectory and in the next laps goes slower in
those points
The car remember where it follows the trajectory perfectly
and tries to go faster in the next laps.

 Overtaking is made by means of modifying the predicted
trajectory, the modification is bigger in straighs than in turns

 To avoid overtakin the car also modifies the trajectoy, trying
to stay in front of the opponent car.


Joseph Alton

Joseph Alton


Joseph Alton

 Bot made specially for the oval leg, and is only designed to
be competent on this leg.
 Plan to change and improve the bot as I learn and to meet
the demands of the other tracks

 Warm-up Learning
Car goes slowly around the track recording in each
segment (e.g. 0 metre, 1 metre, 2 metre) whether it is
turning or not
This information can then be loaded up in qualifying or
finals to help find a more appropriate target speed.
Straight = fast, Turn = slower
During the race, the driver looks ahead to the next or the
next two segments depending on the speed


COBOSTAR

Thies Lönneker and Martin V. Butz
University of Würzburg

http://www.coboslab.psychologie.uni-wuerzburg.de

CIG-2008 Champ

Luigi Cardamone
Politecnico di Milano


Scoring process: Warm-up Qualifying

 Scoring process involves three motor high-speedway tracks:
Kiwi
Mango
Triangle

 Kiwi is a (renamed and slightly modified) track of TORCS

 Mango and Triangle has been developed by the organizers

 Each controller raced for 100000 game ticks in the warm-up
stage and then its performance is computed in the qualifying
stage as the distance covered within 10000 game ticks


Qualifying: Kiwi

MR.Racer

Joseph

Jorge Muñoz

COBOSTAR

Cardamone

Autopia

0 2000 4000 6000 8000 10000 12000 14000 16000 18000


Qualifying: Mango

MR.Racer

Joseph

Jorge Muñoz

COBOSTAR

Cardamone

Autopia

0 2000 4000 6000 8000 10000 12000 14000 16000


Qualifying: Triangle

MR.Racer

Joseph

Jorge Muñoz

COBOSTAR

Cardamone

Autopia

0 2000 4000 6000 8000 10000 12000 14000 16000


Qualifyng summary

Competitor Kiwi Mango Triangle Total
COBOSTAR 10 10 8 28
Autopia 8 8 10 26
Mr. Racer 6 6 6 18
Jorge Muñoz 4 5 5 14
Cardamone 5 4 4 13
Joseph 3 3 3 9


What about qualifying?
COBOSTAR is the fastest driver
(despite not optimized neither for high-speedways nor for
noisy sensors)

Experiments without noise shows that

Noise is not critical in high-speedways
for driving alone

Some controllers (MR. Racer and Muñoz) perform even
better with noise

35

The Three GPs

For each track we run 5 races
with random starting grids

Each race is scored using the F1 point system
(10 to first, 8 to second, 6 to third, …)

Two points to the controller with lesser damage

Two points for the fastest lap of the race


Final Results

Competitor Kiwi Mango Triangle Total
Autopia 12 12 10 34
Jorge Muñoz 5.5 8 9 22.5
Mr. Racer 9 4 3 16
Cardamone 6 4.5 5.5 16
Joseph 4 5 6.5 15.5
COBOSTAR 4 4.5 5.5 14


What about the race?
Results do not confirm the qualifying performance

COBOSTAR is the worst performing controller!

The results suggest that noise is much more relevant for the
opponent perception that for driving alone

Races without noise show that AUTOPIA and COBOSTAR
being the top controllers (as expected from qualifyng)

Cardamone’s controller seems slighlty more reliable (as it is
completely based on a Neural Network)

2010 Simulated Car Racing Championship @ GECCO-2010

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie 2010 Simulated Car Racing Championship @ GECCO-2010

Ähnlich wie 2010 Simulated Car Racing Championship @ GECCO-2010 (20)

Mehr von Daniele Loiacono

Mehr von Daniele Loiacono (20)

2010 Simulated Car Racing Championship @ GECCO-2010