Introduction to Genetic Algorithms

1. INTRODUCTION TO
GENETIC
ALGORITHMS
BY
AHMED MEDHAT OTHMAN
@amedhat3

2. AGENDA
1. Introduction
2. Natural Inspired Computing
3. Classical Computation vs. bio-inspired computing
4. Evolution in the real world
5. Collaborative discussion
6. Problem solving
7. Genetic Algorithms (GA)
8. Anatomy of a GA
9. Advantages of GA
10. Limitations of GA
11. Related techniques
12. Summary
2

3. INTRODUCTION
3

4. INTRODUCTION
4

5. INTRODUCTION
5

6. INTRODUCTION
6

7. INTRODUCTION
7

8. INTRODUCTION
8

9. INTRODUCTION
• The only intelligent systems on this planet are biological.
• Biological intelligences are designed by natural
evolutionary processes.
• They often work together in groups, swarms, or flocks.
• They don't appear to use logic, mathematics, complex
planning, complicated modeling of their environment.
• They can achieve complex information processing and
computational tasks that current artificial intelligences
find very challenging indeed.
9

10. NATURAL INSPIRED
COMPUTING
10

11. NATURAL INSPIRED
COMPUTING
• In other words Biologically Inspired Computing.
• Biological organisms cope with the demands of their
environments.
• They uses solutions quite unlike the traditional human-
engineered approaches to problem solving.
• They exchange information about what they’ve discovered
in the places they have visited.
• Bio-inspired computing is a field devoted to tackling
complex problems using computational methods modeled
after design principles encountered in nature.
11

12. CLASSICAL COMPUTATION
VS.
BIO-INSPIRED COMPUTING
12

13. CLASSICAL COMPUTATION
VS.
BIO-INSPIRED COMPUTING
• Classical computing is good at:
• Number-crunching
• Thought-support (glorified pen-and-paper)
• Rule-based reasoning
• Constant repetition of well-defined actions.
• Classical computing is bad at:
• Pattern recognition
• Robustness to damage
• Dealing with vague and incomplete information;
• Adapting and improving based on experience
13

14. CLASSICAL COMPUTATION
VS.
BIO-INSPIRED COMPUTING
• Bio-inspired computing takes a more evolutionary
approach to learning.
• In traditional AI, intelligence is often programmed from
above. The Programmer create the program and imbues it
with its intelligence.
• Bio-inspired computing, on the other hand, takes a more
bottom-up, decentralized approach.
• Bio-inspired computing often involve the method of
specifying a set of simple rules, a set of simple organisms
which adhere to those rules.
14

15. EVOLUTION IN THE
REAL WORLD
• Each cell of a living thing contains
chromosomes - strings of DNA.
• Each chromosome contains a set of
genes - blocks of DNA.
• Each gene determines some aspect
of the organism (like eye colour).
• A collection of genes is sometimes
called a genotype.
• A collection of aspects (like eye
characteristics) is sometimes called
a phenotype.
15

16. EVOLUTION IN THE
REAL WORLD
• Reproduction involves recombination of genes from parents
and then small amounts of mutation (errors) in copying.
• The fitness of an organism is how much it can reproduce
before it dies.
• Evolution based on “survival of the fittest”.
16

17. COLLABORATIVE
DISCUSSION
• What we can learn from nature?
• Applications of nature in the engineering field.
17

18. PROBLEM SOLVING
• Suppose you have a problem.
• You don’t know how to solve it.
• What can you do?
• Can you use a computer to somehow find a solution?
• This would be nice! Can it be done?
18

19. PROBLEM SOLVING
Brute-Force Solution
A “blind generate and test” algorithm:
Repeat
Generate a random possible solution
Test the solution and see how good it is
Until solution is good enough
19

20. PROBLEM SOLVING
Can we use this Brute-Force idea?
• Sometimes - YES:
• if there are only a few possible solutions
• and you have enough time
• then such a method could be used
• For most problems - NO:
• many possible solutions
• with no time to try them all
• so this method cannot be used
20

21. PROBLEM SOLVING
Search Techniques
Calculus Base
Techniques
Guided random search
techniques
Enumerative
Techniques
BFSDFS Dynamic
Programming
Tabu Search Hill
Climbing
Simulated
Annealing
Evolutionary
Algorithms
Genetic
Programming
Genetic
Algorithms
Fibonacci Sort
21

22. GENETIC ALGORITHMS
22
Initialize Population
satisfy constraints ?
Evaluate Fitness
Select Survivors
Output Results
Randomly Vary Individuals
Yes
No

23. GENETIC ALGORITHMS
How do you encode a solution?
• Obviously this depends on the problem!
• GA’s often encode solutions as fixed length “bitstrings”
(e.g. 101110, 111111, 000101)
• Each bit represents some aspect of the proposed solution
to the problem
• For GA’s to work, we need to be able to “test” any string
and get a “score” indicating how “good” that solution is.
23

24. GENETIC ALGORITHMS
• The set of all possible solutions [0..1000] is called the
search space or state space.
• In this case it’s just one number but it could be many
numbers.
• Often GA’s code numbers in binary producing a bitstring
representing a solution.
• We choose 1,0 bits which is enough to represent 0..1000
24

25. GENETIC ALGORITHMS
• Example: encoding 4 parameters
• Param1 value = 1000 = 8
• Param2 value = 1011 = 11
• Etc.,
25
Binary Representation

26. GENETIC ALGORITHMS
Search Space
• For a simple function f(x) the search space is one
dimensional.
• But by encoding several values into the chromosome
many dimensions can be searched e.g. two dimensions
f(x,y).
• Search space can be visualised as a surface or fitness
landscape in which fitness dictates height.
• Each possible genotype is a point in the space.
• A GA tries to move the points to better places (higher
fitness) in the the space.
26

27. GENETIC ALGORITHMS
Fitness landscapes
27

28. GENETIC ALGORITHMS
Implicit fitness functions
• Most GA’s use explicit and static fitness function
• Some GA’s (such as in Artificial Life or Evolutionary Robotics) use
dynamic and implicit fitness functions - like “how many obstacles
did I avoid”
Individual’s fitness
Average fitness of population
28

29. GENETIC ALGORITHMS
Example - Drilling for Oil
• Imagine you have to drill for oil somewhere along a single
1 km desert road.
• Problem: choose the best place on the road that produces
the most oil per day.
• We could represent each solution as a position on the
road.
• Say, a whole number between [0..1000]
29

30. GENETIC ALGORITHMS
Where to drill for oil?
30
0 500 1000
Road
Solution2 = 900Solution1 = 300

31. GENETIC ALGORITHMS
In GA’s these encoded strings are sometimes called
“genotypes” or “chromosomes” and the individual bits are
sometimes called “genes”
31
512 256 128 64 32 16 8 4 2 1
900 1 1 1 0 0 0 0 1 0 0
300 0 1 0 0 1 0 1 1 0 0
102
3
1 1 1 1 1 1 1 1 1 1
Convert to binary string

32. GENETIC ALGORITHMS
32
0 1000
Road
Solution2 = 900
(1110000100)
Solution1 = 300
(0100101100)
OIL
Location
30
5

33. ANATOMY OF GA
Selecting Parents
• Many schemes are possible so long as better scoring
chromosomes more likely selected
• Score is often termed the fitness
• “Roulette Wheel” selection can be used:
• Add up the fitness's of all chromosomes
• Generate a random number R in that range
• Select the first chromosome in the population that - when all
previous fitness’s are added - gives you at least the value R
33

34. ANATOMY OF GA
34
Example population
No. Chromosome Fitness
1 1010011010 1
2 1111100001 2
3 1011001100 3
4 1010000000 1
5 0000010000 3
6 1001011111 5
7 0101010101 1
8 1011100111 2

35. ANATOMY OF GA
35
Roulette Wheel Selection
1 2 3 1 3 5 1 2
0 18
21 3 4 5 6 7 8
Rnd[0..18] = 7
Chromosome4
Parent1
Rnd[0..18] = 12
Chromosome6
Parent2

36. ANATOMY OF GA
36
Crossover - Recombination
0100101100
1110000100
Crossover
single point -
random
0100000100
1110101100
Parent1
Parent2
Offspring1
Offspring2
With some high probability (crossover rate) apply
crossover to the parents.
(typical values are 0.8 to 0.95)

37. Why does crossover work?
• A lot of theory about this and some controversy
• The idea is that crossover preserves “good bits” from
different parents, combining them to produce better
solutions
• A good encoding scheme would therefore try to preserve
“good bits” during crossover and mutation
37
ANATOMY OF GA

38. ANATOMY OF GA
38
Mutation
1011011111
1010000000
Offspring1
Offspring2
1011001111
1000000000
Offspring1
Offspring2
mutate
Original offspring Mutated offspring
With some small probability (the mutation rate) flip each bit in
the offspring (typical values between 0.1 and 0.001)

39. ANATOMY OF GA
39
Many Variants of GA
• Different kinds of selection (not roulette)
• Tournament
• Elitism, etc.
• Different recombination
• Multi-point crossover
• 3 way crossover etc.
• Different kinds of encoding other than bitstring
• Integer values
• Ordered set of symbols
• Different kinds of mutation

40. ADVANTAGES OF GA
• Concepts are easy to understand
• Genetic Algorithms are intrinsically parallel.
• Always an answer; answer gets better with time
• Inherently parallel; easily distributed
• Less time required for some special applications
• Chances of getting optimal solution are more
40

41. LIMITATIONS OF GA
• The population considered for the evolution should be
moderate or suitable one for the problem (normally 20-30 or 50-
100)
• Crossover rate should be 80%-95%
• Mutation rate should be low i.e. 0.5%-1% assumed as best
• The method of selection should be appropriate
• Writing of fitness function must be accurate
41

42. RELATED TECHNIQUES
• Genetic programming
• Evolutionary programming
• Swarm intelligence
• Ant colony optimization
• Particle swarm optimization
• Intelligent Water Drops
• Bees algorithm
• Neural Networks
42

43. SUMMARY
43
Representation Binary strings
Recombination N-point or uniform
Mutation Bitwise bit-flipping with fixed
probability
Parent selection Fitness-Proportionate
Survivor selection All children replace parents
Speciality Emphasis on crossover

44. 44

45. THANKS
• Contact information
• Email: amedhat3@gmail.com
• Website: http://amedhat.info
• Twitter: @amedhat3
• Linkedin: linkedin.com/in/amedhat
45