Creative Evolutionary Algorithms: Evolutionary Art & Music

1. Creative Evolutionary
Algorithms:
Evolutionary Art & Music
Francisco Fernández de Vega
University of Extremadura, Spain
fcofdez@unex.es

2. “...Although the ultimate goal may be the creation of
artif icial intelligence that is capable of producing high
quality works of art without human intervention (...)
Personally I’d rather man stay ahead and in control of the
very aspects of life that make it worth living, and rely on
machines to help us with those things that they are
currently best at, relieving us of the mundane and
tedious, rather than taking over our most enjoyable
pastimes and leaving creativity to man and God!”
M. Mumford, Fellow of the Royal British Photographic
Society. Excerpt from XY foreword, Parlamento de
Extremadura, 2014. ISBN 978-84-96757-50-9.

3. Summary
● Introduction: Creativity & EAs.
● Art & Computers.
● Evolutionary Art.
● Unplugged Evolutionary Algorithms.
● Evolving music.

4. Summary
● Introduction: Creativity & Eas.
● Art & Computers.
● Evolutionary Art.
● Unplugged Evolutionary Algorithms.
● Evolving music.

5. ● Main goal for an Evolutionary Algorithm:
– Finding the best (or good enough)
solution.
Creativity and EAs

6. ● Quality assessment in an EA:
– Every candidate solution must be
evaluated.
Creativity and EAs

7. ● Quality assessment in an EA:
– Every candidate solution must be
evaluated..
– A final evaluation is required to
accept/reject the output of the run.
Creativity and EAs

8. Creativity and EAs
● EAs can be creative when providing
solutions.

9. Creativity and EAs
● What is creativity?
Wikipedia: Creativity is a phenomenon
whereby something new and valuable is
created.
It has to do with novelty and originality...
but that's not enough: random processes
typically produce uninteresting results.

10. Creativity and EAs
● Novelty search? (K. Stanely)
– Does not reward progress.
– Rewards being different.
Novel? Addition of an anti-freeze fish gene to strawberries give it this
blue hue

11. Creativity and EAs
● Being different: Diversity is useful.
● Evolution needs diversity (check
diversity in EAs).
Ursem, R. K. (2002, September). Diversity-guided evolutionary algorithms. In International Conference on Parallel Problem
Solving from Nature (pp. 462-471). Springer, Berlin, Heidelberg.
Alfaro-Cid, E., Merelo, J. J., Fernandez de Vega, F., Esparcia-Alcázar, A. I., & Sharman, K. (2010). Bloat control operators and
diversity in genetic programming: A comparative study. Evolutionary Computation, 18(2), 305-332.

12. Creativity and EAs
● Computational Creativity?
– A subfield of AI with some goals:
● To provide computational perspective
of human creativity, in order to help us
to understand it.
● To enable machines to be creative.
● To produce tools which enhance
human creativity.
(On impact and evaluation in Computational Creativity: A discussion of the Turing Test and an
alternative proposal, A. Pease & S. Colton)

13. Summary
● Introduction: Creativity and EAs.
● Arts & Computers.
● Evolutionary Art.
● Unplugged Evolutionary Algorithms.
● Evolving music.

14. Summary
● Introduction: Creativity and EAs.
● Arts & Computers.
● Evolutionary Art.
● Unplugged Evolutionary Algorithms.
● Evolving music.

15. Generative Art and
Computers
● Cybernetic Serendipity Exhibition,
London, 1968.
http://cyberneticserendipity.net
/

16. How to put creativity
within computers
● AARON: Probably the first robot artist.
● Expert system writen in LISP.

17. The Painting Fool
● About me: I'm The Painting
Fool: a computer program, and
an aspiring painter.
http://www.thepaintingfool.com

18. Art and Computers
● Two main questions:
– How can we encode the main
ingredients for creativity within an
algorithm?
– How can we measure aesthetic quality?
The persistence of Memory, S. Dalí.

19. Art and Computers
Where do artists find their inspiration?
Gaudi

20. Art and Computers
● Two main problems to address:
– Motivational force (& inspiration).
– Measuring/being aware of aesthetic
quality.

21. Art and Computers
● My motivational force...

22. Art and Computers
● Motivational force & inspiration:
– Cultural tradition.
– Tangible forms for spiritual concepts.
– Recording history.
– Visual language to convey teachings.
– Tell a story.
– Reflecting beauty of nature.
– Express the ideal or expose the real.
– Provoke others to think.
– Experimenting with formal elements and
quality of medium.

23. Art and Computers
● Measuring aesthetic quality:
– Many proposals, such as:
● Birkhoff and the Aesthetic Measure
● The Golden Ratio
● Zipf’s Law
● Fractal Dimension
● Gestalt Principles
● The Rule of Thirds
– … but, do they guarantee the quality of
the result?

24. ● Useful in both steps of evaluation?:
– Fitness evaluation step within the EA
loop.
– Final evaluation is required to
accept/reject the output of the run.
Art and Computers

25. Art & Computers
A Turing Test
for Creativity?
Colton, S. (2008). Creativity Versus the Perception of Creativity in
Computational Systems. In AAAI Spring Symposium: Creative Intelligent
Systems (pp. 14-20).

26. Creativity and EAs
● Is it enough Turing
Test to asses the
artistic value of a
work?
● What happens with
human art that
seems computer
generated art?
– Check for
instance
Schoenberg
serialism and
antonality.

27. Creativity and EAs
● Some authors consider that creativity
must only be assessed by means of the
output (second stage of evaluation):
– “For the purposes of setting up an initial
framework, we shall adopt the
assumption that the internal workings
of a program are not part of the
relevant data”.
Ritchie, G. 2007. Some empirical criteria for attributing creativity to a computer
program. Minds and Machines 17:67–99.

28. Art and Computers
● Measuring aesthetic quality...is not easy
Cezanne
Monet
Van Gogh

29. Art and Computers
● Introduction.
● Art & Computers.
● Evolutionary Art.
● Unpplugged Evolutionary Algorithms.
● Evolving music.

30. Art and Computers
● Introduction.
● Art & Computers.
● Evolutionary Art.
● Unpplugged Evolutionary Algorithms.
● Evolving music.

31. Evolutionary Art
● A hot topic in computer science...

32. Interactive
Evolutionary
Algorithms
● Standard EA:
– Fitness Evaluation.
– Selection.
– Crossover+Mutation.
● Interactive EA:
– Fitness Evaluation (human made).
– Selection.
– Crossover+Mutation.

33. Evolutionary Art
● W. Latham, Formsynth, 1984.

34. Interactive EAs
● IEA: An evolutionary Algorithm with a
human being in charge of fitness
evaluation.

35. Evolutionary Art
● Collaborative works...:
– Picbreeder (Secretan et al).
– EvoEco (Kowaliw, Dorin, McCormack).
EvoEco.

36. Evolutionary Art
● Evolutionar Art applications =
Evolutionary Design:
– Endless forms (Clune & Lipson).

37. Evolutionary Design:
Genetic Terrain Programming
● Genetic Terrain Programming:
– Interactive GP.

38. Evolutionary Design
● Some of the terminals:

39. Previous Work

40. Previous work
● Genetic Terrain Programming.

42. Generative Art and
Computers
● GTP technology applied in videogames.

43. Evolutionary Art
● Open problems in Evolutionary Art
(McCormak):
– Useful genotypes and phenotypes.
– Fitness Functions embodying human
aesthetic evaluation.
– Evolutionary Art complying standards
for evolutionary art.
– Ecosystems that recognize their own
creativity.
– Developing Evolutionary Art theories.
McCormack, J. (2005). Open problems in evolutionary music and art. In Applications
of Evolutionary Computing (pp. 428-436). Springer Berlin Heidelberg.

44. Our Goal
● More closely working with artist.
● Involving them in evolutionary art
experiments.
● Learning from them to better understand
creativity from the EA Point of view.
● Providing technology that helps.

45. Our Goal
● How to reach our goal:
– Technology: Developing an easy to use
software tool (EVOSPACE).
– Developing a new methodology:
Unplugged EAs.

46. Summary
● Introduction.
● Art & Computers.
● Evolutionary Art.
● Unplugged Evolutionary Algorithms.
● Evolving music.

47. Summary
● Introduction.
● Art & Computers.
● Evolutionary Art.
● Unplugged Evolutionary Algorithms.
● Evolving music.

48. Unplugging EAs
● Standard EA.
– Fitness
Evaluation.
– Selection.
– Crossover +
Mutation.
● Interactive EA.
– Fitness
Evaluation.
– Selection.
– Crossover +
Mutation.

49. Unplugging EAs
● Quality assessment in an EA:
– Every candidate solution must be
evaluated (Interactive EAs).
– A final evaluation is required to
accept/reject the output of the
run (?).

50. Unplugging EAs
● Interactive EA.
– Fitness
Evaluation.
– Selection.
– Crossover +
Mutation.
● Unplugged EA.
– Fitness
Evaluation.
– Selection.
– Crossover +
Mutation.

51. Unplugging EAs
● Our Team:
– 5 artists.
– 1 coordinator.
● Way of working:
– #1 Artists decide the initial population.
– #2 Working isolated, they evaluate,
select parents, and apply crossover +
mutation creating a new individual.
– #3 Individuals sent to the coordinator by
email.
– #4 The coordinator anonymously share
them in a dropbox folder.

52. Unplugging EAs
● The first experiment:
– 10 weeks: October 2012 - January
2013.
– 5 artists x 10 weeks (1 work per week)=
50 paintings.

53. Unplugging EAs
● First step: Seeding the initial population:
– Choosing a well-known painting...
Question #1: Which painting would you have
selected for the initial population?

54. Unplugging EAs
● What did artists selected for the initial
population?
Botticelli, Birth of Venus; Schiele, Sitting Woman with Legs Drawn Up; Millais, Ophelia;
Schiele, Seated Man; Leonardo Da Vinci, The last supper.

55. Unplugging EAs
● Second Step (every week).
– Artists select two parents.
– Decide mutation and crossover to
apply.
– Generate a new painting.

56. Unplugging EAs
● Question #2: Can you figure out the
parents for any of the paintings?

57. Unplugging EAs

58. Unplugging EAs
● #3 Individuals sent to the coordinator,
● #4 anonymously shared in a dropbox
folder... and going back to step #1

59. Unplugged EAs
● Three first generations.

60. Unplugging EAs
Comparing initial and latest generations.

61. Unplugging EAs
● What kind of mutation and crossover
operations have been applied?

62. Unplugging EAs
G7-5 by L. Navarro

63. Unplugging EAs
- Self-reflection process: it tries to not
only reason about the work to be
produced, but also on how the evolutionary
process is working -converging- and the
best way to influence and change the
behavior of the algorithm itself so that
more diversity is added.

64. Unplugging Eas.
G9-3: By C. Cruz

65. Unplugging EAs
● Artists feel a need to to use and
manipulate natural media.
● The diversity observed is much grater
than that generated with available IEAs.
● This observation, expressed by the
audience, is aligned with some of the
problems described by McCormack
when referring to evolutionary art: “there
is still a large distance between
evolutionary art and the art that human
artists can produce.”
McCormack, J.: Open problems in evolutionary music and art. In: EvoWorkshops, pp.
428{436 (2005)

66. Unplugging EAs
● Quality assessment in an EA:
– Every candidate solution must be
evaluated (IEA).
– A final evaluation is required to
accept/reject the output of the run.

67. Unplugging EAs
● Final Evaluation: Involving other actors
in the EA world:
– Audience.
– Galleries.
– Museums.
– Art Critics.

68. Unplugging EAs
● Analyzing audience response is part of
the creativity assessment:
– Survey based approach.
– Non-intrusive methods (kinnect based
approach).

69. Unplugging EAs
● Analyzing audience response – main
conclusions:
– Survey produce fatigue – better use
non-intrusive methods:
● Amount of data collected grows.
● Information collected is coherent with
surveys.
● Data can be compared with artists
creativity displayed.
Analyzing Evolutionary Art Audience Interaction by Means of a
Kinect Based Non-intrusive Method. F. Fernánez de Vega, M.
García, JJ. Merelo, G. Aguilar, C. Cruz, P. Hernández. To appear in Volume
785 of the Studies in Computational Intelligence series.

70. Unplugging EAs
● Final Evaluation: Involving other actors
in the EA world:
– Audience.
– Galleries.
– Museums.
– Art Critics.

71. Galleries & Museums
● Art exhibits in three different galleries
worldwide.
Back Gallery Project – Vancouver Gallery Louchard – Paris MC Gallery – Manhattan - NY

72. International Competitions
● Best way to evaluate the results of an
Evolutionary Art project.
● Submitted our work to:
– ACM GECCO Evolutionary Art, Design
and creativity competition 2013
winner.
– 2017 re:artiste Show Your World
International Art Competition and
juried exhibition.

73. International
Competitions
● XY: Winner ACM
Gecco 2013
Evolutionary Art
Design and
Creativity
competition.
● Finalist 2017
re:artiste show your
world competition.

74. Summary
● Introduction.
● Art & Computers.
● Evolutionary Art.
● Unplugged Evolutionary Algorithms.
● Evolving music.

75. Summary
● Introduction.
● Art & Computers.
● Evolutionary Art.
● Unplugged Evolutionary Algorithms.
● Evolving music.

76. Music is a nice
gameboard for EAs
● Many problems available.
● Just a sample:
– 4-part harmonization problem.

77. 4-part problem
Harmonization
Chromosome includes:
chord progression + SATB
voices.
Two stages Genetic
Algorithm evolves first the
progression and then the
voices.

78. 4-part problem
Harmonization

79. Another music problem:
automatic transcription
● Automatic transcription of piano music
by means of Genetic Algorithms.

80. Automatic
transcription
● Remember, evaluate in the appropriate
context, and add a final evaluation step:
– ISMIR 2011, MIREX Competion: 3rd
position in piano transcription.
Gustavo Reis, Francisco Fernández de Vega, Aníbal Ferreira:
Automatic Transcription of Polyphonic Piano Music Using Genetic
Algorithms, Adaptive Spectral Envelope Modeling, and Dynamic
Noise Level Estimation.
IEEE Trans. Audio, Speech & Language Processing 20(8): 2313-
2328 (2012)

81. Automatic
transcription
● Remember, evaluate in the appropriate
context, and add final evaluation step :
– ISMIR 2011, MIREX Competion: 3rd
position in piano transcription.
Gustavo Reis, Francisco Fernández de Vega, Aníbal Ferreira:
Automatic Transcription of Polyphonic Piano Music Using Genetic
Algorithms, Adaptive Spectral Envelope Modeling, and Dynamic
Noise Level Estimation.
IEEE Trans. Audio, Speech & Language Processing 20(8): 2313-
2328 (2012)

82. Summary
● Try to add diversity, novelty and
creativity to your algorithm.
● Evaluate it in the appropriate context.
● If dealing with art and music add an
extra evaluation step for the final result.

84. Residence for European Researchers.
Available for researchers collaborating with our group (free of charge).

85. Thank you.
Find me at GP bibliography:
http://www.cs.bham.ac.uk/~wbl/biblio/gp-
html/FranciscoFernandezdeVega.html
fcofdez@unex.es