1. E N G A G I N G T H E W O R L D
Automatic Calibration of Modified FM Synthesis
to Harmonic Sounds using Genetic Algorithms
Matthieu Macret - Philippe Pasquier - Tamara Smyth

2. Replicating sounds of musical instruments
• Problem frequently addressed in Computer
Music
• Various synthesis techniques developed over
the past century (Additive, FM, granular...)
2

3. Issues / Limitations
• Parametric Synthesis techniques
• Selecting suitable controls and synthesis
parameters can be unintuitive, manual and
time consuming
3

4. Automatic calibration
• Genetic algorithms
• Neural networks
• Cellular automata
• Particle Swarm Optimization
• etc.
4

5. Scope of our research
• Genetic algorithms (GA)
• Modified FM synthesis
• Musical harmonic sounds
5

6. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
6

7. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
7

8. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
7

9. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
7

10. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
7

11. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
7

12. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
7

13. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
7

14. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
8

15. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
8

16. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
8

17. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
8

18. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
8

19. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
8

20. FM synthesis
Classic
FM
synthesis:
Chowning
(AES
Journal
1973)
Modified
FM
synthesis:
Lazzarini,
V.
and
Timoney
(AES
Journal
2010)
8

21. Synthesis Model
9

22. Synthesis Model
9

23. Synthesis Model
9

24. Synthesis Model
9

25. Synthesis Model
9

26. Synthesis Model
Insure
the
harmonicity
of
the
sound
Scale
spectum
on
the
target
sound
spectrum
9

27. Parameters to optimize
k1 I1 k2 I2 ... kN IN
Param. Bounds Type Use
k [0,10] Integer
I [0,20] Double
10

28. Input parameters
• number of carriers
• harmonic target sound
11

29. Genetic Algorithm
12
Image
from
www.genePc-­‐programming.com

30. Initialization
13
PopulaPon
size
=
100

31. Initialization
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN INk1 I1 k2 I2 ... kN IN
13
PopulaPon
size
=
100

32. Evaluation
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN INk1 I1 k2 I2 ... kN IN
14

33. Evaluation
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN INk1 I1 k2 I2 ... kN IN
14

34. Evaluation
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN INk1 I1 k2 I2 ... kN IN
14

35. Evaluation
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN INk1 I1 k2 I2 ... kN IN
6
3 1
2
4
2
7
6
4
14

36. E N G A G I N G T H E W O R L D
Fitness function

37. Analysis parameters
Analysis window size 10
ms
Overlapping 5
ms
Zero padding factor 4
Spectral resolution 20
Hz
16

38. Target
Sound
Fitness function
17

39. Target
Sound
Fitness function
17

40. Target
spectrum
A
Hz
18

41. Target
spectrum
A
Hz
18

42. Target
spectrum
A
Hz
Target
vector
18

43. k1 I1 k2 I2 k3 I3
19

44. k1 I1 k2 I2 k3 I3
19

45. k1 I1 k2 I2 k3 I3
Hz
A
19

46. k1 I1 k2 I2 k3 I3
Hz
A
19

47. k1 I1 k2 I2 k3 I3
Hz
A
19

48. k1 I1 k2 I2 k3 I3
Hz
A
19

49. k1 I1 k2 I2 k3 I3
Hz
A
Hz
A
19

50. k1 I1 k2 I2 k3 I3
Hz
A
Hz
A
19

51. k1 I1 k2 I2 k3 I3
Hz
A
Hz
A
19

52. 20

53. System
of
equaPons
to
solve
Unknown
Least
Squares
ApproximaPons
Fitness
Func:on
+=
approximaPon
error
21

54. Target
Sound
22

55. Target
Sound
22

56. Target
Sound
22

57. 1st
Carrier
envelop
3st
Carrier
envelop
2st
Carrier
envelop
Synthesized
sound:
23
Fitness
Func:on
=
sum(approximaPon
errors)

58. Termination
criteria
• Reach 300
generations
• Weighted change in
the fitness < 1.E-10
over 50 generations
24

59. GeneraPon
0:
Size
=
100
2
elites
children
0.8
x
98
=
78
children
by
crossover
98-­‐78
=
20
children
by
mutaPon
GeneraPon
1:
Size
=
100
25
Generating the next population

60. k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN INk1 I1 k2 I2 ... kN IN
6
3 1
2
4
2
7
6
4
26
Binary selection tournament

61. k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN INk1 I1 k2 I2 ... kN IN
6
3 1
2
4
2
7
6
4
26
Binary selection tournament

62. k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN INk1 I1 k2 I2 ... kN IN
6
3 1
2
4
2
7
6
4
26
Binary selection tournament

63. Crossover
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
k1 I1 k2 I2 ... kN IN
Parents
Children
Crossover
point
27

64. Gaussian Mutation Operator
k1 I1 k2 I2 ... kN IN
Add
a
random
number
from
a
gaussian
distribuPon
28

65. Termination
criteria
• Reach 300
generations
• Weighted change in
the fitness < 1.E-10
over 50 generations
29

66. Evaluations
1.Musical Instrument Sounds (University of
Iowa Electronic Music Studios)
2.Comparison between ClassicFM and ModFM
30

67. Insure
the
harmonicity
of
the
sound
Scale
spectum
on
the
target
spectrum
31

68. Musical Instrument Sounds
• AltoSax
• Bass
• Bassoon
• Clarinet
• Cello
• Flute
• Violin
32
• Horn
• Oboe
• Piano
• Tenor Trombone
• Trumpet
• Tuba
• Viola

69. Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 368 180 121
nb
Gen
to
converge 69 138 58
Results Flute - ModFM
33

70. Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 368 180 121
nb
Gen
to
converge 69 138 58
Results Flute - ModFM
33

71. Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 368 180 121
nb
Gen
to
converge 69 138 58
Results Flute - ModFM
33

72. Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 368 180 121
nb
Gen
to
converge 69 138 58
Results Flute - ModFM
33

73. Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 368 180 121
nb
Gen
to
converge 69 138 58
Results Flute - ModFM
33

74. Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 368 180 121
nb
Gen
to
converge 69 138 58
Results Flute - ModFM
33

75. Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 368 180 121
nb
Gen
to
converge 69 138 58
Results Flute - ModFM
33

76. Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 368 180 121
nb
Gen
to
converge 69 138 58
Results Flute - ModFM
33

77. Flute - Harmonic 1 34

78. Flute - Harmonic 4 35

79. Flute - Harmonic 6 36

80. Flute - Harmonic 9 37

81. Results Clarinet - ModFM
38
Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 20.8 8.50 3.9
nb
Gen
to
converge 71 64 75

82. Results Clarinet - ModFM
38
Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 20.8 8.50 3.9
nb
Gen
to
converge 71 64 75

83. Results Clarinet - ModFM
38
Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 20.8 8.50 3.9
nb
Gen
to
converge 71 64 75

84. Results Clarinet - ModFM
38
Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 20.8 8.50 3.9
nb
Gen
to
converge 71 64 75

85. Results Clarinet - ModFM
38
Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 20.8 8.50 3.9
nb
Gen
to
converge 71 64 75

86. Results Clarinet - ModFM
38
Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 20.8 8.50 3.9
nb
Gen
to
converge 71 64 75

87. Results Clarinet - ModFM
38
Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 20.8 8.50 3.9
nb
Gen
to
converge 71 64 75

88. Results Clarinet - ModFM
38
Target
Number
of
carriersNumber
of
carriersNumber
of
carriers
Target
2 4 6
Best
fitness
score 20.8 8.50 3.9
nb
Gen
to
converge 71 64 75

89. 39
Clarinet Spectra
20.8
8.50
3.9
Fitness

90. Stats ClassicFM - ModFM
40
ParametersParameters
ClassicFMClassicFM ModFMModFM
mean SD mean SD
Fitness
Gen.
to
conv.
51.94 77.96 52.60 87.52
99.24 69.24 87.60 42.57

91. Trumpet - 6 carriers 41

92. Trumpet - 6 carriers 41

93. Trumpet - 6 carriers 41

94. Trumpet - 6 carriers 41

95. Trumpet - 6 carriers 41

96. Trumpet - 6 carriers 41

97. Trumpet - 6 carriers 41

98. Stats Evolution 42

99. Contributions
• Apply GA to a new FM synthesis technique
• Automatize completely the process
• Refine the parameters of the GA
• Evaluation on a large set of sounds
• Comparison between ClassicFM and ModFM
43

100. Future work - GA
• Remove constraints:
–Non harmonic sounds
–Remove constraint on the synthesis method
–Other fitness function
44

101. Future work - GP
• Go further!
• Evolve not only the
parameters but also
the synthesis model
45

102. http://www.metacreation.net/mume2012/
46Chair
Philippe
Pasquier
Co-­‐Chairs
Arne
Eigenfeldt,
Olivier
Brown

103. E N G A G I N G T H E W O R L D
Thank you !
Matthieu Macret - Philippe Pasquier - Tamara Smyth
mmacret@sfu.ca - pasquier@sfu.ca - tamaras@sfu.ca