SlideShare ist ein Scribd-Unternehmen logo
1 von 24
Downloaden Sie, um offline zu lesen
"MACHINE LISTENING:
L'INTELLIGENCE
ARTIFICIELLE POUR LES
SONS ET LA MUSIQUE"
COLLOQUE IMT - L’INTELLIGENCE
ARTIFICIELLE AU COEUR DES
MUTATIONS INDUSTRIELLES.
APRIL 4TH, 2019
Gaël RICHARD
Professor, Head of the Image, Data, Signal department
11/04/2019
2
MACHINE LISTENING ?
A well established domain for speech …
Speech Recognition
« What a great workshop !»
Speech Emotion Recognition
« The person who’s speaking is happy but anxious»
Speaker Recognition
« Gaël Richard is speaking»
Speech Translation
« Quel excellent colloque ! »
« What a great workshop ! »
MACHINE LISTENING FOR AUDIO AND MUSIC
Goal: to extract « information » from the audio signal
11/04/2019
3
Music recognition (or Audio ID): identifying the music recording
Music Information retrieval
Music transcription, Music similarity, Music genre recognition,
Musical instrument recognition, Music recommendation, Autotagging,
Audio Scene recognition: sound recorded in streets, in subway station, in office,….)
Audio Source separation
Demixing music, Singing voice extraction, source localisation,
Audio Event recognition: Speech, Music, Car noises, birds, …
Audio Sound capture: Localisation, Dereverberation, Denoising, …
Music Emotion recognition: Sad vs happy vs dance music, ….
Bioacoustics: Wildlife monitoring, biodiversity, species id…
SOME APPLICATIONS …
11/04/2019
4
Music streaming,
music recommendation Music education
Music Identification, Audio Fringerprint Karaoke, speech to rap conversion
Sound recognition,
smarthomes, smart hearables
bmat
Audeering
Stratégie de marque
musicale, Supervision
musicale (pub.; films)
Bioacoustics
Vocal separation, music separation
SOURCE SEPARATION
11/04/2019
5
SOURCE SEPARATION FOR REMIXING
11/04/2019
6
SOURCE SEPARATION
(LEGLAIVE & AL.)
11/04/2019
7
Model for Audio sources
Model for reverberation
AN EXAMPLE OF MODEL FOR AUDIO SOURCES
Non Negative Matrix factorization
11/04/2019
8
SOURCE SEPARATION - S. LEGLAIVE
(EXAMPLE REMIXED BY RADIO FRANCE)
11/04/2019
9
S. Leglaive, R. Badeau, G. Richard, "Multichannel Audio Source Separation with Probabilistic Reverberation
Priors", IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 24, no. 12, December 2016
Simon Leglaive, Roland Badeau, Gaël Richard, Separating Time-Frequency Sources from Time-Domain
Convolutive Mixtures Using Non-negative Matrix Factorization. WASPAA , Oct. 2017 New Paltz, US.
AUDIO IDENTIFICATION OU AUDIOID
Audio ID = find high-level metadata from a music recording
Challenges:
Efficiency in adverse conditions (distorsion, noises,..)
Scale to “Big data” (bases > millions of titles)
Rapidity / Real time
Product example :
Audio
identification
Information of the
recording (title, artist,
etc.., …)
REAL-TIME AUDIO IDENTIFICATION
(FENET & AL.)
Audio recordings recognition
• Identical
• Approximate (live vs studio)
• Real time demonstrator
• For music recommendation, second screen applications, …
11/04/2019
11
Sébastien Fenet, Yves Grenier, Gaël Richard: An Extended Audio Fingerprint Method with Capabilities for
Similar Music Detection. ISMIR 2013: 569-574
SOUND EVENTS AND ACOUSTIC SCENE RECOGNITION
11/04/2019
12
SOME APPROACHES
11/04/2019
13
AN EXEMPLE OF A RECENT APPROACH
11/04/2019
14
V. Bisot, R. Serizel, S. Essid, G. Richard, "Feature Learning with Matrix Factorization Applied to Acoustic Scene
Classification", IEEE/ACM Transactions on Audio, Speech, and Language Processing, (2017), Special Issue on
Sound Scene and Event Analysis.
EXAMPLE FOR SCENE CLASSIFICATION
UNSUPERVISED NMF FOR ACOUSTIC SCENE RECOGNITION
UNSUPERVISED NMF FOR ACOUSTIC SCENE RECOGNITION
EXAMPLE WITH DNN: ACOUSTIC SCENE RECOGNITION
V. Bisot & al., "Feature Learning with Matrix Factorization Applied to Acoustic Scene Classification", IEEE/ACM
Transactions on Audio, Speech, and Language Processing, (2017),
V. Bisot & al., Leveraging deep neural networks with nonnegative representations for improved environmental sound
classification IEEE International Workshop on Machine Learning for Signal Processing MLSP, Sep 2017, Tokyo,
TYPICAL PERFORMANCES OF ACOUSTIC SCENE RECOGNITION (CHALLENGE
DCASE 2016)
A Mesaros & al. Detection and Classification of Acoustic Scenes and Events: Outcome of the DCASE 2016
challenge IEEE/ACM Transactions on Audio, Speech, and Language Processing 26 (2), 379-393
MUSIC INFORMATION RETRIEVAL
Major topics:
• Music transcription (Multiple F0 estimation, Beat/Downbeat detection, instrument classification,
…),
• Music recommendation
• Source separation,
• Multimodal music processing
11/04/2019
20
MIR: AN EXAMPLE WITH DOWNBEAT ESTIMATION
(DURAND & AL. 2017)
MIR: AN EXAMPLE WITH DOWNBEAT ESTIMATION
(DURAND & AL. 2017)
S Durand & al., "Robust Downbeat Tracking Using an Ensemble of Convolutional Networks", IEEE/ACM
Transactions on Audio, Speech, and Language Processing, Vol 25, N°1, 2017
DOWNBEAT ESTIMATION: DÉMO
Examples at the output of each network
https://simondurand.github.io/dnn_audio.html
Other audio example
JBB (Downbeat)
JBB (Tatum)
Exemple (Downbeat)
Exemple (Tatum)
MUSIC INFORMATION RETRIEVAL
Some Current trends:
11/04/2019
24
Context-aware music
Recommendation
with
Text-Informed Lead
Vocal Extraction
with
EEG-driven
music processing
With
Context-driven
Style transfer
With Conditional audio
generation
With
5 new PhD grants within project «New Frontiers in Music Information Processing (MIP-
Frontiers), funded by the European Unions Horizon 2020 research and innovation program
under the Marie Skodowska-Curie grant agreement No. 765068.

Weitere ähnliche Inhalte

Ähnlich wie Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - "Machine Listening: L'intelligence artificielle pour les sons et la musique"

Ml conf2013 teaching_computers_share
Ml conf2013 teaching_computers_shareMl conf2013 teaching_computers_share
Ml conf2013 teaching_computers_shareMLconf
 
1. paul dwyer internet radio emma
1. paul dwyer internet radio emma1. paul dwyer internet radio emma
1. paul dwyer internet radio emmaPaul Dwyer
 
Taking A Look At Audio Compression
Taking A Look At Audio CompressionTaking A Look At Audio Compression
Taking A Look At Audio CompressionPatty Buckley
 
A Computational Framework for Sound Segregation in Music Signals using Marsyas
A Computational Framework for Sound Segregation in Music Signals using MarsyasA Computational Framework for Sound Segregation in Music Signals using Marsyas
A Computational Framework for Sound Segregation in Music Signals using MarsyasLuís Gustavo Martins
 
Explain Briefly The Physiological Process Of Hearing In...
Explain Briefly The Physiological Process Of Hearing In...Explain Briefly The Physiological Process Of Hearing In...
Explain Briefly The Physiological Process Of Hearing In...Sue Ganguli
 
Analysis Synthesis Comparison
Analysis Synthesis ComparisonAnalysis Synthesis Comparison
Analysis Synthesis ComparisonJim Webb
 
Towards User-friendly Audio Creation
Towards User-friendly Audio CreationTowards User-friendly Audio Creation
Towards User-friendly Audio CreationJean Vanderdonckt
 
The Impact Of Digital Audio On The Audio Engineering Industry
The Impact Of Digital Audio On The Audio Engineering IndustryThe Impact Of Digital Audio On The Audio Engineering Industry
The Impact Of Digital Audio On The Audio Engineering IndustryAngie Jorgensen
 
The Impact Of Digital Audio On The Audio Engineering Industry
The Impact Of Digital Audio On The Audio Engineering IndustryThe Impact Of Digital Audio On The Audio Engineering Industry
The Impact Of Digital Audio On The Audio Engineering IndustryJessica Robles
 
Nithin Xavier research_proposal
Nithin Xavier research_proposalNithin Xavier research_proposal
Nithin Xavier research_proposalNithin Xavier
 
Digital Composition
Digital CompositionDigital Composition
Digital CompositionJosh White
 
The Impact Of Technology On The Recording Studios
The Impact Of Technology On The Recording StudiosThe Impact Of Technology On The Recording Studios
The Impact Of Technology On The Recording StudiosKaren Thompson
 
'Music and Noise Fingerprinting and Reference Cancellation Applied to Forensi...
'Music and Noise Fingerprinting and Reference Cancellation Applied to Forensi...'Music and Noise Fingerprinting and Reference Cancellation Applied to Forensi...
'Music and Noise Fingerprinting and Reference Cancellation Applied to Forensi...owrpresentations
 
Archiving and disseminating sound archives – 3. Analysis and treatment of sou...
Archiving and disseminating sound archives – 3. Analysis and treatment of sou...Archiving and disseminating sound archives – 3. Analysis and treatment of sou...
Archiving and disseminating sound archives – 3. Analysis and treatment of sou...Phonothèque MMSH
 
The And Its Effects On America
The And Its Effects On AmericaThe And Its Effects On America
The And Its Effects On AmericaVictoria Leon
 
Audio Processing and Music Recognition
Audio Processing and Music RecognitionAudio Processing and Music Recognition
Audio Processing and Music RecognitionMrinmoy Dalal
 
Alpan Aytekin-Game Audio Essentials
Alpan Aytekin-Game Audio EssentialsAlpan Aytekin-Game Audio Essentials
Alpan Aytekin-Game Audio Essentialsgamedevelopersturkey
 

Ähnlich wie Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - "Machine Listening: L'intelligence artificielle pour les sons et la musique" (20)

Ml conf2013 teaching_computers_share
Ml conf2013 teaching_computers_shareMl conf2013 teaching_computers_share
Ml conf2013 teaching_computers_share
 
1. paul dwyer internet radio emma
1. paul dwyer internet radio emma1. paul dwyer internet radio emma
1. paul dwyer internet radio emma
 
Sound is not speech
Sound is not speechSound is not speech
Sound is not speech
 
Taking A Look At Audio Compression
Taking A Look At Audio CompressionTaking A Look At Audio Compression
Taking A Look At Audio Compression
 
A Computational Framework for Sound Segregation in Music Signals using Marsyas
A Computational Framework for Sound Segregation in Music Signals using MarsyasA Computational Framework for Sound Segregation in Music Signals using Marsyas
A Computational Framework for Sound Segregation in Music Signals using Marsyas
 
Explain Briefly The Physiological Process Of Hearing In...
Explain Briefly The Physiological Process Of Hearing In...Explain Briefly The Physiological Process Of Hearing In...
Explain Briefly The Physiological Process Of Hearing In...
 
Analysis Synthesis Comparison
Analysis Synthesis ComparisonAnalysis Synthesis Comparison
Analysis Synthesis Comparison
 
Towards User-friendly Audio Creation
Towards User-friendly Audio CreationTowards User-friendly Audio Creation
Towards User-friendly Audio Creation
 
Group 6 Presentation
Group 6 PresentationGroup 6 Presentation
Group 6 Presentation
 
The Impact Of Digital Audio On The Audio Engineering Industry
The Impact Of Digital Audio On The Audio Engineering IndustryThe Impact Of Digital Audio On The Audio Engineering Industry
The Impact Of Digital Audio On The Audio Engineering Industry
 
The Impact Of Digital Audio On The Audio Engineering Industry
The Impact Of Digital Audio On The Audio Engineering IndustryThe Impact Of Digital Audio On The Audio Engineering Industry
The Impact Of Digital Audio On The Audio Engineering Industry
 
Nithin Xavier research_proposal
Nithin Xavier research_proposalNithin Xavier research_proposal
Nithin Xavier research_proposal
 
Digital Composition
Digital CompositionDigital Composition
Digital Composition
 
The Impact Of Technology On The Recording Studios
The Impact Of Technology On The Recording StudiosThe Impact Of Technology On The Recording Studios
The Impact Of Technology On The Recording Studios
 
'Music and Noise Fingerprinting and Reference Cancellation Applied to Forensi...
'Music and Noise Fingerprinting and Reference Cancellation Applied to Forensi...'Music and Noise Fingerprinting and Reference Cancellation Applied to Forensi...
'Music and Noise Fingerprinting and Reference Cancellation Applied to Forensi...
 
Archiving and disseminating sound archives – 3. Analysis and treatment of sou...
Archiving and disseminating sound archives – 3. Analysis and treatment of sou...Archiving and disseminating sound archives – 3. Analysis and treatment of sou...
Archiving and disseminating sound archives – 3. Analysis and treatment of sou...
 
The And Its Effects On America
The And Its Effects On AmericaThe And Its Effects On America
The And Its Effects On America
 
Audio Processing and Music Recognition
Audio Processing and Music RecognitionAudio Processing and Music Recognition
Audio Processing and Music Recognition
 
Alpan Aytekin-Game Audio Essentials
Alpan Aytekin-Game Audio EssentialsAlpan Aytekin-Game Audio Essentials
Alpan Aytekin-Game Audio Essentials
 
Ibac2007 riede
Ibac2007 riedeIbac2007 riede
Ibac2007 riede
 

Mehr von I MT

Colloque Healthcare 4.0 : "Soufflez, c’est dépisté"
Colloque Healthcare 4.0 : "Soufflez, c’est dépisté" Colloque Healthcare 4.0 : "Soufflez, c’est dépisté"
Colloque Healthcare 4.0 : "Soufflez, c’est dépisté" I MT
 
Colloque Healthcare 4.0 : "Accompagnement des troubles du sommeil : la recher...
Colloque Healthcare 4.0 : "Accompagnement des troubles du sommeil : la recher...Colloque Healthcare 4.0 : "Accompagnement des troubles du sommeil : la recher...
Colloque Healthcare 4.0 : "Accompagnement des troubles du sommeil : la recher...I MT
 
Colloque Healthcare 4.0 : "HospiT'Win"
Colloque Healthcare 4.0 : "HospiT'Win"Colloque Healthcare 4.0 : "HospiT'Win"
Colloque Healthcare 4.0 : "HospiT'Win"I MT
 
Colloque Healthcare 4.0 : « Vous ne devriez pas autant être sur les écrans ! »
Colloque Healthcare 4.0 : « Vous ne devriez pas autant être sur les écrans ! » Colloque Healthcare 4.0 : « Vous ne devriez pas autant être sur les écrans ! »
Colloque Healthcare 4.0 : « Vous ne devriez pas autant être sur les écrans ! » I MT
 
Colloque Healthcare 4.0 : "NeuroLife : Interfaces Cerveau Machine pour la San...
Colloque Healthcare 4.0 : "NeuroLife : Interfaces Cerveau Machine pour la San...Colloque Healthcare 4.0 : "NeuroLife : Interfaces Cerveau Machine pour la San...
Colloque Healthcare 4.0 : "NeuroLife : Interfaces Cerveau Machine pour la San...I MT
 
Colloque Healthcare 4.0 : "La protection de données en santé"
Colloque Healthcare 4.0 : "La protection de données en santé"Colloque Healthcare 4.0 : "La protection de données en santé"
Colloque Healthcare 4.0 : "La protection de données en santé"I MT
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « EIT Health : un tremplin europ...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « EIT Health : un tremplin europ...Colloque IMT - 15/10/2019 - Healthcare 4.0 – « EIT Health : un tremplin europ...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « EIT Health : un tremplin europ...I MT
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Modélisation semi-analytique ...
Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Modélisation semi-analytique ...Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Modélisation semi-analytique ...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Modélisation semi-analytique ...I MT
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Développement de la caméra XE...
Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Développement de la caméra XE...Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Développement de la caméra XE...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Développement de la caméra XE...I MT
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Le suivi géographique des per...
Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Le suivi géographique des per...Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Le suivi géographique des per...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Le suivi géographique des per...I MT
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Pilotage intelligent du servic...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Pilotage intelligent du servic...Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Pilotage intelligent du servic...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Pilotage intelligent du servic...I MT
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Quantification of idiopathic i...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Quantification of idiopathic i...Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Quantification of idiopathic i...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Quantification of idiopathic i...I MT
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Contrôle...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Contrôle...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Contrôle...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Contrôle...I MT
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Interopé...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Interopé...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Interopé...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Interopé...I MT
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - TeraLab,...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - TeraLab,...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - TeraLab,...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - TeraLab,...I MT
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Session ...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Session ...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Session ...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Session ...I MT
 
Colloque IMT - L'IA au cœur des mutations industrielles - Session Robotique, ...
Colloque IMT - L'IA au cœur des mutations industrielles - Session Robotique, ...Colloque IMT - L'IA au cœur des mutations industrielles - Session Robotique, ...
Colloque IMT - L'IA au cœur des mutations industrielles - Session Robotique, ...I MT
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Focus CE...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Focus CE...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Focus CE...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Focus CE...I MT
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - L'IA pou...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - L'IA pou...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - L'IA pou...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - L'IA pou...I MT
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles -PPC dans ...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles -PPC dans ...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles -PPC dans ...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles -PPC dans ...I MT
 

Mehr von I MT (20)

Colloque Healthcare 4.0 : "Soufflez, c’est dépisté"
Colloque Healthcare 4.0 : "Soufflez, c’est dépisté" Colloque Healthcare 4.0 : "Soufflez, c’est dépisté"
Colloque Healthcare 4.0 : "Soufflez, c’est dépisté"
 
Colloque Healthcare 4.0 : "Accompagnement des troubles du sommeil : la recher...
Colloque Healthcare 4.0 : "Accompagnement des troubles du sommeil : la recher...Colloque Healthcare 4.0 : "Accompagnement des troubles du sommeil : la recher...
Colloque Healthcare 4.0 : "Accompagnement des troubles du sommeil : la recher...
 
Colloque Healthcare 4.0 : "HospiT'Win"
Colloque Healthcare 4.0 : "HospiT'Win"Colloque Healthcare 4.0 : "HospiT'Win"
Colloque Healthcare 4.0 : "HospiT'Win"
 
Colloque Healthcare 4.0 : « Vous ne devriez pas autant être sur les écrans ! »
Colloque Healthcare 4.0 : « Vous ne devriez pas autant être sur les écrans ! » Colloque Healthcare 4.0 : « Vous ne devriez pas autant être sur les écrans ! »
Colloque Healthcare 4.0 : « Vous ne devriez pas autant être sur les écrans ! »
 
Colloque Healthcare 4.0 : "NeuroLife : Interfaces Cerveau Machine pour la San...
Colloque Healthcare 4.0 : "NeuroLife : Interfaces Cerveau Machine pour la San...Colloque Healthcare 4.0 : "NeuroLife : Interfaces Cerveau Machine pour la San...
Colloque Healthcare 4.0 : "NeuroLife : Interfaces Cerveau Machine pour la San...
 
Colloque Healthcare 4.0 : "La protection de données en santé"
Colloque Healthcare 4.0 : "La protection de données en santé"Colloque Healthcare 4.0 : "La protection de données en santé"
Colloque Healthcare 4.0 : "La protection de données en santé"
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « EIT Health : un tremplin europ...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « EIT Health : un tremplin europ...Colloque IMT - 15/10/2019 - Healthcare 4.0 – « EIT Health : un tremplin europ...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « EIT Health : un tremplin europ...
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Modélisation semi-analytique ...
Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Modélisation semi-analytique ...Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Modélisation semi-analytique ...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Modélisation semi-analytique ...
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Développement de la caméra XE...
Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Développement de la caméra XE...Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Développement de la caméra XE...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Développement de la caméra XE...
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Le suivi géographique des per...
Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Le suivi géographique des per...Colloque IMT - 15/10/2019 - Healthcare 4.0 –  « Le suivi géographique des per...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Le suivi géographique des per...
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Pilotage intelligent du servic...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Pilotage intelligent du servic...Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Pilotage intelligent du servic...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Pilotage intelligent du servic...
 
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Quantification of idiopathic i...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Quantification of idiopathic i...Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Quantification of idiopathic i...
Colloque IMT - 15/10/2019 - Healthcare 4.0 – « Quantification of idiopathic i...
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Contrôle...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Contrôle...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Contrôle...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Contrôle...
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Interopé...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Interopé...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Interopé...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Interopé...
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - TeraLab,...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - TeraLab,...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - TeraLab,...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - TeraLab,...
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Session ...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Session ...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Session ...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Session ...
 
Colloque IMT - L'IA au cœur des mutations industrielles - Session Robotique, ...
Colloque IMT - L'IA au cœur des mutations industrielles - Session Robotique, ...Colloque IMT - L'IA au cœur des mutations industrielles - Session Robotique, ...
Colloque IMT - L'IA au cœur des mutations industrielles - Session Robotique, ...
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Focus CE...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Focus CE...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Focus CE...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - Focus CE...
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - L'IA pou...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - L'IA pou...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - L'IA pou...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - L'IA pou...
 
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles -PPC dans ...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles -PPC dans ...Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles -PPC dans ...
Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles -PPC dans ...
 

Kürzlich hochgeladen

SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxSUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxNaveenVerma126
 
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxIT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxSAJITHABANUS
 
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratoryدليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide LaboratoryBahzad5
 
solar wireless electric vechicle charging system
solar wireless electric vechicle charging systemsolar wireless electric vechicle charging system
solar wireless electric vechicle charging systemgokuldongala
 
Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxKISHAN KUMAR
 
me3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Ame3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Akarthi keyan
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxrealme6igamerr
 
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfSummer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfNaveenVerma126
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvementVijayMuni2
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfJulia Kaye
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid BodyAhmadHajasad2
 
Dev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingDev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingMarian Marinov
 
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...soginsider
 
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Amil baba
 
Nodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxNodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxwendy cai
 
Gender Bias in Engineer, Honors 203 Project
Gender Bias in Engineer, Honors 203 ProjectGender Bias in Engineer, Honors 203 Project
Gender Bias in Engineer, Honors 203 Projectreemakb03
 
ChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai searchChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai searchrohitcse52
 
cloud computing notes for anna university syllabus
cloud computing notes for anna university syllabuscloud computing notes for anna university syllabus
cloud computing notes for anna university syllabusViolet Violet
 

Kürzlich hochgeladen (20)

Litature Review: Research Paper work for Engineering
Litature Review: Research Paper work for EngineeringLitature Review: Research Paper work for Engineering
Litature Review: Research Paper work for Engineering
 
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docxSUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
SUMMER TRAINING REPORT ON BUILDING CONSTRUCTION.docx
 
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxIT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
 
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratoryدليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
 
solar wireless electric vechicle charging system
solar wireless electric vechicle charging systemsolar wireless electric vechicle charging system
solar wireless electric vechicle charging system
 
Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptx
 
me3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part Ame3493 manufacturing technology unit 1 Part A
me3493 manufacturing technology unit 1 Part A
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
 
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfSummer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvement
 
Présentation IIRB 2024 Marine Cordonnier.pdf
Présentation IIRB 2024 Marine Cordonnier.pdfPrésentation IIRB 2024 Marine Cordonnier.pdf
Présentation IIRB 2024 Marine Cordonnier.pdf
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
 
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
Engineering Mechanics  Chapter 5  Equilibrium of a Rigid BodyEngineering Mechanics  Chapter 5  Equilibrium of a Rigid Body
Engineering Mechanics Chapter 5 Equilibrium of a Rigid Body
 
Dev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & LoggingDev.bg DevOps March 2024 Monitoring & Logging
Dev.bg DevOps March 2024 Monitoring & Logging
 
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...Transforming Process Safety Management: Challenges, Benefits, and Transition ...
Transforming Process Safety Management: Challenges, Benefits, and Transition ...
 
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
 
Nodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptxNodal seismic construction requirements.pptx
Nodal seismic construction requirements.pptx
 
Gender Bias in Engineer, Honors 203 Project
Gender Bias in Engineer, Honors 203 ProjectGender Bias in Engineer, Honors 203 Project
Gender Bias in Engineer, Honors 203 Project
 
ChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai searchChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai search
 
cloud computing notes for anna university syllabus
cloud computing notes for anna university syllabuscloud computing notes for anna university syllabus
cloud computing notes for anna university syllabus
 

Colloque IMT -04/04/2019- L'IA au cœur des mutations industrielles - "Machine Listening: L'intelligence artificielle pour les sons et la musique"

  • 1. "MACHINE LISTENING: L'INTELLIGENCE ARTIFICIELLE POUR LES SONS ET LA MUSIQUE" COLLOQUE IMT - L’INTELLIGENCE ARTIFICIELLE AU COEUR DES MUTATIONS INDUSTRIELLES. APRIL 4TH, 2019 Gaël RICHARD Professor, Head of the Image, Data, Signal department
  • 2. 11/04/2019 2 MACHINE LISTENING ? A well established domain for speech … Speech Recognition « What a great workshop !» Speech Emotion Recognition « The person who’s speaking is happy but anxious» Speaker Recognition « Gaël Richard is speaking» Speech Translation « Quel excellent colloque ! » « What a great workshop ! »
  • 3. MACHINE LISTENING FOR AUDIO AND MUSIC Goal: to extract « information » from the audio signal 11/04/2019 3 Music recognition (or Audio ID): identifying the music recording Music Information retrieval Music transcription, Music similarity, Music genre recognition, Musical instrument recognition, Music recommendation, Autotagging, Audio Scene recognition: sound recorded in streets, in subway station, in office,….) Audio Source separation Demixing music, Singing voice extraction, source localisation, Audio Event recognition: Speech, Music, Car noises, birds, … Audio Sound capture: Localisation, Dereverberation, Denoising, … Music Emotion recognition: Sad vs happy vs dance music, …. Bioacoustics: Wildlife monitoring, biodiversity, species id…
  • 4. SOME APPLICATIONS … 11/04/2019 4 Music streaming, music recommendation Music education Music Identification, Audio Fringerprint Karaoke, speech to rap conversion Sound recognition, smarthomes, smart hearables bmat Audeering Stratégie de marque musicale, Supervision musicale (pub.; films) Bioacoustics Vocal separation, music separation
  • 6. SOURCE SEPARATION FOR REMIXING 11/04/2019 6
  • 7. SOURCE SEPARATION (LEGLAIVE & AL.) 11/04/2019 7 Model for Audio sources Model for reverberation
  • 8. AN EXAMPLE OF MODEL FOR AUDIO SOURCES Non Negative Matrix factorization 11/04/2019 8
  • 9. SOURCE SEPARATION - S. LEGLAIVE (EXAMPLE REMIXED BY RADIO FRANCE) 11/04/2019 9 S. Leglaive, R. Badeau, G. Richard, "Multichannel Audio Source Separation with Probabilistic Reverberation Priors", IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 24, no. 12, December 2016 Simon Leglaive, Roland Badeau, Gaël Richard, Separating Time-Frequency Sources from Time-Domain Convolutive Mixtures Using Non-negative Matrix Factorization. WASPAA , Oct. 2017 New Paltz, US.
  • 10. AUDIO IDENTIFICATION OU AUDIOID Audio ID = find high-level metadata from a music recording Challenges: Efficiency in adverse conditions (distorsion, noises,..) Scale to “Big data” (bases > millions of titles) Rapidity / Real time Product example : Audio identification Information of the recording (title, artist, etc.., …)
  • 11. REAL-TIME AUDIO IDENTIFICATION (FENET & AL.) Audio recordings recognition • Identical • Approximate (live vs studio) • Real time demonstrator • For music recommendation, second screen applications, … 11/04/2019 11 Sébastien Fenet, Yves Grenier, Gaël Richard: An Extended Audio Fingerprint Method with Capabilities for Similar Music Detection. ISMIR 2013: 569-574
  • 12. SOUND EVENTS AND ACOUSTIC SCENE RECOGNITION 11/04/2019 12
  • 14. AN EXEMPLE OF A RECENT APPROACH 11/04/2019 14 V. Bisot, R. Serizel, S. Essid, G. Richard, "Feature Learning with Matrix Factorization Applied to Acoustic Scene Classification", IEEE/ACM Transactions on Audio, Speech, and Language Processing, (2017), Special Issue on Sound Scene and Event Analysis.
  • 15. EXAMPLE FOR SCENE CLASSIFICATION
  • 16. UNSUPERVISED NMF FOR ACOUSTIC SCENE RECOGNITION
  • 17. UNSUPERVISED NMF FOR ACOUSTIC SCENE RECOGNITION
  • 18. EXAMPLE WITH DNN: ACOUSTIC SCENE RECOGNITION V. Bisot & al., "Feature Learning with Matrix Factorization Applied to Acoustic Scene Classification", IEEE/ACM Transactions on Audio, Speech, and Language Processing, (2017), V. Bisot & al., Leveraging deep neural networks with nonnegative representations for improved environmental sound classification IEEE International Workshop on Machine Learning for Signal Processing MLSP, Sep 2017, Tokyo,
  • 19. TYPICAL PERFORMANCES OF ACOUSTIC SCENE RECOGNITION (CHALLENGE DCASE 2016) A Mesaros & al. Detection and Classification of Acoustic Scenes and Events: Outcome of the DCASE 2016 challenge IEEE/ACM Transactions on Audio, Speech, and Language Processing 26 (2), 379-393
  • 20. MUSIC INFORMATION RETRIEVAL Major topics: • Music transcription (Multiple F0 estimation, Beat/Downbeat detection, instrument classification, …), • Music recommendation • Source separation, • Multimodal music processing 11/04/2019 20
  • 21. MIR: AN EXAMPLE WITH DOWNBEAT ESTIMATION (DURAND & AL. 2017)
  • 22. MIR: AN EXAMPLE WITH DOWNBEAT ESTIMATION (DURAND & AL. 2017) S Durand & al., "Robust Downbeat Tracking Using an Ensemble of Convolutional Networks", IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol 25, N°1, 2017
  • 23. DOWNBEAT ESTIMATION: DÉMO Examples at the output of each network https://simondurand.github.io/dnn_audio.html Other audio example JBB (Downbeat) JBB (Tatum) Exemple (Downbeat) Exemple (Tatum)
  • 24. MUSIC INFORMATION RETRIEVAL Some Current trends: 11/04/2019 24 Context-aware music Recommendation with Text-Informed Lead Vocal Extraction with EEG-driven music processing With Context-driven Style transfer With Conditional audio generation With 5 new PhD grants within project «New Frontiers in Music Information Processing (MIP- Frontiers), funded by the European Unions Horizon 2020 research and innovation program under the Marie Skodowska-Curie grant agreement No. 765068.