Activity report of the MM team (Telecom-ParisTech) for the National Evaluation process (HCERES 2018)

1. LTCI4-6/12/2018 MM - Multimédia
MM - Multimédia
Résponsable : M. Cagnazzo

2. LTCI4-6/12/2018 MM - Multimédia
Activity report 2013-2018
2

3. LTCI4-6/12/2018 MM - Multimédia
Team evolution
3
M. Cagnazzo
F. Dufaux
A. Fiandrotti
B. Pesquet-Popescu
G. Valenzise
C. Concolato
J. Le Feuvre
J.-C. Dufourd
J.-C. Moissinac
Ch. Clavel
G. Chollet
L. Likforman
M. Sigelle
C. Pelachaud
Compression and
transmission of images
and videos
Interactive services
delivery and
orchestration
Multimodal content and
interactions
Current team member
Former team member / retired / on-leave
2013 2014 2015 2016 2017 2018

4. LTCI4-6/12/2018 MM - Multimédia
Team identity
■ Current permanent staff
• Five professors and associate professors: Marco Cagnazzo, Jean-Claude
Dufourd, Attilio Fiandrotti, Jean Le Feuvre, Jean-Claude Moissinac
• One research engineer: Aurélien David (mission)
■ Non-permanent staff: 7 PhD students, 3 engineers
■ Team running
• Monthly meetings (permanent staff)
• Scientific coordination: seminars, conference and defense rehearsals,
ML-Compression study group, PhD’s day...
4

5. LTCI4-6/12/2018 MM - Multimédia
Research activities
■ All the life cycle of multimedia documents and signals is concerned:
• Acquisition, coding, transmission, transport, interactivity
■ Three main areas:
• Compression and transmission of images and videos
• Interactive services delivery and orchestration
• Multimodal content and interactions
■ These areas are linked and several collaborations exist, within the
team and within the lab
5

6. LTCI4-6/12/2018 MM - Multimédia
Compression and transmission of visual data
People: M. Cagnazzo, F. Dufaux, A. Fiandrotti, B. Pesquet-Popescu, G. Valenzise
■ Image and video compression: Rate-distortion optimization, rate
control, semantic coding
• Historical research topic, theoretical and practical contributions
• Collaboration with IMAGE (satellite image compression)
■ Network coding, distributed video coding, linear video coding:
• Robust transmission of video over unreliable networks, joint source-
channel coding
19/11/20
18
6
HEVC-SCC at 0.018 bpp, 33.1 dB Proposed at 0.007 bpp, 38.2 dB

7. LTCI4-6/12/2018 MM - Multimédia
Compression and transmission of visual data
People: M. Cagnazzo, F. Dufaux, A. Fiandrotti, B. Pesquet-Popescu, G. Valenzise
■ Immersive video: multiview formats, high-dynamic-range
video, hologram compression, view synthesis, integral
imaging, quality, features
• Compression and transmission
• Disparity and depth estimation
• Free navigation and
view synthesis
■ Three books published in the period
■ 8 defended and 3 on-going PhD thesis
7

8. LTCI4-6/12/2018 MM - Multimédia
Interactive Services Delivery and Orchestration
People: C. Concolato, J.-C. Dufourd, J. Le Feuvre, J.-C. Moissinac
■ Streaming
• Hybrid broadband / broadcast activity
─ Editor spec MPEG-2 hybrid sync
• Tiled-based video delivery (360 immersive,
panoramic)
• Ultra-low Latency streaming over HTTP
─ Editor spec MPEG-CMAF
• 4 R&D projects awards (Celtics, NAB)
■ GPAC
• Codec, player and streaming for interactive multimedia services
• Open software (LGPL): >17000 visits/months, >4000 dwnl/month
• World premieres: MPEG-DASH player, MPEG-HEVC player,
autostereoscopic screen support …
• More than 300 scientific articles using GPAC
8

9. LTCI4-6/12/2018 MM - Multimédia
Interactive Services Delivery and Orchestration
People: C. Concolato, J.-C. Dufourd, J. Le
Feuvre, J.-C. Moissinac
■ Orchestration
• A new MPEG activity
• Multi-source (uncoordinated),
multi-output
• Editor and main contributor of
architecture
■ Semantic Web
• Data aggregation
• Collaboration with the DIG team
• “Data & Musée” project
9

10. LTCI4-6/12/2018 MM - Multimédia
Multimodal Content and Interaction
People: G. Chollet, Ch. Clavel, L. Likforman, C. Pelachaud, M. Sigelle
■ Hidden Markov models and recurrent neural network-based
architectures for handwriting recognition
■ Markov models for modeling wireless networks
(collaboration with RMS)
■ Neural networks for speech recognition
■ Multimodal socio-emotional interaction
• Different aspects of human-agent interactions
• GRETA platform, socio-emotional agent
• Collaboration with IMAGES
10

11. LTCI4-6/12/2018 MM - Multimédia
Main results and achievements
■ Total scientific production (6 Equivalent Full-time Researcher)
• 61 journal papers, 252 conference articles, 27 books and chapters, 28 patents
■ Scientific production per Equivalent Full-time Researcher
• 10.2 journal papers, 42 conference articles, 4.5 books and chapters, 4.7 patents
■ Examples: Book on Emerging Technologies for 3D Video (Wiley),
chapter on Multimedia Streaming
─ Contributions from several research areas in the team:
compression, transmission, quality, display, streaming, …
■ International recognition
• Two IEEE Fellowship, one Eurasip Fellowship, one Honoris Causa PhD
• One Editor in Chief
• Impact on MPEG and W3C standardization committees
■ Society interaction
• Demos: Futur.e.s, Roland Garros ‘13 and ‘15
• GPAC community animation and transfer towards the industrial world
11

12. LTCI4-6/12/2018 MM - Multimédia
Strategy 2019-2024
12

13. LTCI4-6/12/2018 MM - Multimédia
Team strategy: strengths
■ Common strengths of the team
• Strong expertise
• Wide collaboration network
(academic and industrial)
• IP protection and valorization
■ Specific strengths: Compression and transmission of visual data
• Recognized expertise in video compression and immersive video
• Strong publication record
■ Specific strengths: Interactive Services Delivery and Orchestration
• Impact on standardization bodies (MPEG, DVB, W3C…)
• Streaming and Media Delivery
─ Worldwide visibility through GPAC and R&D projects
13

14. LTCI4-6/12/2018 MM - Multimédia
Team strategy: the environment
■ Video traffic and video applications are dominating the digital
ecosystem…
• … and it is not going to change soon, see Cisco white papers
■ Immersive formats gathers more and more interest
• 360 video, super-multiview, holograms
• MPEG ad-hoc groups
■ Methodology derived from machine learning (and in particular,
deep learning) is successfully applied to many image and video-
related fields
• including video super-resolution,
restoration, analysis, compression
but also streaming
14
Gaming
Web data
File sharing
Video

15. LTCI4-6/12/2018 MM - Multimédia
Team strategy: scientific goals
■ Compression and transmission of visual data
• Compression, with particular focus on immersive formats and semantic
coding, hologram streaming
• Free navigation, view synthesis
• Linear video coding (joint source-channel coding) for 5G networks
• Enlarge the domain and the methodology: Video analysis and modeling
─ ML-Compression study group
15
Concatenate Conv. Layer 1 Conv. Layer 2

16. LTCI4-6/12/2018 MM - Multimédia
Team strategy: scientific goals
■ Interactive Services Delivery and Orchestration
• 6DOF/Immersive Media
─ Taking advantage from team expertise on immersive media
• Next-Generation broadcast
• Orchestration
• Semantic web
■ The team is aware of the risks inherent to a strong link with the
industrial world
• But this is also a strength: opportunity for collaborations
• Scientific quality of industrial partnership
16

17. LTCI4-6/12/2018 MM - Multimédia
Means to achieve the goals
■ European and national research projects
• E.g., an ITN on ultra-low latency immersive video compression and
streaming will be submitted in Jan, with contributions from the
team’s two scientific areas
■ Team growth
• Models and Learning for Video (will be open soon)
• Immersive Media Streaming
■ Capability of recruiting excellent PhD, interns, engineers
• The Multimedia Networking M2R has been a good lever in this sense
17

18. LTCI4-6/12/2018 MM - Multimédia
Thanks for your attention!
18

19. LTCI4-6/12/2018 MM - Multimédia
Main results and achievements
■ Organization of the 2014 IEEE International Conference on Image
Processing (ICIP)
■ The most important conference on image processing in the world
■ Great involvement of all the team
• General co-chair, vice-general chair, award chair, electronic media
chair, local arrangement chair and 20+ volunteers
■ Very successful conference
• Record attendance (1700+ attendees)
• Record paper submission
19