Spatial Conferencing HP Baumeister, Director, Fraunhofer USA Redmond, 16 Oct 2014

1. © Fraunhofer IIS 1
Spatial Conferencing
Redmond, 16 Oct 2014
Spatial Conferencing
HP Baumeister

2. © Fraunhofer IIS 2 S p atial Conferencing
Telepresence Rooms:
High End Conferencing

3. © Fraunhofer IIS 3 S p atial Conferencing
General Reaction:
“As if speaking to someone in the same room”

4. © Fraunhofer IIS 4 S p atial Conferencing
General Reaction:
“As if speaking to someone in the same room”
Why?
In part, because the audio is “transparent”, and comes from the appropriate
direction!
“Spatial Audio”

5. © Fraunhofer IIS 5 S p atial Conferencing
“Spatial Audio”
• audio: 3 channels of Fraunhofer AAC-LD @ 48kHz
• video: 3 x H.264 @ 1080p

6. © Fraunhofer IIS 6 S p atial Conferencing
Telepresence for Everyone:
Spatial Conferencing
 Bring the “telepresence experience” to smartphones
 Using regular headsets/earphones

7. © Fraunhofer IIS 7 S p atial Conferencing
Telepresence for Everyone:
Spatial Conferencing
“Billions of smartphone and tablets”

8. © Fraunhofer IIS 8
Spatial Conferencing
Spatial Conferencing: Key Elements
Highest Quality Audio: “Full-HD Voice”
“Spatialization”

9. © Fraunhofer IIS 9
Spatial Conferencing
Spatial Conferencing: “Full-HD Voice”
14 kHz+ audio bandwidth

10. © Fraunhofer IIS 10
Spatial Conferencing
Full-HD Voice Audio Quality
source: http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_52/Docs/S4-090080.zip (Feasibility study on EVS audio bandwidth, Ericsson 3GPP/SA4 01/2009)
0
1
2
3
4
5
Mean Opinion Score
Fair
Good
Excellent
Bad
Poor
NB
WB
SWB

11. © Fraunhofer IIS 11
Spatial Conferencing
Spatial Conferencing “Full-HD Voice”
14 kHz+ audio bandwidth
Very low delay
Multi-channel capability

12. © Fraunhofer IIS 12
Spatial Conferencing
Spatial Conferencing
“Full-HD Voice”
ELD = Enhanced Low Delay
AAC-ELD is a low latency mode of the widely adopted music codec AAC.
CD-like audio quality for telephone calls
full audio bandwidth up to 20 kHz
low coding delay down to 15 ms, crucial for natural conversations
optimized for a bit-rate range of 24 kbit/s to 64 kbit/s per channel
AAC-ELDv2 enhances bit-rate range down to 24 kbit/s stereo
Most widely adopted codec offering Full-HD Voice today
FaceTime
Native in iOS
Native in Android

13. © Fraunhofer IIS 13
Spatial Conferencing
Spatial Conferencing for Everyone “Spatialization”
Geometric Localization of Participants
With or without Video……

14. © Fraunhofer IIS 14
Spatial Conferencing
Spatial Conferencing “Head Related Transfer Function”

15. © Fraunhofer IIS 15
Spatial Conferencing
Spatial Conferencing “Head Related Transfer Function”
Credit: University of Maryland

16. © Fraunhofer IIS 16
Spatial Conferencing
Spatial Conferencing “Out of Head Experience”
Stereo = “Inside Head”
Spatial Rendering = “outside Head”

17. © Fraunhofer IIS 17
Spatial Conferencing
Audio Conferencing
MCU

18. © Fraunhofer IIS 18
Spatial Conferencing
Audio Conferencing “single channel” – “everyone is in the same location”
MCU

19. © Fraunhofer IIS 19
Spatial Conferencing
Spatial Conferencing “Spatialization” – “participants can be localized as if in the room”
MCU
positioning & out of head
Full-HD Voice (spatialized)

20. © Fraunhofer IIS 20
Spatial Conferencing
Spatial Conferencing MCU

21. © Fraunhofer IIS 21
Spatial Conferencing
3-Point conference via IP
Full Band Audio, 44.1 kHz
Immersive, spatial experience
Spatial Conferencing Demo Setup
3x Nexus 5
-Android 4.4.3.2.1
-VoIP App using native ELD
standard
in-ear headset
Mac-Mini running MCU with binaural processing
64 kbps mono
128 kbps
stereo

22. © Fraunhofer IIS 22 S p atial Conferencing
Developed and evaluated in collaboration with
Spatial Conferencing Trials
PSTN Full-HD Voice
“I found it really amazing, because one really
thought you are sitting in the same room”

23. © Fraunhofer IIS 23
Spatial Conferencing
Member Companies should consider spatial conferencing in future systems
Define uses cases
Interface Specification / Standardisation
Interoperability
Spatial Conferencing IMTC Opportunities

24. © Fraunhofer IIS 24
Spatial Conferencing
See me: HP Baumeister
Call me: 408 573 9903
 Mail Me hp.baumeister@dmt.fraunhofer.org
Thank you! Questions? Live Demo?

25. © Fraunhofer IIS 25
Spatial Conferencing
Spatial Conferencing “Spatialization”
Full-HD Voice (mono)
MCU
positioning & out of head
Full-HD Voice (spatialized)