This document discusses the future of telepresence and how new technologies can make it more accessible. It proposes that: 1. Using scalable video coding (SVC) and a video routing unit (VRU) can replace traditional MCUs, enabling near-zero delay, high quality experiences without quality loss from multiple encodings. 2. SVC allows co-existence of low-end and high-end systems by adapting video quality based on available bandwidth. 3. The VRU architecture is simpler and more scalable than MCUs. Combined with SVC and standard hardware, these technologies can solve long-standing videoconferencing problems and make telepresence affordable and ubiquitous.

1. The Future of Telepresence Alex Eleftheriadis, alex@vidyo.com

2. What is Telepresence? Wikipedia: Telepresence refers to a set of technologies which allow a person to feel as if they were present, to give the appearance that they were present, or to have an effect, at a location other than their true location. Alternatively: Videoconferencing sucks! What can you do to make it better if you have all the money you need?

3. Lessons Learned Elsewhere Computing Networking Communications

4. DEC VAX (1977, 0.5 MIPS) VAX 11/780 (first two cabinet sections), shown with Unibus expansion cabinet (middle cabinet section), two tape drives, two RP05 or RP06 removable pack disk drives, a DECwriter printing terminal, and a VT52 CRT terminal

5. iPhone 3G (2008, 1250 MIPS)

6. Lessons Learned Elsewhere Computing Fast Personal Portable Networking Scalable Ubiquitous Communications Reliable Ubiquitous Mobile

7. Telepresence Today Extremely expensive Not scalable Not portable Not personal Very high quality => Traditional videoconferencing: same, but sacrifices quality for low cost.

9. Total delay under 250ms latencyLegacy video conferencing quality Quality of Experience 10 hours/month Type Desktop Executive Room Telepresence Cost $100’s $1000’s $10,000’s $100,000’s

10. Key Videoconferencing System Characteristics Rate Matching Support for heterogeneous endpoints and access networks Personalized Layout Each user selects the resolution and users he/she wants to see Low Delay Interactivity requires <250msec end-to-end Error Resilience Tolerance to packet loss rates >10% Error Localization An error in one user should not affect other users Complexity Lower complexity = Lower cost Low complexity allows to integrate with existing network services (e.g., in cheap routers) => How can we have all that, but with high quality and scalability?

11. H.264 SVC Scalable Video Coding (SVC) Developed by JVT (=Joint Video Team of ITU and ISO) SVC is Amendment 3 to H.264 AVC Most of SVC is in Annex G of H.264 AVC CFP April 2004, Consented in Nov 2007 RTP payload format nearly completed Architecture centered on VidyoRouter™ (“VRU”)

12. Temporal Structure of Non-Scalable Codecs

13. Temporal Scalability

14. Spatial Scalability Prediction Coding Multiplex Prediction Coding SVC Bitstream H.264/AVC-compatible base layer Prediction Coding Scale Scale

15. Temporal + Spatial Scalability

16. QVGA 30 fps Temporal + Spatial Scalability

17. VGA 15 fps Temporal + Spatial Scalability

18. What does the VidyoRouter do? High Resolution High Frame Rate 2 Mbps High Resolution Medium Frame Rate VidyoRouter™ 1 Mbps Medium Resolution Medium Frame Rate 500 Kbps High Resolution High Frame Rate Low Resolution Low Frame Rate 150 Kbps

19. Traditional MCU Architecture High Cost High Delay Quality Loss from Cascaded Encodings ENCODE ENCODE DECODE DECODE COMPOSE ENCODE DECODE

20. VidyoRouter™vs. MCU VidyoRouter is simply an application layer router Nearly zero-delay (~20 msec), allows interactive multi-point sessions Eliminates video quality loss due to transcoding Rate matching and personal layout are simple routing decisions Error localization and robustness up to 20% packet loss rates Computing power that rides the Intel® curve Endpoint processing of the video from different participants makes the VidyoRouterhighly scalable Similar complexity to other network appliances

21. Significant Impact Minor or No Impact Base Layer Conventional Coding Scalable Coding Enhancement Layer SVC+VR Eliminates Error Resilience Problem

22. Temporal Scalability

23. Concluding Remarks Videoconferencing and telepresence will remain “small”, unless our technology enables scaling with high quality and low cost. Video coding scalability and the video router architecture allow: Replacement of the MCU with a much simpler, scalable device – the Video Routing Unit (VRU) Very high error resilience (=high quality user experience) Co-existence of lower-end systems with telepresence systems Use of standard (Intel) hardware components – riding the Intel curve These concepts finally solve 30-year-old packet video problems, and are the key ingredients for making telepresence “personal”.

24. IMTC SuperOp! Room

25. IMTC SuperOp! Room

26. Concluding Remarks Videoconferencing and telepresence will remain “small”, unless our technology enables scaling with high quality and low cost. Video coding scalability and the video router architecture allow: Replacement of the MCU with a much simpler, scalable device Very high error resilience (=high quality user experience) Use of standard (Intel) hardware components – riding the Intel curve These concepts finally solve 30-year-old packet video problems, and are the key ingredients for making telepresence “personal”. Can I get the same benefit from simulcasting? Or without two spatial layers? Or ….? Partially. If you use part of the recipe, you get only part of the flavor. Wait … So, the future of telepresence is … Vidyo?

27. Thank You!