DevEX - reference for building teams, processes, and platforms
Dynamics of Video
1. Today and tomorrow of visual communications,
building on 20 years of technology breakthroughs
October 8 – 10
Porto, Portugal
International Multimedia Telecommunications Consortium
Dynamics of Video
in Mobile Environments
Chienchung Chang
Vice President, Head of Multimedia R&D
Qualcomm Inc.
2. IMTC 20th Anniversary Forum – Porto, Portugal
Behind the Smartphone Frenzy:
Multimedia
1/3+
of daily media interactions
occur on a smartphone
3/4
Worlds population has
access
to a mobile device
604M
Facebook users are mobile
3. IMTC 20th Anniversary Forum – Porto, Portugal
• Millions of new consumers with video enabled device (phone/tablet)
• Higher availability of digital content services and social sharing platforms
(Hulu, Netflix, Amazon, Facebook, YouTube…)
• Improved capability of devices (higher resolution display, better camera and
video engine in devices)
Mobile Video Dominance
3
0
2
4
6
8
10
12
2012 2013 2014 2015 2016 2017
M2M Video File sharing Web/Data
Cisco® VNI report
Mobile Data Traffic (Exabytes [1018] per month)
5. IMTC 20th Anniversary Forum – Porto, Portugal
Data Growth Driven by Digital Video
5
1990 1995 2000 2005 2010 2013
1080p 30fps
1980x1020
QCIF 15fps
176x144
160 x
6. IMTC 20th Anniversary Forum – Porto, Portugal
Data Growth Driven by Digital Video
6
2015 2020
1080p 30fps
1980x1020
2013
8K 120fps
7680x4320
4K 60fps
3840x2160
8 x
8 x
7. IMTC 20th Anniversary Forum – Porto, Portugal
MPEG-2
•Standard in 1996
•DVD, Digital TV
MPEG-4
•Standard in 1999
AVC (H.264)
•Standard in 2003
•Bluray, Digital TV
HEVC (H.265)
•Standard in 2013
25% better 25% better ~40% better
100 MBbyte MPEG-2 video compressed with newer codecs:
MPEG-2 MPEG-4 AVC (H.264) HEVC (H.265)
100MB 75MB 56MB 33MB
Data for high quality video increased by 150x in last 22 years
With 4K and 8K video, data will increase by 64x in next 10 years
Video codec compression efficiency has only increased by 3x
The Need for Better Compression
8. IMTC 20th Anniversary Forum – Porto, Portugal
• Mobile industry now in the driver seat
– Driving the international standards
– Friendly to embedded platforms:
power, computation, HW/SW implementation etal.
– Mass digital contents are consumed on personal
screens
• Mobile and PC, TV, CE devices converge in video
experience
Video Codec Evolution
8
MPEG-2
H.262 (‘95)
MPEG-4 ASP (‘00/’11)
H.263++ (‘00)
MPEG-4 AVC
H.264 HP (‘03/’05)
HEVC MP (’13)
-40%-44%-20%
VCEG Initiates H.265 -
KTA SW (Jul’06)
MPEG’s CfE Results
(Apr’09)
-71%
9. IMTC 20th Anniversary Forum – Porto, Portugal
Drivers of High Resolution Video
Driving Beyond “Retina” Displays
qVGA
hVGA
WVGA
DVGA
HD
FHD
‘05 ‘07 ’08-’09 ’10 ‘11 ’12-’13 ‘14
130~180 ppi
180~310 ppi
326 ppi
329 ppi
450~520 ppi
132ppi
498ppi
244ppi
10. IMTC 20th Anniversary Forum – Porto, Portugal
Drivers of High Resolution Video
Resolution Upper Limits for Smartphones & Tablets
0
100
200
300
400
500
600
2 3 4 5 6 7 8 9 10 11
Resolution(PPI)
Screen Size (inches)
QVGA
HVGA
VGA
WVGA
qHD
XGA
720HD
1080HD
QXGA
WQXGA
4K
Eye resolution SP: 573ppi
Eye resolution Tablet: 382ppi
11. IMTC 20th Anniversary Forum – Porto, Portugal
Drivers of High Resolution Video
MiracastTM WiFi Display
Video
Gaming
Browsing
Photos
Social Networking
Consumers want to share content from
smaller devices to larger displays.
Video projected via MiracastTM WiFi Display requires the video stream to
be simultaneously decoded on the smartphone/tablet and re-encoded and
streamed to the larger display
Video Compression is key to lower overall network bandwidth traffic
12. IMTC 20th Anniversary Forum – Porto, Portugal
Video Technology Considerations for Mobile
Widely used codecs and file formats supported
Wide Range of Codec & Format Support
• HW accelerated codec for higher performance
• Scalable covering from QCIF to 4K video
• Transcode, trans-rate HD in real time
• HFR (high frame rate) encode in HD
Scalable Performance & Rich Features
• Pre-processing for encode path
Rotation, zoom, image stabilization, overlay
• Post-processing for decode path
Scaling, sharpening, frame rate conversion, de-
interlacing, de-blocking, de-noising, error
concealment
CE Grade Video Quality
All major frameworks:
DirectShow, OpenCore, StageFright,
Flash, Silverlight
OpenMax IL level access to HW codecs
Streaming protocol support:
RTSP, RTP, HTTP Live Streaming, HTTP Adaptive
Streaming, DASH
Multimedia Framework Support
13. IMTC 20th Anniversary Forum – Porto, Portugal
• 1080p30 is ubiquitous in smartphones(de facto)
• Next-gen imaging format: 4K (UHD-1) video is imminent.
• Visual quality (VQ) emerges as a key metric
• Internet traffic growth fuels the need of
– More efficient data compression : HEVC, SHVC and VP9
evolutions will set new bar in mobile video
– Delivery mechanisms : DASH
• As mobile and TV video experience converges, emerging
technologies gain importance:
Miracast, Alljoyn, DIAL, AirPlay
• High quality video post-processing is needed to
compensate the gap of video and display resolution.
Mobile Video Dynamics
13
14. IMTC 20th Anniversary Forum – Porto, Portugal
Adaptive Video Streaming
Quality is adapted to changing network conditions.
Videos encoded to
different streams at
different quality
Stored or
generated on-
demand
Device selects
optimal stream
Low
Video SegmentsQuality
Med
High
Time
Internet
(CDN)
6.3 billion mobile connections worldwide Mobile phone has become the most used product on the planet. In 2010, the average person looks at the phone 150 times per day (i.e. every six and a half minutes of every hour we are awake!) It is expected that Mobile web will overtake PC web in 2014
Here in this table we are seeing %48 CAGR for video streaming which is higher than both file sharing traffic, as well as web data including browser and similar applications using internet connectionsInternet video continue to grow and will reach %58 of all internet traffic by 2015Petabyte = 1M GB or 1000 Terabytes
In order to compare how each new generation of codecs improve video compression efficiency let’s assume we have a digital video file that is about 100MB and encoded with MPEG2 codec. Starting from original source file if I were to go back and encode same video using MPEG-4 Advanced Simple Profile I would have same video for 75MB. Similarly 56 MB with H.264 High profile and 33MB with upcoming HEVC standard. So in summary in the span of 20 yearsData required for presenting highest quality video increased by 1500xWhereas despite numerous advancements in video coding standards codec compression efficiency from H.261/H.262 to HEVC only got 3x or 4x better.
Video codec standards now are more embedded friendly in HEVC than H.264, like CABAC (context-based adaptive binary arithmetic coding)which everyone has hard time in implementing it in mobile video.
For video technology we are putting our focus on Covering a wide range of standard and proprietary codecs and formats for improved multimedia experienceScalability across MSMs. In the past we used mainly DSP architecture in the past . For scalability with optimized power scheme we are migrating to all hardware accelerated architecturesEnhancing visual quality through post and pre-processing. This is another area we put tremendous investment with silicon and algorithm developmentFor easy 3rd party integration we are supporting all popular frameworks including opencore, stagefright, directshow