This is a talk at Taipei Video Tech #5. The presentation illustrates how a Japanese streaming service handles chaotically supplied contents for 24/7 linear streaming. It also describes the background behind the fact that unique circumstances between the video streaming industry and broadcasting industry in Japan create the chaos.

1. A Japanese Way to Maintain Constant Quality
on Streaming Chaotically Supplied Contents
Yusuke Goto, AbemaTV, Inc.
- 2019.12.27 Taipei Video Tech #5 -

2. Yusuke Goto
五藤 佑典
https://ygoto3.com/
@ygoto3_
● Majored in Graphic Design at California State University, San Bernardino
● Software engineer @ CyberAgent, Inc. and AbemaTV, Inc.
● AbemaTV Development Division
○ Lead New Device team
○ Lead Streaming Client team
○ A Developer Expert in video technology and product design

3. Everything about video tech in Japan is unique.
Unique practices, original specifications ...

4. Everything about video tech in Japan is unique.
Unique practices, original specifications ...

5. A Japanese video streaming service, AbemaTV.
Its workflow is also affected
by the unique practices and specifications.

7. is a company and an internet TV station

8. The Company
Internet media
● Blogging platform
● Music streaming
● Game
● Curation
● Ad product
Development
● Ad agency
Broadcasting
● News
● Documentary
● Drama
● Anime
● Sports
● Music
● Movies
Internet TV station

9. The Service

10. https://abema.tv/

11. No.1 Japanese Internet TV Service

12. No.1 Japanese Internet TV Service

13. No.1 Japanese Internet TV Service
2.3 Tbps
(via Akamai’s Adaptive Media Delivery)
Japan’s highest traffic

14. 20 Channels * 24/7 + 18,000 Original Episodes
20 channels 24/7

15. 20 Channels * 24/7 + 18,000 Original Episodes
All Free
No need to sign up. Just visit and watch.
*premium features are additionally available
100% Professional
The contents are 100% produced by
professional experts in the industry.
Completely Programmed
Streamed 24/7 like broadcasting.
A Variety of Categories
Original live contents / drama / news /
anime / sports / etc.

16. We Do Everything E2E
● Studio work
○ Shooting
○ Editing
● Production
○ Pre-production
○ Post-production
● Mastering
● Development
○ The streaming system
○ The applications
○ The ad system
● Ad production

17. Our workflow is complex ...

18. Recording Editing
MP4MXF
Content
Operation
Packaging
HLS
MPEG
-DASH
MP4

19. Recording Editing
Buying
Converting
MP4MXF
MP4
MPEG 2
MOV
MXF
MP4
AVI
WMV
Content
Operation
Packaging
HLS
MPEG
-DASH
MP4

20. Recording Editing
Buying
Converting
MP4MXF
MP4
MPEG 2
MOV
MXF
MP4
AVI
WMV
Content
Operation
Packaging
HLS
MPEG
-DASH
MP4
Ad agents / Post-production
MP4
Ad
Operation
MP4

21. Recording Editing
Buying
Converting
MP4MXF
MP4
MPEG 2
MOV
MXF
MP4
AVI
WMV
Content
Operation
Packaging
HLS
MPEG
-DASH
MP4
Ad agents / Post-production
MP4
Ad
Operation
MP4

22. If you don’t do anything ...

23. -24.0
LUFS
Linear Streaming and Server-Side Ad Insertion
CFR
Progressive
Interlaced
-18.0
LUFS
VFR
16:00 - 17:00
Slot A - Purchased
17:00 - 18:00
Slot B - Recorded
18:00 - 20:00
Slot C - Live
20:00 - 21:00
Slot D - Recorded

24. Example : AbemaTV’s Ad Regulation for Submission
Duration 15s/30s/60s/90s/120s/150s/180s/300s
Container Format MP4
Frame Size Full HD (1920 x 1080)
DAR 16:9
Video Bitrate (avg.) 20 Mbps
Video Encoding Mode 2-pass encoding
Scan Type Progressive scan
Video Codec H.264
Frame Rate 29.97 fps
Profile High Profile / Main Profile
Audio Channels Stereo 2ch
Audio Codec AAC-LC
Audio Bitrate 320 Kbps
Audio Bit depth 16 bits
Audio Sampling Freq 48 kHz
Audio Peak Level >= -3 dDBFS
Loudness (avg.) -18.0 LUFS ± 1LU

25. However, it’s not easy to follow the regulation

26. Sources Come from Everywhere
● Broadcasters
○ Their workflow is much simpler and
fixed
● Masters for BD/DVD
○ Too much info
● Old contents
○ Legacy codec
■ Can’t handle
w/ the latest software
● Sources originally for another purpose
○ Digital signage
○ Web video
■ Fewer framerate
■ VFR
○ Archives of game streaming video
○ Hardware-dependent codec
■ Via hardware-recording
● Multi-language sources
○ Multi-track
■ Primary tracks platform-dependent
● etc.
○ All the video sources that
are not optimized for streaming

27. Chaotic Sources
MP4
H.264
AAC-LC
MOV
ProRes 422
Linear PCM
MXF
XDCAM HD
422
Linear PCM
AVI
Canopus HQ
DV
Linear PCM
WMV
MPEG 2
MPEG 2
Linear PCM
MPEG Audio
Recording
Buying
Ad agents / Post-production

28. Mastering for AbemaTV
MPEG2
MXF
MOV
MP4
AVI
WMV
MPEG-2 TS
fMP4
Chaotic

29. Need a mezzanine

30. Mastering for AbemaTV
MPEG-2 TS
fMP4
MP4
(mezzanine)
MPEG2
MXF
MOV
MP4
AVI
WMV
Chaotic

31. Why MP4 ?
● Good for our needs
○ Small file size
○ Good compression for 4:2:0 chroma subsampling
■ Good enough as a de facto media format
○ Decodable everywhere
■ Playable everywhere
○ Fast transcoding
○ Simple structure
■ Easy to handle GOP
■ Easy to convert sources to I-frame-only data

32. Why MP4 ?
● MP4 already seems to be the standard for delivery
○ Major media solution providers primarily support MP4
■ AWS Elemental MediaLive
● mp4 only
■ Harmonic
● mov/mxf/mp4 - (mp4 is the stablest)
■ We do not use these solutions, but good to have other options

33. However, it’s not easy to generate a good-quality mp4

34. MP4 is hard to handle
● MP4 is an effectively compressed format
○ Easily succeeds negative factors from the original sources
■ Broken decode order
■ Wrong header info
Not all the content operators are video experts

35. Auto QC / Auto transcoding

36. Recording
Buying
MXF
MPEG 2
MOV
MXF
MP4
AVI
WMV
Ad agents / Post-production
MP4
Auto QC / Auto transcoding
through MAM

37. Recording
Buying
MXF
MPEG 2
MOV
MXF
MP4
AVI
WMV
Ad agents / Post-production
MP4
Auto QC / Auto transcoding
through MAM

38. A MAM solution is good, but ...
● Does not support all the media formats we receive
● MAM companies are usually not familiar with the Japanese manner
○ e.g., DALET is a French company
○ 30 fps / NTSC ? No, PAL does not use it

39. A semi-auto QC / semi-auto converting step before MAM

40. Recording
Buying
MXF
MPEG 2
MOV
MXF
MP4
AVI
WMV
Ad agents / Post-production
MP4
Auto QC / Auto transcoding
through MAM
Semi-auto QC / Semi-auto converting
Semi-auto QC / Semi-auto converting
Semi-auto QC / Semi-auto converting

43. Judging the Scan Type
Headers tell lies

44. Judging the Scan Type
Headers tell lies

45. Judging the Scan Type
ffmpeg -ss {duration/2} -t {duration*0.1} -vf idet ...
isInterlaced = (TFF + BFF) / (TFF + BFF) + Progressive > 0.2

46. -18.0
LUFS
Now it’s constant
CFR
Progressive
Progressive
-18.0
LUFS
CFR
16:00 - 17:00
Slot A - Purchased
17:00 - 18:00
Slot B - Recorded
18:00 - 20:00
Slot C - Live
20:00 - 21:00
Slot D - Recorded

47. Loudness
-18.0 LUFS
● AbemaTV’s original specification
○ ARIB T032 specifies -24.0 LUFS
■ Not enough for mobile devices
○ Optimized for playback on mobile devices
○ Follows AES’ Recommendation for Loudness of Audio Streaming &
Network File Playback
■ -16.0 to -20.0 LUFS

48. Constant media formats = Constant quality?

49. Constant media formats = Constant quality?
Not enough

50. Various Kinds of Platforms to Support

51. Various Kinds of Platforms to Support
● iPhone
● iPad
● Apple TV
● Android Smartphone
● Android Tablet
● Android TV
● Fire Tablet
● Amazon Fire TV
● Google Chrome
● Mozilla Firefox
● Internet Explorer
● Microsoft Edge
● Apple Safari
● Google Chromecast
● Google Daydream View
● Amazon Echo
● LINE Clova Desk
● SONY BRAVIA
● Panasonic VIERA
● TOSHIBA REGZA
● Sharp AQUOS
● FUNAI
● LEOPALACE21
Life Stick
● CCCAIR Air Stick
● ...

52. Buffering Capacities Differ by Platform
Chrome Firefox Safari Edge Chromecast
Video 150 MB 100 MB 290 MB Unknown 30 MB
Audio 12 MB 15 MB 14 MB Unknown 2 MB
https://developers.google.com/web/updates/2017/10/quotaexceedederror

53. Buffering Capacities Differ by Platform
Chrome Firefox Safari Edge Chromecast
Video 150 MB 100 MB 290 MB Unknown 30 MB
Audio 12 MB 15 MB 14 MB Unknown 2 MB
https://developers.google.com/web/updates/2017/10/quotaexceedederror

54. QuotaExceededError
https://cs.chromium.org/chromium/src/media/base/demuxer_memory_limit.h

55. Beauty > Bitrate

56. Beauty > Bitrate
$ ffmpeg -maxrate ..k -bufsize ..k ..

57. Detecting the Risk of Exceeding the Memory Limit

58. Detecting the Risk of Exceeding the Memory Limit

59. Need to know buffering transition
Detecting the Risk of Exceeding the Memory Limit

60. Visualizing the Buffering Transition

61. df = pandas.DataFrame(keyframes)
...
ddf = df.diff()
df['diff'] = ddf['pos'] / ddf['time']
df.index = pandas.TimedeltaIndex([pandas.Timedelta(minutes=x) for x in df['time']])
up_sampled = df['pos'].resample('S').mean().interpolate('linear')
upx = up_sampled.copy(deep=True)
upx.index -= pandas.Timedelta(f'{buffer_length}s')
upx = upx[upx.index > pandas.Timedelta('0s')]
x = pandas.DataFrame()
x[f'{buffer_length} seconds'] = upx - up_sampled

62. Conclusion
● To keep constant quality on streaming chaotically supplied contents
○ Developed a unique and original specifications
○ Partly adopt parts of various standardized specifications
○ Developed tools and workflow to patch gaps

63. Yusuke Goto
五藤 佑典
https://ygoto3.com/
@ygoto3_
Thank you !