Apple TV offers a friendly SDK, full of familiar view controllers and Foundation classes, with everything an iOS developer needs to develop their own streaming channel. Except for… you know… the streaming part. In this session, we'll look at how Apple's HTTP Live Streaming video works -- from flat files or live sources -- and how to get it from your computer to a streaming server and then to an Apple TV. We'll also look at common challenges for building streaming channel apps, like serving metadata, protecting content, and supporting single sign-on.

1. Building a Streaming
Apple TV app
Chris Adamson (@invalidname)
CocoaConf DC • September, 2016
Slides will be available at slideshare.net/invalidname
Code will be available at github.com/invalidstream

6. “The future of TV is apps.”
“The future of apps is TV.”

7. Video
(how do you even?)

9. Digital Media
• Codecs — How we encode (represent) audio or
video in a digital form
• AAC, Apple Lossless, H.264, ProRes
• Containers — How we deliver encoded media
• MP3, .aac, Shoutcast, .mov, .mp4

10. Codecs
• From “coder / decoder”
• Simplest: PCM (audio), M-JPEG (video)
• Modern: AAC, H.264/AVC
• Next gen: H.265 (HEVC), VP9

11. Inter-frame compression

12. Containers: QuickTime
• Dates back to the early 1990s
• Originally used Mac dual-fork files
• Content agnostic: nearly any media content can
be contained in a QuickTime file
• MPEG-4 adopted QuickTime as basis of its file
format (1998)

13. QuickTime: Great for Editing

14. QuickTime: OK for playback
• Resolve all references into a single file (flatten)
or export (possibly re-encoding) to a file
• Dual-fork approach abandoned to support
QuickTime for Windows (and later, Mac OS X)

15. QuickTime: Bad for
Streaming
• Metadata at end of file meant you needed the
whole file
• First solution: “Quick-Start”, which put metadata
at beginning of file
• Later: QuickTime Streaming Server
• Also-ran to RealPlayer, Windows ASF, Flash

16. Streaming
• Different from progressive download in that the
client does only wants the media for the current
playback time
• Live sources are possible

17. Bad old streaming
• Real, ASF, Flash, QTSS – socket-based
connections
• Hard to scale
• Bad for mobile
• Not on port 80; frequently blocked

18. Adaptive streaming
• Send “stream” as a series of small files over
HTTP, expect client to reassemble them
• Easy to scale
• Good for mobile / inconsistent networks
• Runs on port 80; never blocked

19. HTTP Live Streaming
• Apple’s de facto standard for adaptive streaming
• Supported on iOS (iPhone, Apple TV) from day
one
• Also: QuickTime, Safari, Microsoft Edge, Flash,
Android, Roku, etc
• Standards-based competitor: MPEG-DASH

20. Demo

22. Streaming playback
@IBAction func handlePlayButtonTapped(_ sender: AnyObject) {
guard let demoItem = demoItem else { return }
self.playerVC?.player?.pause()
let playerVC = AVPlayerViewController()
playerVC.showsPlaybackControls = true
let player = AVPlayer(url: demoItem.url)
playerVC.player = player
present(playerVC, animated: true)
player.play()
self.playerVC = playerVC
}
This is identical to local file playback

23. HLS, how do you even?
• Source media is split into short (~10 second)
segments, as MPEG-2 transport stream (.ts) files
• iOS 10 / macOS Sierra support “Fragmented
MP4”, which is potentially MPEG-DASH
compatible
• An .m3u8 playlist provides a manifest of files to
be played back

24. #EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:10.00000,
fileSequence0.ts
#EXTINF:10.00000,
fileSequence1.ts
#EXTINF:10.00000,
fileSequence2.ts
#EXTINF:10.00000,
fileSequence3.ts
#EXTINF:10.00000,
fileSequence4.ts
#EXTINF:10.00000,
fileSequence5.ts
#EXTINF:10.00000,
fileSequence6.ts
#EXTINF:10.00000,
fileSequence7.ts
#EXTINF:10.00000,
fileSequence8.ts
#EXTINF:10.00000,
fileSequence9.ts
#EXTINF:7.40000,
fileSequence10.ts
#EXT-X-ENDLIST

25. Streaming Tools

26. mediafilesegmenter
• Takes a source MP4 file and splits it into
segment files and an .m3u8 playlist
• Resulting directory can be hosted on any http
server to provide stream

27. Demo

29. Why is this better than a flat
file?
• Easier to skip around
• Can be encrypted / DRM’ed
• Also, about that bitrate…

30. Variant Playlists
• Allow you to encode at multiple bitrates
• Client determines at runtime if it is keeping up,
and whether to request a different variant for the
next 10 sec segment
• Instead of “buffering” stalls, viewers just see
degraded image/sound quality

32. variantplaylistcreator
#EXTM3U
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1269452,BANDWIDTH=1304420,CODECS="mp4a.40.2,
avc1.4d4016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
wifi-high/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=5172486,BANDWIDTH=5330176,CODECS="mp4a.40.2,
avc1.4d401f",RESOLUTION=960x540,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
broadband-high/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2599260,BANDWIDTH=2677271,CODECS="mp4a.40.2,
avc1.4d4016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
broadband-low/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=757070,BANDWIDTH=782983,CODECS="mp4a.40.2,
avc1.42e016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
wifi-low/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=474503,BANDWIDTH=507149,CODECS="mp4a.40.2,
avc1.42e015",RESOLUTION=428x240,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
cellular-high/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=224956,BANDWIDTH=243197,CODECS="mp4a.40.2,
avc1.42e015",RESOLUTION=428x240,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONE
cellular-low/prog_index.m3u8
#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=33400,BANDWIDTH=33547,CODECS="mp4a.
40.2",CLOSED-CAPTIONS=NONE
audio-only/prog_index.m3u8

33. Demo

36. Compressor’s HLS bitrates
Table 1
Broadband high 5000
Broadband low 2500
Wi-fi high 1250
Wi-fi low 735
Cellular high 450
Cellular low 225
Audio only 32

37. TN 2224
https://developer.apple.com/library/ios/technotes/tn2224/_index.html

38. TN 2224 Bitrates

39. Livestreaming!

42. HLS livestreams
• Exactly the same as the VOD streams we’ve
seen before
• Support variant playlists and encryption
• Difference: the playlist doesn’t have a #EXT-X-
ENDLIST
• Client knows to refresh the playlist periodically
to fetch new segment files

43. mediastreamsegmenter
• Works just like mediafilesegmenter to create the
segment files and a playlist
• Takes as input an MPEG-2 transport stream
• Nothing in Apple’s developer toolchain can
actually create such a stream

44. Streaming servers
• Services that create the stream files from an
input stream you provide
• Can also transcode to lower bitrates and
provide a variant playlist
• Can also transmux to non-HLS formats
• Generally you send them RTMP

45. Open Broadcast Software
https://obsproject.com

46. Demo

47. Wowza
http://wowza.com

48. Wowza
• Top streaming service provider
• Wowza Streaming Engine — runs on your
server (real or virtual)
• Wowza Streaming Cloud — streaming as a
service

52. Wirecast
http://telestream.net/wirecast/

53. Wirecast
• Proprietary streaming production software
• Mixes shots, captures from cameras, iOS
devices, or screen, highly customizable
• Prices start at $500
• Watch for a 1/3 off sale on Black Friday

54. Demo

55. Securing Streams
• With -k option, mediafilesegmenter and
mediastreamsegmenter will AES-128 encrypt
segments, using keys from the specified location
• FairPlay Streaming for HLS provides severe
DRM.
• See WWDC 2015 session “Content Protection
for HTTP Live Streaming”

56. Takeaways
• Video streaming is the best thing the Apple TV
does
• HLS streams are just flat files on a web server
• Create them with mediafilesegmenter or a
streaming server
• For lifestreams RTMP to the streaming server
with OBS, Wirecast, etc.

57. Building a Streaming
Apple TV app
Chris Adamson (@invalidname)
CocoaConf DC • September, 2016
Slides will be available at slideshare.net/invalidname
Code will be available at github.com/invalidstream