Overview of Apple\'s HTTP Live Streaming (HLS)

1. (Apple) HTTP Live Streaming
(HLS)
Dynamically adapting video
streaming to network conditions
Auro Tripathy

2. Outline
• The basics of adaptive streaming
• A Wireshark view
• New tags defined by the HLS protocol
• A comparison with other adaptive
streaming protocols
November 8, 2011 Auro 2

3. HLS Streaming Architecture
Content Preparation
HTTP Content Server
Media Media
Encoder Segmenter Index HTTP
File Cache
Segment
November 8, 2011 Auro 3

4. The Basics of HLS
• Protocol for transmitting alternate versions
(i.e., bitrates) of the stream
• Media can be served soon after its created
– Not actually “live”, but near real-time
• Specify a presentation with the URI of a
playlist file
• A playlist file is an ordered list of media URI
and info tags
• Each media URI refers to a media file which
is a segment of a single contiguous file
November 8, 2011 Auro 4

5. How does the client play the
stream?
• Client first obtains the playlist file
– Initial playlist file may contain alternates (see
next slide)
• Client decides which alternate playlist file
to play
• Gets and plays each media file in the
playlist.
November 8, 2011 Auro 5

6. Server Provides Alternate
Playlists to Clients
Server
Alternate 1
Index
File TS Segment
Index Alternate 2
File Index
w/alternates File TS Segment
Alternate 3
Index
File TS Segment
November 8, 2011 Auro 6

7. M3U format – the History
• M3U stands for MP3 URL.
• Files that contain the .M3U file-extension are
MP3 playlist files
– commonly referred to as Winamp playlist files
• Important to remember, this file does not
contain media, only references to media
• Files whose name end with .m3u are
encoded with US-ASCII.
• The unicode version of "m3u" is "m3u8",
which uses UTF-8 unicode characters
November 8, 2011 Auro 7

8. Extended M3U
• Defines two tags: EXTM3U & EXTINF
• First line must be #EXTM3U
– This distinguishes it from a M3U file
• EXTINF is a record marker that describes the
media file identified by the URI that follows it.
• Example
#EXTM3U
#EXTINF:<duration>, <title>
http://media.example.com/entire.ts
November 8, 2011 Auro 8

9. Server Tasks to Achieve HLS
• Encode video in H.264 format and Single
Program Transport Stream “container”
– Encode in different bitrates
– taken from live feed or from a file
• Use a stream segmenter to generate short
“chunks” of content – typically 10 seconds
each - and generate a playlist file (m3u or
m3u8) indicating from where to download the
chunks
• Serve via HTTP server, and provide
appropriate caching for performance
November 8, 2011 Auro 9

10. Who supports HLS?
• Apple, of course
– Authored the protocol
http://tools.ietf.org/html/draft-pantos-http-live-
streaming-07
• Support available in Android 3.0
– http://developer.android.com/sdk/android-3.0-
highlights.html
November 8, 2011 Auro 10

11. Deconstructing HLS with
Wireshark

12. Server Offering Multiple Playlist Files
Different encodings of the same presentation
Client request
Server offers four
alternative playlists at
200 Kbps, 311.1 Kbps,
484.4 Kbps & 737.8 Kbps
November 8, 2011 Auro 12

13. Client fetches the first Playlist File
Client request 200 Kbps
playlist
Duration of segment in
seconds; appears once
Segment URLs
November 8, 2011 Auro 13

14. Note - Playlist file ends with a #EXT-X-ENDLIST
Indicates no more media files
will be added to the Playlist file
November 8, 2011 Auro 14

15. Client fetches the second Playlist File
November 8, 2011 Auro 15

16. Client fetches the 3rd Playlist File
November 8, 2011 Auro 16

17. Client fetches the 4th Playlist File
November 8, 2011 Auro 17

18. Client requests a Media File to Play
November 8, 2011 Auro 18

19. Summarizing the Sequence
Server Client
Respond with four alternate playlists
Location of all segments in Playlist
Location of all segments in Playlist
Respond with A/V data
November 8, 2011 Auro 19

20. VLC Plays HLS Test File
http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8
November 8, 2011 Auro 20

21. New Tags defined in HLS

22. HLS defines the following new tags
Each with it own attribute list
• EXT-X-TARGETDURATION
• EXT-X-MEDIA-SEQUENCE
• EXT-X-KEY
• EXT-X-PROGRAM-DATE-TIME
• EXT-X-ALLOW-CACHE
• EXT-X-PLAYLIST-TYPE
• EXT-X-STREAM-INF
• EXT-X-ENDLIST
• EXT-X-DISCONTINUITY
• EXT-X-VERSION
November 8, 2011 Auro 22

23. Tag – EXT-X-TARGETDURATION
• EXT-X-TARGETDURATION:<s>
– Specifies the maximum media file duration
– Appears only once in the playlist file
– <s> is an integer indicating the duration in
seconds
November 8, 2011 Auro 23

24. Tag – EXT-X-MEDIA-SEQUENCE
• Each media file URI in a playlist must have
a unique integer number
• Sequence # := Sequence # of URI that
preceded it + 1
• Tag indicates the sequence number of the
first URI that appears in the playlist file
• Format
EXT-X-MEDIA-SEQUENCE:<number>
November 8, 2011 Auro 24

25. Tag – EXT-X-KEY
• Contains information to decrypt media files that
follow it
• Format
#EXT-X-KEY:<attribute-list>
• METHOD attribute specifies encryption method
(AES-128 or NONE)
• URI attribute identifies the Key file
– Key file contains the cipher key to decrypt subsequent
media files in the Playlist
– Quoted string with URI
• IV attribute specifies the initialization vector to be
used with the key
November 8, 2011 Auro 25

26. Tag – EXT-X-PROGRAM-DATE-
TIME
• Associated with the next media file
• Should indicate a time zone
• Format
EXT-X-PROGRAM-DATE-TIME:<YYYY-MM-DDThh:mm:ssZone>
November 8, 2011 Auro 26

27. Tag – EXT-X-ALLOW-CACHE
• Indicates whether a client MAY or MUST-
NOT cache downloaded media files for
later playback.
• Tag applies to all segments in the playlist
• Format
#EXT-X-ALLOW-CACHE:<YES|NO>
November 8, 2011 Auro 27

28. Tag - EXT-X-PLAYLIST-TYPE
• Optional tag provides mutability
information about the playlist file
• Format:
#EXT-X-PLAYLIST-FILE:<Event|VoD>
November 8, 2011 Auro 28

29. Tag – EXT-X-ENDLIST
• Indicates no more media files will be
added to the Playlist file
• Occurs only once
• Format:
#EXT-X-ENDLIST
November 8, 2011 Auro 29

30. Tag – EXT-X-STREAM-INF
• Tag indicates that the next URI in the playlist
file indentifies another playlist file
• Format/example
#EXT-X-STREAM-INF:<attribute-list>
<URI of playlist file>
• Defined attributes
– BANDWIDTH
– PROGRAM-ID
– CODECS
– RESOLUTION
November 8, 2011 Auro 30

31. Tag – EXT-X-DISCONTINUITY
• Tag indicates an encoding discontinuity
between the media file that follows it and the
one that preceded it
• Reason
– File format change
– Number and type of tracks
– Encoding parameters
– Encoding sequence
– Timestamp sequence
• Format
– #EXT-X-DISCONTINUITY
November 8, 2011 Auro 31

32. Tag – EXT-X-VERSION
• Tag indicates the compatibility version of
the Playlist file
• One occurrence
• Format
#EXT-X-VERSION:<n>
November 8, 2011 Auro 32

33. Media Files
• Media files are formatted as MPEG-2
transport streams
• It’s a singe program transport stream
• Should have a PAT/PMT at the start of
each files
• Client should be able to handle any
encoding parameter change such as a
resolution change
November 8, 2011 Auro 33

34. Key Files
• The EXT-X-KEY tag has the URI and the
IV attributes to identify the key file and the
Initialization Vector
• If the IV is not present, client must use the
sequence number of the media file as the
IV
• Note, varying the IV increases the strength
of the cipher
November 8, 2011 Auro 34

35. Conclusion – Lots of upside with
the protocol
• Uses the HTTP protocol
– Firewall-friendly
– HTTP server are ubiquitous thus reducing
operational complexity
– Unifies the delivery for three screens
(TV/Mobile/Tablet)
– Scales, no session state on the HTTP server
• Chunked transfers allow the client to
compensate/optimize in near real-time for
varying network conditions
November 8, 2011 Auro 35

36. Conclusion – Potential
Downsides
• Choice of MPEG-2 single program
transport streams
– 188 byte packet structure with 4-byte header
creates non-negligible overhead
• VCR-like trick-modes (seek to a time
forward or backward) can cause
noticeable delays if the segment size is
large
November 8, 2011 Auro 36