Video Streaming with NGINX

Keine Notizen für die Folie

Thank you Andrew. Over the years different methods have been devised to deliver video content. In this webinar we will take a look at some of these methods and then I will talk about how you can use NGINX for video delivery.

Today the methods I will be talking about are file download, progressive download, pseudo-streaming, adaptive bitrate streaming and RTMP.

I’m going to start by discussing how these methods have evolved over time, focusing on HTTP and so for that I will cover file download, progressive download, pseudo streaming and adaptive bitrate streaming.

We will start with the simplest way of delivering video content and that is to download the complete media file to your local hard drive and then playing the video. The process is very straight forward. The file is downloaded from the video server to the local machine and once the download completes the video can be played. This means that you have to wait for the entire file to be downloaded before playback can begin and if the file is large or your link is slow this can take a very long time and if you are unlucky and run into connection errors, the download may never complete. So there is obviously room for improvement.

An improvement over simply downloading a file is progressive download. In this scenario, the meta data for the media file (that is the information needed by the video player to know how to play the video) is downloaded first, then the download of the rest of the file begins and after a certain amount of data is buffered by the player it is able to start playing the video. The file will continue to download and as long as the file downloads faster then the video plays, you will experience smooth playback. If your link is slow, then the playback may stall while the player waits for more data to be buffered. I’m sure you have all encountered this phenomena. So this method does not handle slow connections well and you aren’t able to use trick play features such as fast forward, jumping to a certain part of the video, etc.

The key advantage to this approach over file download is that you don’t have to wait for the download to complete before playing the video so the video can be played while the file is still being downloaded. This is made possible by moving the meta data to the front of the file so the player has the information it needs to play the file. So this a step in the right direction, but still is not ideal.

Pseudo-streaming is an advance on simple progressive download in that the player can ask for specific byte ranges, thus enabling some trick play features and also the player can buffer the data without necessarily having to write the data to disk.

The download no longer has to be continuous from the start to the end of the file. The player can now stop the stream and move to a new point in the data.

A big step forward came with the advent of adaptive bitrate streaming. This delivers the media in a different way then the progressive download and pseudo-streaming methods we have been talking about. Here a media file is repackaged as a series of smaller segments, each with just a few seconds of video. A playlist or manifest file is then created that acts as a table of contents for the segments. What adaptive bitrate refers to is that there can be different versions of the same video, each encoded with a different bitrate. Think of a having a high definition and low definition version of the same video and even versions in between. The high definition version would be appropriate for a high bandwidth link and the low definition version appropriate for a low bandwidth link. With adaptive bitrate streaming the player can switch seamlessly between bitrates depending on the speed of the connection and can easily skip forward or backward through the segment files. By being able to adapt to network conditions using different bitrates the client can see a better quality of service, and the player can also more easily recover from network interruptions.

Now let’s look at two popular ways of doing adaptive bitrate streaming over HTTP and how they work. HTTP Live Streaming, or HLS, was developed by Apple. With HLS, the mp4 file is re-packaged into a playlist with the extension of m3u8 and a series of short segment files with an extension of ts. The playlist is used to identify the segments. A player first requests the playlist and then streams the segment files. HTTP Dynamic Streaming, or HDS, comes from Adobe and also requires that the content to be re-packaged, but in this case into a manifest file, an index file and a fragment file. If you have encoded the video with multiple bitrates, then you will have a master playlist or manifest file that points to a playlist or manifest file for each bitrate.

Hopefully this diagram will clarify things. This shows the file structure used by HLS, assuming there are 2 bitrates available. The master playlist points to a playlist for each bitrate and these playlists point to the segment files. I only show the first 2 segment files. The number of segment files will depend on the length of the segments and the total length of the video. If you only have a single bitrate, then there would be no master playlist.

Here we see the file structure for HDS. Again I have assumed 2 bitrates. So there is a master manifest that points to the manifest for each bitrate. These manifest then point to an index which points to the fragment file. Again if you have only a single bitrate there will not be a master manifest.

Another way of doing video streaming that doesn’t use HTTP, is to use the Real Time Messaging Protocol or RTMP. This can support both live video and video on demand and is supported across various desktop browsers by means of a Flash plugin, and it is also supported by many other software programs.

Now that we covered a bit about video streaming in general, let’s take a look at how you can use NGINX to support your video streaming requirements.

First off, what is NGINX? NGINX is a high performance web server of static content, including video and also a full reverse proxy or Application Delivery Controller with many advanced features. For dynamic content it can connect to application servers over HTTP, FastCGI, uWSGI, Passenger and other methods.

Started as on open source project about 10 years ago, NGINX has gained a justified reputation as a high performance web and proxy server and is seeing widespread adoption, now powering over 140 million websites.

Including over 20% of the top 1 million busiest websites and almost 40% of the top 1,000 busiest websites, which now makes NGINX the most used web server for the top 1,000 websites.

As I said, NGINX started out as an open source project and that continues today, but in August of last year we released the first commercial version of NGINX, NGINX Plus, built on top of the open source version.

NGINX Plus brings advanced features, mainly around enterprise level load balancing and video streaming and also comes with support. As we go through this presentation, we will talk about some of the NGINX Plus features specific to video streaming.

There are four builds of NGINX Plus that are available. NGINX Plus is the core build and all the code in this build has been developed by the Nginx team in Moscow, the same team that develops the open source version, and is fully supported. The NGINX Plus Lua build adds the popular 3rd party module that allows you to do scripting in the Lua programming language. The NGINX Plus Extras build adds several additional 3rd party modules. These 3rd party modules are supported on a best effort basis. And finally there is the NGINX Plus Streaming Media Server. This is a stripped down version of NGINX Plus, with just those features needed for video streaming.

So now let’s talk about how you can use NGINX to do video streaming.

We’ll start with HLS. NGINX Plus includes a HLS module. You may remember that when I talked about HLS previously I mentioned that the content needs to be re-packaged into a playlist and segment files. One of the nice features of the NGINX Plus HLS module is that it can do this re-packaging for you, on-the-fly. So you can take your existing mp4 files and start serving them with HLS without having to do anything to the files. Just point NGINX Plus at your mp4 files and NGINX Plus will do the rest. If you want to support multiple bitrates, then all you need to do is create a master playlist pointing to the playlist for each bitrate. These playlist files won’t actually exist, but NGINX will create them automatically at run time.

Now I will start showing demos using NGINX to deliver video using some of the methods I have been talking about. First, let’s see HLS in action. You will see that if we look in the /mp4 directory, we just have the file catanddog.mp4, but if we go to our browser and enter catanddog.mp4.m3u8, which is the URI of the playlist, we see our video even though that file doesn’t exist. This is because NGINX created it on the fly. If we look in the access log, we will see the request for the m3u8 file along with the segment files, which were also created on-the-fly by NGINX.

Show /mp4
Browser http://vidtest/mp4/catanddog.mp4.m4u8
Show /etc/nginx/logs/hls.access.log

Now let’s move on to HDS. HDS is a different then HLS in that the mp4 encoded files must be re-packaged with an Adobe tool, like f4fpackager. Once this is done the f4f module that comes with NGINX Plus will handle the HDS streaming requests.

If we look in the hds directory where I have put the HDS files, we will see the manifest, index and segment file. To use HDS we need to embed a flash player in a webpage, which is what we have done here and we see the same video we saw during the HLS demo. If we look at the access log for these requests we will see many request for cat1-Frag1, cat1-Frag2 and so on and the HDS module has the logic to know how to handle those requests.

Show /hds
Browser http://test-f4f.html
Show /etc/nginx/logs/hds.access.log

Now, adaptive bitrate streaming is great and I’ve talked about some of its advantages, but not all client devices may support it, so you may still want to support pseudo-streaming and that is something supported in NGINX open source, but NGINX Plus adds the ability to control the bandwidth used by a client over a connection. mp4_limit_rate is multiplied by the bitrate, so in this example we will allow 1.2 times the encoded bitrate for each connection. mp4_limit_rate_after tells NGINX Plus to delay starting to limit the bandwidth for the specified time after the download begins. So in this example, 15 seconds. This allows the client to buffer an initial amount of video before before the limit kicks in.

As we saw when we looking in the /mp4 directory, we had just the one mp4 file. For the previous demo, we requested the playlist, but if request the file directly, then the video will be played using pseudo-streaming.

Browser http://vidtest/mp4/catanddog.mp4

One alternative to HTTP streaming I mentioned was RTMP and NGINX has a module for this too.

Some of the notable features of this module are that, although it does not support transcoding (converting live stream to a different codec, size, bitrate, etc.), it does support external process execution using transcoders such as ffmpeg, avconv, etc. These processes usually connect to NGINX as a subscriber, receive the stream and publish it back transcoded under a different name.

Also for certain streaming events (publisher connect/disconnect or subscriber connect/disconnect) the module can make HTTP subrequests. This allows you to add business logic. For example to check a database entry and drop the RTMP connection if the check fails. RTMP secure links use this feature as well.

And when a client publishes the stream, the module can push the stream to another server. Or when a subscriber tries to play a missing stream the module can pull the stream from a remote server.

With the proliferation of devices, each with different functionality, you are probably going to have to support multiple delivery methods. For example, for Flash players you are going to need to support HDS and/or RTMP and for iPhones you will want to support HLS and since not all devices may support these delivery methods you will probably also want to still support psuedo-streaming. Here is an example configuration that will take the same mp4 file and serve it via pseudo-streaming if a client requests has an extension of mp4 since it is requesting the file directly and will serve it via HLS if the client request has an extension of mp4.m3u8 since the client is requesting an HLS playlist. You can also have a configuration that has NGINX serving HDS, RTMP, etc. For example, the configuration I am using also support HLS.

As you saw when I did the demos, if you have access logging turned on, like I do, you will get an entry for each individual request during the streaming. That was good for my demos, to show you what was going on, but for a production system, with possibly thousands, tens of thousands or even more clients, you can image how many log entries would result. This can cause very high disk usage and excessive disk operations and also is too granular to use for billing if that is necessary. NGINX Plus introduces the ability to aggregate log entries based on criteria you specify so you get higher level log entries rather then one for each segment. This results in less disk storage, less disk I/O and data that is much more useful for billing.

Here is an example configuration that does log aggregation for HDS, logging based on client IP and user agent and once a given client IP and user agent is inactive for 30 seconds the session will be considered complete and the log entry will be written.

When I showed you the HLS demo previously, I showed you the detailed access log, but I also had log aggregation configured. Here again is the detailed log, and here is the aggregate log for the same session.

Show /etc/nginx/logs/hls.access.log
Show /etc/nginx/logs/hls.log

Here is the full configuration file I used for all the demonstrations I’ve shown.

Show /etc/nginx/conf.d/video.conf

As we come to the end, let’s do a quick review. We talked about a number of ways that can be used to deliver video, including HLS, HDS, pseudo-streaming, progressive download and RTMP. And given the vast array of different mobile clients, desktop browsers and other clients, you will probably need to support many of these methods. NGINX Plus supports all of them, and does it in a way that is easy to configure.

Now we will open the floor for questions.

Thank you for taking the time to attend this session. I hope it was of use to you and I hope you attend some of our future webinars. You will find additional information about video streaming at nginx.com. And it is easy to test out either the open source version of NGINX or NGINX Plus.

Video Streaming with NGINX

NGINX, Inc.

Video Streaming with NGINX