Scaling API-first – The story of a global engineering organization
Realtime video streaming the open source way
1. MADRID · NOV 27-28 · 2015
Realtime video streaming
MADRID · NOV 27-28 · 2015
the opensource way
Iván Belmonte
2. MADRID · NOV 27-28 · 2015
Github: ibelmonte
Sysdivision
Twitter: @sysdivision
Website: http://sysdivision.com
Twitter: @ibelmonte
Iván Belmonte
3. MADRID · NOV 27-28 · 2015
Real time weather reporting system for outdoor sports.
2 main elements:
- HD cameras in surfing spots (beaches)
- Weather stations in surfing spots
GOAL:
Weather forecast typically fails. Spotfav gives you a
window to your spot, to check the weather conditions
before moving.
4. MADRID · NOV 27-28 · 2015
What forecast says the day before
5. MADRID · NOV 27-28 · 2015
What you expect when you take your car to go to the spot
6. MADRID · NOV 27-28 · 2015
What’s actually going on
7. MADRID · NOV 27-28 · 2015
IMPORTANT NOTE:
I’m a backend developer/sysadmin
I know nothing about video and audio processing
In this project I am A VICTIM
8. MADRID · NOV 27-28 · 2015
Video manipulation and transmission is complex
- Plenty of difficult concepts: encoding, muxing,
transcoding, encapsulation, bitrates, fps…
- Different media servers, different players, some codecs
may not be supported
- Several transmission protocols depending on your needs:
RTP, RTSP, RTMP, HLS, HDS, Shoutcast, HTML5…
Added complexity: LIVE streaming
9. MADRID · NOV 27-28 · 2015
Looking for a standard setup
Most common case: H264 over RTSP
Affiliates: get this if you want to get
connected to our network
10. MADRID · NOV 27-28 · 2015
There are several options
Commercial:
Adobe media server, Wowza, Flussonic, Flumotion…
SaaS:
Ustream, Justin.tv (defunct)…
Open source:
ffmpeg, ffserver, libav, Live555, Red5, crtmpd, vlc,
Erlyvideo…
11. MADRID · NOV 27-28 · 2015
OUR SCENARIO
1 Camera, H264 over RTSP
1. The goal
Embed the camera in a website and also in a smartphone app
2. Nice to have
Not permanently connected (connect on demand)
Tolerant to network cuts
3. Science fiction
Be able to filter by referral (avoid other websites to
embed our cameras)
12. MADRID · NOV 27-28 · 2015
STEP 1: WEB player (RTMP)
13. MADRID · NOV 27-28 · 2015
ROUND 1
A javascript serving a Directx (wtf?) script.
14. MADRID · NOV 27-28 · 2015
ROUND 1
Cons:
- The camera has a limited pool of connections available.
Visitors exceeding this poll get rejected.
Works?
- In some browsers
15. MADRID · NOV 27-28 · 2015
ROUND 2
A VLC cli process reading the RTSP, transcoding and
publishing an RTMP stream in a socket
17. MADRID · NOV 27-28 · 2015
ROUND 2
Works?
- YES… for 1 camera.
18. MADRID · NOV 27-28 · 2015
ROUND 2
Works?
- YES… for 1 camera.
What if more cameras?
- 1 VLC process for each camera.
19. MADRID · NOV 27-28 · 2015
ROUND 2
Round 2 version with multiple cameras
20. MADRID · NOV 27-28 · 2015
ROUND 2
Cons:
- A lot of processes in the server eating memory as hungry
animals.
- VLC is unable to recover from network cuts.
- Each VLC process opens a socket in a different port, which
in my case is a mess as I’m hosting the VLC processes inside
a container and then routing the outside in.
- VLC is permanently connected to the camera
21. MADRID · NOV 27-28 · 2015
AND THEN ONE DAY…
Adobe updates flash player, breaks my streamings
22. MADRID · NOV 27-28 · 2015
BROKEN STREAMING
Two possible reasons:
1. Something is wrong with my VLC command
vlc-wrapper -vvv h,p://camera.ip.address/cgi-bin/faststream.jpg?stream=full&fps=16&size=640x480&preview" -sout
'#transcode{vcodec=FLV1,vb=800,ab=32,fps=16,width=640,height=480,acodec=none}:duplicate{dst=std{access=h,p{mime=video/x-
flv},mux=ffmpeg{mux=flv},dst=LINUX.BOX.IP.ADDRESS:8080/stream.flv},dst=display}'
2. Something is wrong with the new version of Flash
23. MADRID · NOV 27-28 · 2015
BROKEN STREAMING
Two possible reasons:
1. Something is wrong with my VLC command
vlc-wrapper -vvv h,p://camera.ip.address/cgi-bin/faststream.jpg?stream=full&fps=16&size=640x480&preview" -sout
'#transcode{vcodec=FLV1,vb=800,ab=32,fps=16,width=640,height=480,acodec=none}:duplicate{dst=std{access=h,p{mime=video/x-
flv},mux=ffmpeg{mux=flv},dst=LINUX.BOX.IP.ADDRESS:8080/stream.flv},dst=display}'
2. Something is wrong with the new version of Flash
Of course option 2. I contacted Adobe:
- ME: your upgrade has a bug and it broke my streamings
- ADOBE: …(silence)
24. MADRID · NOV 27-28 · 2015
ROUND 3
A super cool media server capable to encapsulate H264 RSTP
into RTMP for flash players: Erlyvideo.
25. MADRID · NOV 27-28 · 2015
ROUND 3
Pros:
- Written in Erlang: amazingly light and fast
- Connects to the source on demand
- One process, several cameras
- Tolerant to network cuts (due to the previous point)
- Referrer filtering
Cons:
- Documented in russian (partially translated into
english). Support forums also 80% russian.
- HLS support (for smartphone apps) only supported in its
new commercial version: Flussonic.
26. MADRID · NOV 27-28 · 2015
STEP 2: Smartphone app (HLS)
27. MADRID · NOV 27-28 · 2015
Nginx RTMP module
An Nginx module capable to read streams in several formats
as input, store them and then serve them encapsulated in
other protocols (eg: HLS)
28. MADRID · NOV 27-28 · 2015
Nginx RTMP module
Works?
- YES, perfectly
Cons:
- Not very well documented
- Not tolerant to network cuts
- Unable to regain lost connectivity
33. MADRID · NOV 27-28 · 2015
What I would do now if starting from scratch
1. Do it the opensource way: there are powerful tools
2. Look for a single solution for both web and mobile
(probably libav, ffmpeg or Red5)
3. Spend time looking for a solution capable to serve also
HTML5 formats (Mp4, Webm, Ogg)
