SMPTE UK Presentation

1. Using IT Equipment In Live Broadcast
@openbroadcastsy - #ITinBroadcast
Kieran Kunhya – kierank@obe.tv

2. What I’ll talk about today
• IT in Live Broadcast Television (orders of magnitude
differences for latency/bitrate compared to OTT).
• History of IT in Live Broadcast
• Being a “broadcast manufacturer” around modern
software development.
• Using IT equipment in live broadcast
• Can’t explain everything (Wikipedia your friend! -
WestminsterGuest Wi-Fi)
• Dense topics – so some attempt of lightheartedness

3. Who are we? What do we do?
• Founded in 2012
• Globally Distributed Team of 4, average age 27
• No traditional broadcast background (3x
Physics, 1 CompSci)
• Main business, encoders and decoders for
news/sport. Entirely around IT equipment.
• Open Source vast majority of code (Redhat-like)
• Developers of FFmpeg, VLC etc.
• Goal: All code written in house (99.99% there)

4. What we do (2)
• Operate a satellite downlink
centre, with nine racks and
fibre connectivity.
• Lets us develop software in
a real-world environment
(DevOps?).

5. What we do (3)
• Drastically lower the
barrier to entry to
broadcast television.
• Make the broadcast apps
room look exactly like an IT
server room.
• Vertically integrated
broadcast manufacturer?

6. Traditional hardware architecture
• Single or limited function
cards with onboard FPGAs
or DSPs doing majority of
work.
• Smaller CPU to perform
control aspects.

7. A true software architecture
Standard CPU performs all non-
physical data processing
ASI
SDI
MADI
AES/EBU
IP
etc.
ASI
SDI
MADI
AES/EBU
IP
etc.

8. A true software architecture
Standard CPU performs all non-
physical data processing
ASI
SDI
MADI
AES/EBU
IP
etc.
ASI
SDI
MADI
AES/EBU
IP
etc.

9. Technological changes
• Intel Nehalem (2008/09)
• Realtime 1080i encoding
• Only possible with overclocking before
• Low cost SDI/ASI boards
• Full 10-bit, VANC
• Commodity hardware
Everything available next-day!

10. Early trailblazers
• BBC News Raven
• Software-based EVS for bureaus/sat-trucks
• Played-out News bulletins at 2012 Olympics
• Free (French ISP)
• National IPTV service entirely software
driven
• Encoding/Transcoding/Remux
• 5m+ subscribers
• A lot of ISP tech built in-house
• Not just TV, STB, DSLAM etc

11. Myth 1: IT hardware is too large and too
power hungry
Units reside in street furniture
outside Downing Street,
Buckingham Palace etc. Low
depth (200mm) chassis, low
power CPU.
Encode/decode, ASI
processing, Intercom.

12. Myth 2: IT hardware isn’t powerful enough
And another 17 more
cropped out!
• Huge processing power available
• 432 CPUs (small supercomputer)
• Costs falling dramatically
• Powerful enough for example to do high-
bitrate VC-2 encoding/decoding, very high
density H.264 decoding, ASI processing.
• ~2-3x less rackspace for equivalent
functionality compared to hardware
• Processing power for new functions

13. Building broadcast tech the IT-way
• Can’t have expensive Tektronix/Phabrix etc. in 3 countries
• Built in-house SDI analyser (based of Dektec Dtu-351)
• Unified with IP-codebase
• Testing, testing, testing
• Fuzz testing, intelligent mutation of data to cause crashes
• Continuous process, happens 24/7 (billions of
iterations)
• Process used to find Heartbleed
• Heavy unit testing

14. Building broadcast tech the IT-way
Unit test physical hardware
Large number of combinations of
• OS Version (kernel etc)
• Network card driver
• SDI card model
• SDI card driver

15. IT and IP in the Studio

16. Standards in software context
SMPTE 2022-6 – SDI over IP
• Pixels spanning packets
• Costly CRC
TR-03 – separate RTP
streams for video/audio
• Pixels no longer span
packets (RFC4175)
• Soon SMPTE 2110

17. Pixel formats
Only YUV 4:2:2 domain (as example)!
• Planar 10b – main working format
• Planar 8b - preview quality
• UYVY 10b (16-bit aligned) – SDI datastream
• Apple v210 – hardware
• Contiguous 10-bit – 2022-6/TR-03 packing
Tricky to work
with in software.

18. Handwritten (no intrinsics!) SIMD for every mapping (and others).
• 5-15x speed improvements compared to C
• Do it once, make it fast once and for all (until new
CPU…)
• Generic conversion library a difficult problem
• Intermediate pixel format(s) always a compromise
• Add special cases until you’ve done them all!
Pixel formats

20. One does not simply…
Bypass the operating system
DPDK, Netmap, Registered I/O
A presentation in itself. See BBC
R&D at UKNOF:
https://www.youtube.com/watch?v=yL
L8wl8YUwA

21. Kernel Bypass
No network stack – You are the network stack
• Craft Ethernet, IP, UDP headers yourself
• No ARP, hardcoded MAC
• Handle most of this in userspace
• Lets you do pixel processing directly to/from
Network Card memory.

22. SMPTE 2022-6 in the real-world
MISSING THE POINT
• 2x SDI ports per 10Gbps port (Wastes 8.5Gbps duplex – 17Gbps)
• Definitely not COTS, long lead-times, single supplier

23. SMPTE 2022-6 in the real-world
Issues with 2022-6 receivers not accepting software streams
• Unrealistic tolerances to jitter
• Network induced and source induced
• Interoperability tests in closed, controlled environment
• Limited number of hardware manufacturers
• Tradeshow demos don’t represent reality
More detailed work needed on real-world kit and (loaded) networks.
No metrics exist.

25. IT-based live production today
• Dual-capable, SDI/ASI/AES and IP
• Make it look the same to end-users, buttons,
SNMP etc.
• Incremental improvements like a web browser
• Allow users to understand multifunctionality
• Integrate seamlessly (e.g MPEG-TS)

26. IT-based live production tomorrow
• A BIG COMPUTE RESOURCE
• Live and offline on the same CPU
• Remote kit can be processing low-priority batch jobs
• Flash override on live broadcasts
• Possible today but no orchestrator (microservices?)
• FIMS too old (SOAP lol)
• Control network and CPU resources

27. Decode
MPEG-TS
multicast
Orchestrator goes and finds CPU
resources somewhere, onsite or
otherwise.
Spin me up a
multiviewer
Transcode
this file
Not possible with
SDI because SDI
associated with a
connector on a
machine.

28. The V-word, Virtualisation
• OS provides priorities to run multiple applications
already
• Many vendors on Windows (often without good
reason)
• Getting > 1Gbps of data out of VM nontrivial.
• Live migration of complex, stateful broadcast
applications not easy. (Not a web server…)
• Complexity may outweigh benefits.

29. Using IT networks for contribution
• Trend towards using generic, unmanaged, IP networks for
broadcast contribution.
• Regular in News (including cellular)
• Growing use for Sport and Channel contribution
• Packet loss and Jitter (relative to traditional metrics)
• Do it properly, using UDP. TCP not suited.
• Variations on three main techniques.

30. Using IT networks for contribution (1)
• SMPTE 2022-1/2 FEC
• XOR based matrix (adds 2 * matrix latency)
• Basic but wide support (albeit many broken implementations)
Row FEC Column FEC

31. Using IT networks for contribution (2)
• Retransmits (aka ARQ)
• Receiver requests sender
to transmit a copy of lost
packet.
• Affected by round-trip
latency
• Negative
acknowledgment
Sender Receiver

32. Using IT networks for contribution (3)
• Dual-pathing (SMPTE 2022-7)
• Hitless Switching
Path1
Path2
Out

33. Example of IT and Unmanaged IP
• Three nights of the One
Show on BBC1 delivered
using public internet
• Reverse vision video on
same unit.
• Next step (in UK), long-form
programming delivered over
cellular?

34. Summary
• IT equipment is on-air in the live broadcast chain today
• The advent of IP-based production provides an opportunity to use
multifunction IT equipment and gain massive flexibility.
• Standards need to reflect this
• The use of generic connectivity is increasing, and not just for
financial reasons.
• Going to be some very high-profile announcements in coming
months