1. Shared HD workflows for Post and Broadcast
Trevor Francis
Worldwide Marketing Manager
Quantel Ltd
Newbury, UK
Trevor.francis@quantel.com
1

2. Quantel’s principal markets
News
Sports Post
Digital
Intermediate
(Film)
All require sharing of media and projects
Quantel delivers solutions to four principal market sectors: News, Sports, Post
Production and Digital Intermediate for the film industry. All four require
collaborative workflows to deliver the speed and efficiency required by profitable
business.
Each of the four has forward-looking requirements to meet the changing
demands of the industry. For broadcasters, some key technology drivers are
High Definition, multi-platform delivery and the adoption of standardised IT
components wherever possible.
For Film and Post Production, 3D stereoscopic is rapidly accelerating as the next
must-have. The traditional Post industry is forced to re-invent itself as the
capabilities of inexpensive software bring compositing, complex effects,
animation and even colour correction to even the smallest production house.
2

3. Quantel: teamworking on Avatar
Quantel is proud of its association with the production of Avatar: Every frame of
the film as well as all the trailers and promotions have passed through LA post
house Modern Video Film’s set of three Quantel Pablo Stereo3D systems. The
work completed on Pablo includes conforming and Stereo3D checking,
adjustment and quality control. Crucially, the Pablos have also handled all the
Stereo3D English subtitling required for the Na'vi language used by some
characters in the film.
3

4. Production timescales
Months…. years
Film DI
Weeks… months
Post
Minutes
TV / Web
The production timescales around the branches of the industry vary enormously.
Complex projects like Avatar may be months or even years in Post Production.
For TV Post, lead times of many weeks or months are common. For live
broadcasting it’s far more immediate, with Sport and News requiring a job to be
completed in a handful of minutes – or less.
4

5. File sizes
2k, 4k
1080i, 1080p
Compressed SD, HD
H264, Quicktime etc
The file sizes, file types, compression and wrapper formats are also diverse. For
Film, 2k and 4k are normal. TV Post is rapidly heading upwards to 1080p –
doubling the data rates for 1080i. Mainstream broadcasting may be
predominantly still SD around the world, but the move to HD is accelerating
rapidly, with 1080p moving up for Sport.
Much smaller files are in use both for the acquisition and distribution of content
via the internet. With consumer HD cameras in daily use around the world, added
to the millions of camera-equipped mobile telephones, we’re all now potential
photo-journalists, so the broadcasters must be able to accept this content into
their production and playout systems.
Most of us still watch TV on TV receivers, but the number using phones,
computers and even games machines to screen content is growing fast. These
are important outlets for the broadcasters. They have to provide files suited to
these distribution channels.
5

6. A solution for Film and high-end Post
Conforming, Grading, Effects, Compositing, Deliverables
• Up to 4K image sizes
• Uncompressed media
• Shared workflows
• 3D
To begin our discussion of shared workflow solutions, let’s look at a solution for
Film and high-end Post Production. Typical workflows will include conforming,
grading, effects, compositing and deliverables.
These are typically using uncompressed media files of up to 4K. So sharing these
isn’t trivial. Copying and moving files of this size is enormously time-consuming
and resource-hungry.
6

7. A big SAN will fix it, right?
?
The idea of enabling the various applications to share a common pool of storage
using a SAN is a good one. No time is wasted when transferring between
applications and as a consequence there is considerable operational flexibility
about what happens and when. However, while SANs bring many benefits, there
are many real-world issues that arise:
7

8. The non-workflow workflow diagram
More Media Managers Media Managers
××Guaranteed playback?
Guaranteed playback?
VFX VFX VFX VFX
××lots of copies
lots of copies
××inefficient use of storage
inefficient use of storage
On
Conform
line
××parent child deletion protection
parent child deletion protection
On ××metadata incompatibility
Retouch metadata incompatibility
SAN
SAN line
××essence incompatibility
essence incompatibility
Central
Retouch Store Colour
××even more copies
even more copies
Format ××creative changes
convert creative changes
Film Tape
Scan Scan
Format Rec Out
convert
Media Management
SAN performance may not be adequate to support enough real time clients without imposing
unrealistic limits such as leaving 50% of disk space free for internal mechanisms to guarantee
performance.
SAN performance under fault conditions, e.g. rebuilding failed disks may be an automatic process
but it can impact the number of real-time concurrent users
Providing multiple streams of 2K and 4K remains a challenge in some systems. All the above
make providing a mission critical shared environment almost impossible so local storage is often
introduced where absolute reliability is required, i.e. client review, ingest and playout. Of course
this adds inefficiency, increases costs and produces more copies of media to be managed.
Applications may copy data to reorder it (edit it), this quickly leads to not only inefficient use of
disk space but also adds a management overhead to manage all the copies of data.
Different applications may support different file formats, resolutions, colour spaces or even bit-
depths. These application-imposed limits mean media must be converted before it can be shared
meaning new file conversion applications and more copies of data to be stored.
Incompatible metadata between different applications and the inability to roundtrip dark metadata
mean flexibility is lost when work moves between systems and external asset management needs
to be introduced to track what happened where. This becomes a big overhead once the usual
creative changes part way through the process start to occur.
These issues are well understood by manufacturers of high-end edit systems and are the reason
why close-coupled direct attach storage is still widely used in practice with SAN based solutions.
8

9. Teamworking in Post
NAS/SAN
LAN
Audio
Shared Workspace
An ideal sharing infrastructure for post and DI would suffer from none of
the above drawbacks. An ideal infrastructure would:
Provide deterministic performance for specified clients even at 4K when
required without unnecessary overheads (no leave 50% free
requirement)
• Provide guaranteed performance under severe fault conditions
• Never copy media for re-ordering
• Guaranteed fast access allows live ‘play’ of edits
• Provide high bandwidth open ‘files and folders’ access to media in the
pool for clients not requiring deterministic access
• Solve application file format, resolution, colour space and bit-depth
issues without requiring multiple copies of media
• Keep track of media relationships
• e.g. know which four layers in a composite relate through to the final
flattened result.
• Provide facilities for storing and managing both open and dark
metadata
• Scale to any extent
9

10. Quantel ‘Dylan’ storage
High-speed access
Fault tolerance
No-copy editing
Frame-based media management
High Speed Access Dylan storage will support continual playout of real
time 4K, i.e. over one GByte/second (GB/s) and also error-free dual 2K
stereoscopic playback. The performance is maintained even with the disk
over 90% full and irrespective of where the media is physically located on
the disk media. The disk never requires defragmenting so all the
bandwidth is available to the user at all times.
Fault tolerance In addition to dual power supplies the storage provides
auto-rebuild of hot swapped disks without compromising performance.
Where required dual parity protection can be implemented to protect
against the failure of a second disk in an array while another is rebuilding.
Again this does not impact performance.
No copy editing The disk is capable of playing out media in any order in
real-time. It does not need to copy or move media in order to play out
edited selections of shots. Multiple edited clips can refer to the same
media. This technique makes storage incredibly efficient as the media
only exists once on the disk irrespective of how many times it is used.
Frame-based media management To implement no-copy editing
requires a strong media management mechanism or inefficiencies are
easily reintroduced. The media management tracks each usage of every
frame, this ensures that long rushes are not locked from deletion simply
because a few frames are used elsewhere in an edit and also that each
user can safely delete unused material as the system will not delete
frames if they are being used in other clips. Both the above mechanisms
are completely transparent to the user, the storage manages the process
without any user input allowing the user more creative (billable) time and
to spend less (non-billable) time on media and disk management.
10

11. ‘Genetic Engineering'
NAS/SAN
LAN
Audio
‘GenePool’ Workspace
Genetic Engineering infrastructure allows multiple Quantel systems to
share the same ‘GenePool’ workspace. This shared workspace replaces
the direct attach storage. Each system accessing the pool retains exactly
the same capabilities as it would have with direct attach storage – there
are no compromises that have to be made to benefit from the shared
infrastructure.
In order to increase reliability and minimise system components there is
no storage controller required, each mainframe manages its own storage
just as it does with direct attach workspace. Audio is stored in a separate
disk array connected by fibre channel to each mainframe. New
mechanisms are provided to ensure each system knows the current state
of the free space so there is no risk of two systems attempting to write to
the same free space. This inter-system communication takes place over
the LAN whenever space needs to be allocated. This method also
automatically provides fault tolerance should a mainframe become
temporarily unavailable.
Due to the fibre channel topology each mainframe has access to the same
disk bandwidth as it would have with direct attach storage so its
performance remains unchanged. In the above diagram each system can
reliably and deterministically play 4K (or could play for example Stereo3D)
without any impact on the systems’ operation. So for example one system
can be doing a client-attended grading session, another can be performing
quality control and the third playing out a 4K stream as HDRGB to
HDCAMSR for a high quality deliverable. There is only ever one copy of
any particular media and each system can, in theory, have independent
access to it.
11

12. Teamworking in Post
NAS/SAN
LAN
Movie
‘Sam’
Audio
File System
Presentation
Shared Workspace
If the GenePool infrastructure only provided Quantel to Quantel sharing it
would be a useful development for the post and DI pipeline but this is only
half the story. As has already been stated Post, DI and Broadcast
Creative Services today are about teamworking and the GenePool
infrastructure integrates any system capable of working with media in
standard file systems.
A shared access product, Sam, provides open read/write access to media
in the GenePool.
Using CIFS (Common Internet Filing System) third party applications,
running Linux (2.6 Kernel) or Windows XP/Vista or Mac OSX can gain
read/write access to media in the shared pool. There is no modification or
special API required, any application that can access media over a
network will be able to access the shared pool ‘out of the box’. The Sam
CIFS server supports multiple simultaneous clients each accessing the
media at the same time without blocking. Depending on configuration and
the network Sam can deliver several hundred MB/s to support 3rd party
applications. Of course any file i/o activity does not impact on the
performance of the Quantel mainframes connected to the shared pool.
12

13. Shared inventory view
Sam
Access to: flattened files and ingredients
Storage automatically tracks and manages media relationships
Virtualise asset in different formats
The above four layer composite is presented not only as a flattened file but also
as a series of ingredients. This allows 3rd party systems to not only access the
media itself but also to access the history of how the media was created, all
without any external media management system. This is a truly powerful
environment for VFX or Broadcast Creative services.
13

14. File virtualisation
Sizing: high quality up or down-res
Colorspace: YUV to RGB, 601 to 709
Bit depth: 16, 10 and 8 bit with dynamic rounding
DPX handling: data packing/unpacking
Colorspace Bit Depth
Shared High YUV-RGB Dynamic
Nx Disk DPX CIFS
Workspace Quality RGB-YUV Rounding
2Gb FC interface Packing Presentation
Pool Sizing 601-709 On
709-601 truncation
A further aspect of Sam is to virtualize media, initially to different
resolutions and a common RGB colorspace but in the future to different
file formats and bit depths. For example a clip that contains media
originally from Digital Betacam (SD 4:2:2) from film (2K 4:4:4) and from
.r3D Red camera files, can be presented to the outside world as if it was
all HD RGB in dpx, Tiff or non compressed Quicktime files. Sam uses
purpose-built hardware to handle the computationally intensive
conversions on-the-fly as the media is read by the 3rd party system.
As always, no new media is created on disk during these operations.
Each application simply has access to the media in the format it needs it
when it needs it without having to explicitly manage file conversions. How
the clip is virtualised is controlled by modifying an associated XML file.
A ‘distribute’ function on each workstation then allows media to be re-
formatted and made available for third party systems without any
rendering – for example an HD clip can be made available in SD to a
desktop application that cannot play back HD.
14

15. Typical Post application
A VFX facility can use multiple desktop VFX systems each accessing media in
the GenePool. Media does not have to be transferred to the desktop systems
they can work direct on the media in the shared pool, this greatly simplifies media
management and also makes it straightforward to see an overall view of the
current state of the work in progress as all the media is one location and can be
instantly conformed for viewing. Full history remains available and files are never
flattened making changes much easier than today.
15

16. Locking image qualities together
Work done on one file type reflected in others
Files may be real or virtual
Requires
Robust identity model
Virtual filing system
This is crucial for a scalable broadcast workflow
Web Desktop SD HD
Everything we’ve discussed so far relies on a robust relationship between
different qualities of media, whether those files are real or virtual. All
Quantel solutions are built on an identity model, which is the subject of
another paper
[http://www.quantel.com/repository/files/whitepapers_tracking_media_ass
ets.pdf] which provides this relationship.
This is critical to Quantel’s broadcast server workflow, where broadcast
and proxy media are held side-by-side on the same storage.
16

17. Broadcast solution requirements
Short timescales
Lots of users
Different skills
Teamworking in Post
Range of file formats
Different sources
Multi-platform output
SD, HD….3D?
News is only news if it’s current. So a solution must be capable of publishing
content quickly. Sports highlighting has even faster reaction times.
In broadcast installations there’s typically a (relatively) large number of users,
with different roles and skill-sets.
Incoming content will be in a wide variety of file formats, image sizes, frame-
rates. Lots of sources will arrive at the same time and must be accommodated.
Outlets to TV, Web, Mobile must be provided – each with the rapid response
required by news itself.
Most TV news broadcasting in the world is still SD and all broadcasters have
legacy archives of SD content - but acquisition is turning rapidly to HD and must
be handled efficiently.
It’s probably a fair prediction that 3D news is a way off, but sport will be there
soon and other genres, like home shopping channels, can see the benefits.
17

18. Shared workflow for live TV
Sources Workflow Output
Journalists
Editors Producers
Cameras Production Graphics
Directors Managers
Anchors
Let’s remind ourselves of the workflow requirements for live broadcasting:
Content comes from diverse sources; is processed by a cross-discipline team
and output to a range of platforms. TV is still predominant, but outlets to fixed and
mobile computers/ hand-held devices are growing rapidly.
The diagram shows a relationship between the different groups – they don’t act
as an individuals, they’re a team. They need instant, uncontested access to the
media. They need to be able to share their work within the groups and with the
wider team.
18

19. Collaborative working
Desktop
Database Desktop
Editors
Desktop
Editors
Desktop
Editors
Desktop
Proxy files Editors
Editors
Broadcast File
Video Proxy File
Craft
Craft
Editors
Broadcast files
Editors
Transcode
?
Dylan
storage
Broadcast and Proxy files locked together
Large numbers of workstations with instant, uncontested access
Collaborative workflow
How do we make content available to large numbers of users distributed around
a production facility?
The Quantel broadcast server is built around the same ‘Dylan’ storage described
above.
Video ingest is fed to a dual encoder which simultaneously creates a broadcast-
quality and proxy file, linked by a single ID. Both files are stored on the same
storage with a separate database to hold both technical and essence metadata.
Client workstations are supported over gigabit Ethernet connections. Desktop
editors working with proxy files; craft editors working with full-quality media. The
Identity model guarantees a frame-accurate relationship between proxy and
broadcast content.
19

20. Make it fast
Use an Identity model to represent every frame of stored media
1 2 3 4 5 6 7 8 9 10
Edits can be pointers to the stored frames
1 2 3 4 5 6 7 8 9 10
In Out
Managed rendered sequences as ‘Deltas’ and add only these to the storage
3 4 5 6 7 8 A B C
Use these techniques to keep data copying and movement to a minimum
It is beyond the scope of this paper to cover the subject in exhaustive detail, but it
is necessary to highlight the design background of a fast, responsive shared
workflow solution.
We make a shared workflow solution fast and responsive by minimising the
requirement to copy and move large media files.
Every incoming file – whether acquired as video or via a network connection – is
split into individual frames. Each file is assigned a unique ID and each frame is
indexed by counting from the start of the file. Thus, every frame can be
referenced, as can every piece of metadata attached to a frame or range of
frames.
Edit decisions can then be referred to those frames, allowing a simple subclip to
be created by simply storing the In and Out frames.
In this way, edits based solely on server-held content are ready to play or export
immediately with no new files created.
Any new sequences, like effects, transitions or keys, create what we call ‘Delta
Frames’, the difference between the original media and the edited clip. Only
these new frames are added to the storage. The database will register a logical
clip which comprises the original file plus the deltas and use this information on
playback or file export.
20

21. SD and HD
Held natively
Edited together
Converted on output as required
SD output
SD sources
HD sources
HD output
Auto size & ARC
Editing
There are advantages to holding SD and HD content in native file format
and resolution, notably efficiency of storage.
Editing workstations connected to the Quantel server may assemble a
timeline of cuts, transitions or keys using any combination of SD and HD
content. The edits are created and stored using the pointers plus ‘Delta’
method described above. Video ports and FTP export modules are
configurable and may be formatted to be SD (4:3 or 16:9) or HD. Content
is up or down rezzed on the fly during the playback / export process. File
conversion and MXF wrapping is also applied live on export..
21

22. People make TV
Different jobs
Different skill-sets
Teamworking in Post
Graduated application set
Viewing
Basic cut editing
News editing
Craft editing / effects
Final Cut Pro
It’s important to recognise that TV is not made by technology; it’s made by people
using technology.
Quantel offers a range of editing tools, targeted at the different tasks which make
up TV production and the range of editing skills held by the operators.
So, a simple viewing and logging tool, with basic drag and drop editing capability.
A cuts-only editor, with sufficient functionality to start and finish a news or sports
story. No video modifications in here, but audio tools including level controls and
voice-over recording.
A news editor equipped for all but the most sophisticated features. Including tools
to fix the kind of problems news people hit every day. Like disguising faces; fixing
colour-balance errors or adjusting badly-exposed shots.
Finally, a full effects editor with independent storage and hardware processing.
In response to demand, Quantel have added Apple’s Final Cut Pro to the
workflow.
22

23. Efficiency
What are the components
of a typical News/Sports
broadcast?
Headlines, teases, bumpers, LVOs
Packaged stories
The design of our range of editing tools was considered alongside the
real-life requirements of live TV production.
A news broadcast, for example contains many items of different lengths;
different degrees of sophistication and different timescales.
Many items – like headline sequences, the stories voiced live by the
anchor (known as LVO – Live Voice-Over- in some parts of the world) and
the ‘tease’ items shown just before a commercial break – are short. These
are perfectly suited to a simple drag and drop desktop workstation. The
training requirement is minimal, so a producer can learn to use it
alongside their scripting tools and take pressure off the craft editing
resources.
Our Viewer or Cuts-only editor is also a perfect rough-cutting or ‘offline’
editor. Edits begun on the desktop may be accessed by the craft editors –
including Final Cut Pro – for finishing.
23

24. Efficiency
Sharing and delegating
Jobs
View, Log
Editing Skills
Heads, Opens, Bumpers, Review
View View View View
LVO’s, simple edits
Cut Cut Cut
Package editing
Edit Edit Edit
Animations, features, promos
FCP Edit +
A hierarchy of editing tools can be used to ‘filter’ the jobs flowing from the
production.
Some may be started and finished at the simplest level – like the
headlines described above.
Others may be passed on to a higher level of competence for finishing.
This is an important contrast with the ‘traditional’ idea of an editing pool
staffed entirely with trained editors with sophisticated workstations. This is
extremely wasteful of resources. Our model allows for larger numbers of
simpler, inexpensive workstations, with each contributing to the stream of
tasks and leaving the craft facilities to dedicate themselves to high-quality
productions. Thus, the quality of the output may be increased.
24

25. Quantel sQ server workflow
ENPS
Project sharing
Playlist control
Status feedback
sQ View sQ Cut
Desktop viewer Desktop editor
PCR control
Proxy media
Record SD video
Control HD video
Quantel sQ server
Broadcast media File Output WWW
sQ Edit sQ Edit Plus Final Cut Pro
News editor Craft editor Craft editor
File
Transcode
Archive
Putting all this together – the shared workflow model is superficially very
different from Post, with some common technical elements and design
philosophy.
Content is acquired and stored on a common pool. A range of
workstations – with a common project structure – is arranged around this
storage.
Separate, less-agile storage may be used for mid- or long-term archiving,
using an MXF file exchange to off-load and restore content.
The shared storage in the server is designed to be scalable on three
levels:
• Numbers of clients
• Number of Input/Output nodes
• Quantity of on-line storage
Users may work with newsroom management tools, like ENPS, to build
rundowns, write scripts etc. The rundowns may be linked to PCR playout
using the MOS protocol.
Video playout may be specified to be SD, HD or both at the same time.
Separate FTP outputs may be scripted to deliver content in any file format,
image size and wrapper, suitable for web or mobile distribution.
25

26. Conclusion
Quantel has shared workflow solutions tailored to the
requirements of the Post and Broadcast users
Quantel philosophy is to empower creativity while keeping
the engineering where it belongs – in the machine room
Teamworking in Post
26

27. Thank you
Trevor Francis
Worldwide Marketing Manager
Quantel Ltd
Newbury, UK
27