Today’s complex media workflows increasingly involve end-to-end, customized and automated operations, often with a combination of both cloud and on-premise storage. They require the integration of multiple geographically dispersed teams and processes, and the ability to publish to multiple screens and platforms. This presentation explores how infrastructure-agnostic technologies can enable these new workflows by allowing large digital content to be ingested, accessed and distributed at maximum speed and with full security - regardless of storage type or network distance.
Bridging On-Premise and Cloud Deployments for Broadcast IT
1. BRIDGING ON-PREMISE AND CLOUD
DEPLOYMENTS FOR BROADCAST IT
Moving the worlds data at maximum speed
2. PRESENTER AND AGENDA
PRESENTERs
AGENDA
Michelle Munson
President and Co-Founder
michelle@asperasoft.com • Industry Trends
• Big Data Challenges
Andrea Di Muzio
Sr. Solutions Architect
• Solutions
dimuzio@asperasoft.com
• Use Cases
• Q&A
2
3. ASPERA’S MISSION
Creating next-generation transport technologies
that move the world’s digital assets at maximum speed,
regardless of file size, transfer distance and network conditions.
3
4. ASPERA PRODUCT PORTFOLIO
TRANSFER WEB MANAGEMENT & SYNCHRONIZATION
CLIENTS APPLICATIONS AUTOMATION
High-speed transfers File sharing, collaboration Transfer management, Scalable, multi-directional,
for web, desktop and mobile and exchange applications monitoring and automation multi-node synchronization
TRANSFER SERVERS
High-speed file transfer servers for on premise, private, public, and hybrid cloud deployments
FASP™ TRANSPORT
Innovative, patented, highly efficient bulk data transport technology, unique and core to all Aspera products
4
5. TRENDS
Big Data Explosion
• 90% of data today file-based or unstructured
• Mix of file sizes—but larger and larger files the norm
Diversity of IP Networks—Media, Bandwidth Rates, and Conditions
• Variable bandwidth rates (slow to super-fast)
• Bandwidth rates increasing—costs decreasing
• Network media remains diverse (terrestrial, satellite, wireless)
• Conditions vary—all networks prone to degradation over distance
Global Workflows—moving Big Data over WANs
• Teams are geographically dispersed
• Over distance, network conditions degrade
• Contemporary TCP acceleration solutions not designed for big data transfer and replication
Cloud Computing Grows Up
• Amazon Web Services (AWS) S3 cloud storage – 2010: 262 billion objects, 2012: 905 billion objects
• More choices: Microsoft Azure, OpenStack, HP Cloud
• No longer a niche – Netflix (transcoding), MTV (global video distribution), BGI (genomic sequencing)
5
6. CHALLENGES WITH TCP AND ALTERNATIVE TECHNOLOGIES
Distance degrades conditions on all networks
• Latency (or Round Trip Times) increase
• Packet losses increase
• Fast networks just as prone to degradation
TCP performance degrades with distance
• Throughput bottleneck becomes more severe with
increased latency and packet loss
TCP does not scale with bandwidth
• TCP designed for low bandwidth
• Adding more bandwidth does not improve throughput
Alternative Technologies
• TCP-based - Network latency and packet loss must be low
• Modified TCP - Improves on TCP performance but insufficient for fast networks
• UDP traffic blasters - Inefficient and waste bandwidth
• Data caching - Inappropriate for many large file transfer workflows
• Data compression - Time consuming and impractical for certain file types
• CDNs & co-lo build outs - High overhead and expensive to scale
6
9. FASP™: HIGH-PERFORMANCE TRANSPORT
Maximum transfer speed
• Optimal end-to-end throughput efficiency
• Transfer performance scales with bandwidth independent of transfer
distance and resilient to packet loss
Congestion Avoidance and Policy Control
• Automatic, full utilization of available bandwidth
• On-the-fly prioritization and bandwidth allocation
Uncompromising security and reliability
• Secure, user/endpoint authentication
• AES-128 cryptography in transit and at-rest
Low Overhead
• Less than 0.1% overhead on 30% packet loss
• High performance with large files or large sets of small files
Resulting in
• Transfers up to thousands of times faster than FTP
• Precise and predictable transfer times
• Extreme scalability (concurrency and throughput)
9
10. FASP 3 – UNIVERSAL HIGH-SPEED DATA TRANSPORT
FILE SIZE INDEPENDENT
Maximum transfer speed, optimal
bandwidth utilization and maximum I/O
throughput on any storage platform ALL PARADIGMS & DEPLOYMENT MODELS
regardless of number or size of files
fasp 3 Transfer, replicate or synchronize your data in all
types of scenarios: one-to-one, hub and spoke,
DISTANCE INDEPENDENT unidirectional and multi-directional
INFRASTRUCTURE AGNOSTIC
Locate your big data anywhere and move it
seamlessly with high-speed, secure transfers
CLOUD HYBRID ON PREMISE
10
11. ASPERA ON DEMAND HIGH SPEED DATA INGEST
WITH DIRECT-TO-CLOUD
Cloud
11
12. ASPERA ON DEMAND
AVAILABLE ON AMAZON WEB SERVICES AND WINDOWS AZURE
Transfer Servers – fasp servers as VM images for high speed transfer with cloud storage.
• Direct integration with Amazon S3 and Windows Azure BLOB object storage for line speed ingest and distribution
• Offers unlimited scale out with additional transfer server instances.
• Flexible Storage Options – Choose between local, Elastic block or Cloud storage (e.g. S3 / Azure Blob)
• Fast, robust and secure - Automatic resume, encryption, encryption-at rest, private multi-tenant storage roots, and
dynamic bandwidth control.
• Fully interoperable with all Aspera clients.
• Base transfer server, or Web-enabled application platform with support for Connect browser plug-in for SDK
integration with 3rd party cloud applications.
Web Applications – Add-ons to the base Transfer Server (AWS Only)
• Aspera Shares –Multi-node file ingest and sharing, provides access to content across any location (on cloud,
other clouds, or on-premise) with powerful security and access model
• Aspera faspex – Global Person-to-person file delivery, collaboration, exchange and distribution
• Perfect for Hybrid deployments: Easily Publish, Store, or Receive content on-premise or on cloud storage
Optional Clients
• Optional add-ons to extend the content ingestion and distribution to mobile devices and email clients
• Supports Faspex iOS Client, Outlook Plug-in, Cargo Automatic Downloader, and Mobile Uploader
Licensing and Availability
• Tiered usage (up to 1PB a month) based on GB transferred; No bandwidth caps, no user counts, no limit on
number of ―servers‖ (or ―instances‖) one can run
• Month-to-month pay as you go, or annual and multi-year terms
• Available now: http://cloud.asperasoft.com 12
13. ORCHESTRATOR
OVERVIEW
Web-based application and SDK for creating and managing automated workflows, from simple file
forwarding, to complex process orchestration.
Applications and Use Cases
• Advanced contribution and automation
• High volume processing and transformation
• Ad ingest and insertion for video-on-demand
• End-to-end content preparation and distribution
Features and Benefits
• Intuitive Web based graphical workflow designer
• Real time monitoring of active workflows
• Open architecture for full customization
• Unattended and interactive workflows, with ad hoc and recurring (scheduled)
execution
• Extensive and growing library of plug-ins for 3rd party product integration
• Highly scalable in support of the most demanding file-based workflows – on
promise, on cloud, or hybrid
• Integration with the Aspera high-speed file transfer environment and Aspera Console
for reporting
13
14. OPEN PLATFORM 3RD PARTY LIBRARY
FREE TO ORCHESTRATOR USERS
Database / Stores Antivirus
Transcoding
Ad Insertion & Media Management Email
MediaInfo
FTP/SFTP SCP
Video Quality Control Encryption File Transfer
ImageMagik
Watermarking Image Manipulation Media Management IT Management
15. USE CASE: INTRA-CLOUD TRANSFERS
ACROSS SAME OR DIFFERENT CLOUD INFRASTRUCTURE
THE SOLUTION
Node
• Data migration from one region to fasp
another or from one provider to another
• Transfer database or application logs
from one region to another for DR or
Business Continuity
fasp
fasp
Node Node
US West US East
15
16. USE CASE: INGEST AND SHARING
HYBRID ACROSS PUBLIC & PRIVATE CLOUDS
THE SOLUTION
• Shares Web app transparently communicates
with Aspera server Nodes and displays
content in a single user interface
• User browses authorized content across
multiple shares
• Independent high-speed data transfers
to/from Datacenter, AWS S3, and Windows Client, NY, NY
Azure BLOB, transparent to user
Shares
fasp fasp
DMZ
Node
Node
Datacenter, Emeryville, CA
16
17. USE CASE: PERSON-TO-PERSON
HYBRID ACROSS PUBLIC & PRIVATE CLOUDS
1. Login to faspex On Demand
2. Remote browsing of remote storage (cloud or on premise)
3. fasp high-speed upload to FoD and S3
4. faspex notification to recipient
5. fasp high-speed download Direct-from-S3
faspex
Person-to-
Person
fasp
4
HTTP –
1 multipart
3
Remote Storage Connect
Connect
On Premise or Browser
Browser
On Cloud fasp Plug-in
Plug-in AWS
2 S3 5
Ireland
Browser, San Diego Datacenter, NY Browser, London
17
18. CUSTOMER EXAMPLE: CLOUD-BASED ENCODING
Ultimate Fighting Championship® (UFC®) is the world’s leading promoter of mixed martial arts (MMA), with programming broadcast to
half a billion homes throughout 150 countries.
Challenge: Transferring high-resolution video content from venue to host site in the cloud for scale-out transcoding
HTTP –
fasp multipart
2
3
Aspera Connect Connect Server
browser plug-in running on EC2 Parallel Transcoding
14 EC2 instances: 3 min
1
4 Herndon, VA
THE SOLUTION
1. Video Broadcast Capture
2. High-speed upload of content directly into AWS S3
3. Transfer completion triggers parallel transcoding in the cloud and
`
device-specific videos are saved to S3
4. Using AWS CloudFront, content is delivered to viewers’ devices
18
19. DEMO AT ASPERA STAND K09: END-TO-END WORKFLOW
INGEST, TRANSFER, DISTRIBUTE AND ARCHIVE GLOBALLY, IN MINUTES
THE SOLUTION
AUTOMATE
1. Content is transferred and ingested to an on-premise Aspera server
2. Aspera high-speed transfer Direct-to-S3 with Aspera On Demand
3. Multiple parallel transcoding jobs, with output stored back to S3
4. Faspex packages are created, sent and downloaded by the customer
Aspera Aspera 5. Media files are archived to Azure BLOB or AWS S3 / Glacier
Console Orchestrator 6. Entire process is MINUTES, works GLOBALLY, and is SECURE
BLOB
DMZ Media Customer
3
Transcoding Service
4 5
Faspex
Media Company On Demand
Aspera Node
1
S3 Glacier
Aspera Node
2
INGEST TRANSFORM DISTRIBUTE ARCHIVE
20. HYBRID WORKFLOW - COMBINES ON-PREM AND ON CLOUD
1. Ingest original content to on-premise
Aspera Server (and trigger workflow)
2. Run MediaInfo plug-in to extract video
details
3. Notify Operator of ingestion
availability and present options for
transcoding on-premise, on-
cloud, and archive
a) Transcode in the cloud
(Zencoder.com or other)
b) Transcode on premise (ffmpeg or
other)
4. Transcode and publish files to faspex
(S3-S3 transfer) or on-premise to S3
transfer
5. Received files are downloaded by
faspex client (Cargo or Connect)
6. Archive original video to alternate
cloud for long-term storage
20
21. STEP 1: INGEST AND TRIGGER WORKFLOW
1 GB high-speed
transfer globally in
under 30 seconds
Ingest original content to on-
premise Aspera Server
(triggers start of workflow) 21
22. STEP 2: EXTRACT VIDEO DETAILS USING MEDIAINFO
Ingested content is stored on-premise – ingest
directory is monitored by Orchestrator, video details
are automatically extracted using MediaInfo Plug-in
22
23. STEP 3: SELECT ON-PREMISE OR ON-CLOUD
TRANSCODING
Select option for transcoding on-premise, on-cloud Select option for post-transcode archive
(operator or programmatic) 1. On-cloud (Azure or AWS S3 / Glacier)
1. Transcode in the cloud (Zencoder.com or other)
2. On-Premise
2. Transcode on premise (ffmpeg or other)
23
24. STEP 4: TRANSCODE AND PUBLISH FILES TO FASPEX
Transcode job completes, and output files are published to
recipient using faspex APIs (running on-premise or on-cloud) 24
25. STEP 5: GLOBAL DISTRIBUTION TO CLIENT IN MINUTES
Received files are downloaded by faspex client at high-speed with encryption in transit and at-rest:
1 GB in 41 seconds (Cargo automatic download or user-initiated with Connect browser plug-in) 25
27. USE CASE: AUTOMATED END-TO-END WORKFLOW
ACROSS HYBRID ACROSS PUBLIC & PRIVATE CLOUDS
THE SOLUTION
AUTOMATE
1. Content is transferred and ingested to an on-premise Aspera server
2. Aspera high-speed transfer Direct-to-S3 with Aspera On Demand
3. Multiple parallel transcoding jobs, with output stored back to S3
4. Faspex packages are created, sent and downloaded by the customer
Aspera Aspera 5. Media files are archived to Azure BLOB or AWS S3 / Glacier
Console Orchestrator
BLOB
DMZ Media Customer
3
Transcoding Service
4 5
Faspex
Media Company On Demand
Aspera Node
1
S3 Glacier
Aspera Node
2
INGEST TRANSFORM DISTRIBUTE ARCHIVE
29. THANK YOU FOR JOINING!
Michelle Munson Andrea Di Muzio
Aspera President and CEO Aspera Sr. Solutions Architect
FOR MORE INFORMATION ON ANY ASPERA PRODUCT, PLEASE CONTACT:
sales@asperasoft.com
Hinweis der Redaktion
Orchestrator allows you to create graphical workflow models for moving and processing filesThe run-time engine executes the models while controlling and monitoring each step of the workflowA lot of workflows involve third-party productsSo in addition to Orchestrator’s core workflow engine, there’s an open, plug-in architecture to support third party products along with a lot of other standard processing functionsAnd here’s an example of some of Orchestrator’s plug-ins….PAUSE