These slides were presented on the Streaming Media West conference in 2016. This talk is also a reference for the blog post "Using Microservices to Encode and Publish Videos at The New York Times" at The New York Times Open blog.
- Streaming Media West 2016: http://streamingmedia.com/Conferences/West2016/
- Open Blog: http://open.blogs.nytimes.com/2016/11/01/using-microservices-to-encode-and-publish-videos-at-the-new-york-times/
7. The Flaws
1. The whole process relies on a filename
2. Several watch folders
3. Unstable transcoding
4. Hard to scale
5. Two CMSs are required to publish a video.
8. The Flaws
1. The whole process relies on a filename
2. Several watch folders (hard to maintain)
3. Unstable transcoding (proprietary software)
4. Hard to scale (cluster in-house)
43380_1_saudi-documentary_wg_720p.mp4
9. The Flaws
1. The whole process relies on a filename
2. Several watch folders (hard to maintain)
3. Unstable transcoding (proprietary software)
4. Hard to scale (cluster in-house)
43380_1_saudi-documentary_wg_720p.mp4
Production ID
10. The Flaws
1. The whole process relies on a filename
2. Several watch folders (hard to maintain)
3. Unstable transcoding (proprietary software)
4. Hard to scale (cluster in-house)
43380_1_saudi-documentary_wg_720p.mp4
Version
11. The Flaws
1. The whole process relies on a filename
2. Several watch folders (hard to maintain)
3. Unstable transcoding (proprietary software)
4. Hard to scale (cluster in-house)
43380_1_saudi-documentary_wg_720p.mp4
Slug
12. The Flaws
1. The whole process relies on a filename
2. Several watch folders (hard to maintain)
3. Unstable transcoding (proprietary software)
4. Hard to scale (cluster in-house)
43380_1_saudi-documentary_wg_720p.mp4
Resolution
13. The Flaws
1. The whole process relies on a filename
2. Several watch folders
3. Unstable transcoding
4. Hard to scale
5. Two CMS to publish a video
15. The Flaws
1. The whole process relies on a filename
2. Several watch folders
3. Unstable transcoding
4. Hard to scale
5. Two CMS to publish a video
17. The Flaws
1. The whole process relies on a filename
2. Several watch folders
3. Unstable transcoding
4. Hard to scale
5. Two CMS to publish a video
19. The Flaws
1. The whole process relies on a filename
2. Several watch folders
3. Unstable transcoding
4. Hard to scale
5. Two CMSs to publish a video
21. The Challenges
1. Fast encoding. News Clips are very time sensitive
2. Scalability
3. Reliability
4. Company-wide video transcoding solution
5. High-quality transcoding
360°, Drone footage, Live Action
26. Media Factory Client
1. EcmaScript 6 Library
2. Upload Files from Browser to S3
http://github.com/TTLabs/EvaporateJS
3. Trigger Jobs on Media Factory API
4. Used by other teams
30. Media Factory API
1. All NYTimes business logics
Interact with our internal APIs
Define which presets will be used based on
job type (Standard, 360)
http://github.com/nytimes/video-presets
32. Transcoding API
1. Integration with multiple encoding cloud
services
2. Reliability & Scalability
3. Define a common Job specification
4. Define a common Preset specification
35. Transcoding API
1. Integration with multiple encoding cloud
services
2. Reliability & Scalability
3. Define a common Job specification
4. Define a common Preset specification
5. http://nyti.ms/mediafactory
37. Distribution API
1. Fast file transfer leveraging Aspera protocol
2. Plan to support distribution to our partners
Youtube, Facebook, other partners
3. Plan to support to multiple CDNs
38. Just a recap!
Distribution API
Acquisition API
Transcoding API
Storage
Media Factory Client
Media Factory API
CDN
Database
39. Future
1. Video Quality
Classify into groups
VMAF, Reingest on the Media Factory API
2. Open Source
Distribution API
Acquisition API
3. Snickers (http://github.com/snickers/snickers)