This document describes a GPU renderfarm system integrated with an asset management and production system (AMPS) to efficiently manage rendering assets and accelerate the rendering process for CG movie production. The system allows artists around the world to upload and manage assets through AMPS. Rendering jobs are submitted online to a GPU renderfarm and monitored. Experiments showed nearly linear speedups from adding more GPU nodes, with render time decreasing from over 5 hours on one GPU to under 1.5 hours on two nodes with six GPUs total. Future work includes direct job submission from authoring tools and support for more tools, renderers, and heterogeneous renderfarm configurations.

1. GPU Renderfarm with Integrated Asset
Management & Production System (AMPS)
Tackling two main challenges in CG movie production
Multi-plAtform Game Innovation Centre (MAGIC), Nanyang Technological University (NTU), Singapore
Presenter: Dr. Qiu Jie

2. Two main challenges in CG movie production
• How to manage the rendering assets efficiently
– Large amount of digital assets
– Huge load of resources and common outsourcing practice
• How to perform CG rendering in a relatively shorter time
– CPU renderfarm systems usually require substantial hardware setup, high maintenance cost, and
power consumption
• Not affordable for small studios
– Rendering algorithms can benefit from parallelization
• GPU renderfarm has more cores, smaller size, and lower cost comparing to CPU one
2

3. • AMPS (Asset Management & Production System)
– Online resource management (e.g. for archiving, retrieving, tracking, without location constraint)
– Can be adopted for other industries
• Integration of GPU renderfarm with AMPS
– Ability to manage the rendering assets efficiently
– Streamlining the rendering pipeline (i.e. sending assets to the renderfarm online via web browser)
– Lower rendering cost and faster rendering process
Our solutions
3

4. • Not flexible to satisfy the needs in all or various media production companies and
studios (e.g. Shotgun and TACTIC)
• Lack of resource management features such as revision control, workflow management,
flexible accessing / sharing rights management (e.g. Dropbox, Box.net)
• No linkage with rendering solutions
Existing asset and workflow management solutions
4

5. • CPU-based renderfarms
– GreenButton (RenderMan Pro Server, YafaRay, LuxRender)
• Provides external software to upload assets, monitor progress, download results, but does not have complete asset management
features
• GPU-based renderfarms
– Render Street, renderFlow, ultrarender (Cycles and/or iRay renderers)
• Rendering jobs are done in the company servers
– Sheep it! Render Farm (Cycles)
• Rendering job is distributed among the clients/users
– OTOY (Octane)
• Utilizes NVIDIA GRID
Existing renderfarm services
5

6. Proposed workflow
6
3D scenes
& other
data
Render
request
3D scenes
&
Render
request
Rendered
results
Rendered
results &
other
data
HP SL250s Gen 8 * 2
NVIDIA TESLA K20X * 6

7. • Back-end
– For storing data (e.g. projects, assets, users,
workflows, etc.)
– Provides back-end functions
– Deployed as a combination of web services and
MVC (Model-View-Controller) infrastructure
• Front-end
– Provides a user-friendly interface to access and
manipulate assets
– In various flavors : web-based (internet
browser), PC client apps, mobile client apps
Asset management component
7

8. Web-based Front-End
8

9. Native client apps
9

10. • Asset
– Media asset in production (e.g. images, videos, sounds, 3d models, etc.)
– Version control support (i.e. revisions are kept for each asset)
– Can be shared to internal and/or external users
• User
– Each user has roles (that can be created/deleted/edited and assigned to different projects)
– Two types, real user or virtual user
• Real user : regular user, such as artist, director etc.
• Virtual user : interface between AMPS and external application / solution e.g. renderfarm.
AMPS Components
10

11. • Workflow
– Production tracking feature
– Consists of workflow Steps
– Multiple workflow support
– Assets can be assigned to steps or transferred to another workflow
• Project
– Typically corresponds to a real-life project such as an animation, film, or game project
– Consists of a set of assets, asset revisions, asset folders, project roles, and users etc.
AMPS Components
11

12. Request asset info (3)
Download assets request (5)
Renderfarm component
12 Web browser
Rendering
Request (1)
AMPS
Thin app
Backburner
Manager
Send job
(8)
Trigger & inputs (2)
Assets (6)
Job &
Assets (10)
Assets
(7)
Assets
(9)
Job &
Assets (10)
Job &
Assets (10)
Results (12)
Results (13)
Rendering results (14)
Results (11)
Results (11)
Results
(11)
Asset info (4)
Manager Node Rendering Nodes

13. • Consists of a manager node and a set of rendering nodes
• Manager node consists of :
– Thin app : our in-house developed software, to facilitate communication between AMPS and the renderfarm
– BackBurner Manager : sending assets to rendering nodes and distributing rendering tasks
• Each rendering node consists of :
– BackBurner Server : to receive assets and rendering jobs from manager node, and to return the rendering
results back to the manager node
– Rendering software : software to perform rendering
Renderfarm component
13

14. • Serve as the middleman between AMPs and the rendering nodes
• Responsibilities :
– Awaiting and receiving rendering requests from AMPS
– Requesting additional information supporting rendering assets (such as textures) from AMPS
– Downloading rendering assets from AMPS to a temporary folder
– Sending rendering jobs to the Backburner Manager
– Awaiting and monitoring a temporary folder for the rendering results
– Uploading the rendering results to AMPS
Thin App
14

15. • Rendering job submission is received by Thin App through the network (local or internet) using TCP
protocol
• The job submission message is sent in JSON format, comprises :
– Action type (“exec function”) : specifies what action to do
– Array of 3D scene files to be rendered (“data array”) : extendable for submitting multiple rendering job
simultaneously
– Each object (“item”) in the array corresponds to one rendering job. It consists of :
• Information related to the main 3D scene file (e.g. its Asset ID in AMPS database, file size, file name ,etc.)
• Asset ID of each dependent asset (e.g. textures, mocap file, etc.)
• Rendering parameters (e.g. output resolution, output filename, folder ID in AMPS to store the output, etc.)
Thin App
15

16. Rendering job submission
16

17. • The system is used in the Asia’s first live-action Mecha feature film “The Boy And His Robot”,
the first fully GPU (Graphics Processing Unit) rendered feature film in post-production
• Experiment scenario
– Artists from various places (e.g. Singapore, China, Russia, France etc.) create and upload assets to
AMPS, managing assets using AMPS
– Movie director in Singapore evaluates their works through AMPS
– Revisions and final approval from the director
– The 3D assets can then be submitted to the renderfarm
Experiment
17

18. • Rendering specifications :
– Resolution : 1920 x 1080
– Polygons : 752,608
– Frames : 85
– GPU RAM consumption : 2,814 MB
– Rendering : Path Tracing, 1,250 samples per pixels
and maximum 3 bounces
• Manager node specifications :
– OS : Windows 7 Enterprise
– CPU : Intel Core i7 920 2.67 GHz & 2.67 GHz
– RAM : 6 GB
Rendering experiment
18
• Rendering node specifications :
– OS : Windows Server 2012
– Renderer : OctaneRender Autodesk 3DS Max Plug-in
(ver. 1.18), is hosted in Autodesk 3DS Max 2014
– CPU : Intel Xeon E5-2660 2.20 GHz & 2.19 GHz
– RAM : 64 GB
– GPU : 3x Tesla K20X

19. Hardware Configuration
Rendering
Time
One node, with one K20X GPU 5:58:45
One node, with two K20X GPUs 3:11:46
One node, with three K20X GPUs 2:15:34
Two nodes, each with one K20X GPU 3:00:24
Two nodes, each with two K20X GPUs 1:36:37
Two nodes, each with three K20X GPUs 1:08:47
Rendering Experiment
19

20. • Rendering speed up for one node :
– 1 GPU -> 2 GPUs : 1.87x (~2.0x)
– 1 GPUs -> 3 GPUs : 2.65x (~3.0x)
– Similar trend for two nodes
• Rendering time for two nodes vs one (each node with the same # GPUs) is almost 0.5 : 1
– Currently one frame can be handled by only one GPU node
• Total acceleration : 1 node with 1 GPU only -> 2 nodes with 3 GPUs per node : 5.22x
Rendering experiment
20

21. • Conclusion
– Our system can aid movie production pipeline in term of efficiency and saving rendering cost
– We can obtain rendering time acceleration that scales almost linearly with the number of GPU nodes
• Future Work
– AMPS plug-in for 3D authoring tool (e.g. 3DS Max) so that the rendering job can be submitted directly
from the authoring tool
– Support for other 3D authoring tools (e.g. Maya and Blender) and other rendering software
– Support for heterogeneous GPU renderfarm (rendering nodes with different OS, brand of GPUs, and
maybe geographically separated)
Conclusion and Future Work
21

22. • Singapore National Research Foundation under its IDM Futures Funding Initiative and
administered by the Interactive & Digital Media Programme Office, Media Development
Authority
• Ministry of Education Singapore for the Tier-2 research funding support
• HP and NVIDIA for providing server nodes and GPUs
• Richmanclub Studios for the 3D models, test-bedding activities and feedbacks
• OTOY for providing the OctaneRender
Acknowledgements
22

23. Demonstration & Discussion