Docker + GCE + etcd + ray tracing

1. Docker + GCE + etcd
+
ray tracing
@syoyo
Sunday, April 27, 14

2. ray tracing
Sunday, April 27, 14

3. Problem
• A lot of computing
resource required
• Imagine 60fps
photorealistic ray
tracing for OculusVR
• ~ 10K nodes
• Efficient deploy,
operation for the
renderer(ray tracer)
Sunday, April 27, 14

4. Why docker
• Hoping...
• Fine resource control
• CPU limit, memory limit, etc
• Easy versioning management of the
renderer
• Application sandboxing
Sunday, April 27, 14

5. My contribution (1/2)
Sunday, April 27, 14

6. Why etcd?
• Hoping...
• scalable management of cluster
information
• node IP addr, redis addr, etc.
• up to 10K nodes possible(?)
Sunday, April 27, 14

7. Why GCE
• CPU-intensive instance available
• fastVM boot, minute-level charge
• CoreOS image avaiable
• 282.0.0(Docker 0.10)
Sunday, April 27, 14

8. Render A Render B Render C Render D
Render E
etcd
Architecture
Sunday, April 27, 14

9. Container
• Busybox(8MB)
• Our renderer(single
binary. 72MB)
• node.js(11 MB)
• ~ 90 MBBusybox
Renderer(Ray tracer)
node.js
(application frontend)
Sunday, April 27, 14

10. Dockerfile
FROM syoyo/aobench
MAINTAINER Syoyo Fujita(syoyo@lighttransport.com)
ADD libdl.so.2 /lib64/
ADD librt.so.1 /lib64/
ADD lte /bin/lte
ADD worker.js /home/default/worker.js
ADD node /bin/node
ADD node_modules /home/default/node_modules
Sunday, April 27, 14

11. Render A Render B Render C
etcd
etcd: 172.17.42.1:4001
webfrontend
redis
Sunday, April 27, 14

12. Get redis info
from etcd
http.get(etcdHost + '/v2/keys/redis-server', function(res) {
if (res.statusCode != 200) {
console.log('failed to get redis infor from etcd.');
process.exit(-1);
}
res.on('data', function(chunk) {
var j = JSON.parse(chunk);
var redisURL = url.parse(j['node']['value']);
var redisServerAddr = redisURL['hostname']
var redisPort = redisURL['port']
var redisClient = redis.createClient(redisPort,
redisServerAddr);
Sunday, April 27, 14

13. Task processing
function loop() {
redisClient.brpop('render-q', timeout, function(err,
reply) {
// kick the ray tracer
exec(...);
// resubmit event loop
setTimeout(loop, 100);
});
}
Sunday, April 27, 14

14. • http://lighttransportengine.com/shader/
Sunday, April 27, 14

15. Future work
• ~10K nodes
• Leverage PCI-ex accelerator(GPU, Xeon
Phi) in Docker container.
• Confirmed InfiniBand working.
• Automation using Docker API
Sunday, April 27, 14