4. @bridgetkromhout
architecture
● All services in AWS
● Python (Django) main website
● Go microservices (video analytics ingest, on-the
fly image processing, bookmarking…)
● Upstreams routed via nginx
● Celery + SQS for async tasks
5. @bridgetkromhout
scale
● Streaming delivery through Akamai
● peak load at 10s of thousands of
requests per second to the website
● The sine wave has a 10-20x differential
throughout the week
8. private registry:
the official party line
https://github.com/docker/docker-
registry#quick-start
S3 is a storage engine option
but… a central registry server didn’t scale well
for us
@bridgetkromhout
9. private registry: dramafever
@bridgetkromhout
# this goes in /etc/default/docker to control
docker's upstart config
DOCKER_OPTS="--graph=/mnt/docker --insecure-
registry=local-repo-alias.com:5000"
● local-repo-alias.com in DNS with A record to 127.0.0.1
● OS X /etc/hosts: use the boot2docker host-only network IP
12. what even is flate?!
Pulling repository local-repo-alias.com:5000/www
4dda2b433370: Error pulling image (prod) from
local-repo-alias.com:5000/www, flate: corrupt
input before offset 54393671 flate: corrupt
input before offset 54393671
d497ad3926c8: Error downloading dependent layers
2014/12/07 02:34:54 Error pulling image (prod)
from local-repo-alias.com:5000/www, flate:
corrupt input before offset 54393671
@bridgetkromhout
13. registry rewrite coming!
DOCKER_OPTS="--graph=/mnt/docker --
insecure-registry=local-repo-alias.
com:5000 -e STORAGE_REDIRECT=true"
...until we get to the promised go lan(d|g), there’s a
workaround for the flate errors we’re seeing:
@bridgetkromhout
16. weekly base builds
FROM local-repo-alias.com:5000/www-base
● include infrequently-changing
dependencies
○ ubuntu packages
○ pip requirements
○ wheels
● other builds can start from these images
(so they’re faster):
@bridgetkromhout
24. before summer 2014
Vagrant for local development
chef-solo provisioner
17 minutes to install everything
@bridgetkromhout
25. now: boot2docker
devs pull down images built on jenkins
mysql image is built with fixtures
can run master or qa image (or even prod)
can build new local images from Dockerfiles
@bridgetkromhout
26. local registry for dev
docker run -d -p 5000:5000 --name
docker-reg -v ${DFHOME}:${DFHOME} -e
DOCKER_REGISTRY_CONFIG=${DFHOME}
/config/docker-registry/config.yml
public_registry_image
@bridgetkromhout
27. $ boot2docker ssh date -u
Mon Nov 24 16:09:02 UTC 2014
$ date -u
Tue Nov 25 01:43:49 UTC 2014
@bridgetkromhout
time is what turns kittens into cats
29. Devs can use their preferred editing environment:
-v ${DFHOME}/www:/var/www
We still want logs, too, so we expose those for the dev here:
-v ${DFHOME}/www/run:/var/log
volume mounting & our fork
@bridgetkromhout
Until 1.3 we ran a forked boot2docker
We needed to mount local files into the VM
30. containerizing front-end
useful for building front-end apps on Jenkins
also allows consistent testing
RUN apt-get install -y nodejs nodejs-legacy npm
RUN npm install -g grunt-cli@0.1.13
RUN npm install -g bower@1.3.8
RUN npm install -g phantomjs@1.9.7-14
ADD bower.json /var/www/dependencies/bower.json
RUN cd /var/www/dependencies && bower install --
allow-root
--config.interactive=false --force
@bridgetkromhout
38. $ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
local-repo-alias.com:5000/mysql dev b0dc5885f767 2 days ago 905.9 MB
local-repo-alias.com:5000/www dev 82cda604a4f1 2 days ago 1.092 GB
local-repo-alias.com:5000/micro local bed20dc84ea1 4 days ago 10.08 MB
google/golang 1.3 e3934c44b8e4 2 weeks ago 514.3 MB
public_registry_image 0.6.9 11299d377a9e 6 months ago 454.5 MB
scratch latest 511136ea3c5a 18 months ago 0 B
$
ever-smaller images
@bridgetkromhout
