Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2dljkbl.
Adrian Trenaman provides detailed examples of how HBC Digital has used Docker and where container technology has paid off and, pragmatically, what aspects of the technology haven't panned out as they thought they would. Filmed at qconnewyork.com.
Adrian Trenaman works as SVP Engineering at HBC Digital. He leads the engineering and infrastructure teams for Gilt in New York and Dublin. He is an experienced, outspoken software engineer, communicator and leader with over 20 years of experience working with teams throughout Europe, US and Asia in diverse industries such as financial services, telecoms, retail, and manufacturing.
1. How containers have panned out
Adrian Trenaman, Raconteur & SVP Engineering, Gilt / HBC Digital
Q-Con, New York, June 2016
@gilttech @adrian_trenaman @hbc_tech
2. InfoQ.com: News & Community Site
âą 750,000 unique visitors/month
âą Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
âą Post content from our QCon conferences
âą News 15-20 / week
âą Articles 3-4 / week
âą Presentations (videos) 12-15 / week
âą Interviews 2-3 / week
âą Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
hbc-containers
3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
12. m > n
This is fundamentally a packing problem.
We have n machines, and we have m services to deploy.
13. 1
Itâs also an isolation problem
Any given service / team / engineer shouldnât be able to take out someone elseâs
work in production.
14.
15. Itâs also an impedance mismatch problem.
Developers often think of machines as something thatâs all theirs, magically
provided by the hardware fairy.
18. â Scalable, performant use of machine resources.
â Solves the impedance mismatch: developers see âa machineâ
â Limits the damage a single engineer can do.
â Infra/Devops engineer embedded into a tightly knit engineering team
â Static infrastructure
â Potential for resource hogging
LXC @ Gilt Japan
23. â Immutable deployment :)
â DNS + ELB traffic migration :)
â Slow to set up / tear down environments :(
â Potentially expensive under continuous deployment :(
â Open-source, but in-house. âA snowflake in the makingâ â
ION-Roller deployment:
24.
25. 6
âWe could solve this now, or, just wait six months, and Amazon will provide a
solutionâ
Andrey Kartashov, Distinguished Engineer, Gilt.
29. github.com/gilt/nova- deployment
Instance_0 - v1.0.0
Instance_1 - v1.0.0
Instance_2 - v1.0.0
Live Traffic
Instance_3 - v1.0.0 Canary
Instance_4 - v1.0.0
Dark
Canary
Elastic Load
Balancer (ELB)
live
Elastic Load
Balancer (ELB)
dark
$> nova deploy common DarkCanary
1.0.1
Instance_4 - v1.0.1
$> nova deploy common Canary 1.0.1
Instance_3 - v1.0.1
$> nova deploy common Production
1.0.1
Instance_0 - v1.0.1
Instance_1 - v1.0.1
Instance_2 - v1.0.1
CodeDeployS3bundle
30. â No docker registry (shock! gasp!) :)
â Less boilerplate code :)
â Immutable deployment (on mutable infrastructure) :)
â Leverage AWS tooling :)
ïŒ Next up? Integrate with Code Pipeline :?
Nova deployment:
31. Fighting bit rot, chaos-monkey style
With long running mutable AMIs, itâs possible for bit-rot to creep in.
Think security vulnerability.
Novel approach: every day, kill and restart your oldest AMI randomly.
â Pick up latest AMI with fixes
â Fail early, noisily and loudly if thereâs a problem without a production outage.
Vulnerability in container? Cut a new release against a fixed base-image.
33. Sundial - running batch jobs with Docker & ECS
â Job dependencies (allows us to break large jobs into smaller jobs)
â Ease of viewing logs and debugging failures
â Automatic rescheduling of failed tasks within a job
â Isolation between jobs
â Low cost of setup and maintenance, as few moving parts as possible for Infra
teams to manage
http://github.com/gilt/sundial
34. Sundial: processes
A process in Sundial is a grouping of tasks (jobs) with dependencies between
them.
Schedule: Either manually triggered, continuous schedule, or cron schedule
Overlap strategy: if previous iteration hasnât completed, do we
Wait
Terminate previous iteration
Run in parallel
When a process kicks off, all tasks with no dependencies kick off.
When a task finishes, any tasks blocked by that task will kick off.
35.
36. ECS is getting really attractive...
Weâre prototyping using for customer-facing services on our mobile team:
â Less configuration / moving parts than MST/Nova
â Automatic rollout
â Easy integration with IAM, CloudWatch, ECR
But:
â IAM roles at instance level not container level
â Tension between CF stack templates and deployment updates
â ELBs require fixed ports: we want to define the listening port.
38. Using docker as a
local build platform
The problem: keeping up
with different versions /
combinations of build
tools is crazy hard.
Why not use Docker for
build, using a versioned
build container?
docker-machine
Build Container
docker
39. Lesson #1
Containers have let us separate what we deploy (JVM, RoR, âŠ)
from how and where we deploy it (mst, nova, EC2, Triton)
and This Is Good.
40. Lesson #2
Itâs still a wild-west in terms of how containers are deployed.
Different teams have different needs - be sensitive to that.
42. Lesson #4
The competitive advantage: containers let us deploy quickly, frequently and safely
to production, which help us innovate faster.
Thatâs it.