IPExpo talk about Chef Delivery and Reliable Infrastructure

1. BUILDING AND MANAGING
RELIABLE
INFRASTRUCTURE
Mandi Walls, Chef Software
IPExpo Europe, October 8, 2015
With Chef and Chef Delivery

2. WHO IS CHEF?
• Founded about 8 years ago
• Seattle-based, with offices in London and San
Francisco
• Customers include web-natives, retail, financial
services
• Core product, Chef, is client/server configuration
management

3. WHOAMI?
• Director of Consulting and Customer Success
EMEA
• Joined Chef in November 2011
• @lnxchk
• These slides will be uploaded
• http://www.slideshare.net/lnxchk

4. CORE CHEF
• Sophisticated toolset to produce reliable
infrastructure
• Tested, versioned, recorded

5. THE CODED BUSINESS
• Infrastructure as Code – the underlying idea
that computing components should be
describable in software, versioned, and tested
• Expanded to include risk-reduction, value
acceleration, and expanded trust

6. EASIER TO FIX BUGS
BEFORE CODE GETS TO
PRODUCTION

7. PUSHING MORE INTO
PRE-DEPLOY
• Testing with ChefSpec, Test Kitchen, Food
Critic
• Additional system controls via Chef Audit
• Don’t get to production and discover that the
security rules prevent the app from working
• Full 360° view of the application, no
assumptions, everything documented in code

8. NEED A TOOL THAT MAKES
SUBMITTING, APPROVING,
AND BUILDING FAST AND
EASY

9. WHY DELIVERY?
• Production of a repeatable pipeline for
software delivery
• Software isn’t creating value at the end of the
build; it creates value after deployment
• Application of key concepts to all code,
including the code that builds other code

10. NOT JUST APPLICATION
CODE
• We’re now working with infrastructure code,
too
• It should be auditable and traceable
• Incorporate good change management
behaviors
• Reduce risk

11. CHEF DELIVERY

12. HOW IT WORKS
CLI
Web
Browser
Job Dispatch
Push
Chef Server
erlang
Delivery
erlang
pgSQL
DB
git
SCM
Build NodeBuild NodeBuild Nodes
U-071982-C

13. U-071982-C
THE PIPELINE CONCEPT
• The build server should reflect modern
distributed infrastructures
• Preference for smaller, independent, loosely
coupled but dependent services to deliver a
fully functioning application
• Individual software projects ship on their own
schedule, but must integrate!

14. FAILURE TO SUCCESSFULLY
INTEGRATE MEANS THE BUILD
SHOULD FAIL BEFORE
PRODUCTION

15. STABLE PIPELINE
Steps
Create a new
change1
Test Change
Locally2
Verification Tests
Submit
Change3
Review
Change4
Build Artifacts
Acceptance Tests
Approve
Change5
Release Process
Deliver
Change6
manual automated
U-071982-C

16. THE SHARED PIPELINE
• Across various products and teams
• Coordination, integration
• New releases can trigger refreshes on
dependent components

17. COMMON PIPELINE
One
Pipeline
Delivery Pipeline - Shared
union rehearsal delivered
Acceptance Pipelines - Private
review approve deliverChange
Cookbook [A]
review approve deliverChange
Cookbook [B]
review approve deliverChange
Application [A]
review approve deliverChange
Application [B]
U-071982-C
Infrastructure & Applications

18. UNIFIED SHAPE
• The stages are fixed. No change goes to
production without flowing through the stages
• Stages include specific phases to reduce
confusion and increase predictability and
stability – these are made up of tasks you’re
probably already doing
• Your goal is to build software that has value
for you, not bikeshed on the pipeline. We did
that.

19. COMMON PIPELINE
Steps
manual automated
Verify
Lint
Syntax
Unit
Build
Merge
Lint
Syntax
Unit
Quality
Security
Publish
Provision
Deploy
Smoke
Functional
Compliance
Acceptance Union
Provision
Deploy
Smoke
Functional
Compliance
Rehearsal
Provision
Deploy
Smoke
Functional
Compliance
Delivered
Provision
Deploy
Smoke
Functional
Compliance
Stages
customizable
Verify Build
Acceptance
Union
Rehearsal
Delivered
Submit
Change3
Review
Change4
Approve
Change5
Deliver
Change6
Chef Delivery
Create a new
change1
Test Change
Locally2
Workstation
U-071982-C

20. CUSTOMIZE ACTIVITIES
• Within the phases, the actions taken are
customized to meet the needs of the
application
• Syntax checking, usability testing, etc, can
make use of existing tools
• Test nodes can be provisioned on a variety of
platforms and environments to meet
application needs

21. BUILD COOKBOOK
├── recipes
├── default.rb
├── lint.rb
├── syntax.rb
├── unit.rb
├── quality.rb
├── security.rb
├── publish.rb
├── provision.rb
├── deploy.rb
├── smoke.rb
└── functional.rb
U-071982-C

22. BUILD COOKBOOK
• Treating the definition of the build process
with the same care as the code that is going
through it
• Allows for versioning of the build definition,
tracking of changes, linking of new build
definition to new components

23. PHASE EXECUTION
log "Running unit"
repo = node['delivery_builder']['repo']
execute “run my junit tests" do
command "mvn test"
cwd repo
end
U-071982-C

24. COOKBOOK WORKFLOW
U-071982-C
Supermarket
Chef Server
review approve deliverChange
Cookbook
Node Node Node
Node Node Node
Node Node Node

25. APPLICATION WORKFLOW
U-071982-C
review approve deliverChange
Application
Node Node Node
Node Node Node
Node Node Node
Deploy
1 2 3
2 2 3
3 3 3

26. U-071982-C
CREATE NEW PIPELINES
• Project code repository
• Build cookbook *versioned!*
• Project configuration file *also versioned!*

27. CONFIG.JSON
{
"version": "1",
"build_cookbook": "./.delivery/build_cookbook",
"build_nodes": {
"default": [“name:*buildnode-*”],
"unit": [“name:*-unittest*”],
"lint": [“name:*-linttest*”],
"build": [“name:*-builder*”]
}
}
U-071982-C

28. • Delivery gives you a mechanism to create
fully-described build process for complex
applications
• The build process itself benefits from the same
version control and software practices the
code uses
• Prevent bugs and regressions from getting to
production with full integration retrospective
testing

29. MORE ABOUT DELIVERY
https://www.chef.io/delivery/
VISIT US
BOOTH GG8

30. UPCOMING EVENTS
• Next London Chef Meetup: October 20
• http://www.meetup.com/Chef-Users-London
• Chef Cookbook Workflow @ AWS Popup Loft
• https://www.chef.io/blog/event/chef-cookbook-workflow-
2/
• Chef Community Summit
• November 3-4 in London
• https://www.chef.io/summit/london/

31. NEW TO CHEF?
• https://learn.chef.io/
• Online trainings, in-person classroom training,
dates announced regularly

32. U-071982-C