From the Open Source North conference, June 9, 2016:
Donnie Berkholz will present an introduction to DevOps, then open it up to questions and discussion. Topics will include Docker and microservices. Wherever you are in your DevOps journey, there will be something for you in this session.
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
DevOps 101+: From collaboration to microservices
1. DevOps 101+:
From collaboration to microservices
Donnie Berkholz, Ph.D.
Research Director — Development, DevOps, & IT Ops
Open Source North, June 2016
8. Polyglot
programming
There’s no obvious choice for the right
language, based on community
adoption.
8
Donnie Berkholz Source: http://redmonk.com/dberkholz/2014/05/02/github-language-trends-and-the-fragmenting-landscape/
19. Where are we today?
19
10.0%
54.7%
27.9%
6.8%
0.7%
Highly Manual
Manual with Limited Automation Tools
Automated with Manual Exception Handling
Policy Based Automation and Orchestration
Other
n = 843
Source: 451 VotE Cloud, Q3 2015
23. DevOps tools in use still vary widely
23
33.83%
37.31%
39.30%
39.30%
40.80%
40.80%
44.28%
45.77%
51.24%
51.74%
63.18%
Infrastructure as a Service
Build and continuous integration…
Application…
Traditional middleware and…
Release management
Platform as a Service
QA planning and automation tools
Configuration management &…
Performance Monitoring and…
Project management tools
Testing
Source: 451 Research/Red Hat, Q1 2016, n=201
26. 26
Aren’t they just like VMs? No.
Source: 451 Research, “Now Shipping: The Docker and containers ecosystem rapidly takes shape”
27. Containers vs VMs: no clear approach
27
451 Research, Voice of the Enterprise: Software-Defined Infrastructure, Q4 2015
10.9%
14.6%
9.0%
Containers Run Separately from VMs
Containers Run On Top Of VMs
Containers Are Replacing VMs
n = 458
30. 56.1%
10.7%
3.9%
4.2%
2.1%
22.9%
31.5%
10.2%
8.4%
9.4%
4.7%
35.8%
Discovery and Evaluation
Running Trials/Pilot Projects
In Test and Development Environment
Initial Implementation of Production
Applications
Broad Implementation of Production
Applications
No Plans
Q1 2015 Q3 2015
Docker is not just a toy
30
14.1%}
Source: 451 VotE Cloud, 2015; Q1 n=991; Q3 n=960
of cloud-using orgs
Prod in 3Q15:
31. 56.1%
10.7%
3.9%
4.2%
2.1%
22.9%
31.5%
10.2%
8.4%
9.4%
4.7%
35.8%
Discovery and Evaluation
Running Trials/Pilot Projects
In Test and Development Environment
Initial Implementation of Production
Applications
Broad Implementation of Production
Applications
No Plans
Q1 2015 Q3 2015
Docker is not just a toy
31
Source: 451 VotE Cloud, 2015; Q1 n=991; Q3 n=960
32.7%}of cloud-using orgs
Pilot+ in 3Q15:
33. 33
Loosely coupled services
“ The only
communication allowed
[at Amazon] is via
service interface calls
over the network.”
– Steve Yegge, Google, Oct 2011,
paraphrasing Jeff Bezos memo
https://plus.google.com/+RipRowan/posts/eVeouesvaVX
34. Loosely coupled teams
“ One of the biggest changes is that we no longer have
an official ‘architecture’ team. Instead, we have made
‘architecture’ an ‘ingredient’ on each of our teams.”
34
http://tech.gilt.com/post/102628539834/making-architecture-work-in-microservice
– Lauri Apple, Gilt Groupe, 14 Nov 2014
37. Container orchestration is limited (∴ adoption immature)
37
451 Research, Voice of the Enterprise: Cloud, Q3 2015
9.4%
36.1%
39.9%
14.6%
Currently use
Considering using in the next two years
Not familiar with these tools
Have no plans to use in the next two years
n = 534
44. Real-world example #2: REA (realestate.com.au)
44
http://techblog.realestate.com.au/a-microservices-implementation-retrospective/
45. REA microservices timeline
45
0
20
40
60
0 6 12 18 24
Microservices
Months
http://yowconference.com.au/slides/yow2014/SkurrieBottcherEvans-MonolithsToMicroservices.pdf
“ Microservices is a
long term strategy.”
– Evan Bottcher,
ThoughtWorks/REA,
9 Dec 2014
46. Real-world example #3:
Ctrip (Chinese travel site)
46
http://www.slideshare.net/yang75108/micro-service-architecture-c-trip-v11
47. Real-world example #3:
Ctrip (Chinese travel site)
47
http://www.slideshare.net/yang75108/micro-service-architecture-c-trip-v11
48. Real-world example #3:
Ctrip (Chinese travel site)
48
http://www.slideshare.net/yang75108/micro-service-architecture-c-trip-v11
49. The cloud-native movement is just about to take off
49
32%
32%
13%
13%
9%
Developing and running web-based
applications
Migrating legacy workloads and
applications to the cloud
Developing and running cloud
native applications
Managing legacy workloads,
applications and assets on the cloud
Testing new technologies and
methods
Source: 451 Research/Red Hat, Q1 2016, n=201
50. From primitives to platforms
50
ServerlessPaaSCaaS
Container
orchestration
IaaS /
Containers
OpinionatedFlexible
53. Some content from this presentation
is Creative-Commons licensed.
https://creativecommons.org/licenses/by-sa/
https://creativecommons.org/licenses/by/
53
0 talks at Glue last year, 12 this year
1 proposal at OSCON last year, 30 this year
Intro myself and credibility re containers/microservices
Q: “Allocate 100 points among the following five goals as they relate to your company or organization?”
Overall business goals
Enabled by cloud, open source, self-service IT
Does your company struggle with increased demands for shorter release cycles, with business managers expecting weekly, daily or even hourly releases and updates?
Cathedral, indulgences to bazaar
Open source, cloud, DigitalOcean
Shadow IT: IT as service provider, as vendor
Changing sales models – 7 red lines, demand gen (content mktg) vs lead gen
This one was hosting and cloud service providers
Languages, databases, frameworks
… What’s driving the way we build technology?
Excepting Angular and React
OODA loops
Culture: Collaboration, Sharing
Automation: Digital
Measurement: KPIs
As compared to ITIL: push down authority, plan iteratively, peer review, collaboration, automation
Developers vs DevOps
Release management – quarters to weeks to days to hours
Gary Gruver, HP
Little’s Law – work in system proportional to arrival speed plus time in system. Reduce batch sizes. Lean.
For cloud users: In general how would you describe your organization’s workload or service provisioning process?
Why?
Can’t improve what you can’t measure
Cattle vs pets (scale, transience)
Nagios (NetSaint): 1999 (MN co)
Community, UX
Overall 65.1%
Overall 39.6%
Data, not just code
Note this is screened for DevOps-leaning audience
Most orgs doing testing, but few automated pipelines
Q: “Approximately how often do you deploy software applications to production?”
Audience: People influential in cloud-services development
The next step in DevOps
How do we cope with these demands for agility, scalability, automation, transience?
Q30. What is your organization’s strategy (if any) around the use of containers (e.g. Docker)?
How does this change the infrastructure you’ve invested in for virtualization?
VotE shows most orgs are largely moved to virtualization, a minority to automation, few to orchestration/private cloud
Immutable infrastructure
Make sure to clarify difference between Docker and K8s
Of cloud-using orgs
Of cloud-using orgs
Business-defined separations.
Bounded context based on cross-organizational empathy.
Steve Yegge memo — Amazon must be SOA, or you’re fired.
DevOps + microservices
Bounded contexts, empathy defined
DevOps is how you build and run microservices.
Nomad out of HashiCorp, new competitor to Mesos/Kubernetes
Azure Container Service, building on Docker & Mesos. Beta by EOY 2015
Also note PaaS providers moving to containers
Windows catching up quickly with DevOps and containers
No need to leave the Microsoft half of your environment behind
Q16. What are your organization’s plans regarding a container orchestration tool?
Hard to add new features, slow provisioning, SPOF
Worked well: template project, resilience, idempotence, automation, exposing verbs
Wrote their own testing library to test consumers
Problems: right-sizing, code sharing (inheriting from common git repo and adding files worked)
Fit service into existing infra, don’t fit infra into service
Conway’s law – services fit org
Skill, ops overhead, complexity, testability
Fit service into existing infra, don’t fit infra into service
Conway’s law – services fit org
Skill, ops overhead, complexity, testability
Fit service into existing infra, don’t fit infra into service
Conway’s law – services fit org
Skill, ops overhead, complexity, testability
Q: “What are the top two uses of cloud-based platforms in your organization?”
Audience: Those responsible for or influencing cloud-services decisions
Most private PaaS options have bought into containers at this point
The future of service delivery.
DevOpsDays Minneapolis, July 20-21. $249, or $199 if you’re a startup, self-financed, etc