With an ever-increasing array of tools and technologies claiming to 'enable DevOps', how do we know which tools to try or to choose? In-house, open source, or commercial? Ruby or shell? Dedicated or plugins? It transpires that highly collaborative practices such as DevOps and Continuous Delivery require new ways of assessing tools and technologies in order to avoid creating new silos. Matthew Skelton shares his recent experience of helping many different organisations to evaluate and select tools to facilitate DevOps; the recommendations may surprise you. A talk given at DevOps Exchange (#doxlon) meetup group on 24th July 2014: http://www.meetup.com/DevOps-Exchange-London/events/194288152/

1. How to choose tools
for DevOps and
Continuous Delivery
Matthew Skelton, Skelton Thatcher Consulting
DevOps Exchange meetup group,
24 July 2014, London, UK
#doxlon

2. Collaboration
Learning
Singleton tools
Conway’s Law

3. Matthew Skelton
•15 years building &
operating software systems
•Cybernetics + Neuroscience
• control engineering
• psychology
• ‘network’ interactions
@matthewpskelton

4. PIPELINE
Conference
London Continuous
Delivery meetup
group (#londoncd)

5. Help orgs to adopt and sustain
good engineering practices
Interim CTO/Head of X, tech strategy,
architecture, workshops, delivery

6. Recent clients
•Tourism
•Betting & gambling
•Travel booking
•Financial data & reporting
•Healthcare

7. Common themes
•Online since ~1999
•Successful in their sector
•Large, central core database(s)
•Non-aligned goals
•Need to adopt DevOps and
Continuous Delivery

8. Common needs
•Technology selection ticklists
•Team interactions
•Collaboration opportunities
•Tools as catalysts

10. Continuous Delivery
•A scientific approach to
changing software systems
(Dave Farley)
•Regular, rapid, repeatable,
reliable changes

11. Continuous Delivery
•Quality
•Safety
•Reliability
•Psychology
•Effectiveness

13. DevOps
“Highly effective, daily collaboration
between software developers and IT
operations people to produce
relevant, working systems” *
*also QA/Testing, IT Service Desk,
Programme Management,
Commercial, Marketing, etc.

14. Not DevOps
“Automation”
“Build & Release”
“Infrastructure Development”
“System Administration”

16. Collaboration

17. 1. Version Control

24. 2. Deployment Pipeline

28. Challenges were:
•Limited Git skills in Service team
•Manual deployments
•‘Snowflake’ servers
•No CI
•Risks: security, automation, etc

29. http://bit.ly/thetrainline-weekly-deploy

30. What we did
•Built a walking skeleton pipeline
•Modelled security roles and stages
•Included manual steps (at first)
•Walked people through steps
•Finally: opened firewall so
everyone could see the UI

31. Results
•Security: happy
•Service team: relieved (& happy)
•Developers: won over (& happy)
•Business: surprised (& happy)

32. 3. Log Aggregation

33. LogStash
•ElasticSearch+ LogStash+ Kibana
•ELK
•In Production, Pre-Prod, Test
•On developer machines!!!

34. http://rashidkpc.github.io/Kibana/images/screenshots/searchss.png

35. Collaboration
& tool choice
Value collaboration as
a key criterion
Orthogonal to main purpose (?)
“How does [the use of] this tool
help people to collaborate?”

36. Learning

41. Learning & tool choice
Bring people with you
Appreciate current skills
Prefer achievable gains now
Avoid fear of too-scary tools

42. Singleton tools
(or the ‘Prize Bull’ approach)

44. Singleton tools
•Special database server
•Costly log aggregation
•Costly monitoring
•Server configuration

45. http://www.walpapershddownload.com/highland-cattle-wallpapers/

46. “Better features”?
Optimise globally across the
teams that need to collaborate

47. Singleton tool
Breaks feedback (learning) loop
from Production
Makes CI/CD more difficult
Underestimates value of
collaboration and learning

48. Conway’s Law

49. Mel Conway, 1968
“organizations which design
systems ... are constrained to
produce designs which are copies
of the communication structures
of these organizations”
http://www.melconway.com/Home/Conways_Law.html

51. DevOps Topologies

52. DevOps Topologies

53. DevOps Topologies

54. DevOps Topologies

55. Conway & Tool Choice
See the organisation as a system
Separate tools for separate teams
Shared tools for collaborative
teams
http://bit.ly/DevOpsTopologies

56. Conway’s Law
•Allan Kelly - @allankellynet
•https://vimeo.com/channels/londoncd

58. How to choose tools
for DevOps
Value collaboration aspects
Avoid a learning mountain:
evolve tooling
Avoid Production-only tools
Consider Conway’s Law
(this list is incomplete!)

59. Further reading
HighOps Operability eBook
bit.ly/OperabilityEbook
Build Quality In
buildqualityin.com

60. Thank you
matthewskelton.net / @matthewpskelton
skeltonthatcher.com
HT: @Squire_Matt, @alan_parkinson