Scaling API-first â The story of a global engineering organization
Â
Quality Assurance in a DevOps World
1. Quality Assurance
in a
DevOps World
Enabling Trust
Kit Plummer
http://kitplummer.github.com
kplummer@maestrodev.com MaestroDev
Tuesday, February 22, 2011 1
2. DevOps is an ecosystem of...
â˘People (relationships)
⢠Processes (activities)
⢠Product(s)
MaestroDev
Tuesday, February 22, 2011 2
3. EcosystemâŚ
⢠Applied on top of columnar
organizations:
â Development/SW eng./design/etc.
â Quality assurance/testing/delivery
â Product eng./operations/customers
MaestroDev
Tuesday, February 22, 2011 3
4. Current DevOps thinking is...
⢠aligned with the Agile Manifesto,
⢠and prescribes itself as
a Cultural (organizational) and
Professional (personal) movement
.
MaestroDev
Tuesday, February 22, 2011 4
9. Trust is the cornerstone...
⢠Patrick Debois' math - based on
Covey's "4 Cores":
⢠trust = Ć(character,competence)
⢠character = Ć(integrity, intent)
⢠competence = Ć(capabilities,
results)
MaestroDev
Tuesday, February 22, 2011 9
10. Trust is a QA deliverable...
⢠Results are the indicator
⢠Testing is the apparatus
⢠Proof of quality is the target
MaestroDev
Tuesday, February 22, 2011 10
16. QA Frontend
⢠Agile
â QA/Testers on
team
â Stories -> UAT
â Continuous
Integration
MaestroDev
Tuesday, February 22, 2011 16
17. QA Backend
⢠Test harness(es)
â Integration
â Functional
â Non-functional
MaestroDev
Tuesday, February 22, 2011 17
18. QAâs DevOp Priorities
⢠Testing must be a ďŹrst-class citizen
MaestroDev
Tuesday, February 22, 2011 18
19. QAâs DevOp Priorities
⢠Testing must be a ďŹrst-class citizen
âWeâre not worried about
testing...our biggest
problem is automating
deploymentâ
MaestroDev
Tuesday, February 22, 2011 18
23. Push QA into Ops and Dev
⢠*DD
â Behavior DD (Agile QA?)
â Pulls non-functional from Ops
â Pushes non-functional to Devs
⢠Less Unit, Mo Integration, Less Mock
⢠Automate functional, but to a point
⢠Ops and QA enable Dev testing
MaestroDev
Tuesday, February 22, 2011 22
24. Push QA into Ops and Dev
⢠*DD
â Behavior DD (Agile QA?)
âBDD has reduced our
â Pulls non-functional from Ops
prerelease bugs by 80%,
â Pushes non-functional to Devs
in a side-by-side
⢠Less Unit, Mo Integration, Less Mock
⢠Automate functional, but to a point
comparoâ
⢠Ops and QA enable Dev testing
MaestroDev
Tuesday, February 22, 2011 22
25. Alternative Perspective
⢠Building missiles
â Platform shall have a maximum range of
100km
MaestroDev
Tuesday, February 22, 2011 23
27. Quality Quality
⢠Tests must be good
â More to it than just assertions
⢠True negatives? False Positives?
â Testing for Failure
⢠Brittle Tests? Avoid testing content
⢠Changes should come with tests
(open sourcey way)
MaestroDev
Tuesday, February 22, 2011 25
28. Truth isnât Good Enough
⢠Regressions (read blame)
â A single version change affects truth
⢠Measuring/Monitoring
â Earlier the better
⢠Trending (continuous improvement)
â Continuous improvement
MaestroDev
Tuesday, February 22, 2011 26