Most of the people think that quality in software development is limited to manual testing on the latest stage before releasing a product. That might be true 20 years ago in the industrial era. But current world is much more dynamic than before. Time to market became the most crucial metric nowadays. Releasing code to production need to be done faster and faster. How to maintain quality on a sufficient level in this fast paced environment? How to find a time to work on quality improvements? Those are two main questions I want to answer during this talk. Do not expect a silver bullet or even receipt to success. But definitely expect a lot of information about continuous delivery/deployment/improvements with a case studies and lessons we learned at Spotify.
Spotify Engineering Culture:
https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1/
https://labs.spotify.com/2014/09/20/spotify-engineering-culture-part-2/
Scaling Agile @ Spotify
http://blog.crisp.se/2012/11/14/henrikkniberg/scaling-agile-at-spotify
Scaled Agile @ Spotify
http://vimeo.com/111131934
1. December 6, 2014
Andrii Dzynia
Road Manager to Continuous Delivery
@adzynia
Quality Built In
2. 2
Spotify brings you the right music for
every moment!
Started in 2006 (in Sweden)
Now 1500+ employees, 500+ engineers
Over 30 million songs available
Over 20,000 songs added every day
5 development centres across the globe
11. 11
Step 1 - Prototyping phase
Technical spike investigation
Hacking different Back-End solutions
Ad-hoc design discussions
Stub implementation of Front End
Getting early user’s feedback on Front End
12. 12 WebApp
with
stubs
Sketches
on
whiteboard
Whatever works
to get end-user
understanding
13. 13
Step 2 - Setting delivery pipeline
Add more unit tests
Add more integration tests
Add more functional tests
Automate deployment configuration
Add dashboards and system monitoring
Clarifying End User Needs
23. Testable micro-services architecture 23
Unit Tests
Component Tests
Functional
Tests
Integration Tests
System/End-2-End Tests
Unit Tests
Component Tests
Functional
Tests
JS Unit Tests
JS Component Tests
UI
Functional
Tests
24. 24
Unit Tests
Check logic of minimal code snippet
with no or mocked dependencies
https://github.com/mockito/mockito
https://github.com/junit-team/junit
25. 25
Component Tests
Very much like unit tests but emulating
external components
e.g. cassandra database
https://github.com/jsevellec/cassandra-unit
26. 26
Fake back-end for client apps
Client App
Fake Back-End Server
Spotify Back End
https://github.com/azagniotov/stubby4j
https://github.com/dreamhead/moco
http://jsonstub.com
27. 27
Integration Tests
Check an integration between components
You do not need whole system to check the integration
https://github.com/spotify/helios/blob/master/docs/testing_framework.md
28. 28
Create Test Environment ‘on the fly’
@docker containers
https://www.docker.com
29. 29
Functional Tests
Check services API behaviour
or
End user use cases
http://hc.apache.org
http://seleniumhq.org
34. 34
Was not covered during this talk …
Load Testing
Gradual Rollouts
A/B Testing
Feature Toggles
Monitoring/Alerting
Information Radiators
… but stay tuned
35. 35
Test Engineering Roles and Responsibilities
Technical Test Engineer Test Engineer
! = Test Automator ! = Manual Tester
~ Software Engineer in Test
~ Context Driven Tester
36. 36
Technical Test Engineer
Work as a software developer
Advocate testability of the product
Argue on software design with software engineers
Help with building new/injecting existed development tools
37. 37
Development Productivity Tools
e.g. cli control over amazon cloud http://dashing.io
Docker orchestration
https://github.com/spotify/helios
Cassandra Unit
https://github.com/jsevellec/cassandra-unit
Spoticloud
Jenkins job-dsl-plugin
https://github.com/mikaellanger/job-dsl-plugin
Dashboards
38. 38
Test Engineer
Knows business domain
Focused on exploring the product
Free to use any programming language to test specific use case
Free to use any programming language to automate his work
39. 39
Mind Map as a Tool
https://www.mindmup.com
Product tree
Scenarios
Playbooks
Checklists
Session notes
40. Quality Engineers
40
@jamesmarcusbach calls us Test Jumpers
TEST JUMPERS: ONE VISION OF AGILE TESTING
http://www.satisfice.com/blog/archives/1372
“RESPONSIBLE TESTER”
http://www.satisfice.com/blog/archives/1364
OMEGA TESTER
http://www.satisfice.com/articles/omega_tester.pdf
42. 42
Break Uncertainty
test ideas during healthy
discussions
test requirements via end
users collaboration
discuss system design and
conner cases earlier in the
planing/design meetings
define definition of done
43. 43
Use Google Docs collaboration for finding group consensus
But have responsible person to figure out what consensus means in each case
44. Social Programming
44
write “checks” during implementation
shape your thoughts via pair discussions
peer review before merging to master
45. 45
Not mentioned Engineering Practices
User Stories
Planning meeting
Standup meeting
Open workspace
TDD
Refactoring
etc……
http://www.extremeprogramming.org/rules.html
51. 51 Ready for a journey?
Make Quality explicit
Find Quality promoters
Define your way of Quality
improvements
Set right Quality constraints
Share results as early as possible
Keep looking further quality improvements
Probably you will never end :)