Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
December 6, 2014 
Andrii Dzynia 
Road Manager to Continuous Delivery 
@adzynia 
Quality Built In
2 
Spotify brings you the right music for 
every moment! 
Started in 2006 (in Sweden) 
Now 1500+ employees, 500+ engineers...
What is Quality? 
3
4 
Quality is a “state” …. 
when expectations matches the reality. 
Not just software defects, but stakeholders expectatio...
Why Quality is Important? 
5
Cost of Bug/Change 
6
What is a way to build high 
quality product? 
7
8 Do not release until it’s perfect
9 
No one is perfect.
Example 
Implementing internal back office system
11 
Step 1 - Prototyping phase 
Technical spike investigation 
Hacking different Back-End solutions 
Ad-hoc design discuss...
12 WebApp 
with 
stubs 
Sketches 
on 
whiteboard 
Whatever works 
to get end-user 
understanding
13 
Step 2 - Setting delivery pipeline 
Add more unit tests 
Add more integration tests 
Add more functional tests 
Automa...
14 
http://blog.crisp.se/2013/02/05/yassalsundman/continuous-delivery-vs-continuous-deployment
15 
Step 3 - Connecting the dots 
Connect back-end and front-end parts 
into 
End-2-End system 
Get user’s feedback and it...
16 
Keep releasing small, usable, incremental pieces
17 
What is under the hood ?
18
Organic Structure 
19 
@joakimsunden Scaled Agile @ Spotify
20 
Test Automation Pyramid
21 
How to avoid Ice-Cream effect?
22 
Be strict with this rule.
Testable micro-services architecture 23 
Unit Tests 
Component Tests 
Functional 
Tests 
Integration Tests 
System/End-2-E...
24 
Unit Tests 
Check logic of minimal code snippet 
with no or mocked dependencies 
https://github.com/mockito/mockito 
h...
25 
Component Tests 
Very much like unit tests but emulating 
external components 
e.g. cassandra database 
https://github...
26 
Fake back-end for client apps 
Client App 
Fake Back-End Server 
Spotify Back End 
https://github.com/azagniotov/stubb...
27 
Integration Tests 
Check an integration between components 
You do not need whole system to check the integration 
htt...
28 
Create Test Environment ‘on the fly’ 
@docker containers 
https://www.docker.com
29 
Functional Tests 
Check services API behaviour 
or 
End user use cases 
http://hc.apache.org 
http://seleniumhq.org
30 
Model Based Testing 
http://graphwalker.org
32 
Code Review 
& 
Human Testing 
on 
Pull Request
Delivery Pipeline 
33
34 
Was not covered during this talk … 
Load Testing 
Gradual Rollouts 
A/B Testing 
Feature Toggles 
Monitoring/Alerting ...
35 
Test Engineering Roles and Responsibilities 
Technical Test Engineer Test Engineer 
! = Test Automator ! = Manual Test...
36 
Technical Test Engineer 
Work as a software developer 
Advocate testability of the product 
Argue on software design w...
37 
Development Productivity Tools 
e.g. cli control over amazon cloud http://dashing.io 
Docker orchestration 
https://gi...
38 
Test Engineer 
Knows business domain 
Focused on exploring the product 
Free to use any programming language to test s...
39 
Mind Map as a Tool 
https://www.mindmup.com 
Product tree 
Scenarios 
Playbooks 
Checklists 
Session notes
Quality Engineers 
40 
@jamesmarcusbach calls us Test Jumpers 
TEST JUMPERS: ONE VISION OF AGILE TESTING 
http://www.satis...
41 
Any other ways 
to improve 
quality?
42 
Break Uncertainty 
test ideas during healthy 
discussions 
test requirements via end 
users collaboration 
discuss sys...
43 
Use Google Docs collaboration for finding group consensus 
But have responsible person to figure out what consensus me...
Social Programming 
44 
write “checks” during implementation 
shape your thoughts via pair discussions 
peer review before...
45 
Not mentioned Engineering Practices 
User Stories 
Planning meeting 
Standup meeting 
Open workspace 
TDD 
Refactoring...
Employee Early Builds Testing 
46
47 
Hackathons 
One week to do whatever 
you want 
Do it alone or with a team 
Crazy ideas comes out
48 
QA engineer at Spotify 
http://continuousdelivery.com/2014/02/visualizations-of-continuous-delivery/
How to check you are on 
a right path ? 
49
That is easy to spot 50
51 Ready for a journey? 
Make Quality explicit 
Find Quality promoters 
Define your way of Quality 
improvements 
Set righ...
52 
@adzynia 
adzynia@spotify.com 
Spotify Engineering Culture 
part 1 
part 2
and … 
Stop saying QA 
when you mean Testing 
Want to join the band? 
Check out http://www.spotify.com/jobs or @Spotifyjob...
Quality Built In @ Spotify
Nächste SlideShare
Wird geladen in …5
×

Quality Built In @ Spotify

24.235 Aufrufe

Veröffentlicht am

Most of the people think that quality in software development is limited to manual testing on the latest stage before releasing a product. That might be true 20 years ago in the industrial era. But current world is much more dynamic than before. Time to market became the most crucial metric nowadays. Releasing code to production need to be done faster and faster. How to maintain quality on a sufficient level in this fast paced environment? How to find a time to work on quality improvements? Those are two main questions I want to answer during this talk. Do not expect a silver bullet or even receipt to success. But definitely expect a lot of information about continuous delivery/deployment/improvements with a case studies and lessons we learned at Spotify.

Spotify Engineering Culture:
https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1/
https://labs.spotify.com/2014/09/20/spotify-engineering-culture-part-2/

Scaling Agile @ Spotify
http://blog.crisp.se/2012/11/14/henrikkniberg/scaling-agile-at-spotify

Scaled Agile @ Spotify
http://vimeo.com/111131934

Veröffentlicht in: Ingenieurwesen, Technologie
  • This is Amazing! He Won the Lotto Jackpot 7 Times, and Doesn't Mind Revealing His Secrets? ♣♣♣ http://t.cn/Airf5UFH
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • I seemed to underperform in my mock exams - achieving D's/E's but after following your strategy and advice, I achieved a 'B' grade in my final GCSE maths exam. I was chuffed because this result enabled me to study A-Level Chemistry. I've used your revision principles again and this has helped me immensely in this subject. Thank you so much Jeevan.. my 'B' grade will definitely help me in applying for a Pharmacy' course at University...➤➤ http://ishbv.com/jeevan91/pdf
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Jeevan's revision guide has helped understand questions which I did not understand before in class. It has really helped me learn things the easy way. It's straightforward and shows you how to get a top grade in GCSE maths, in a step-by-step format... ◆◆◆ http://t.cn/AirrSv7D
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • I didn't think failing my maths back in high school would come back to haunt me but it did! After graduating from Edge Hill University, I was looking to undertake a PGCE in primary teaching. However, one of the requirements was a pass in GCSE maths so I had no choice but to re-take it. After failing it twice, my confidence was very low and I didn't think I could qualify for the PGCE. After running a Google search on passing GCSE maths, I came across Jeevan's revision system. The feedback it received looked very good so I decided to give it a try... And it was one of the best decisions I've ever made. After going through Jeevan's guide, I managed to grasp the entire subject and I passed my next GCSE maths exam, with ease. He (Jeevan) provided me with all the tools I needed to prepare for the exam. Together with his guidance, my pass was a foregone conclusion. Thank you so much Jeevan! You have potentially changed my life. I recommend anyone seeking a pass grade in GCSE maths to purchase this fantastic package!♣♣♣ https://bit.ly/33W8jmf
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • I didn't think failing my maths back in high school would come back to haunt me but it did! After graduating from Edge Hill University, I was looking to undertake a PGCE in primary teaching. However, one of the requirements was a pass in GCSE maths so I had no choice but to re-take it. After failing it twice, my confidence was very low and I didn't think I could qualify for the PGCE. After running a Google search on passing GCSE maths, I came across Jeevan's revision system. The feedback it received looked very good so I decided to give it a try... And it was one of the best decisions I've ever made. After going through Jeevan's guide, I managed to grasp the entire subject and I passed my next GCSE maths exam, with ease. He (Jeevan) provided me with all the tools I needed to prepare for the exam. Together with his guidance, my pass was a foregone conclusion. Thank you so much Jeevan! You have potentially changed my life. I recommend anyone seeking a pass grade in GCSE maths to purchase this fantastic package!♣♣♣ http://t.cn/AirrSv7D
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

Quality Built In @ Spotify

  1. December 6, 2014 Andrii Dzynia Road Manager to Continuous Delivery @adzynia Quality Built In
  2. 2 Spotify brings you the right music for every moment! Started in 2006 (in Sweden) Now 1500+ employees, 500+ engineers Over 30 million songs available Over 20,000 songs added every day 5 development centres across the globe
  3. What is Quality? 3
  4. 4 Quality is a “state” …. when expectations matches the reality. Not just software defects, but stakeholders expectations as well
  5. Why Quality is Important? 5
  6. Cost of Bug/Change 6
  7. What is a way to build high quality product? 7
  8. 8 Do not release until it’s perfect
  9. 9 No one is perfect.
  10. Example Implementing internal back office system
  11. 11 Step 1 - Prototyping phase Technical spike investigation Hacking different Back-End solutions Ad-hoc design discussions Stub implementation of Front End Getting early user’s feedback on Front End
  12. 12 WebApp with stubs Sketches on whiteboard Whatever works to get end-user understanding
  13. 13 Step 2 - Setting delivery pipeline Add more unit tests Add more integration tests Add more functional tests Automate deployment configuration Add dashboards and system monitoring Clarifying End User Needs
  14. 14 http://blog.crisp.se/2013/02/05/yassalsundman/continuous-delivery-vs-continuous-deployment
  15. 15 Step 3 - Connecting the dots Connect back-end and front-end parts into End-2-End system Get user’s feedback and iterate
  16. 16 Keep releasing small, usable, incremental pieces
  17. 17 What is under the hood ?
  18. 18
  19. Organic Structure 19 @joakimsunden Scaled Agile @ Spotify
  20. 20 Test Automation Pyramid
  21. 21 How to avoid Ice-Cream effect?
  22. 22 Be strict with this rule.
  23. Testable micro-services architecture 23 Unit Tests Component Tests Functional Tests Integration Tests System/End-2-End Tests Unit Tests Component Tests Functional Tests JS Unit Tests JS Component Tests UI Functional Tests
  24. 24 Unit Tests Check logic of minimal code snippet with no or mocked dependencies https://github.com/mockito/mockito https://github.com/junit-team/junit
  25. 25 Component Tests Very much like unit tests but emulating external components e.g. cassandra database https://github.com/jsevellec/cassandra-unit
  26. 26 Fake back-end for client apps Client App Fake Back-End Server Spotify Back End https://github.com/azagniotov/stubby4j https://github.com/dreamhead/moco http://jsonstub.com
  27. 27 Integration Tests Check an integration between components You do not need whole system to check the integration https://github.com/spotify/helios/blob/master/docs/testing_framework.md
  28. 28 Create Test Environment ‘on the fly’ @docker containers https://www.docker.com
  29. 29 Functional Tests Check services API behaviour or End user use cases http://hc.apache.org http://seleniumhq.org
  30. 30 Model Based Testing http://graphwalker.org
  31. 32 Code Review & Human Testing on Pull Request
  32. Delivery Pipeline 33
  33. 34 Was not covered during this talk … Load Testing Gradual Rollouts A/B Testing Feature Toggles Monitoring/Alerting Information Radiators … but stay tuned
  34. 35 Test Engineering Roles and Responsibilities Technical Test Engineer Test Engineer ! = Test Automator ! = Manual Tester ~ Software Engineer in Test ~ Context Driven Tester
  35. 36 Technical Test Engineer Work as a software developer Advocate testability of the product Argue on software design with software engineers Help with building new/injecting existed development tools
  36. 37 Development Productivity Tools e.g. cli control over amazon cloud http://dashing.io Docker orchestration https://github.com/spotify/helios Cassandra Unit https://github.com/jsevellec/cassandra-unit Spoticloud Jenkins job-dsl-plugin https://github.com/mikaellanger/job-dsl-plugin Dashboards
  37. 38 Test Engineer Knows business domain Focused on exploring the product Free to use any programming language to test specific use case Free to use any programming language to automate his work
  38. 39 Mind Map as a Tool https://www.mindmup.com Product tree Scenarios Playbooks Checklists Session notes
  39. Quality Engineers 40 @jamesmarcusbach calls us Test Jumpers TEST JUMPERS: ONE VISION OF AGILE TESTING http://www.satisfice.com/blog/archives/1372 “RESPONSIBLE TESTER” http://www.satisfice.com/blog/archives/1364 OMEGA TESTER http://www.satisfice.com/articles/omega_tester.pdf
  40. 41 Any other ways to improve quality?
  41. 42 Break Uncertainty test ideas during healthy discussions test requirements via end users collaboration discuss system design and conner cases earlier in the planing/design meetings define definition of done
  42. 43 Use Google Docs collaboration for finding group consensus But have responsible person to figure out what consensus means in each case
  43. Social Programming 44 write “checks” during implementation shape your thoughts via pair discussions peer review before merging to master
  44. 45 Not mentioned Engineering Practices User Stories Planning meeting Standup meeting Open workspace TDD Refactoring etc…… http://www.extremeprogramming.org/rules.html
  45. Employee Early Builds Testing 46
  46. 47 Hackathons One week to do whatever you want Do it alone or with a team Crazy ideas comes out
  47. 48 QA engineer at Spotify http://continuousdelivery.com/2014/02/visualizations-of-continuous-delivery/
  48. How to check you are on a right path ? 49
  49. That is easy to spot 50
  50. 51 Ready for a journey? Make Quality explicit Find Quality promoters Define your way of Quality improvements Set right Quality constraints Share results as early as possible Keep looking further quality improvements Probably you will never end :)
  51. 52 @adzynia adzynia@spotify.com Spotify Engineering Culture part 1 part 2
  52. and … Stop saying QA when you mean Testing Want to join the band? Check out http://www.spotify.com/jobs or @Spotifyjobs for more information

×