Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Software Quality Lab
Markus Unterauer
Consultant and trainer
Lessons Learned from
measuring software development
processes
www.software-quality-lab.com | improve your quality
At the end of this session you know …
Agenda
 Introduction to process...
www.software-quality-lab.com | improve your quality
The Why? and first ideas on the How?
Introduction to process measureme...
www.software-quality-lab.com | improve your quality
Measurement as basis for all improvements
Why measure processes
Slide ...
www.software-quality-lab.com | improve your quality
Proper metrics show the way to our goal
Why measure processes
Slide 5
...
www.software-quality-lab.com | improve your quality
Boundaries show, when to act
Why measure processes
Slide 6
Upper bound...
www.software-quality-lab.com | improve your quality
Define proper metrics with G-Q-M
GQM for defining metrics
1. Goal
 Wh...
www.software-quality-lab.com | improve your quality
Example
GQM for defining metrics
Slide 8
Metric
Question
Goal We want ...
www.software-quality-lab.com | improve your quality
Question: Are we finding the right bugs?
Goal: Improve quality of our ...
www.software-quality-lab.com | improve your quality
So much for the theory, let‘s turn to practise …
Experience from pract...
www.software-quality-lab.com | improve your quality
Situation at our customer
Experience from practise
Slide 11
http://pix...
www.software-quality-lab.com | improve your quality
Goals for our customer
Experience from practise
 How bad is it really...
www.software-quality-lab.com | improve your quality
First attempt …
Experience from practise
 Our customer wanted to meas...
www.software-quality-lab.com | improve your quality
Some sources of values
Experience from practise
Slide 14
• Different l...
www.software-quality-lab.com | improve your quality
Everything very difficult
Experience from practise
Slide 15
http://pix...
www.software-quality-lab.com | improve your quality
New approach GQM
Experience from practise
1. Define two goals
 Ask ma...
www.software-quality-lab.com | improve your quality
Step 1: Define Goals
Experience from practise
1. We want to improve qu...
www.software-quality-lab.com | improve your quality
Step 2: Ask questions
Experience from practise
1. We want to improve q...
www.software-quality-lab.com | improve your quality
Step 3: Find metrics
Experience from practise
1. We want to improve qu...
www.software-quality-lab.com | improve your quality
Step 4: Define target values and boundaries
Experience from practise
S...
www.software-quality-lab.com | improve your quality
Step 5: Try out and validate
Experience from practise
Slide 21
http://...
www.software-quality-lab.com | improve your quality
Step 6: Reporting
Experience from practise
Slide 22
Trend for past the...
www.software-quality-lab.com | improve your quality
Document metrics in a metric handbook
Experience from practise
Slide 2...
www.software-quality-lab.com | improve your quality
From practise
Lessons Learned
Slide 24
www.software-quality-lab.com | improve your quality
Tips for starting your quality metric system
Lessons Learned
 Define ...
www.software-quality-lab.com | improve your quality
Questions?
Lessons Learned from measuring software development process...
Your Partner in Software Quality and Testing
Software Quality Lab GmbH
www.software-quality-lab.com
Consulting | Service |...
Nächste SlideShare
Wird geladen in …5
×

Lessons learned from measuring software development processes

664 Aufrufe

Veröffentlicht am

Get an impression on how we used GQM to define metrics on how to measure and improve quality and productivity in software development. Maybe our lessons learned might help you in your projects

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Lessons learned from measuring software development processes

  1. 1. Software Quality Lab Markus Unterauer Consultant and trainer Lessons Learned from measuring software development processes
  2. 2. www.software-quality-lab.com | improve your quality At the end of this session you know … Agenda  Introduction to process measurement  Experience from practise  Lessons learned from the project Slide 2
  3. 3. www.software-quality-lab.com | improve your quality The Why? and first ideas on the How? Introduction to process measurement Slide 3
  4. 4. www.software-quality-lab.com | improve your quality Measurement as basis for all improvements Why measure processes Slide 4 Define goal and path to reach it Discover, where we are Identify gap, to where we should be Find corrective measures Go next step Goal reached 
  5. 5. www.software-quality-lab.com | improve your quality Proper metrics show the way to our goal Why measure processes Slide 5 Source:DchicviaFlickr,https://www.flickr.com/photos/agenciamodels/6118171675
  6. 6. www.software-quality-lab.com | improve your quality Boundaries show, when to act Why measure processes Slide 6 Upper boundary Lower boundary Target value Measured value Time
  7. 7. www.software-quality-lab.com | improve your quality Define proper metrics with G-Q-M GQM for defining metrics 1. Goal  What should be done for whom?  Which aspects do we want to change? What is the context? e.g. „Improve the quality for all new projects“ 2. Question  Which questions guide us to the measurement goal? e.g. „Are we finding the right bugs?“ 3. Metric  How can we answer the questions using metrics? e.g. „Ratio of bugs found per severity class“ Slide 7 Define goals Find questions Answer questions
  8. 8. www.software-quality-lab.com | improve your quality Example GQM for defining metrics Slide 8 Metric Question Goal We want to improve the quality of our products! Are we finding the right bugs? Ratio of bugs per severity No. of bugs found after release Do we find the bugs instead of the users? Ratio of bugs found by QA vs. Users
  9. 9. www.software-quality-lab.com | improve your quality Question: Are we finding the right bugs? Goal: Improve quality of our products  Ratio of bugs per severity class  Number of bugs found after release  Per severity class  Ratio of recurring bugs  Average number of bugs found per test case  Number of bugs found by test level  Unit tests  Integration tests  System tests  User acceptance tests  Bugs found by type  Coding bug  Bad requirements  UI bug  … Slide 9
  10. 10. www.software-quality-lab.com | improve your quality So much for the theory, let‘s turn to practise … Experience from practise Slide 10
  11. 11. www.software-quality-lab.com | improve your quality Situation at our customer Experience from practise Slide 11 http://pixabay.com/en/desert-road-sand-forward-winds-609316/
  12. 12. www.software-quality-lab.com | improve your quality Goals for our customer Experience from practise  How bad is it really?  What should we do?  Set up an improvement cycle Slide 12 http://en.wikipedia.org/wiki/PDCA#/media/File:PDCA_Cycle.svg
  13. 13. www.software-quality-lab.com | improve your quality First attempt … Experience from practise  Our customer wanted to measure …  ~120 metrics  in 12 process fields with 21 processes  on 4 levels  from 14 different sources and source system  But …  for more than half of all metrics we could not get values  no one really knew, what to do with the metrics  management did not even look at it  effort for getting numbers was extremly high Slide 13
  14. 14. www.software-quality-lab.com | improve your quality Some sources of values Experience from practise Slide 14 • Different levels of detail • Errors and missing data • Bookings on different categories Time recording • Each project with different structure • Not comparable • Incomplete Project planning • Only for parts of all sourcecode • Difficult to query and analyze Source Control • Manually built Excel sheets by Project Managers • Each project with different structure • Not comparable Controlling sheets • Very high effort • Ambiguous questions Questionaires and interviews Good at first glance, but bad in detail …
  15. 15. www.software-quality-lab.com | improve your quality Everything very difficult Experience from practise Slide 15 http://pixabay.com/en/squirrel-reading-books-surprise-304021/
  16. 16. www.software-quality-lab.com | improve your quality New approach GQM Experience from practise 1. Define two goals  Ask management, what they would like to achieve  Define reporting 2. Ask questions 3. Find metrics to answer question  Only automatically measurable ones  Only one per question 4. Define target values and boundaries  Get target values from industry benchmarks 5. Try out and validate 6. Report to management Slide 16 http://pixabay.com/en/shield-directory-panic-fear-trust-492992/
  17. 17. www.software-quality-lab.com | improve your quality Step 1: Define Goals Experience from practise 1. We want to improve quality to the customer 2. We want to increase developer productivity Slide 17 Images from www.pixabay.com
  18. 18. www.software-quality-lab.com | improve your quality Step 2: Ask questions Experience from practise 1. We want to improve quality to the customer  Do we find bugs instead of the customer?  Are we testing enough?  Are we testing effectively? 2. We want to increase developer productivity  Do our developers spend too much time on bug fixing?  Are we developing fast enough?  Is the quality of our source code high enough? Slide 18 Images from www.pixabay.com
  19. 19. www.software-quality-lab.com | improve your quality Step 3: Find metrics Experience from practise 1. We want to improve quality to the customer  Do we find bugs instead of the customer? • Ratio of bugs found by us to bugs found by customer  Are we testing enough? • Number of test cases tested per developer day  Are we testing effectively? • Number of bugs found per tester day 2. We want to increase developer productivity  Do our developers spend too much time on bug fixing? • Effort-ratio for bug fixing per developer month  Are we developing fast enough? • Number of changed LoC per developer day  Is the quality of our source code high enough? • Number of bugs found by customer per developer month Slide 19 Images from www.pixabay.com
  20. 20. www.software-quality-lab.com | improve your quality Step 4: Define target values and boundaries Experience from practise Slide 20 Metric Target Value Current value Conclusion Changed LoC per developer day 30 – 40 LoC per day 123 LoC per Day Too high. Results in poor quality. Effort-ratio for bug fixing 25 - 35% 115% Far too high. Low productivity. Ratio of bugs found by customer 1 – 3% of all bugs 68% of all bugs Far too high. Testing not effective. … … … … Target range taken from literature and publications
  21. 21. www.software-quality-lab.com | improve your quality Step 5: Try out and validate Experience from practise Slide 21 http://pixabay.com/en/calculator-calculation-insurance-385506/
  22. 22. www.software-quality-lab.com | improve your quality Step 6: Reporting Experience from practise Slide 22 Trend for past thee months Textual comment from Quality Manager
  23. 23. www.software-quality-lab.com | improve your quality Document metrics in a metric handbook Experience from practise Slide 23 Metric Effort-ratio of bug fixing per developer month Goal Developers should produce high quality code and spend as much time for implementing new features as possible Benefit Fast development of new features High quality Formula Ratiobug fixing = Effortbug fixing / Total Effortdevelopment Method Automatic calculation from timesheets Datasource Time recording system Interval Monthly Responsible Process Manager
  24. 24. www.software-quality-lab.com | improve your quality From practise Lessons Learned Slide 24
  25. 25. www.software-quality-lab.com | improve your quality Tips for starting your quality metric system Lessons Learned  Define metrics top down, not bottom up  Goal > Question > Metric  Not: „Hey, look how many things we can measure!“  Concentrate on 5 - 10 metrics, e.g. the one presented before  Define lower and upper boundary together with your team  … or measure without boundaries for some iterations and define boundaries then  Start with automatically measurable metrics  When defining metrics, always have in mind, if you have all data necessary in the quality needed  Define in advance, what to do with the metrics Slide 25 Image:http://pixabay.com/en/bulb-light-lamp-electric-160207/
  26. 26. www.software-quality-lab.com | improve your quality Questions? Lessons Learned from measuring software development processes Slide 26
  27. 27. Your Partner in Software Quality and Testing Software Quality Lab GmbH www.software-quality-lab.com Consulting | Service | Academy | Tool Expertise Markus Unterauer markus.unterauer@software-quality-lab.com

×