Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

[Case study]Utilize STLC data for Process Improvement

データを解析しプロセス改善につかった事例

  • Als Erste(r) kommentieren

[Case study]Utilize STLC data for Process Improvement

  1. 1. [Case study] Utilize STLC data for Process Improvement Jun 30th, 2020 Rajat Dayma Service Quality Assurance Group Leisure Product Dept. Commerce Company Rakuten, Inc.
  2. 2. 2 1. Self Introduction 2. Today’sTheme & Goal 3. What is STLC Data 4. a. Data collection and analysis 5. b. Metrix and analytics flow 6. Actual improvement: deployment issue 7. Future 8. Conclusion Agenda
  3. 3. 3 1. Self Introduction 2. Today’sTheme & Goal 3. What is STLC Data 4. a. Data collection and analysis 5. b. Metrix and analytics flow 6. Actual improvement: deployment issue 7. Future 8. Conclusion Agenda
  4. 4. 4 [Career Summary] > I am from India having more than 13+ years of experience in QA field > Worked in Banking, Finance, Insurance domains > Having around 7-8 years' experience working with Japanese clients > ISTQB Foundation level and JLPT N2 certified > Worked on service and product based companies both Self-Introduction
  5. 5. 5 1. Self Introduction 2. Today’sTheme & Goal 3. What is STLC Data 4. a. Data collection and analysis 5. b. Metrix and analytics flow 6. Actual improvement: deployment issue 7. Future 8. Conclusion Agenda
  6. 6. 6 Today’s Theme QA Role = Product and Process Quality https://www.kindpng.com/imgv/ihmmowT_this-free-icons-png-design-of-conveyor-with/
  7. 7. 7 Today’s Theme QA Role = Product and Process Quality Process Product https://www.kindpng.com/imgv/ihmmowT_this-free-icons-png-design-of-conveyor-with/
  8. 8. 8 Today’s Theme QA Role = Product and Process Quality Process https://www.kindpng.com/imgv/ihmmowT_this-free-icons-png-design-of-conveyor-with/ Today’s Goal
  9. 9. 9 1. Self Introduction 2. Today’sTheme & Goal 3. What is STLC Data 4. a. Data collection and analysis 5. b. Metrix and analytics flow 6. Actual improvement: deployment issue 7. Future 8. Conclusion Agenda
  10. 10. 10 STLC Data Plan TestDevelop In the testing phase, we view human error as a bug. As of June 2020, there are 277k of them recorded and many points of improvement to learn from the results It means that you can.
  11. 11. 11 STLC Data Plan TestDevelop In the testing phase, we view human error as a bug. As of June 2020, there are 277k of them recorded and many points of improvement to learn from the results It means that you can.
  12. 12. 12 STLC Data Plan TestDevelop In the testing phase, we view human error as a bug. As of June 2020, there are 277k of them recorded and many points of improvement to learn from the results It means that you can.
  13. 13. 13 STLC Data Plan TestDevelop STLC Database 277k+ results In the testing phase, we view human error as a bug. As of June 2020, there are 277k of them recorded and many points of improvement to learn from the results It means that you can.
  14. 14. 14 1. Self Introduction 2. Today’sTheme & Goal 3. What is STLC Data 4. a. Data collection and analysis 5. b. Metrix and analytics flow 6. Actual improvement: deployment issue 7. Future 8. Conclusion Agenda
  15. 15. 15 Metrix and analytics flow Category Meaning Example FUNCTION Do not work as per specification • Spec says ‘A’ link should be clickable but in app its not working DEGRADE Function worker earlier but not working now properly • In previous R1 execution “Button A” was working fine but in R2 its not working as expected REQUIREMENT Not mentioned or wrong explanation in specification • “A” user should able to purchase only one type of ticket but actually it should purchase all types of tickets PRD is wrong. EXISTING Existing product bug • QA found difference in PC an SP behavior but its expected based on usage UI Usability, Security, Performance issues • If any page is taking more time to load then its treated as usability issues ENVIRONMENT Environment issues which affects for our test execution • Because of wrong deployment of STG we raise the ticket INVALID QA misunderstood functionality • Spec completely misunderstood by QA LATER It’s bug but won’t be fixed this release. Future release • Valid bug wont be fixed in this release and considered for future releases NONFUNCTION (userability)(performance(security) • Performance or security related issues OTHER_SERVICE CWD, Coupon, Salesforce, Dependent on other teams to resolve • Bugs which are dependent on other systems to solve
  16. 16. 16 STLC Database 277k+ results Metrix and analytics flow
  17. 17. 17 STLC Database 277k+ results Metrix and analytics flow
  18. 18. 18 STLC Database 277k+ results Metrix and analytics flow
  19. 19. 19 1. Self Introduction 2. Today’sTheme & Goal 3. What is STLC Data 4. a. Data collection and analysis 5. b. Metrix and analytics flow 6. Actual improvement: deployment issue 7. Future 8. Conclusion Agenda
  20. 20. 20 STLC Data TestDevelop STLC Database We saw a lot of environmental issues that arose during the development phase and the testing phase. So we decided to focus on that.
  21. 21. 21 STLC Data TestDevelop STLC Database Developer We saw a lot of environmental issues that arose during the development phase and the testing phase. So we decided to focus on that.
  22. 22. 22 STLC Data TestDevelop STLC Database Test Environment Developer We saw a lot of environmental issues that arose during the development phase and the testing phase. So we decided to focus on that.
  23. 23. 23 STLC Data TestDevelop STLC Database Test Environment Developer 3rd party system We saw a lot of environmental issues that arose during the development phase and the testing phase. So we decided to focus on that.
  24. 24. 24 STLC Data TestDevelop STLC Database Test Environment Developer 3rd party system We saw a lot of environmental issues that arose during the development phase and the testing phase. So we decided to focus on that.
  25. 25. 25 Proxy Issue 35 % of environment bug treatment time spent on Among them, “Deployment issue” category takes up the most time. Test Environment 3rd party system 3rd PartyTrouble #ofissues cumulativeratio Developer 35% Find the root cause
  26. 26. 26 Proxy Issue Source: xxx 35 % of environment bug treatment time spent on Among them, “Deployment issue” category takes up the most time. Test Environment 3rd party system Wrong setting 3rd PartyTrouble #ofissues cumulativeratio Developer Deployment Issue is the most 35% Find the root cause
  27. 27. 27 Once the target of the problem has been determined, we gather qualitative data through interviews as well as quantitative data. Find the root cause
  28. 28. 28 Ticket Ticket Comment Hearing Note https://ticket.server.com/ticket/browse/BUG-99848 Due to DB data difference. XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Environment settingissue Dev team YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY YYYYY https://ticket.server.com/ticket/browse/BUG-88090 【Expected value 】 is correct. XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XX Environment settingissue Dev team inserted YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-66427 Deployment for group XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Environment settingissue Updating content YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-88726 it is environment problem.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX. Deploytimingissue but unfortunately YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-77062 All the following functions are modified by both our project XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx Deploytimingissue Issue detail) YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-66345 this is environment issue. Because XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Deploytimingissue Issue detail) Multiple YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY YYYYYYYYYYYYYYYYYYYYYYYYYY Once the target of the problem has been determined, we gather qualitative data through interviews as well as quantitative data. Find the root cause
  29. 29. 29 Ticket Ticket Comment Hearing Note https://ticket.server.com/ticket/browse/BUG-99848 Due to DB data difference. XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Environment settingissue Dev team YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY YYYYY https://ticket.server.com/ticket/browse/BUG-88090 【Expected value 】 is correct. XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XX Environment settingissue Dev team inserted YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-66427 Deployment for group XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Environment settingissue Updating content YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-88726 it is environment problem.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX. Deploytimingissue but unfortunately YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-77062 All the following functions are modified by both our project XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx Deploytimingissue Issue detail) YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-66345 this is environment issue. Because XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Deploytimingissue Issue detail) Multiple YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY YYYYYYYYYYYYYYYYYYYYYYYYYY Once the target of the problem has been determined, we gather qualitative data through interviews as well as quantitative data. Find the root cause
  30. 30. 30 Ticket Ticket Comment Hearing Note https://ticket.server.com/ticket/browse/BUG-99848 Due to DB data difference. XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Environment settingissue Dev team YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY YYYYY https://ticket.server.com/ticket/browse/BUG-88090 【Expected value 】 is correct. XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XX Environment settingissue Dev team inserted YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-66427 Deployment for group XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Environment settingissue Updating content YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-88726 it is environment problem.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX. Deploytimingissue but unfortunately YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-77062 All the following functions are modified by both our project XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx Deploytimingissue Issue detail) YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY https://ticket.server.com/ticket/browse/BUG-66345 this is environment issue. Because XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Deploytimingissue Issue detail) Multiple YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY YYYYYYYYYYYYYYYYYYYYYYYYYY Once the target of the problem has been determined, we gather qualitative data through interviews as well as quantitative data. Find the root cause
  31. 31. 31 Calculate ROI : Because the number of man-hours spent on solving the problem and the value of the solution It's to reflect on the economic value or not. Calculate Return on Investment
  32. 32. 32 Calculate ROI : Because the number of man-hours spent on solving the problem and the value of the solution It's to reflect on the economic value or not. Calculate Return on Investment
  33. 33. 33 Test Environment Developer Tester Currently we are losing around 300 hours efforts to solve deployment issue Calculate Return on Investment
  34. 34. 34 Test Environment Developer Tester ①Confirm & Create bug ticket Currently we are losing around 300 hours efforts to solve deployment issue Calculate Return on Investment
  35. 35. 35 Test Environment Developer Tester ①Confirm & Create bug ticket ②Investigate & Re-Deploy Currently we are losing around 300 hours efforts to solve deployment issue Calculate Return on Investment
  36. 36. 36 Test Environment Developer Tester ①Confirm & Create bug ticket ②Investigate & Re-Deploy ③Re-Test Currently we are losing around 300 hours efforts to solve deployment issue Calculate Return on Investment
  37. 37. 37 Test Environment Developer Tester ①Confirm & Create bug ticket ②Investigate & Re-Deploy ③Re-Test ①Confirm Create bug ticket : X hours ②Investigate & Re-Deploy : Y hours ③Preparation & Re-Test : Z hours #of Dep issues / yar Currently we are losing around 300 hours efforts to solve deployment issue Calculate Return on Investment
  38. 38. 38 Test Environment Developer Tester ①Confirm & Create bug ticket ②Investigate & Re-Deploy ③Re-Test 300+ hours Lost ①Confirm Create bug ticket : X hours ②Investigate & Re-Deploy : Y hours ③Preparation & Re-Test : Z hours #of Dep issues / yar Currently we are losing around 300 hours efforts to solve deployment issue Calculate Return on Investment
  39. 39. 39 Calculate ROI : Because the number of man-hours spent on solving the problem and the value of the solution It's to reflect on the economic value or not. Calculate Return on Investment
  40. 40. 40 Pick up any number of projects and test them for additional man-hours and effectiveness Calculate Return on Investment
  41. 41. 41 Pick up any number of projects and test them for additional man-hours and effectiveness Calculate Return on Investment Project A Project B Project C Project D Project E Project F Project G Project H Project I Project J Project K Project L Project M Project N Project O Project P Project Q Project R
  42. 42. 42 Pick up any number of projects and test them for additional man-hours and effectiveness Calculate Return on Investment Project A Project B Project C Project D Project E Project F Project G Project H Project I Project J Project K Project L Project M Project N Project O Project P Project Q Project R
  43. 43. 43 We decided to target this category because this solution has no cost and some of LPD team has already implemented strong solution about that. Objective Key elements Root cause Reduction of environment al issues Deployment issue 3rd PartyTrouble Deploy timing issue Environment setting issueProxy Related Issue Other Solutions Calculate Return on Investment
  44. 44. 44 We decided to target this category because this solution has no cost and some of LPD team has already implemented strong solution about that. Objective Key elements Root cause Reduction of environment al issues Deployment issue 3rd PartyTrouble Deploy timing issue Environment setting issueProxy Related Issue Other Solutions Solution A $ Solution B $$$ Solution C $$ Calculate Return on Investment
  45. 45. 45 We decided to target this category because this solution has no cost and some of LPD team has already implemented strong solution about that. Objective Key elements Root cause Reduction of environment al issues Deployment issue 3rd PartyTrouble Deploy timing issue Environment setting issueProxy Related Issue Other Solutions Solution A $ Solution B $$$ Solution C $$ Solution D $ Solution E $$$ Solution F $$ Calculate Return on Investment
  46. 46. 46 We decided to target this category because this solution has no cost and some of LPD team has already implemented strong solution about that. Objective Key elements Root cause Reduction of environment al issues Deployment issue 3rd PartyTrouble Deploy timing issue Environment setting issueProxy Related Issue Other Solutions Solution A $ Solution B $$$ Solution C $$ Solution D $ Solution E $$$ Solution F $$ Calculate Return on Investment
  47. 47. 47 Set the target
  48. 48. 48 Source: xxx Before After 35% 2.38% Result
  49. 49. 49 Once proven effective, we will apply it to all projects. Adaptation to the all projects Project A Project B Project C Project D Project E Project F Project G Project H Project I Project J Project K Project L Project M Project N Project O Project P Project Q Project R
  50. 50. 50 1. Self Introduction 2. Today’sTheme & Goal 3. What is STLC Data 4. a. Data collection and analysis 5. b. Metrix and analytics flow 6. Actual improvement: deployment issue 7. Future 8. Conclusion Agenda
  51. 51. 51 STLC Database 277k+ results Future Of course, there are still many problems that can be solved with STLC database analysis. So in the future we want to keep improving! Branch merge issueEnvironment Isolation issue Requirement issue
  52. 52. 52 1. Self Introduction 2. Today’sTheme & Goal 3. What is STLC Data 4. a. Data collection and analysis 5. b. Metrix and analytics flow 6. Actual improvement deployment issue 1. Result 01 2. Result 02 3. Summary 7. Future 8. Conclusion Agenda
  53. 53. 53 Conclusion QA Role = Product and Process Quality Process Product https://www.kindpng.com/imgv/ihmmowT_this-free-icons-png-design-of-conveyor-with/

    Als Erste(r) kommentieren

  • kunioyamamoto519

    Jun. 30, 2020

データを解析しプロセス改善につかった事例

Aufrufe

Aufrufe insgesamt

1.520

Auf Slideshare

0

Aus Einbettungen

0

Anzahl der Einbettungen

967

Befehle

Downloads

5

Geteilt

0

Kommentare

0

Likes

1

×