Learn the fundamentals to calculating how effective your team is at finding bugs in your software. Using ServiceNow and Micro Focus ALM to demonstrate, we’ll show you how to automate the flow of information between your ALM and ITSM tools to monitor DDP in real time. Synchronizing incidents to your defect tracking tool is imperative if you want to perform root cause analysis to identify what regression test case “should have” trapped the defect during your last round of regression testing and trace it back. We’ll show you how this integration allows you to: Monitor your testing teams’ effectiveness in real time Demonstrate and improve regression test coverage Protect production environments with early trend detection

1. © Tasktop 2018
Improving Regression Test Effectiveness with Defect Detection Percentage (DDP)
August 22, 2018

2. © Tasktop 2018
Today’s Speaker
Experience
▪ 15+ years of ERP (SAP), Agile/DevOps, SaaS, and Large-Scale IT Project Delivery
▪ Many Shoes filled, including: Product Owner, Test Manager, Consultant, Program QA Lead, and System Analyst
▪ Consulted across 8 industry sectors, including: DoD (Army/Navy/DLA), Public Sector, Energy/Utilities, Financial Services, Retail, Manufacturing,
Healthcare and Real Estate.
• Participated in 10 end-to-end lifecycle SAP ECC and S/4 HANA implementations from 2002 through 2017.
▪ Founded VerticalRent®, Award-Winning Property Management SaaS Platform (2011-2018)
Education & Certifications
▪ B.S. in Operations & Information Systems Management, Penn State University
▪ Certified Scrum Master / Certified Micro Focus ALM.Net and UFT Practitioner
▪ Certified Fair Credit Reporting Act (FCRA) Compliance
Location & Hobbies
▪ Southwest Florida (Fort Myers) | +814.860.6872 mobile | matt.angerer@tasktop.com
▪ Reading and Writing about Technology and Real Estate
▪ Biking, Running, and Landscaping
Matt Angerer
Also Featured in…

3. © Tasktop 2018
Our Agenda for Today
Exploring the Mechanics of Defect Detection Percentage
▪ Industry Feedback
▪ What is DDP?
▪ How is it Calculated?
▪ Implications of DDP Calculation
▪ How Value Stream Integration Can Help
▪ What is Model-Based Integration?
▪ Live Demo of Tasktop Integration Hub
• ServiceNow – SQL Server – Micro Focus ALM
• Sample DDP Dashboards
▪ Q&A

4. © Tasktop 2018
Current Trends in Software QA
and Testing
According to the World Quality Report
According to..
› Survey results indicate that ‘hybrid Agile’ framework is the new norm, with organizations adopting a
combination of SAFe, Scrum, Kanban, XP, DSDM, and Waterfall models in varying combinations.
› 99% of survey respondents say they have difficulties with testing in Agile development. This is extremely
worrying, particularly as challenges persist across-the-board, with little change since last year’s study.
› A common challenge among practitioners are ‘difficulties in identifying the right areas on which test
should focus’ and ‘early involvement of testing team in inception phase or sprint planning.’
› Research shows that there is a large proportion of Test Managers lacking skills in test strategy and
design. This has grown from 22% in 2016 to 32% in 2017. Many QA Leaders and Test Managers have
difficulty with the Eagle’s Eye View required to uncover inefficiencies in their end-to-end process.
› A key recommendation of the World Quality Report is for organizations to define their QA Analytics
Strategy at an enterprise-level.

5. © Tasktop 2018
Are We Measuring the “Right Stuff”?
There’s evidence in the 9th edition of the World Quality Report that your peers are learning to strike the balance. Up to 41% of its sample
shows a return to the main objective of QA & Testing—to improve the overall quality of software, products, and processes. Call it
historic recurrence or mean-reversion, but the leaders are going “back to basics”.

6. © Tasktop 2018
Common QA and Testing Metrics
Tests
• Authored Tests
Manual vs. Automated
• Passed Tests
Manual vs. Automated vs. Exploratory
• Failed Tests
• Tests Not Executed
• Tests Blocked
• …. And the List Goes On!
Defects
• Active Defects
• Defects Fixed Per Day
• Severe Defects
• Rejected Defects
Requirements
• Covered Requirements
• Passed Requirements
• Reviewed Requirements
Test Execution Velocity is commonly measured as well…..

7. © Tasktop 2018
But, what about Effectiveness?
We are NOT talking about the Effectiveness of….
• How easily you can build a new automated test…… or
• How skilled your team is at designing lots of manual tests….. or
• Whether or not a user story is “covered” by a test or not…… or
• How quickly your team re-tests a bug fix to meet your sprint dates…. or
• How many defects your team found in the last release….. or
• How quickly your team can adjust/update automated tests….
We are talking about the effectiveness of PROTECTING your
PRODUCTIVE environment from failures (incidents!).

8. © Tasktop 2018
Measuring Test Effectiveness
ef·fec·tive·ness
the degree to which something is successful in producing a desired result; success.
Synonyms: potency, power
Q: How effective is our testing at protecting our productive environments?
Q: What data can you show to support your team’s testing effectiveness?
Q: Are our QA processes improving the quality of our software over time?
Q: Can you estimate how much (money) the QA team has saved our organization?

9. © Tasktop 2018
Time for a quick poll……
Do you track Defect Detection Percentage (DDP),
AKA Defect Detection Effectiveness (DDE) today?

10. © Tasktop 2018
Here are the basic ingredients….
Bugs Incidents
Do you track bugs and incidents?
Do you track what testing layer a bug was found (e.g., System vs. UAT vs. Regression)?
Do you track what release a bug-fix was moved to production in?
Of course you do!
Then why aren’t you monitoring DDP in real-time for every release?
Do you track the date/time an incident was logged by your end user and/or operations team?

11. © Tasktop 2018
DDP is Simple, Yet Complex
Defects Logged UndetectedIncidents Logged
Testing
Stage(s)
Deploy Stop
The Complexity Stems from Multiple Testing Stages and Systems
Defects Logged in Test Phase
Total Defects + After Deploy Point

12. © Tasktop 2018
DDP is a Simple Calculation
20 + 1220 20 / 32 DDP = 63%

13. © Tasktop 2018
Live Chart
(To Be Seen in Demo)

14. © Tasktop 2018
Live Chart with Cost of Quality
(To Be Seen in Demo)

15. © Tasktop 2018
Implications of DDP
High DDP of 90%+:
• Testing is outstanding!
• Limited Usage of the Application Under Test in a real-world scenario
during your measurement period.
Low DDP (< 65%):
• Ambiguous Requirements resulting in Low Quality Tests?
• Lack of QA & Testing Involvement Early-On in SDLC?
• Limited use of Data Sets for Automated Regression Test Cycles?
• Inexperienced Exploratory Testers?
• Unstable Software with Not Enough Time to Test!

16. © Tasktop 2018
Defects, Vulnerabilities, & Incidents Converge

17. © Tasktop 2018
DDP and Cost of Quality
Cost of Quality
is a powerful way to show your
leadership team the monetary value
associated with your team’s efforts.
It includes:
▪ The Cost of Defect Prevention
▪ The Cost of Defect Detection
▪ The Cost of Failures (both Pre and
Post Production)

18. © Tasktop 2018
How can Value Stream Integration Help?
Value Stream Integration is about
connecting the people and activities
in the software delivery toolchain by
automating the flow of value from
ideation to production.

19. Project
Managers
Business
Analysts
Developers Testers
Business Customers
Operations

20. Project
Managers
Business
Analysts
Developers Testers
Business Customers
Operations
Speed
Quality (Scalable)
Compliance (Traceable)
Demands
Alerts
Issues

21. © Tasktop 2018
PROJECT MANAGERS BUSINESS ANALYSTS DEVELOPERS TESTERS OPERATIONS

22. © Tasktop 2018
PROJECT MANAGERS BUSINESS ANALYSTS DEVELOPERS TESTERS OPERATIONS

23. © Tasktop 2018
Why Model-Based Integration?

24. © Tasktop 2018

25. © Tasktop 2018
Demo -
Tasktop
Integration
Hub

26. © Tasktop 2018
Any Questions?

27. © Tasktop 2018
E-Book and Tableau Packaged Workbook
Check your inbox
tomorrow for your copy of
our brand new e-book.
Using Tableau?
Jump start your DDP report creation with a packaged
workbook of the report from today’s demo. Contact us at
https://www.tasktop.com/contact-us to request.