SlideShare a Scribd company logo
1 of 14
Download to read offline
SOFTWARE TESTING
METRICS
Presenter : P.M.Venkatesh Babu
2
What is a METRIC
• Metrics can be defined as “STANDARDS OF
MEASUREMENT
• Metric is a unit used for describing or
measuring an attribute
• Test metrics are the means by which the
software quality can be measured
• Test metrics provides the visibility into the
readiness of the product, and gives clear
measurement of the quality and
completeness of the product
3
Why we need Metrics ?
• You cannot improve what you cannot measure
• You cannot Control what you cannot measure
 Without measurement it is impossible to tell whether the
process implemented is improving or not
 Metrics helps in taking decisions for next phase of
activities
 Metrics helps in understanding the type of improvement
required and helps in taking decisions on process or
technology change
4
Why Metrics in Software Testing?
There will be certain questions during and after testing such as :
 How long would it take to test ?
 How Bad / Good is the product?
 How many bugs still remain in the product?
 Will testing be completed on time?
 Was the testing done effectively?
 How much effort went into testing the product?
To Answer these questions properly we need some type of
measurements and record keeping to justify the answers.
This is where the testing metrics comes into picture.
Testing Metrics Life Cycle
 Analysis Phase:
• Identify the Metrics which has to be generated
• Define the identified Metrics
 Communication Phase:
• Explain the need of the Metrics to the stakeholders
• Educate the testing team about the data points for generating the Metrics
 Evaluation Phase:
• Capture and verify the data used for generating the Metrics
• Calculate the Metrics based on the data captured
 Reporting Phase :
• Develop the Metrics report and distribute to the stakeholders
• Take feedback from the stakeholders for any improvements
5
6
Types Of Metrics
Base metrics (Direct Measure)
• The Base metrics constitute the raw data gathered by the
test Engineers throughout the testing effort
• The Base metrics are used to provide project status reports
to the Test lead and to the project manager
• The Base metrics provide the input data to feed into the
formulas used to derive Calculated metrics
• Examples of Base metrics are:
# of test cases
# of test cases executed
7
Types Of Metrics (contd..)
Calculated Metrics (Indirect Measure)
• The Calculated Metrics convert the Base metrics data into
more useful information
• The Calculated Metrics are generally prepared by the Test
lead and is used to track the progress of the project at
different levels like at Module level, at Tester level and for
the project as a whole
• The Calculated Metrics provide valuable information that
when used and implemented often times leads to significant
improvements in the Overall SDLC
8
Base Metrics and Testing Phases
TEST METRIC TESTING PHASE
Number of test cases Test Development Phase
Number of Test cases Passed Test Execution Phase
Number of Test cases Executed Test Execution Phase
Number of Test cases Failed Test Execution Phase
Number of Test cases under
Investigation
Test Development Phase
Number of Test cases Blocked Test Dev / Execution Phase
Number of Test cases Re-
executed
Regression Phase
Number of First run Failures Test execution Phase
Total Executions Test Reporting Phase
Total Passes and Failures Test Reporting Phase
Test case Execution time Test Reporting Phase
Test Execution time Test Reporting Phase
9
Calculated Metrics and Phases
The Following Calculated metrics are created at Test Reporting
Phase or Post Test Analysis Phase:
 % of Test cases Passed
 % of Test Coverage
 % of Defects corrected
 % of Test cases Blocked
 % of Rework
 % of Test Effectiveness
 1st
Run Fail Rate
 Defect discovery rate
 Overall Fail rate
10
Test case Defect Density
The number of errors found in test cases v/s test cases developed and
executed
• ( Defective Test cases / Total Test cases ) * 100
Example : Total no of test cases developed is 1360, total test cases executed
is 1280, total no of test cases passed is 1065, total no of test scripts failed
is 215
So Test case Defect Density is :
215 X 100
------------------------------- = 16.8 %
1280
The 16.8 % value can also be called as Test Case Efficiency % which
depends upon the total number of Test cases which found defects
11
Defect Slippage Ratio
No of bugs reported from Production V/S No of defects reported during
execution
No of Defects slipped / ( Number of Defects Raised – Number Defects
Withdrawn) * 100
Example : Customer reported 21 defects, total no of defects found
while testing are, total no of Invalid defects are 17
So Slippage ratio is : [ 21 / (267 – 17) ] X 100 = 8.4%
12
Requirement Volatility Metric
This metric ensures that the requirements are normalized or defined
properly while estimating
No of requirements agreed V/S No of requirements changed
• (No of requirements Added + Deleted + Modified) * 100 / No of
original requirements
Example : SVN 1.3 release has 67 requirements initially, later 7 new
requirements are added, 3 requirements are deleted from initial
requirements and modified 11 requirements
Hence Requirement volatility is calculated as :
(7 + 3 + 11 ) X 100 / 67 = 31.34 %
This means that almost 1/3 of the requirements changed after the initial
identification of requirements
13
Conclusion
• The Test metrics should be reviewed & interpreted on regular basis
throughout the test effort and particularly after the application is
released into production
• There are several key factors in implementing and using the metrics
in the organization, beginning with determining the goal for
developing the metrics, followed by the identification of metrics to be
tracked and ending with sufficient analysis of the resulting data to be
able to make changes to the software development lifecycle
• - So finally Metrics themselves so not create improvements, they do
provide the objective information necessary to understand what
changes are necessary to be implemented
14
Questions ???

More Related Content

What's hot

Software testing metrics | David Tzemach
Software testing metrics | David Tzemach Software testing metrics | David Tzemach
Software testing metrics | David Tzemach David Tzemach
 
Software testing and process
Software testing and processSoftware testing and process
Software testing and processgouravkalbalia
 
Quality Assurance and Software Testing
Quality Assurance and Software TestingQuality Assurance and Software Testing
Quality Assurance and Software Testingpingkapil
 
Regression testing
Regression testingRegression testing
Regression testingMohua Amin
 
Types of software testing
Types of software testingTypes of software testing
Types of software testingTestbytes
 
Software Quality Assurance
Software Quality AssuranceSoftware Quality Assurance
Software Quality AssuranceSachithra Gayan
 
Software testing.ppt
Software testing.pptSoftware testing.ppt
Software testing.pptKomal Garg
 
Presentation On Software Testing Bug Life Cycle
Presentation On Software Testing Bug Life CyclePresentation On Software Testing Bug Life Cycle
Presentation On Software Testing Bug Life CycleRajon
 
Software Testing Metrics
Software Testing MetricsSoftware Testing Metrics
Software Testing MetricsVladimir Arutin
 
Intro to Manual Testing
Intro to Manual TestingIntro to Manual Testing
Intro to Manual TestingAyah Soufan
 
Software quality assurance
Software quality assuranceSoftware quality assurance
Software quality assuranceAman Adhikari
 
Software Testing Introduction
Software Testing IntroductionSoftware Testing Introduction
Software Testing IntroductionArunKumar5524
 
Non Functional Testing
Non Functional TestingNon Functional Testing
Non Functional TestingNishant Worah
 

What's hot (20)

SOFTWARE TESTING
SOFTWARE TESTINGSOFTWARE TESTING
SOFTWARE TESTING
 
Software testing metrics | David Tzemach
Software testing metrics | David Tzemach Software testing metrics | David Tzemach
Software testing metrics | David Tzemach
 
Software testing and process
Software testing and processSoftware testing and process
Software testing and process
 
Quality Assurance and Software Testing
Quality Assurance and Software TestingQuality Assurance and Software Testing
Quality Assurance and Software Testing
 
Regression testing
Regression testingRegression testing
Regression testing
 
Test Levels & Techniques
Test Levels & TechniquesTest Levels & Techniques
Test Levels & Techniques
 
Test cases
Test casesTest cases
Test cases
 
Types of software testing
Types of software testingTypes of software testing
Types of software testing
 
Software Quality Metrics
Software Quality MetricsSoftware Quality Metrics
Software Quality Metrics
 
Software Quality Assurance
Software Quality AssuranceSoftware Quality Assurance
Software Quality Assurance
 
Regression testing
Regression testingRegression testing
Regression testing
 
Software testing.ppt
Software testing.pptSoftware testing.ppt
Software testing.ppt
 
Testing & Quality Assurance
Testing & Quality AssuranceTesting & Quality Assurance
Testing & Quality Assurance
 
Presentation On Software Testing Bug Life Cycle
Presentation On Software Testing Bug Life CyclePresentation On Software Testing Bug Life Cycle
Presentation On Software Testing Bug Life Cycle
 
Software Testing Metrics
Software Testing MetricsSoftware Testing Metrics
Software Testing Metrics
 
Intro to Manual Testing
Intro to Manual TestingIntro to Manual Testing
Intro to Manual Testing
 
Software quality assurance
Software quality assuranceSoftware quality assurance
Software quality assurance
 
Software Testing Introduction
Software Testing IntroductionSoftware Testing Introduction
Software Testing Introduction
 
Introduction & Manual Testing
Introduction & Manual TestingIntroduction & Manual Testing
Introduction & Manual Testing
 
Non Functional Testing
Non Functional TestingNon Functional Testing
Non Functional Testing
 

Viewers also liked

Software Testing Services
Software Testing ServicesSoftware Testing Services
Software Testing ServicesFuad Mak
 
Coding by Example - Tutorial Agiles 2012
Coding by Example - Tutorial Agiles 2012Coding by Example - Tutorial Agiles 2012
Coding by Example - Tutorial Agiles 2012Wildtech
 
Coding standards
Coding standardsCoding standards
Coding standardsMimoh Ojha
 
Software coding & testing, software engineering
Software coding & testing, software engineeringSoftware coding & testing, software engineering
Software coding & testing, software engineeringRupesh Vaishnav
 
Coding standards and guidelines
Coding standards and guidelinesCoding standards and guidelines
Coding standards and guidelinesbrijraj_singh
 
Coding and testing in Software Engineering
Coding and testing in Software EngineeringCoding and testing in Software Engineering
Coding and testing in Software EngineeringAbhay Vijay
 

Viewers also liked (7)

Linux comands for Hadoop
Linux comands for HadoopLinux comands for Hadoop
Linux comands for Hadoop
 
Software Testing Services
Software Testing ServicesSoftware Testing Services
Software Testing Services
 
Coding by Example - Tutorial Agiles 2012
Coding by Example - Tutorial Agiles 2012Coding by Example - Tutorial Agiles 2012
Coding by Example - Tutorial Agiles 2012
 
Coding standards
Coding standardsCoding standards
Coding standards
 
Software coding & testing, software engineering
Software coding & testing, software engineeringSoftware coding & testing, software engineering
Software coding & testing, software engineering
 
Coding standards and guidelines
Coding standards and guidelinesCoding standards and guidelines
Coding standards and guidelines
 
Coding and testing in Software Engineering
Coding and testing in Software EngineeringCoding and testing in Software Engineering
Coding and testing in Software Engineering
 

Similar to Testing Metrics

Testing Metrics and Tools, Analyse de tests
Testing Metrics and Tools, Analyse de testsTesting Metrics and Tools, Analyse de tests
Testing Metrics and Tools, Analyse de testsHervKoya
 
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ... Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...Seapine Software
 
unit-2_20-july-2018 (1).pptx
unit-2_20-july-2018 (1).pptxunit-2_20-july-2018 (1).pptx
unit-2_20-july-2018 (1).pptxPriyaFulpagare1
 
Measurements &milestones for monitoring and controlling
Measurements &milestones for monitoring and controllingMeasurements &milestones for monitoring and controlling
Measurements &milestones for monitoring and controllingDhiraj Singh
 
Software Engineering (Testing Activities, Management, and Automation)
Software Engineering (Testing Activities, Management, and Automation)Software Engineering (Testing Activities, Management, and Automation)
Software Engineering (Testing Activities, Management, and Automation)ShudipPal
 
Software test management
Software test managementSoftware test management
Software test managementVishad Garg
 
Creating Functional Testing Strategy.pptx
Creating Functional Testing Strategy.pptxCreating Functional Testing Strategy.pptx
Creating Functional Testing Strategy.pptxMohit Rajvanshi
 
STLC (Software Testing Life Cycle)
STLC (Software Testing Life Cycle)STLC (Software Testing Life Cycle)
STLC (Software Testing Life Cycle)Ch Fahadi
 
1)Testing-Fundamentals_L_D.pptx
1)Testing-Fundamentals_L_D.pptx1)Testing-Fundamentals_L_D.pptx
1)Testing-Fundamentals_L_D.pptxgianggiang114
 
What is Test Matrix?
What is Test Matrix?What is Test Matrix?
What is Test Matrix?QA InfoTech
 
Health Care Project Testing Process
Health Care Project Testing ProcessHealth Care Project Testing Process
Health Care Project Testing ProcessH2Kinfosys
 
Software Testing Metrics
Software Testing MetricsSoftware Testing Metrics
Software Testing MetricsJatin Kochhar
 
DISE - Software Testing and Quality Management
DISE - Software Testing and Quality ManagementDISE - Software Testing and Quality Management
DISE - Software Testing and Quality ManagementRasan Samarasinghe
 
Fundamentals_of_Software_testing.pptx
Fundamentals_of_Software_testing.pptxFundamentals_of_Software_testing.pptx
Fundamentals_of_Software_testing.pptxMusaBashir9
 
Fundamentaltestprocess windirohmaheny11453205427 kelase
Fundamentaltestprocess windirohmaheny11453205427 kelaseFundamentaltestprocess windirohmaheny11453205427 kelase
Fundamentaltestprocess windirohmaheny11453205427 kelasewindi rohmaheny
 
Metrics based Management
Metrics based ManagementMetrics based Management
Metrics based ManagementSPIN Chennai
 

Similar to Testing Metrics (20)

Testing Metrics and Tools, Analyse de tests
Testing Metrics and Tools, Analyse de testsTesting Metrics and Tools, Analyse de tests
Testing Metrics and Tools, Analyse de tests
 
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ... Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
Use the Windshield, Not the Mirror Predictive Metrics that Drive Successful ...
 
unit-2_20-july-2018 (1).pptx
unit-2_20-july-2018 (1).pptxunit-2_20-july-2018 (1).pptx
unit-2_20-july-2018 (1).pptx
 
Measurements &milestones for monitoring and controlling
Measurements &milestones for monitoring and controllingMeasurements &milestones for monitoring and controlling
Measurements &milestones for monitoring and controlling
 
Software Engineering (Testing Activities, Management, and Automation)
Software Engineering (Testing Activities, Management, and Automation)Software Engineering (Testing Activities, Management, and Automation)
Software Engineering (Testing Activities, Management, and Automation)
 
QACampus PPT (STLC)
QACampus PPT (STLC)QACampus PPT (STLC)
QACampus PPT (STLC)
 
Software test management
Software test managementSoftware test management
Software test management
 
SDET UNIT 3.pptx
SDET UNIT 3.pptxSDET UNIT 3.pptx
SDET UNIT 3.pptx
 
Creating Functional Testing Strategy.pptx
Creating Functional Testing Strategy.pptxCreating Functional Testing Strategy.pptx
Creating Functional Testing Strategy.pptx
 
Test performance indicators
Test performance indicatorsTest performance indicators
Test performance indicators
 
STLC (Software Testing Life Cycle)
STLC (Software Testing Life Cycle)STLC (Software Testing Life Cycle)
STLC (Software Testing Life Cycle)
 
1)Testing-Fundamentals_L_D.pptx
1)Testing-Fundamentals_L_D.pptx1)Testing-Fundamentals_L_D.pptx
1)Testing-Fundamentals_L_D.pptx
 
What is Test Matrix?
What is Test Matrix?What is Test Matrix?
What is Test Matrix?
 
chapter 7.ppt
chapter 7.pptchapter 7.ppt
chapter 7.ppt
 
Health Care Project Testing Process
Health Care Project Testing ProcessHealth Care Project Testing Process
Health Care Project Testing Process
 
Software Testing Metrics
Software Testing MetricsSoftware Testing Metrics
Software Testing Metrics
 
DISE - Software Testing and Quality Management
DISE - Software Testing and Quality ManagementDISE - Software Testing and Quality Management
DISE - Software Testing and Quality Management
 
Fundamentals_of_Software_testing.pptx
Fundamentals_of_Software_testing.pptxFundamentals_of_Software_testing.pptx
Fundamentals_of_Software_testing.pptx
 
Fundamentaltestprocess windirohmaheny11453205427 kelase
Fundamentaltestprocess windirohmaheny11453205427 kelaseFundamentaltestprocess windirohmaheny11453205427 kelase
Fundamentaltestprocess windirohmaheny11453205427 kelase
 
Metrics based Management
Metrics based ManagementMetrics based Management
Metrics based Management
 

Recently uploaded

Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 

Recently uploaded (20)

Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 

Testing Metrics

  • 2. 2 What is a METRIC • Metrics can be defined as “STANDARDS OF MEASUREMENT • Metric is a unit used for describing or measuring an attribute • Test metrics are the means by which the software quality can be measured • Test metrics provides the visibility into the readiness of the product, and gives clear measurement of the quality and completeness of the product
  • 3. 3 Why we need Metrics ? • You cannot improve what you cannot measure • You cannot Control what you cannot measure  Without measurement it is impossible to tell whether the process implemented is improving or not  Metrics helps in taking decisions for next phase of activities  Metrics helps in understanding the type of improvement required and helps in taking decisions on process or technology change
  • 4. 4 Why Metrics in Software Testing? There will be certain questions during and after testing such as :  How long would it take to test ?  How Bad / Good is the product?  How many bugs still remain in the product?  Will testing be completed on time?  Was the testing done effectively?  How much effort went into testing the product? To Answer these questions properly we need some type of measurements and record keeping to justify the answers. This is where the testing metrics comes into picture.
  • 5. Testing Metrics Life Cycle  Analysis Phase: • Identify the Metrics which has to be generated • Define the identified Metrics  Communication Phase: • Explain the need of the Metrics to the stakeholders • Educate the testing team about the data points for generating the Metrics  Evaluation Phase: • Capture and verify the data used for generating the Metrics • Calculate the Metrics based on the data captured  Reporting Phase : • Develop the Metrics report and distribute to the stakeholders • Take feedback from the stakeholders for any improvements 5
  • 6. 6 Types Of Metrics Base metrics (Direct Measure) • The Base metrics constitute the raw data gathered by the test Engineers throughout the testing effort • The Base metrics are used to provide project status reports to the Test lead and to the project manager • The Base metrics provide the input data to feed into the formulas used to derive Calculated metrics • Examples of Base metrics are: # of test cases # of test cases executed
  • 7. 7 Types Of Metrics (contd..) Calculated Metrics (Indirect Measure) • The Calculated Metrics convert the Base metrics data into more useful information • The Calculated Metrics are generally prepared by the Test lead and is used to track the progress of the project at different levels like at Module level, at Tester level and for the project as a whole • The Calculated Metrics provide valuable information that when used and implemented often times leads to significant improvements in the Overall SDLC
  • 8. 8 Base Metrics and Testing Phases TEST METRIC TESTING PHASE Number of test cases Test Development Phase Number of Test cases Passed Test Execution Phase Number of Test cases Executed Test Execution Phase Number of Test cases Failed Test Execution Phase Number of Test cases under Investigation Test Development Phase Number of Test cases Blocked Test Dev / Execution Phase Number of Test cases Re- executed Regression Phase Number of First run Failures Test execution Phase Total Executions Test Reporting Phase Total Passes and Failures Test Reporting Phase Test case Execution time Test Reporting Phase Test Execution time Test Reporting Phase
  • 9. 9 Calculated Metrics and Phases The Following Calculated metrics are created at Test Reporting Phase or Post Test Analysis Phase:  % of Test cases Passed  % of Test Coverage  % of Defects corrected  % of Test cases Blocked  % of Rework  % of Test Effectiveness  1st Run Fail Rate  Defect discovery rate  Overall Fail rate
  • 10. 10 Test case Defect Density The number of errors found in test cases v/s test cases developed and executed • ( Defective Test cases / Total Test cases ) * 100 Example : Total no of test cases developed is 1360, total test cases executed is 1280, total no of test cases passed is 1065, total no of test scripts failed is 215 So Test case Defect Density is : 215 X 100 ------------------------------- = 16.8 % 1280 The 16.8 % value can also be called as Test Case Efficiency % which depends upon the total number of Test cases which found defects
  • 11. 11 Defect Slippage Ratio No of bugs reported from Production V/S No of defects reported during execution No of Defects slipped / ( Number of Defects Raised – Number Defects Withdrawn) * 100 Example : Customer reported 21 defects, total no of defects found while testing are, total no of Invalid defects are 17 So Slippage ratio is : [ 21 / (267 – 17) ] X 100 = 8.4%
  • 12. 12 Requirement Volatility Metric This metric ensures that the requirements are normalized or defined properly while estimating No of requirements agreed V/S No of requirements changed • (No of requirements Added + Deleted + Modified) * 100 / No of original requirements Example : SVN 1.3 release has 67 requirements initially, later 7 new requirements are added, 3 requirements are deleted from initial requirements and modified 11 requirements Hence Requirement volatility is calculated as : (7 + 3 + 11 ) X 100 / 67 = 31.34 % This means that almost 1/3 of the requirements changed after the initial identification of requirements
  • 13. 13 Conclusion • The Test metrics should be reviewed & interpreted on regular basis throughout the test effort and particularly after the application is released into production • There are several key factors in implementing and using the metrics in the organization, beginning with determining the goal for developing the metrics, followed by the identification of metrics to be tracked and ending with sufficient analysis of the resulting data to be able to make changes to the software development lifecycle • - So finally Metrics themselves so not create improvements, they do provide the objective information necessary to understand what changes are necessary to be implemented