1. ASSIGNMENT ON METRICS AND EVALUATIONSASSIGNMENT ON METRICS AND EVALUATIONS
SUBMITTED BY:SUBMITTED BY:
Pavithra.S.Pavithra.S.
Preethi SubramanianPreethi Subramanian
Punithavathi.P.Punithavathi.P.
VidhyaLakshmi.R.VidhyaLakshmi.R.
YaminiPriya.K.YaminiPriya.K.
SOFTWARE TESTING
2. WHY METRICS AND MEASUREMENT?WHY METRICS AND MEASUREMENT?
Metrics derive Info from raw data with a view to help in Decision makingMetrics derive Info from raw data with a view to help in Decision making
in areas likein areas like
Relationship between data pointsRelationship between data points
Any cause and effect correlation between observed data pointsAny cause and effect correlation between observed data points
Any pointers to how data can be used in FutureAny pointers to how data can be used in Future
Project Performance can be tracked by:Project Performance can be tracked by:
Measuring the Right ParametersMeasuring the Right Parameters
Performing right analysis on data measuredPerforming right analysis on data measured
Presenting the results of analysis in an appropriate mannerPresenting the results of analysis in an appropriate manner
3. STEPS IN METRICSSTEPS IN METRICS
Identify What to measureIdentify What to measure
Transform Measurements to metricsTransform Measurements to metrics
Decide operational requirementsDecide operational requirements
Perform metrics analysisPerform metrics analysis
Take actions and follow upTake actions and follow up
Refine measurements and metricsRefine measurements and metrics
4. WHY METRICS IN TESTINGWHY METRICS IN TESTING
It is essential to measure the progress of testing and product qualityIt is essential to measure the progress of testing and product quality
Test case execution productivity:Test case execution productivity:
Days needed to complete testing=total test cases yet to be executedDays needed to complete testing=total test cases yet to be executed
Estimate of total days:Estimate of total days:
Days needed for release=Max(Days needed for testing &fixing defectsDays needed for release=Max(Days needed for testing &fixing defects
Release date:Release date:
Total days needed for fixing defects= (Outstanding defects yet to be fixedTotal days needed for fixing defects= (Outstanding defects yet to be fixed
++
Defects that can be found in future test)Defects that can be found in future test)
//
Defect fixing capabilityDefect fixing capability
5. Metrics test help in identifyingMetrics test help in identifying
When making the releaseWhen making the release
What is the release:What is the release:
Based on defect density across modules & analysis scope ofBased on defect density across modules & analysis scope of
product can be decided to release the product in timeproduct can be decided to release the product in time
Whether product is being released with known quality:Whether product is being released with known quality:
Deciding whether product is released with known quality orDeciding whether product is released with known quality or
will function on predictable way in field.will function on predictable way in field.
6. TYPES OF METRICSTYPES OF METRICS
Project metrics:Project metrics:
Indicates how project is planned and executedIndicates how project is planned and executed
Progress metrics:Progress metrics:
Tracks how different activities are progressingTracks how different activities are progressing
Productivity metrics:Productivity metrics:
Takes into account various productivity numbers used forTakes into account various productivity numbers used for
planning and testing activitiesplanning and testing activities
7. PROJECT METRICSPROJECT METRICS
Projects start with information gatheringProjects start with information gathering
After info collection Planning And tracking Is doneAfter info collection Planning And tracking Is done
In planning scope is estimated that is mapped as size estimate whichIn planning scope is estimated that is mapped as size estimate which
in turn is mapped as Effort estimatein turn is mapped as Effort estimate
This isThis is Baselined effort.Baselined effort.
Two factors to be tracked in each phaseTwo factors to be tracked in each phase
EffortEffort
ScheduleSchedule
8. If something goes wrong with the scope or project progressesIf something goes wrong with the scope or project progresses
the process is re-evaluated. This is calledthe process is re-evaluated. This is called Revised effortRevised effort
The basic measurements used here are:The basic measurements used here are:
The different activities,the initial baseline and schedule used forThe different activities,the initial baseline and schedule used for
each process.each process.
The actual effort and time taken for various activities.The actual effort and time taken for various activities.
The Revused estimate for effort and schedule .The Revused estimate for effort and schedule .
9. Effort Variance:Effort Variance:
Plotting baselined ,revised and actual efforts ,phase-wisePlotting baselined ,revised and actual efforts ,phase-wise
Substantial difference between Baselined and revise effort point toSubstantial difference between Baselined and revise effort point to
incorrect initial estimation.incorrect initial estimation.
Variance%=[(Actual effort – Revised estimate)/Revised estimate]*100Variance%=[(Actual effort – Revised estimate)/Revised estimate]*100
Schedule Variance:Schedule Variance:
Deviation of actual schedule from Estimated scheduleDeviation of actual schedule from Estimated schedule
Calculated at the end of every milestoneCalculated at the end of every milestone
Several phases could be active at the same time based on theSeveral phases could be active at the same time based on the
SDLC usedSDLC used
10. Effort Distribution across phasesEffort Distribution across phases
Variance calculations help in finding out whether commitments are metVariance calculations help in finding out whether commitments are met
The distribution effort across each phases can be estimated at time ofThe distribution effort across each phases can be estimated at time of
planningplanning
11. PROGRESS METRICSPROGRESS METRICS
Projects need to be tracked from two angles:Projects need to be tracked from two angles:
How well project is doing with respect to effort and scheduleHow well project is doing with respect to effort and schedule
How well product is meeting with the quality requirements forHow well product is meeting with the quality requirements for
releaserelease
Defects metrics are classified as:Defects metrics are classified as:
Test defect metricsTest defect metrics
Development defect metricsDevelopment defect metrics
Number of defects found in product are main indicators of qualityNumber of defects found in product are main indicators of quality
12. TEST DEFINE METRICS:TEST DEFINE METRICS:
Defect priorityDefect priority Provides management perspective for order of defectsProvides management perspective for order of defects
Defect SeverityDefect Severity Provides a perspective of impact of the defectProvides a perspective of impact of the defect
Defect find rate:Defect find rate:
Purpose of testing is to find defects early in the cyclePurpose of testing is to find defects early in the cycle
After certain period of testing and fixing plot the results in a Bell Curve.After certain period of testing and fixing plot the results in a Bell Curve.
Defect fix rate:Defect fix rate:
Purpose of development is to fix defects as soon as they arrivePurpose of development is to fix defects as soon as they arrive
Last minute fixing results in slip of deadlines , product quality and also putLast minute fixing results in slip of deadlines , product quality and also put
development at pressuredevelopment at pressure
13. Outstanding defects rate:Outstanding defects rate:
No of outstanding defects =Total defects found – Total defects FixedNo of outstanding defects =Total defects found – Total defects Fixed
Outstanding defects in a well executed project is nearly Zero.Outstanding defects in a well executed project is nearly Zero.
Priority outstanding rate:Priority outstanding rate:
Sometimes defects coming out of testing may be critical and enormous.Sometimes defects coming out of testing may be critical and enormous.
Correspond to extreme and critical classification of defectsCorrespond to extreme and critical classification of defects
Some defects of high priority may require change in design orSome defects of high priority may require change in design or
architecture while some defects may take only little time but enormousarchitecture while some defects may take only little time but enormous
efforteffort
Defect trend:Defect trend:
Consolidating fix rate, find rate, priority outstanding and outstanding intoConsolidating fix rate, find rate, priority outstanding and outstanding into
one chartone chart
14. Defects classification trends:Defects classification trends:
How many are Extreme defectsHow many are Extreme defects
How many are CriticalHow many are Critical
How many are importantHow many are important
Weighted defects trend:Weighted defects trend:
Helps in quick analysis of defectsHelps in quick analysis of defects
Weighted defect =[Extreme*5+Critical*4+Important*3+Minor*2+Cosmetic]Weighted defect =[Extreme*5+Critical*4+Important*3+Minor*2+Cosmetic]
Defect cause distribution:Defect cause distribution:
Why defects occur and what are root causes of the defectWhy defects occur and what are root causes of the defect
What areas must be focused for getting more defects in testing.What areas must be focused for getting more defects in testing.
15. Development defects metricsDevelopment defects metrics
Component wise defect distribution:Component wise defect distribution:
Important to map defects into different componentsImportant to map defects into different components
Knowing the component that produce more defects help in fixing them.Knowing the component that produce more defects help in fixing them.
Defect density and defect removal rate:Defect density and defect removal rate:
Defect density correlates source code and defectsDefect density correlates source code and defects
Defects per KLOC=[Total defects in the product]/[total executable lineDefects per KLOC=[Total defects in the product]/[total executable line
of codes in KLOC]of codes in KLOC]
Defect removal rate= (Defects found by verification activitiesDefect removal rate= (Defects found by verification activities
++
Defects found in unit testing) /(Defects found in test team)Defects found in unit testing) /(Defects found in test team)
**
100100
16. Age analysis of outstanding defects:Age analysis of outstanding defects:
Age is the time defects have been waiting to be fixed.Age is the time defects have been waiting to be fixed.
The time needed to fix a defect is proportional to ageThe time needed to fix a defect is proportional to age
Introduced reopened defects trend:Introduced reopened defects trend:
Fixing a defect sometimes may lead to the malfunctioning of aFixing a defect sometimes may lead to the malfunctioning of a
currently working proces.This is called Introduced Defect.currently working proces.This is called Introduced Defect.
17. PRODUCTIVITY METRICSPRODUCTIVITY METRICS
Purpose:Purpose:
Estimating for new releaseEstimating for new release
Finding out how well a team is progressingFinding out how well a team is progressing
Estimating the number of defectsEstimating the number of defects
Estimating release date and qualityEstimating release date and quality
Estimating cost involved in ReleaseEstimating cost involved in Release
18. Defects per 100 Hours of Testing:Defects per 100 Hours of Testing:
(Total defects found for a period / Total hours spent to get defects)*100(Total defects found for a period / Total hours spent to get defects)*100
Test cases Executed per 100 Hours of testing:Test cases Executed per 100 Hours of testing:
(Total test cases executed for a period / Total Hours spent in last(Total test cases executed for a period / Total Hours spent in last
execution)*100execution)*100
Test cases Developed per 100 Hours of testing:Test cases Developed per 100 Hours of testing:
(Total test cases developed for a period / Total hours spent in test case(Total test cases developed for a period / Total hours spent in test case
Development)*100Development)*100
Defects per 100 test cases:Defects per 100 test cases:
(Total defects found for a period / Total test cases executed for the(Total defects found for a period / Total test cases executed for the
same period)*100same period)*100
19. Defects per 100 failed test cases:Defects per 100 failed test cases:
(Total defects found for a period / Total test case failed due to those(Total defects found for a period / Total test case failed due to those
defects)*100defects)*100
Closed defect distribution:Closed defect distribution:
Making sure the defects are fixed and the customer gets qualityMaking sure the defects are fixed and the customer gets quality
assuranceassurance
20. RELEASE METRICSRELEASE METRICS
Deciding the release of productsDeciding the release of products
Varies from Product to productVaries from Product to product
Provides guidelines in making the DecisionProvides guidelines in making the Decision