Software Defect Repair Times: A Multiplicative Model - PROMISE 2008

1. Software Defect Repair Times: A Multiplicative Model Robert Mullen Cisco Systems Boxborough MA bomullen @ cisco.com Swapna S. Gokhale Univ. of Connecticut Storrs CT [email_address]

6. Comparison of MTTR and Age The chart represents the methods, as practiced. Improvements to either method might remove their weaknesses. Today’s presentation uses the Age perspective. For MTTR perspective see Gokhale/Mullen, ISSRE-2006. exceptions numbers Manage By descriptive analytic Tools present trend Time Scale outliers average Focus Age Distribution MTTR

12. Seven hypothetical factors affecting resolution time There is a 4% Probability the Priority is P1, and if so the Time multiplier is .5, etc Probabilities, as percent, each column totals 100. Time multiplier, selected with appropriate probability Subtle Hard Moderate Obvious DIFFICULTY Misleading Oversights Well Written Complete BUG CLARITY Novice Minimal Moderate Practiced SKILLS Slow Average Fast Superstar SPEED None Substitute P4 Inadequate Remote P3 Workable Shared/Wait P2 Specific Available P1 TOOLS RESOURCES PRIORITY 10 40 40 10 10 40 25 25 20 30 30 20 10 40 40 10 25 10 10 25 20 76 25 30 10 25 40 4 3 2 1 .5 2 1.2 .8 .5 1.7 1.2 .9 .6 3 1.5 1 .5 1.7 3 4 1.4 2 2 .9 1.5 1 .8 1 .5

13. Seven hypothetical factors affecting resolution time Drawn from experience and COCOMO Subtle Hard Moderate Obvious DIFFICULTY Misleading Oversights Well Written Complete BUG CLARITY Novice Minimal Moderate Practiced SKILLS Slow Average Fast Superstar SPEED None Substitute Inadequate Remote P3 Workable Shared/Wait P2 Specific Available P1 TOOLS RESOURCES PRIORITY

14. Seven hypothetical factors: tentative distributions There is a 2% Probability the Priority is P1, and if so the Time multiplier is .5, etc For Severity and the other 6 dimensions there is a probability distribution of levels of difficulty We model the distributions by a discrete distribution with 3 or 4 relative levels of difficulty, each with a given probablility Probabilities add to 1.0, i.e. 100% For each factor, we know the variance of the log 0.077 Var. 0.18 Var. 0.12 Var. 0.87 Var. 0.30 Var. 0.22 Var. 0.04 Var 1.4 0.15 4 0.10 1.7 0.20 10 0.20 3 0.20 4 0.20 1.1 0.35 2 0.20 1.2 0.30 3 0.30 2 0.30 2.5 0.30 3.49 0.79 0.9 0.35 1.5 0.30 0.9 0.30 1 0.45 1 0.40 1.5 0.30 1.78 0.19 0.8 0.15 1 0.40 0.6 0.20 0.5 0.05 0.5 0.10 1 0.20 1 0.02 Value Prob. Value Prob. Value Prob. Value Prob. Value Prob. Value Prob. Value Prob. Tools Resources Skills Speed Difficulty Clarity Severity Process Support Personnel Defect

17. Nine product families

24. Other Lognormal Relationships Trouble Tickets = Discrete-LN SRGM = Cumulative Defects = Laplace Transform of LN Test Strategy Ten x the rare rates will find rare-rare interactions 100 times as fast. Equivalent to Heat/Power/ Temp “corner testing” of HW. Multiplicative Rates Limiting Distribution = Lognormal Triggering Conditions Release Strategy Is it ready? Which is best? States, Usage, Code Repair Strategy Risk vs. Benefit ? Removed IO error IO works UBD User error By book Distant Nearby Local Create Open Read RARE UNCOMMON COMMON ETC ETC ETC