In many cases, we choose solutions to problems without sufficient analysis of the underlying causes. This results in implementing a cover-up of the symptoms rather than a solution to the real underlying problem. When we do this, the problem is likely to resurface in one disguise or another, and we may mishandle it again—just as we did initially. Getting to the root of the problem is the better way to solve the current problem, and save time and money in the future. Alon Linetzki identifies and explains a number of root cause analysis techniques widely used in the industry, gives examples of how to apply them in software testing, demonstrates how to implement them, and discusses how to connect them to our day-to-day testing context. Alon shares how root cause analysis can be an effective tool in defect prevention.
1. TP
Half-day Tutorials
5/6/2014 1:00:00 PM
Root Cause Analysis for
Software Testers
Presented by:
Alon Linetzki
Best-Testing
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
2. Alon Linetzki
Best-Testing
Alon Linetzki, founder and managing director of Best-Testing, has been a coach and consultant in
development, testing, and quality assurance for twenty-eight years. Alon helps organizations
enhance the test engineer’s professional and personal skills, training test managers in optimization
and test process improvement, and optimizing their test operations. His main areas of expertise are
in agile testing and transitioning to agile, exploratory testing, test process Improvement, risk-based
testing, and test automation. Alon is a member of the ISTQB® authors and review team for the
ISTQB® Agile Tester Add-on certification, cofounder of ISTQB® in Israel, leader of the ISTQB®
Partner Program, and founder of the SIGiST Israel conference.
6. March 20144
Limit of responsibility
Best-Testing invests significant effort in updating this material, however, Best-Testing is not responsible of
any errors or material which may not meet specific requirements. The user alone is responsible for
decisions based on the information contained in this material.
Protected Trademark
In this material, protected trademarks appear that are under copyright. All rights to the trademarks in this
material are reserved to the authors.
Best–Testing wishes you success in the course!
Copyrights
8. 6
Alon Linetzki
30 years in IT, 20+ years as a test engineer and a test manager
Certified Scrum Master, Scrum Alliance, 2008
ISQTB® Partner Program leader, ISTQB® Agile Tester
Certification Leader
Specializes in Agile testing test strategy & optimization
test process improvement test management test design
risk based testing test automation Building Smart
Teams
A degree in ‘testing’ – B.Sc. statistics and criminology,
International Speaker worldwide, since 1995
Co-founder of the Israeli Testing Certification Board
(www.itcb.org.il, 2004)
Founder of the SIGiST Israel (testing forum) in Israel
(www.sigist.org.il, 2000)
9. We shall cover…
Introduction
Presenting participants and trainer
Workshop expectations
What is Root Cause Analysis?
How to use RCA for Defect analysis?
7
March 2014
10. We shall cover…
How to Use RCA for analyzing critical problems?
Cause Effect graphing, amended
Case Study #0
Summary
How to use RCA for process improvement?
Case Study #1
Case Study #2
Retrospective
Workshop retrospective
8
March 2014
(*) Bonus topic – only if there is enough time left…
11. Professional Background
Years experience in testing/development/both/other
Responsibilities in current role
Have you implemented RCA? Can you share?
Workshop expectations?
Introduce participants…
9
March 2014
12. Interactive sessions
Discussions, skepticism, curiosity
Examples, demonstration
Thinking, communicating
Keeping an open mind for ideas and concepts
Contribute to the class team
Learn…discuss…review…
Give quick feedback on everything you may think of
Time boxed sessions…breaks…exercises…discussions
Introducing workshop dynamics
10
March 2014
13. There are FAR more slides than we can
cover…
Say if you have something to say… do not
wait till the end of the workshop!
Introducing workshop dynamics
11
March 2014
14. Start : 10:15
Lunch: 11:45 – 12:45
Finish line: 14:15
But… we may have more
breaks if we feel like it…
Logistics…
March 2014
12
16. Root Cause Analysis definition
(My interpretation)
From wiki:
Root cause analysis (RCA) is a class of problem solving
methods aimed at identifying the root causes of problems
or events.
The practice of RCA is predicated on the belief that
problems are best solved by attempting to correct or
eliminate root causes, as opposed to merely addressing
the immediately obvious symptoms.
By directing corrective measures at root causes, it is
hoped that the likelihood of problem recurrence will be
minimized.
14
17. Root Cause Analysis definition
From Bill Willson’s website:
Root cause analysis (RCA) is a methodology for
finding and correcting the most important reasons
for performance problems.
It differs from troubleshooting and problem-solving
in that these disciplines typically seek solutions to
specific difficulties, whereas RCA is directed at
underlying issues
15
18. Root Cause Analysis - definition
Root cause analysis (RCA) is a methodology for finding
and correcting the most important reasons for
performance problems.
It differs from troubleshooting and problem-solving in
that these disciplines typically seek solutions to
specific difficulties, whereas RCA is directed at
underlying issues.
16
19. Root Cause Analysis definition -
summary
As a business process improvement tool, RCA seeks
out unnecessary constraints as well as inadequate
controls.
In safety and risk management, it looks for both
unrecognized hazards and broken or missing barriers.
It helps target CAPA (corrective action and preventive
action) efforts at the points of most leverage.
RCA is an essential ingredient in pointing
organizational change efforts in the right direction.
Finally, it is probably the only way to find the core
issues contributing to your toughest problems.
17
20. Root Cause Analysis
(My interpretation)
A problem solving method/process designed to search
for the root causes of a problem using a predefined
structural thinking process, identifying the underlying
issues, with the expectation that dealing with these
issues will dramatically reduce the likelihood of the
problem to occur.
The process involves data collection, cause charting, root
cause identification and recommendation generation
and implementation
18
21. “Cows falling on a road from a mountain”
– is it a problem or a symptom?
Should we eliminate all cows on
that area?
Should we dig-out the mountain?
Should we rotate the sign?
Should we divert the road
elsewhere?
It seems that sometimes
eliminating the causes is not an
easy task, and finding the problems
is even harder!
19
22. Root causes are underlying causes
The investigator’s goal should be to identify
specific underlying causes
The more specific the investigator can be about
why an event occurred, the easier it will be to
arrive at recommendations that will prevent
recurrence.
Practical RCA – definition guidelines
March 2014
20
23. Root causes are those that can reasonably be
identified
Occurrence investigations must be cost beneficial
It is not practical to keep valuable manpower
occupied indefinitely searching for the root
causes of occurrences…
Structured RCA helps analysts get the most out of
the time they have invested in the investigation.
Practical RCA – definition guidelines
March 2014
21
24. Root causes are those over which management has control
Analysts should avoid using general cause classifications such
as “operator error”, “equipment failure” or “external factor”
Such causes are not specific enough to allow management to
make effective changes
Management needs to know exactly why a failure occurred
before action can be taken to prevent recurrence
We must also identify a root cause that management can
influence:
Identifying “severe weather” as the root cause of parts not
being delivered on time to customers is not appropriate
Severe weather is not controlled by management.
Practical RCA – definition guidelines
March 2014
22
25. Root causes are those for which effective recommendations can
be generated (if cannot be done, keep on searching!)
Recommendations should directly address the root causes
identified during the investigation
If the analysts arrive at vague recommendations such as,
“Improve adherence to written policies and procedures,” then
they probably have not found a basic and specific enough
cause and need to expend more effort in the analysis process.
Practical RCA – definition guidelines
March 2014
23
26. Root causes are underlying causes
Root causes are those that can reasonably be
identified
Root causes are those over which
management has control
Root causes are those for which effective
recommendations can be generated (if
cannot be done, keep on searching!)
Practical RCA – definition guidelines
March 2014
24
28. Root Cause Analysis for Beginners - by James J. Rooney and Lee N. Vanden Heuvel
References – what is Root Cause Analysis?
26
March 2014
29. How to use RCA in Defect Analysis?
March 2014
27
30. Escaping defects occur all the time
We try to contain them, from going to production,
but much efforts should be done in preventing
them going from Design to Code
A set of questions may help out leading us to Data
Collection on those Defects for performing the
RCA – next page…
Defects RCA introduction
March 2014
28
31. Defects RCA guided questions
Data Collection
March 2014
29
Question Who
asks?
Describe circumstances and consequence of the
Problem?
Test Team
Where was the problem introduced? Dev Team
Describe the root cause of the problem Dev Team
What reviews were held for the phase? Dev Team
How and why did the problem escape testing? Test Team
What could be done to prevent this type of error in the
future?
Both
What could be done to find this type of problem in the
future?
Both
John Ruberto – Root Cause for Problems Escape
32. Defect Analysis is the process of analyzing a defect
to determine its root cause.
Defect Prevention is the process of addressing
root causes of defects to prevent their future
occurrence.
RCA – few definitions
March 2014
30
33. Defect Source – Finding defects in the stage they
were introduced and as early in the lifecycle as
possible – Eliminating escaping defects
RCA – few definitions
March 2014
31
35. Canceled Defects Root Cause Analysis
Cancelled defects are not real defects of the
system-under-test
They can be the result of: environment problem
(non product), out of scope, test design or test
execution problem, un-reproducible defect, not
understanding the specifications, etc.
33
37. Code Inspection - Example
March 2014
35
Background:
180 people on the project
3,700 modules interconnecting with each other
Complex infrastructure
25 testers
Using Test Director/Quality Center as reporting
platform
Strategic customer
39. Code Inspection - Example
March 2014
37
Analysis:
Main concern – Complience with design (10%
Medium+)
Investigated RCA:
A few specs were written by same designer
A few modules were written by 3 programmers
Initiated training for designer and programmers
40. Code Inspection - Example
March 2014
38
Background:
Another project,
450 people
working, 55 testers,
4,300 modules,
strategic customer
Focus on:
Performance
Maintainability
Proper algorithm
usage
41. Code Inspection - Example
March 2014
39
Analysis:
Investigated RCA:
Performance defects were on resource usage, memory
leaks
Implementing improper algorithm was mainly due to
specification documents written unclear + implementation
complexity chosen by a few programmers
Initiated training for architects and programmers
42. Adding defect template fields:
Injected, Detected, Removed
Version [build] + Date per field
[Updating the defect lifecycle process (status)]
Changing field CRUD permissions – Dev, Test,
Managers
Defining reports
Updating the Defect Management process
Training the teams
Implementing action plan…
Implementation guidelines –
defects RCA
March 2014
40
48. ‘5ys’ or ‘5 whys’ technique, and the cause-effect diagram.
Presenting a problem,
Asking “why?” it happens, finding the effect that caused it
(1 effect),
Presenting the effect on the diagram,
Asking “why?” it happens… [back to previous step, unless
we ask it for 5 times already]
Done.
Presenting the 5whys Technique
46
49. ‘5ys’ or ‘5 whys’ technique, and the cause-effect
diagram.
Presenting the RCA Technique
47
Cause
Cause Cause Cause
Cause
Problem
Why
#1
Why
#2
Why
#3
Why
#4
Why
#5
Thinking path…
50. ‘5ys’ or ‘5 whys’ technique, and the cause-effect diagram.
1. There is the assumption that a single cause, at each level
of "why", is sufficient to explain the effect in question.
2. What if one of the ‘Why’ is answered wrongly? Maybe
our answer is possible, but what if the actual cause is
something else entirely?
3. When we have found the problem, and draw the route,
how ‘strong’ is this solution? Maybe we should prefer one
over the other?
Challenges: what the method can
not solve48
52. Short structured interview with rep’s of management,
development, release, system, testing, product teams.
Step 1: Draw a cause-effect diagram & exercise the 5whys
Step 2: Investigate the arrows/causes for:
What evidence you have that the cause exist? (relevancy)
What evidence you have that the cause leads to the effect?
(strength)
Is anything else needed together with the cause for the effect to
occur? (strength)
Is there evidence that the cause is contributing to the problem
I’m looking at? How much it contributes? (Impact:
direct/indirect)
Enhancing the Method:
Example project50
53. Enhancing the Method:
Example project51
Type Question Cause-Effect
Relevancy What evidence you have that the cause exist? H/M/L
Strength
(S or W)
What evidence you have that the cause leads to
the effect?
H/M/L
Strength
(S or W)
Is anything else needed, together with the
cause, for the effect to occur?
Yes/No
Impact
(D or I)
Is there a evidence that the cause is contributing
to the problem I’m looking at?
Yes / No
Impact
(D or I)
How much this cause is contributing to a
possible resolution?
Direct /
Indirect
Mark
54. Enhancing the Method:
Example project52
Type Question Cause-Effect
Relevancy What evidence you have that the cause exist? High (3)
Strength
(S or W)
What evidence you have that the cause leads to
the effect?
Medium (2)
Strength
(S or W)
Is anything else needed, together with the
cause, for the effect to occur?
Yes (1)
Impact
(D or I)
Is there a evidence that the cause is contributing
to the problem I’m looking at?
Yes (1)
Impact How much this cause is contributing to a
possible resolution?
Direct (2)
Mark 9
You should mark each arrow using this table.
55. Step 3: Identify the routes leading to the
problem/s,
Step 4: Identify the strength and direction (impact)
they have (calculating the mark for each arrow),
Step 5: Choose the best route to focus on,
[Improve it, and go to the next one].
Enhancing the Method:
Example project53
57. Background:
company was using a very advanced technology, and
a complex product line,
Complex product, uses mechanics, electronics,
hardware, software, devices, cooling device, has water
resistant, has heating resistant, accurate up to
1:1,000,000 cm,
In the last 0.5 year, 50% of released machines
returned from the floor (clients) for fixing,
Example project – Hi-Tech Company
55
58. SQA manager was at a course I gave, and liked
one of the tools,
He thought automation can solve many of his
problems, because:
A lot more tests running,
Identifying more defects before the clients do,
Less products coming back,
Clients are happy!
Example project – Hi-Tech Company
56
59. I investigated their automation
needs,
Followed the steps of the
enhanced method,
Found out their problems might
be elsewhere…
Example project – Hi-Tech Company
57
Lets see the drawing board from
that meeting…
61. The RCA meeting (company exec’s and directors):
At first, the belief was that the primary problem
was:
Partial Test Planning (less tests are executed)
Example project – Hi-Tech Company
59
Lets see an illustration diagram …
62. 1st drawing – RCA meeting
60
Many clients
ask for
different Sw
of the
product
Many
versions
open in
parallel
Complexity of
version control
management is
very high
Defining req’
not good
enough by
client
Spec Lvl 0
No specs
in lvl 1
Spec Lvl
1 not
complete
or does
not fit
Spec Lvl
2 not
written
Good
definition of
Spec Lvl 0
Spec Lvl
1 fits
Spec Lvl
2 fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not full
coverage
Partial test
case planning
and coverage
Partial test
execution
and low
coverage
Our way of thinking1 2
63. 1st drawing – RCA meeting
61
Many clients
ask for
different Sw
of the
product
Many
versions
open in
parallel
Complexity of
version control
management is
very high
Defining req’
not good
enough by
client
Spec Lvl 0
No specs
in lvl 1
Spec Lvl
1 not
complete
or does
not fit
Spec Lvl
2 not
written
Good
definition of
Spec Lvl 0
Spec Lvl
1 fits
Spec Lvl
2 fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not full
coverage
Partial test
case planning
and coverage
Partial test
execution
and low
coverage
Our way of thinking1 2
64. 1st drawing – RCA meeting
62
Many clients
ask for
different Sw
of the
product
Many
versions
open in
parallel
Complexity of
version control
management is
very high
Defining req’
not good
enough by
client
Spec Lvl 0
No specs
in lvl 1
Spec Lvl
1 not
complete
or does
not fit
Spec Lvl
2 not
written
Good
definition of
Spec Lvl 0
Spec Lvl
1 fits
Spec Lvl
2 fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not full
coverage
Partial test
case planning
and coverage
Partial test
execution
and low
coverage
Our way of thinking
1
2
65. 2nd drawing – RCA meeting
63
Many clients
ask for
different Sw
of the
product
Many
versions
open in
parallel
Complexity of
version control
management is
very high
Defining req’
not good
enough by
client
Spec Lvl 0
No specs
in lvl 1
Spec Lvl
1 not
complete
or does
not fit
Spec Lvl
2 not
written
Good
definition of
Spec Lvl 0
Spec Lvl
1 fits
Spec Lvl
2 fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not full
coverage
Partial test
case planning
and coverage
Partial test
execution
and low
coverage
Our way of thinking1 2
66. After a while, we shifted the focus and agreed that
the real problem was actually:
Poor Product Quality
Because that was the reason the clients returned
their product.
And we started RCA from there.
After a while, we started to see the light – real
problems started to crystallize, problems that
involved people and processes
Example project – Hi-Tech Company
64
68. Our way of thinking
3rd drawing – RCA meeting
66
Many clients
ask for
different Sw
of the
product
Many
versions
open in
parallel
SCM - Complexity
of version control
management is very
high
Defining req’ not
good enough by
client
Spec Lvl 0
No
specs in
lvl 1
Spec Lvl 1
not complete
or does not
fit
Spec Lvl
2 not
written
Good
definition
of Spec
Lvl 0
Spec Lvl
1 fits
Spec Lvl
2 fit
Spec Lvl 2
does not
fit/complete
Code written
with low
match to
client req’
Only
Partial Test
planning
and not full
coverage
Partial test
case
planning and
coverage
Partial test
execution
and low
coverage
1 2
Tight
schedule
projectPrioritization
and
compromise
on scope to
clients
Low
Quality
Product
Req’
managemen
t not good
enough
Lack of methods
and techniques
in testing
Low lvl of test
identification
69. We then defined the relevancy, strength and
impact of each arrow (cause),
And calculated the grades for the arrows (which
are not seen here),
Example project – Hi-Tech Company
67
Back to the board…
71. 5th drawing – RCA meeting
69
Many
clients ask
for
different
Sw of the
product
Many
versions
open in
parallel
SCM - Complexity of version control
management is very high
Defining
req’ not
good
enough by
client
Spec Lvl 0
No specs in
lvl 1
Spec Lvl 1
not
complete
or does not
fit
Spec Lvl 2
not written
Good
definition
of Spec Lvl
0
Spec Lvl 1
fits
Spec Lvl 2
fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not
full
coverage
Partial test
case
planning
and
coverage
Partial test
execution
and low
coverage
Our way of thinking1
2
Tight
schedule
project
Prioritization
and
compromise on
scope to clients
Low
Quality
Product
Req’
management
not good
enough
Lack of methods and
techniques in testing
Low lvl of test
identification
S/D
W/D
W/I
S/I
72. We went back to double check the RCA of the
routes leading to the primary problem, marking
the arrows with their grades (from the table,
remember?)
We ended up circling the main causes, that have
initiated the strongest routes that are directly
impacting our problem,
Example project – Hi-Tech Company
70
75. Many
clients ask
for
different
Sw of the
product
Many
versions
open in
parallel
SCM - Complexity of version control
management is very high
Defining
req’ not
good
enough by
client
Spec Lvl 0
No specs in
lvl 1
Spec Lvl 1
not
complete
or does not
fit
Spec Lvl 2
not written
Good
definition
of Spec Lvl
0
Spec Lvl 1
fits
Spec Lvl 2
fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not
full
coverage
Partial test
case
planning
and
coverage
Partial test
execution
and low
coverage
Our way of thinking1
2
Tight
schedule
project
Prioritization
and
compromise on
scope to clients
Low
Quality
Product
Req’
manageme
nt not
good
enoughLack of methods and
techniques in testing
Low lvl of test
identification
S/D
W/D
W/I
S/I
Last drawing – RCA meeting
73
76. Many
clients ask
for
different
Sw of the
product
Many
versions
open in
parallel
SCM - Complexity of version control
management is very high
Defining
req’ not
good
enough by
client
Spec Lvl 0
No specs in
lvl 1
Spec Lvl 1
not
complete
or does not
fit
Spec Lvl 2
not written
Good
definition
of Spec Lvl
0
Spec Lvl 1
fits
Spec Lvl 2
fit
Spec Lvl 2
does not
fit/complete
Code
written
with low
match to
client req’
Only
Partial Test
planning
and not
full
coverage
Partial test
case
planning
and
coverage
Partial test
execution
and low
coverage
Our way of thinking1
2
Tight
schedule
project
Prioritization
and
compromise on
scope to clients
Low
Quality
Product
Req’
manageme
nt not
good
enoughLack of methods and
techniques in testing
Low lvl of test
identification
S/D
W/D
W/I
S/I
74
4/6
Last drawing – RCA meeting
Lets see the routes…
3/3
77. 5 major Root Topics were Identified, explained and prioritized:
1. Produce requirements from client definitions
2. Requirements management
3. Either ‘No Spec Level 1’, or ‘Spec level 1 not matching
requirements’
4. Lack of methods and techniques in testing for development
and testing teams
5. Allot of clients define slightly different requirement for the
SW – allot of specials
We defined a pragmatic corrective actions plan, with priority items.
Example project – Hi-Tech Company
75
78. Major Areas of Concern identified and prioritized:
1. Requirements Management
2. Configuration Management
3. Design Documentation and Flow
4. Testing Methodologies, techniques and tools
Not discussed:
- Release Management
- Risk Management + Risk Based Testing
- Requirements Definition
- Project Management
- Professional Development
Example project – Hi-Tech
Company76
Organization
Language!
80. If we follow the questioning and grading
process, than problems shall be the ‘leafs’
of the cause-effect diagram, with:
A positive answer to the questions:
Relevancy – ‘yes’
Highest strength mark (direct/Indirect),
and
Direct impact on the undesired
outcome (initial hypothesis)
Highest grade per route guideline,
Translate into company language (areas
of concern)
Enables an Immediate vs. long term
action plan
Differentiating Problems from
Symptoms78
Just what our office needs..
82. Cause-effect & RCA may eliminate the problems of the model
when answering the questions and marking them on the diagram
for the best route(s),
(but) We must deal with symptoms some time, in the short term,
We should estimate effort needed (or money required) for each
route, and integrate that into our decision (i.e. 60/40 weight)
The questions (answers) make sure that our analysis is with
minimal deviations, and that the route we take is a ‘strongest’ one,
It is a constructive process that leads to Understanding, Clarity,
Focus and the right Priority setting
Summary
80
83. Further enhancing the mode, we must think of the following:
What about the junctions points (inbound and outbound):
direct impact of routes with those? Indirect? Impact on speed of
performance (bottle-necks)?
What is the ROI of this method within context?
Can we validate a route? Can we tie it to be a successful
problem eliminator?
How much the method is context dependant?
Can we hook it to Test Process Improvement methods or other
Key Performance/Area Indicators?
Other?
Food for Thought…
81
85. RCA and Process Improvement?
March 2014
83
Introduction
Case Study #1
Case Study #2
86. Process improvement should use data and RCA
outputs
CI example… - process was changed, training was
done to relevant people
Defects example… - lessons learned, training was done
to relevant testers, discussions with development
made it clear which information is required more on
the defect template
RCA and Process Improvement?
March 2014
84
88. Analysis:
Small, yet impacting, breaks in work, caused huge
impact on the release (big team of testers)
Breaks were 15-45 min each
Areas of impact were: performance, network, DB
Analyzing test environments
utilization downtime
March 2014
86
89. Improved process:
Defined measurements for monitoring downtime
of test environments
Identify trend of areas of impact on infrastructure
Used it weekly to establish Status of Downtime, to
decrease technical impacts (reached -25% waste
on next release).
Analyzing test environments
utilization downtime
March 2014
87
90. Analyzing development effort
with defects found
March 2014
88
15
26
11
120
65
43
32
187
12
38
210
294
588
2184
504
210
126
1344
336 378
80%
99%
21%
62%
145%
230%
286%
156%
40%
113%
0%
50%
100%
150%
200%
250%
300%
1
50
SC MAF FBF CSM E-Care E-Support Rater AR Reports BL Interface
Investment vs Defects
Defects Dev Effort Def % vs Dev Eff %
~28 years development.
~150 people in development
team.
91. Analysis:
Areas of defects clustering mainly in: Rater, E-
Support, E-Care, AR
Relative % of defects are much higher than % of
development effort (days)
Investigated UT and Staging (integration test)
phases, found missing testing types done, UT &
INT regression testing, performance testing,
interface testing
Further considerations: Complexity, New/Legacy
code, # people touching code
Analyzing development effort
with defects found
March 2014
89
92. Improved process:
Defined Unit test and Integration test process and
strategy guidelines and work instructions
Trained programmers in UT and INT techniques and
new processes
Aligned effort estimation and duration with project
management and product management
Defined coverage criteria for UT and INT
Initiated test automation project on UT and INT levels
Analyzing development effort
with defects found
March 2014
90
94. Root Cause Analysis techniques can help us in
finding the underlying problems (root causes), and
get to deal with real problems
Structured RCA methods support productive
thinking, identifying problems and identify more
symptoms
Cause Effect Graphing (amended), Defects RCA
RCA and process improvement
Summary
Retrospective
March 2014
92
95. “It is not the strongest of the species that
survives, nor the most intelligent but the one
that is most responsive to change”
Charles Darwin
A changing world…
93
96. Or perhaps . . .
94
. . . the one who had anticipated all possible
requirements !