A Novel View of Applying FMECA to Software Engineering

A Novel View of
Applying FMECA to
Software Engineering
Robert W. Stoddard
©2013 ASQ & Presentation Stoddard

http://reliabilitycalendar.org/webina
rs/

ASQ Reliability Division
English Webinar Series
One of the monthly webinars
on topics of interest to
reliability engineers.
To view recorded webinar (available to ASQ Reliability
Division members only) visit asq.org/reliability
To sign up for the free and available to anyone live
webinars visit reliabilitycalendar.org and select English
Webinars to find links to register for upcoming events

http://reliabilitycalendar.org/webina
rs/

ASQ RD Webinar, 10 October 2013

A Novel View of Applying
FMECA to Software
Engineering
by
Robert W. Stoddard

© 2013 Six Sigma IDS, LLC. Confidential and Proprietary.

Version 010

3


Conceptual View of FMECA
FMECA is a structured approach to anticipating what can fail and why
FMECA is a proactive way to prevent issues from occurring
FMECA is considered a mature approach that is implemented in CMMI
High Maturity organizations (Causal Analysis & Resolution)
FMECA may be thought of as a top-down approach that may be
combined with other approaches
– Fault tree analysis and other root cause methods may be instantiated within the
FMECA process
– The results of FMECA may be used, for example, to guide prevention activities,
such as Poka-Yoke


Version 010

4


Original Roots of FMECA
During the 1950’s, FMECA enjoyed initial popularity within the
Aerospace, Nuclear Engineering and Reliability Engineering fields
Considered an essential technique for life-critical applications
During the 1990’s, use of FMECA notably expanded to other industries
and domains


Version 010

5


Example Business Use and Benefits

Kraft Foods
http://www.mcp-cg.com/pdf/FMECA%20Case%20Study%20-%20Kraft%2

Training and Consulting Services Company
http://www.fmeainfocentre.com/examples/f0503_chowdary2.pdf
Biotech Manufacturing
http://www.pharmamanufacturing.com/articles/2009/115.html
Behavioral Health Unit attempt to reduce patient falls
http://www.hcpro.com/QPS-29783-234/Case-study-Use-FMEA-topreventMedicine Prescription Error Reduction
http://www.fmeainfocentre.com/guides/qp0803reiling.pdf
Numerous Case Studies
http://www.fmeainfocentre.com/fmea_abstracts_2006.htm


Version 010

6


Failure Modes and Effects Criticality Analysis
A structured method of using a multi-disciplinary team to brainstorm
and anticipate failure modes and associated potential root causes
Basically, three scores are assessed:
1. Severity
2. likelihood of occurrence
3. likelihood of escaping to the customer
A Risk Priority Number (RPN) is computed by multiplying the three
scores. (Normally, scores greater than 200 warrant prevention and
mitigation planning by the team)
DFMECA = Design FMECA

PFMECA = Process FMECA


Version 010

7


FMECA Flow
Step 1:
Process Step or
Design Component

Step 4:
Id Root
Cause(s)

Step 7:
Calculate
RPN

Step 2: Id
Failure
Mode(s)

Step 5:
Id Probability of
Occurrence of
Root Cause

Step 3:
Id Effect &
Severity

Step 6:
Id Probability of
Escape of Root
Cause

Step 8:
Id Prevention &
Mitigation Actions;
recalculate RPN


Step 9:
Continue Tracking
RPN > 200

Version 010

8


Traditional vs Modified Use of FMECA
Traditional Use

Modified Use

Originally applied to products at
the macro level

Now applied to sub components of
products, and to processes

Originally used only objective
historical data to establish the
three RPN component scores

Now applied with a combination of
available historical data but also
heavy use of subjective expert
opinion of the RPN component scores

Used with additional scrutiny and
columns to discuss things such
as: why the severity is high, why
the root cause was possible, and
which processes enabled the
escape

Now, the additional items of
information to the left are considered
optional. However, they are
recognized as valuable information
that may be added to the FMECA


Version 010

9


General Comments on the RPN Fields
FMECA Field

Comments

Likelihood of
Occurrence

This is normally a scale of 1…10. Although most
organizations find it useful to create text definitions of
each level, one should remember that this scale should
be ratio rather than ordinal. The higher the score, the
more likely the root cause will occur.

Severity

This is also normally a ratio scale of 1…10. The higher
the score, the more severe the consequence of the
failure mode.

Likelihood of
Escape

This is also normally a ratio scale of 1…10. The higher
the score, the more likely the root cause will escape to
be felt by a user or customer.

See Don Wheeler’s commentary:
http://www.qualitydigest.com/inside/quality-insider-column/problems-risk-priority-numbers.html


Version 010

10


Additional Thoughts re Likelihood of Occurrence
Often there may be little data to support a consensus of the likelihood
of occurrence
In these situations, a surrogate for likelihood often works well
For example, likelihood of occurrence of a code issue may correspond
to the McCabe unit cyclomatic complexity value. As such, a profile
of the unit complexity values may be mapped to the 1..10 scale so
that differentiation may occur.


Version 010

11


Additional Thoughts re Likelihood of Escape - 1
Often there may be little data to support a consensus of the likelihood
of escape
In these situations, a simplistic calculation often works well
For example, assume a root cause is in unit A code. The team
considers how the root cause may be found in downstream quality
checks or testing.
Continuing the example, assume that a code inspection, unit test,
integration test and system test can possibly catch the root cause
(e.g. defect)
The team should quickly reach consensus on the probability that the
defect will escape each of the downstream checks or tests.

Version 010

12


Additional Thoughts re Likelihood of Escape - 2
Let’s assume the following escape probabilities:
75% chance of escaping the code inspection
50% chance of escaping the unit test
20% chance of escaping the integration test
90% chance of escaping the system test
So,

0.75 * 0.50 * 0.20 * 0.90 = 7%

and the 0..100% probability scale for escaping through all downstream
activities can then be mapped to the 1..10 scale


Version 010

13


Software Engineering Application
Generally applied with the perspective of the software-only content of a
product
May be driven by a product level FMECA or performed in isolation
May be applied to different levels of abstraction of the software
– Requirements
– Architecture
– Design
– Code

Remains a key tool to make software products and components more
robust of “rainy day” operation of the software
Provides a rich input to the software testing activity

Version 010

14


Enlightened Approach to FMECA of SW
FMECA is a structured approach to identify what can go wrong and
why
Software exists in many forms as it is developed
Software begins as an abstract form of requirements and needs
Software then is defined as an architecture
Software then proceeds through high and low levels of design
Software finally arrives in the form of actual code (source, object,
executable image)
As such, FMECA may be applied to any and all the different
abstractions of software!

Version 010

15


The V Model for Software Development
FMECA results drives test cases
SW Reqts

SW System Test

SW Architecture

SW Performance Test
SW High Level
Integration Test

High Level Design

Detailed Design

Code


SW Low Level
Integration Test
SW Unit Test

Version 010

16


DFMEA on the Requirements

The distinct
requirements and/or
needs identified for
the software


Version 010

17



The distinct
operational scenarios
or use cases


Version 010

18



The distinct quality
and performance
measures for the
overall software


Version 010

19


Question 1
To what degree do you believe SFMECA could benefit your
organization’s software requirements quality?


Version 010

20


Applying DFMEA to Software Architecture
Each View of the Software Architecture may be analyzed via FMECA
Remember, a view is a stakeholder representation of the architecture
structure
FMECA enables a proactive approach to assessing potential
weaknesses or vulnerabilities in the software architecture


Version 010

21


FMECA on the Logical View

The discrete services
that the software
should provide users


Version 010

22


FMECA on the Process View

The different possible
concurrent and
independent
processes, programs,
or networks that
communicate with
each other


Version 010

23


FMECA on the Development View

The Development
Parts of the Software
which are assigned to
developers to work on


Version 010

24


FMECA on the Physical View

The mapping of the
objects from the other
views on the
processing nodes of
the system


Version 010

25


Using Software Architecture FMECA Results
The results of FMECA can be used as follows:
– To adjust software architecture tactics or decisions
– To re-assess the need for fault tolerance and error-handling
– To guide the planning of software testing, especially robust software testing
– To assess the risk of product shipment and warranty exposure

The above actions may be decided based on:
– The prevention and mitigation actions identified during the FMECA exercise
– The RPN scores with values remaining above threshold determined by the team


Version 010

26


Question 2
organization’s software architecture quality?


Version 010

27


Applying DFMEA to Software Design
Each Depiction of the Software Design may be analyzed via FMECA
Possible Software Design Depictions include:
– Software Call Tree
– Object Relationship Diagram
– Entity Relationship Diagram, including all major interfaces
– Control flow chart
– Data flow chart
– McCabe logical path diagram (Cyclomatic, Module Design, Subtrees)
– Detailed State-Transition-Diagram
– Message Sequence Charts
– Network Performance Diagrams, including Markov and Petri-Net Models


Version 010

28


Software Call Tree


Version 010

29


Object Relationship Diagram

Each object has internal states,
inputs, and outputs.

Version 010

30


Entity Relationship Diagram
External System

HW
1

User
SW

SW

1

1

2

User

SW

3

3

External
System 2

HW
2

User
2
Version 010

31


Control Flow Diagram

Objects or Code Processes with control
operations between each.

Version 010

32


Data Flow Diagram

Data Store
1
Objects or Code Processes sending
different data to each other.

Data
Store 2
Version 010

33


McCabe Logical Path
Cyclomatic Complexity =
# Unique logical paths

Module design
complexity =
# unique logical
paths related to code
design interfaces
vv


Version 010

34


State-Transition Diagram

State 2

State 5

State 4

State 1

State 3

Different events cause transitions from one
particular state to another state.

Version 010

35


Message Sequence Chart


Version 010

36


Network Performance Diagrams

Using simulation, such as Discrete Event Simulation, Throughput, cycle
times, waste times, queue lengths are determined

Version 010

37


Question 3
organization’s software design quality?


Version 010

38


Code Inputs and Outputs
Analyze the outputs of the code units for deficient conditions


Version 010

39


Code External Interfaces
Analyze all or a subset of external interfaces for proper operation
Deficiencies in an external call can be failure modes


Version 010

40


Code Constants, Variables, Pointers
Constants may be set or reset incorrectly
Variables can go out of allowable range
Pointers may be set incorrectly or not released when done (e.g.
memory leak)
Obviously only a critical subset of the above items to be analyzed!


Version 010

41


Code Memory Utilization
Memory usage may be analyzed
Reference memory utilization patterns and anti-patterns


Version 010

42


Code Error Handling
Look at existing error handling code
Identify all exceptions to be handled
Question missing exceptions
Question how exceptions are to be handled


Version 010

43


Question 4
organization’s code quality?


Version 010

44


Approach to Populating FMECA Template
Teams have found it less distracting if they populate the FMECA
template column by column
This approach keeps focus on one aspect at a time
The team can then more early assess the total effort needed to finish
the FMECA
Easier to have stops and starts if the FMECA session needs to run
across different face-to-face meetings


Version 010

45


Approach to Reporting Completed FMECA
Teams have found it more communicative to report the results of the
FMECA row by row
Teams will also reorder the rows in decreasing RPN magnitude
Teams may also keep groups of rows that make logical sense to
discuss together


Version 010

46


Relationship to Test Activity
As discussed earlier, FMECA may be an excellent targeting
mechanism for stress and robustness testing!
Test staff have been found to have a rich perspective (detective,
critical, search for ways to break things) that makes them superior
facilitators of FMECAs
FMECAs held during development provide a rewarding mechanism for
development and test staff to talk, instead of the development
followed by throwing the code over the wall to the testers.
The prevention and mitigation thinking promoted by FMECA during
software development can save development engineers a lot of time
and rework by getting it right the first time


Version 010

47


Who Should Facilitate FMECA?
Someone other than the author of the artifact under analysis
The more independent the better!
Sometimes, too much domain knowledge can hinder great facilitation
(too easy for facilitator to get wrapped up in the answers instead of
asking the probing questions)
Test staff are excellent!
Quality and process improvement staff can perform well if they have
some knowledge of the domain and application
Management and supervisors should not facilitate the FMECAs of work
products of their subordinates!


Version 010

48


Change from Peer Reviews & Inspections
Peer Reviews & Formal
Inspections

FMECAs

Require and depend on significant
advance individual preparation

Require less advance preparation
by participants

Focus on established standards
and conventions from a compliance
standpoint (Closed Focus)

Focus on brainstormed failure
modes (Open focus)

Focus only on identifying noncompliances (defects)

Focus on failure modes and
corresponding root causes

Only looks at actual issues or
defects (e.g. they have already
occurred)

Purposely anticipates issues and
defects that have yet to occur


Version 010

49


Patience is Required!
FMECAs can be similar to diving for pearls!
Divers know that they must accomplish 20 dives to find a decent pearl.
Likewise, FMECAs may have to explore 20-30 items before a
valuable recognition of a potential issue is found.
Participants in FMECAs must have patience! They may feel like they
are being peppered with a lengthy list of silly questions.
Management must enable the resources and time to conduct FMECAs;
Show data reflecting avoidance of rework and downstream issues!


Version 010

50


Root Cause vs Underlying Causes
FMECA teams can become mired in root cause discussions. If so, the
team facilitator should accept the identification of an underlying
cause.
Root cause is the real source of the problem
Underlying cause is the proximate cause of the problem but may not
necessarily be the real root cause
We don’t want the FMECA to expend unreasonable time and resources


Version 010

51


Tools to Use for FMECA
Many tools exist to support FMECA.
Free tools and tools for purchase may be identified with a simple
internet search
Many companies are satisfied to use a simple Excel spreadsheet
Others desire a database approach to enable the sharing of data and
results across the organization and/or product lines


Version 010

52


Managing Complexity and Exploding FMECAs
FMECAs must be scoped to meet the needs of the business or
organization
Scoping must first address which products to be analyzed (HW, SW,
management artifacts)
Scoping must also address perceived risk. It might be reasoned that
only certain parts of the product may be analyzed. For the parts
identified, it may be desired to still drive deep in the FMECA to
ensure critical aspects are fully reviewed.
Most organizations find an incremental approach of FMECAs during
product development the most beneficial.
However, FMECAs can still be performed on completed products if one
desires a rich risk assessment!

Version 010

53


Who Should be on the FMECA Team?
The FMECA team should be as multi perspective as possible
If done at the product level, all engineering disciplines should
participate
If done within just software engineering, then different software
perspectives should participate as follows:
–
–
–
–
–
–
–

Software Req’ts analyst
Software Architecture and/or System engineer
Software Design
Software Coding (programmer)
Software Test (all levels of test should participate)
Software technology experts and domain experts
Software process and quality staff

Great facilitators with a detective mindset are needed! Facilitators
need to manage both convergent and divergent thinking styles.
Lastly, facilitators must manage time and patience levels!

Version 010

54


Causing Active Institutional Learning Cycles!
Many organizations have found more advanced and clever uses of
FMECA!
These organizations will use historical results of failure modes and
corresponding root causes to pre-populate FMECA templates. In
this approach, different templates are created based on the phase
of the software lifecycle and nature of the artifact under analysis.
With this approach, individual teams will assess themselves against the
historical failure modes and root causes before venturing into the
normal process of evaluating their artifacts. This helps ensure
cycles of learning within the organization!
This approach has been shown to be superior to large databases of
lessons learned that are supposed to be searched by very busy
development engineers!

Version 010

55


Question 5
Would you be interested in a ½ day to a full day tutorial
consisting of hands-on practice using Software FMECA?


Version 010

56


Summary
FMECA is simple, practical and hopefully easy
FMECA is more rewarding than traditional peer reviews and
inspections
FMECA doesn’t require a tool investment – but rather some team
training
FMECA can literally be applied to almost anything to anticipate, prevent
and mitigate issues


Version 010

57

A Novel View of Applying FMECA to Software Engineering

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (18)

Ähnlich wie A Novel View of Applying FMECA to Software Engineering

Ähnlich wie A Novel View of Applying FMECA to Software Engineering (20)

Mehr von ASQ Reliability Division

Mehr von ASQ Reliability Division (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

A Novel View of Applying FMECA to Software Engineering

Hinweis der Redaktion