Paper given at RAMS 2013 on best practices and consideration when establishing reliability goals for the development team.
The way we communicate goals directly impacts the achievement of the goal. Creating and succinctly stating a reliability goal necessary for the leadership it provides an organization when designing a new product.
The reliability goal statement includes four elements: function, environment, probability of success, and duration. The function definition provides one means to define failure conditions. The environment includes elements such as weather and elements related to use frequency.
The probability of success and duration should always be stated in a couplet. Setting more than one probability and duration couplet further defines reliability over the expected lifespan of the product. And setting multiple couplets enables various focuses of the goal that concern different periods of time or constituents.
Setting and stating a clear and complete reliability goal delineates the boundary between a reliable enough product and one that is not. The goal enables the design team to balance the myriad of other design considerations along with product reliability in a meaningful manner.
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
RAMS 2013 Establishing Product Reliability Goals
1. Establishing Product Reliability Goals
Fred Schenkelberg, Ops A La Carte, LLC
Key Words: reliability, specification, goal, process
SUMMARY & CONCLUSIONS
2 A RELIABILITY GOAL
The way we communicate goals directly impacts the
The common understanding of „reliable‟ involves
achievement of the goal. Creating and succinctly stating a
concepts like trust and consistency. Saying a product is
reliability goal necessary for the leadership it provides an
reliable generally means it works as expected over time. When
organization when designing a new product.
I say my car is reliable, I may mean it starts every morning, on
The reliability goal statement includes four elements:
demand.
function, environment, probability of success, and duration.
These are not useful definitions even if the sentiment is
The function definition provides one means to define failure
correct for a product design engineering team. One way to
conditions. The environment includes elements such as
define reliability that is more useful for engineers is
weather and elements related to use frequency.
“The probability that an item will perform a required
The probability of success and duration should always be
function without failure under stated conditions for a stated
stated in a couplet. Setting more than one probability and
period of time.” [1]
duration couplet further defines reliability over the expected
In this definition, we have four elements that need
lifespan of the product. And setting multiple couplets enables
refinement to fit a particular „item‟. Each of the four elements
various focuses of the goal that concern different periods of
may or may not be well defined for the item and they may or
time or constituents.
may not be stated all in one place.
Setting and stating a clear and complete reliability goal
For the purpose of this paper, we will address the four
delineates the boundary between a reliable enough product
elements as: Function, Environment, Probability of success,
and one that is not. The goal enables the design team to
and Duration.
balance the myriad of other design considerations along with
Creating a succinct reliability goal statement provides a
product reliability in a meaningful manner.
means to clearly communicate the desired reliability or
1 INTRODUCTION performance over time of the product. Rather than an implied,
vague or scattered set of requirements or goals, create a single
Nearly every product has a clear set of functional or
statement with supporting information.
performance objectives, possibly a cost goal and often a time
The reliability goal should be easy to understand and
to market deadline. Product reliability should be part of this
promote the desired behavior and outcomes. As with any goal
initial set of criteria for product design and it should be very
the normal guidelines apply. The goal should be specific,
clearly stated. All too often goals such as „as good as or better
quantifiable and be met within a specific period of time. [2]
than the last product‟, or „5 year life‟, or „2-year warranty‟, is
The goal setting process is a business process and should
all the design team receives as a reliability goal.
reflect the interests and capabilities of the business,
Stating a complete reliability goal includes the references
technology and customer. The intent is to provide a realistic
to product function(s) and expected operating environment
objective that can be achieved with a product that is profitable
and use profiles. It also includes stating the probability of
for the business and meets or exceeds the customer‟s
survival over a time period. This probability and time couplet
expectations. Profitable here may mean the product meets
may have multiple statements in order to consider early life
business objectives, which may or may not include profit,
failure period, a warranty period, and a design life for the
although profit is a common business objective.
product. Each period has specific and often different risks,
considerations and business impact. 3 FUNCTION AND ENVIRONMENT
This paper explores the practical means to establish
The first two elements of a reliability goal to explore are
product reliability goals. The use of field performance of past
the function and environment. The first assists us in defining a
products, market positioning, cost of failures, technical
failure, the second the expected set of external stresses or
capabilities, and engineering judgment all come to play when
conditions. Both are necessary for a complete reliability goal
establishing the reliability goal. Setting a meaningful goal, that
statement.
is measureable during product development, is often one of
the first steps necessary to create a reliable product.
2. 3.1 Function Tests.[3] Another with a focus on the telecommunications
industry equipment is the set of documents provided by the
The function or functions of an item are essentially what
European Telecommunications Standards Institute (ETSI).[4]
the item is to do. For example, a passenger plane transports
Rather than use rated limits or absolute maximum storage
passengers from one location to another, or, a pair of
temperature expectations which would only apply to a very
sunglasses provides UV radiation eye protection from the sun.
few situations and units, I recommend you use the 90 th
The product may be very complex or relatively simple.
percentile values for daily average temperature and daily
The product may include multiple functions or only one. Yet,
average temperature range. The National Climatic Data Center
even with the sunglasses, ultraviolet protection is probably not
[5] has available worldwide weather station daily data
the only function. Style, color, comfort, and other aspects of
readings. Besides temperature there is dew point (related to
the glasses may be as or more important than UV protection.
relative humidity) and rainfall, windspeed and similar
For a reliability goal, select the primary or salient function
information for each day at each of the thousands of weather
of the product and reference the remaining functions in other
reporting stations from around the world. The database access
documents. While I was at Hewlett-Packard Company, I
permits one to select a region or country set of stations.
noticed that many product development teams developed early
Having years of daily or hourly weather data permits one to
in the design process a detailed Product Requirements
use the actual distribution of the values or a specific percentile
Document (PRD), which included sufficient definition of all
of the data, say the 90th percentile, which improves the
functions to deliver to the customer, often including color,
accuracy of any consideration related to the weather.
weight, location of the logo, pages printed per minute in
Depending on the products industry or use expectations,
various printing modes, etc. The PRD was often 50 pages or
the use of standards or tabulated data may prove useful. If not,
more.
then gathering readings from actual or similar products in use
When setting a reliability goal, use the primary function
would provide the necessary information to establish a
to communicate the purpose of the product. Keep it simple and
complete set of environmental conditions.
clear. For products with a detailed PRD or similar document,
The other element of environment is the use profile. How
provide a reference to the full set of expected functions.
something is used as well as how often it is used may directly
3.2 Environment affectits reliability. A long haul international plane may have
one or two flights per day, whereas, a local commuter plane
The product has to work for the customer to be of use. It
may experience a dozen flights per day. The difference in
also has to function where and when the customer is expecting
stresses included the number of take offs and landings, maybe
the product to work. For example, a large passenger aircraft is
altitude, and many other factors that detail the expected
expected to operate around the world, and function properly
frequency and nature of use.
during taxi, take off, level flight and landing. Another example
The use profile may also influence the types of failure
is sunglasses should function when exposed to full sun.
mechanisms and related stresses. A bicycle used by a
The weather (temperature, humidity, contaminants, etc.)
professional during a major race, will likely have stresses
around the product constitutes part of the environment. The
imparted by the rider that are significantly higher than by a
use profile is another element. The weather is a short way of
recreational rider. The maintenance practices, hours of use per
stating the list of stresses that exist external to the product and
day, and other conditions may be quite different also.
may impact the reliability of the product. If a product is
It may be necessary to create different profiles and
expected to operate outdoors in all weather, then include
associated environmental conditions. The aircraft example
condensation, rain, and hail as expected conditions.
may include details related to long haul and commuter and
The weather should include all stresses that the product is
cargo uses for the same aircraft. Each type of use would have
exposed to during storage, transport, installation and use. And,
slightly different expected frequency of use, loading
all stresses that related to an expected failure mechanism
conditions, weather conditions, and possibly maintenance
should have sufficient information to permit characterization
practices.
of the products performance when exposed to those stresses.
The same considerations apply for the design of a bicycle.
For example, the sunglasses material should not deteriorate or
A bicycle that is rarely maintained or used may be more prone
change color when exposed to sunlight. Knowing how much
to corrosion related failure. The professional bike may have a
sunlight, expected spectrum and intensity, would assist in the
higher probability for metal fatigue related failure.
design, evaluation of alternatives, and validation &
Understanding and detailing the full set of environmental
verification testing.
(weather and use) conditions provides the design team a clear
The environmental set of conditions are not just weather,
set of expectations. The weather and use permit the evaluation
it may include radiated emissions, power line anomalies,
of materials and designs to function within the customers‟
insect or animal exposure, dust, or something as possibly
world.
damaging as the interior of a purse or briefcase.
One freely available standard that lists a wide range of 4 PROBABILITY AND DURATION
environmental conditions is the MIL-STD-810G
The probability and duration for a reliability goal
Environmental Engineering Considerations and Laboratory
complete the goal statement. I recommend always pairing and
3. stating the two elements together to avoid any confusion. In practice, this line does not have to be straight. We may
Stating 98% probability of success over 2 years is easier to want very few failures during the first month, a relatively low
understanding than stating only one of the following: chance of failure over the warranty period, and a modest
Two year life failure rate out to the expected useful life of the product. The
2% failure rate
The first implies duration and leaves open the expected
number of units that would survive the two years without
failure. The second example, may apply to one year, one
month, one hour or some other duration. Neither is complete.
4.1 Probability
There are a few ways to state the probability of success. I
like probability of success or reliability as its positive
(proportion not failing); it‟s easy to multiply with expected
number of shipping units to estimate the number of units still
operating over the related duration; and it is difficult to
misunderstand.
Another way to state the probability term is to use the
probability of failure, which is the complement of the
probability of success, F(t) = 1 - R(t), where F(t) is the Figure 1 CDF lot of example to assist goal setting
cumulative distribution function or probability of failure; and,
R(t) is the reliability function or the probability of success.
Other ways to state the probability include annualized
failure rate (AFR) or mean time between failure (MTBF). I
don‟t recommend these as they assume the failure rate is
constant (which it often is not) and one must use the
exponential distribution when the failure rate is constant. The
R(t) or F(t) do not assume a particular distribution.
Setting this number is a balance between the desire to
have no failures whatsoever, the cost, technology and
complexity of the product, and business profit or related
objectives. The „positioning‟ of the product as a high quality
and reliability product may or may not command a higher
price. This may depend on the marketing, competition, brand
perception and consumer acceptance. Some makes charge
more for their cars in part due to the brand perception of
reliability. Even if Yugo built a very reliable car it would have Figure 2 CDF plot with multiple slopes
to overcome the brand perception and earn market acceptance.
As a consumer I do not want my specific unit to fail and next section discusses these and other duration periods.
as a reliability engineer I understand that there is a finite risk Figure 2 shows three different slopes on the CDF plot that
that my unit will fail. If it fails quickly, I may return the reflect the balance between all the elements related to
product and buy a competitor‟s product as a replacement. If it probability of success.
fails after many years of service, I am likely to buy the same In this example, figure 2, the reliability at 30 days is
brand or model. This implies a changing acceptance of failure 99.8%, and at one year is 98%, and at five years is greater than
over time. 90%.
To set an appropriate reliability goal, it may be necessary Establishing the probability values is to some extent an
to establish multiple couplets of probability and duration for exercise in trial and error. Understanding the customer
the product. The various probabilities of success define the expectations may provide one view, which has to balance with
line between the decreasing reliability over time, and the cost of the solution and available technology. One option
thecustomers‟ acceptance of a failure. One way to represent to bracket the probability values is to review past products
this goal is to draw a line on a cumulative distribution (CDF) field reliability performance, or estimate the reliability of
plot (assuming a particular distribution, in this case Weibull), similar products in the field. Another method is to conduct
see figure 1. reliability analysis and testing on competitor products. The
At any point in time, the plot in figure 1 provides the intent is to find an acceptable level of product reliability for
probability of failure (one minus the probability of success). the customer and the producer.
For example, at 30 days the reliability is 99.7%, and at one
year it is 97%, and at five years is less than 90%.
4. 4.2 Duration Useful life is the period of time the product is expected to
function or operate for the customer. It is often well beyond
The discussion in section 4.1 alluded to a few types of
the warranty period. While there isn‟t an obligation to repair
durations: early life, warranty period, and useful life. In
or replace the product if it fails, there may be an impact to
addition, durations such as economical life, mission time, and
repeat sales, to sales of other products offered by the
many others may apply. In this paper we will explore firs three
company, or to the purchase of consumables associated with
mentioned. The discussion for the establishment of each may
the product. The last example is illustrated well by inkjet
apply directly to many other types of durations.
printers, which have a long expected useful life, such that the
Early life failures (ELF), also called infant mortality, is
consumer will continue to purchase inkjet cartridges. The first
defined by the producer to reflect the time period over which
two examples are related to brand image and the purchase
product failures are included in a different group for analysis
decision based on perception of product reliability.
and tracking. The reasons for doing this include, detection of
The useful life may have technology limitations, may
supply chain or assembly issues, detection of shipping or
have industry or market expectations (possibly also contract
installation issues, and to provide focus on the failures that
obligations), or it may have practical business limitations.
occurred very early in the expected life of the unit. Another
Consider the washing machines built in the 1950‟s. They were
reason is customers with ELF may reduce the brand‟s
cast iron, very simple in design and very durable. They also
reliability image, may complain more to potential customers,
washed clothes just fine. Some are probably still in service, as
and may have their reliability expectation eroded.
is the one in my grandparents‟ home. Today‟s machines are
The ELF period is variable and is often in the range from
lighter (less transport costs), feature rich (marketing?), and
installation or first use, up to 3 months of use by the customer.
tend to last about 10 to 12 years. One reason for this reduction
Depending on the product and market the actual period of time
in useful life is the business desire to have customers buy
may or may not start with first use, as it may be difficult to
another washing machine every 10 to 15 years. “Let‟s not
actually determine. Instead starting the clock with shipment
make it too reliable,” might be heard in discussions of such
date may serve as surrogate.
product lines.
The intention is to identify those failures that occur very
The useful life duration is the reliability goal elements
early. The focus on these helps to isolate and resolve issues
that address how long the product should last. The useful life
quickly. Also, the ELFs tend to have root causes unrelated to
is a major concern for the selection of materials and
wear out or end of useful life type failure mechanisms. Setting
components, and determining if and how to implement repairs.
the duration is local policy and enough duration to include at
least a month of use by the customer.
Warranty period is the length of time the manufacturer is 5 EXAMPLE
financially responsible to repair or replace a faulty unit. It is
Consider a bicycle as a product under development. It
beyond the scope of this paper to discuss the types of
needs to have a clear reliability goal established. Even without
warranties, so we will only focus on the broad definition.
mentioning reliability, some questions may arise from the
As warranty is an expense to the company it is of
design team: What type of bicycle? Intended use? Size and
financial interest. Warranty Week [6] reports publicly traded
weight options? Custom accessories or fittings for customer
companies spend roughly 2 to 4% of net revenue on warranty
selected accessories? Color? And many more.
claims. Therefore, one way to establish a warranty period is to
These natural questions begin to develop the function and
determine the products‟ expected failure rate and associated
environmental elements of a reliability goal. Let‟s say the new
cost of warranty over various time periods. Then set the
product is to be a road bike suitable for the amateur triathlete.
warranty such that the warranty expense is reasonable.
The full list of specifications for the model may evolve into a
In reality the above financial concern is balanced by
complete list of specifications called a product requirements
marketing, customer expectations and in some cases local
document. Succinctly let‟s say the function is a triathlon road
laws. If the current market expects a 1-year warranty, offering
bicycle.
a shorter warranty period may signal an inferior product.
The environment may need further refinement. Is the bike
Whereas, offering a longer warranty may indicate a better
to be available only in certain markets? What is the price point
product or one worth „taking a chance on‟ as it is „under
and expected level of riders? Will it be maintained by a
warranty‟. There are no hard and fast rules for setting a
professional staff or by the customer only? How many miles
warranty period and often is already established for the market
and over what terrain is expected?
the product enters.
The answers to these environmental questions may
Setting the warranty period as one of the durations
initially take the form of just an outline. And, later evolve to
permits the entire organization to understand the financial
include temperature distributions, miles per day distributions,
impact of product reliability on the business bottom line. One
maintenance frequency expectations, torque loading
way to discuss warranty within the company is to calculate the
distributions, and more. To summarize for the goal, we can
warranty cost per unit shipped. This has the advantage of
say the bike is expected to be used by amateur triathletes in
placing warranty in the same units as most of the products bill
Olympic to Ironman distance races, and including their
of materials list of component costs.
training rides.
5. Consider the probability of success and duration as 6.1 Market
couplets. The distributors and resellers are concerned with
Part of marketing a product includes setting expectations
ELF within 1 month of purchase. Discussions with resellers
for customers concerning product reliability. This may be
and with customers and review of customer satisfaction
done with clear reliability claims or comparisons to competitor
surveys for the product line and industry suggest keeping the
products.
ELF to less than 1% over the first month is not damaging to
Often the marketing team is the voice of the next
the brand‟s reputation nor significantly impact brand future
customer. Especially for new products or for new markets, the
sales. Therefore, the first couplet is stated as 99% probability
marketing team‟s input is essential for a well-positioned
of success over first month of ownership.
product within that marketplace.
The industry‟s warranty period is commonly one year for
parts and workmanship (doesn't cover normal use or 6.2 Reliable Enough
accidents). A brief discussion with senior management reveals
Using more expensive materials and technologies may
there is not a desire to differentiate on reliability with this
provide a more robust and reliable product. The washing
product so the one-year warranty is fine. A review of similar
machine example illustrates the possibility of creating a
models suggests that a 1.75% failure rate over a year is the
product that lasts an undesirably long time.
company‟s average for the past few models of triathlon bikes.
It may be possible to create a cell phone that would last
Also, a discussion with the project manager suggests the
for 20 years without failure. Given the expectation of new
weight of the bike is the primary design goal, and lowering
features enticing the customer to buy a new phone every two
warranty cost is a major concern. Therefore, the second
years negates the necessity or desire for a 20-year phone.
couplet may be stated as 98.25% probability of success over
Create the goal that is in line with customer and market
the one-year warranty period.
expectations, balanced with cost and actual market drivers for
The third of the couplets addresses the useful life of the
purchase. If reliability is not the most important factor for
bicycle. A serious amateur may replace their bike every year,
purchase, it should not be the primary concern or constraint
while others may use the same equipment for many years. A
for the design team.
(hypothetical) survey conducted at regional and national
triathlon events found most bikes are 3 years old or newer, 6.3 Safety
while a few are as much as 10 years old. Let‟s say for this
Product reliability and safety are related. One of the
paper that 90% of triathletes are riding bikes less than 5 years
considerations in setting a reliability goal concerning safety is
old. Therefore, a reasonable useful life probability/duration
considering how the product fails. If it has a reasonable
couplet is set as 95% probability of success over 5 years.
possibility of unavoidably failing in an unsafe manner (i.e.
In summary, we can state a reliability goal with all four
ignition) then one option is to reduce the chance of failure.
elements included and three probability of success (reliability)
The increase in reliability to avoid un-safe failure modes
and duration couplets.
may be feasible, yet it may preclude features, cost or other
Design and build a triathlon road bicycle intended for use
objectives. Of course it is preferred to only permit product
in training and races for triathlons ranging from Olympic to
failures into a safe mode. Another possibility is to add fail-safe
Ironman distances, with 99% reliability over the first month,
features to the product to mitigate the harm possibility caused
with 98.25% reliability over the one-year, and, with 95%
by un-safe failure modes.
reliability over the 5 years. Details for product specifications
and functions are in the PRD and environmental and use 6.4 Customer Expectations
details are in the Environment document.
Product failures, degraded performance, fracture, bugs,
6 CONSIDERATIONS and related are part of the risk we encounter with every
product we employ. Failures happen.
A goal statement is a balance between the desired and the
The expectation is not for a totally failure free experience
possible. A well-crafted goal will provide sufficient
with every product, yet each person does have an internal
information and guidance to enable decision-making during
gauge for what is acceptable. Understanding you customer
the development process that leads to attainment of the goal. It
expectations related to reliability is part customer satisfaction,
must be measureable, which is sometimes difficult for
part brand loyalty, part market share and part the nature of
reliability. A well-crafted goal also is clear, concise and easily
your customers (i.e. early adopters may have more tolerance
understood.The balance for a reliability goal is between „never
for failure).A consideration for the reliability goal is the
fails‟ and „fails too often or early‟.
perception of customer acceptance of the products reliability.
Each consideration or factor that impacts the setting of the
goal should be part of the discussion. The lack of any 6.5 Technology and Materials
reliability goal permits everyone in the organization to
Consider, for example, alow density poly(ethylene) melts
prioritize product reliability according to their local objectives.
at a relatively low temperature (98-115°C) compared to Nylon
Setting a goal that is tied to the organizations strategy,
66 (255-265°C), thus enabling the correct choice of material
includes all stakeholders and is clearly stated enables the
based on temperature when compared to use temperature. Of
entire organization to strive toward the same objective.
6. course, polymer selection as with all materials may consider Considerations and Laboratory Tests, US Department of
many different factors. For reliability, the consideration is the Defense, 31 October 2008.
amount of margin the materials and technologies have beyond 4. ETSI En 300 119 series Environmental Engineering (EE):
the expected use stresses. Environmental conditions and environmental tests for
Sometimes there are relatively obvious and low cost telecommunications equipment; Part 1-0: Classifications
improvements, sometimes there are no choices. Adjust the of environmental conditions; Introduction – and
reliability goal to accommodate any reliability limitations associated documents, freely available at www.etsi.org,
related to technology or a material expected within the product visited July 29th, 2012.
helps to communicate the priority reliability has in this area. 5. National Climatic Data Center, U.S. Department of
Commerce, as of July 6, 2012,
http://www7.ncdc.noaa.gov/CDO/cdoselect.cmd?dataseta
6.6 Value
bby=GSOD&countryabby=&georegionabby=
The final judge of a product value is the customer or end 6. Eric Arnum, Warranty Week newsletter,
user. If the product provides sufficient value (benefits www.warrantyweek.com, visited on July 30th, 2012.
outweigh costs) then the product is successful. This judgment 7. Quote attributed to Charles Coonradt at Chatsworth
is always in context of other possible solutions. Also, product Consulting Group blog, visited on July 31 st, 2012,
reliability may provide a major element of the value, or it may http://chatsworthconsulting.com/2011/06/20/want-more-
not be a consideration. The way customers perceive value is a of-it-measure-it/
major influence on product reliability goal setting. If we
BIOGRAPHIES
understand the value placed on product reliability by those
purchasing the product we have a better opportunity to meet or Fred Schenkelberg
exceed their expectations. 15466 Los Gatos Blvd #109-371
Another value to consider is the impact a reliability goal Los Gatos, CA, 95032, USA
and it‟s measurement has on the design process. “You have to
e-mail: fms@opsalacarte.com
measure what you want more of.” Charles Coonradt [7] The
simple idea is to clearly communicate what you want from the Fred Schenkelberg is a reliability engineering and
organization. If reliability is not included or not clear, then it is management consultant with Ops A La Carte, LLC, with areas
possibly considered not important. Setting and achieving a of focus including reliability engineering management training
goal also enables the entire organization to plan appropriately and accelerated life testing. Previously, he co-founded and
for warranty accruals, manufacturing capacity, spare stocking, built the HP corporate reliability program, including
etc. Guessing the reliability objectives can cause expensive consulting on a broad range of HP products. He is a lecturer
mistakes in these areas. with the University of Maryland teaching a graduate level
course on reliability engineering management. He earned a
REFERENCES Master of Science degree in statistics at Stanford University in
1996. He earned hisbachelor‟s degrees in Physics at the
1. Patrick D. T. O‟Connor and Andre Kleyner, Practical United State Military Academy in 1983. Fredis the immediate
Reliability Engineering, 5th Edition, Chichester, Wiley, Past-Chair of the American Society of Quality Reliability
2012, p 1. Division, active with IEEE and IEC reliability standards
2. Joseph M. Juran and A. Blanton Godfrey, Juran‟s Quality development teams. Fred is also the founder of the No MTBF
Handbook, 5th Edition, New York, McGraw-Hill, 1999, p movement and website nomtbf.com. He is a Senior Member of
13.9. ASQ and IEEE. He is an ASQ Certified Quality and
3. MIL-STD-810G, Environmental Engineering Reliability Engineer.