Gartner Research Director Thomas Murphy notes that software quality is often a poor misnomer for the current practice of risk management applied by most companies. Many organizations use risk management to mitigate delivery risk, typically at the expense of application quality. Learn about the importance of focusing on application structural quality to reduce business disruption risk in this Gartner-CAST paper.

1. S f r R s Ma a e n:
ot e ik
wa n g me t
AP i r o I E e uie
rme fr T x c t s
v

2. Software Risk Management:
A Primer for IT Executives
Controlling the Structural Drivers of Business Risk
Featuring research from

3. Excerpt from:
Predicts 2010: Agile and Cloud Impact
Application Development Directions
2 The following section is excerpted from Predicts 2010: Agile and Cloud
Predicts 2010: Agile and Cloud Impact Impact Application Development Directions. To view the full note, click
Application Development Directions anywhere in the section.
4 Strategic Planning Assumption: Through 2015, the shift toward
Software Risk Management cloud architecture will create demand for new skills, practices and
objectives for software quality.
6
About CAST Analysis By: Thomas Murphy
Key Findings: Software quality is often a poor misnomer for the
current practice of risk management applied by most companies
when it comes to practices and scheduling in software projects. The
focus is not to drive quality, but to mitigate risk. While this is a viable
approach, it also goes together with a concept that quality equals the
absence of defects. Although this is theoretically true, the application
is often too narrow to say that from this, quality software is delivered.
The International Organization for Standardization (ISO) produced a
standard (9126) that is generally ignored, because quality costs, and
often is not seen as providing a return on investment.
However, as organizations seek to drive down maintenance costs and
adapt to the shorter project life cycles found in agile practices, there
is a need to focus efforts on a broader quality definition. In addition,
organizations will need to invest in additional tools and skills to deal
with increasingly complex distributed applications. Development
frameworks may hide some of the complexity of creating these
applications, but it won’t help with the testing of applications.
We are seeing strong growth now in tools that support a more
automated test lab environment. This includes:
• Virtual lab management
• Virtualization of services
Application Structural Quality is the Key to Software Risk Management is published by CAST Inc. Editorial supplied by CAST Inc. is independent of Gartner analysis. All Gartner research is © 2010 by Gartner, Inc. and/or
its Affiliates. All rights reserved. All Gartner materials are used with Gartner’s permission and in no way does the use or publication of Gartner research indicate Gartner’s endorsement of Cast’s Software products and/
or strategies. Reproduction and distribution of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner
disclaims all warranties as to the accuracy, completeness or adequacy of such information. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations
thereof. The reader assumes sole responsibility for the selection of these materials to achieve its intended results. The opinions expressed herein are subject to change without notice.
2

4. 3
• Improved tools for test data management, including subsetting Market Implications: The shift first toward SOA, then to rich Internet
and data masking applications has stressed the ability of testing tools to keep up with
technology shifts, and for testing teams to keep up with the pace
• Integration into life cycle tools to improve traceability and of technology and application changes. The complexity of testing
automation of workflow, and to close gaps in the common bugs scenarios requires vendors to also deliver a broader spectrum of
that cannot be reproduced in tester/developer interaction tools. This is resulting in a number of new companies and products
coming to market, and will also result in increased acquisition activity
However, these are all just improvements to business as usual. as existing market leaders look to fill out their solutions.
While ALM tools provide better accountability to requirements, quality
software has a variety of attributes not directly connected to normal While many organizations will be attracted to the promise of reuse
requirements, including: from SOA, success will be limited because of the lack of skills and
structure to support reusable assets. Reuse requires a view toward
• Understandability governance, ownership and quality.
• Completeness Because software quality can’t be tested at the end, organizations
will need to look at facilities and practices that drive quality through
• Conciseness the development life cycle. This will include using practices from agile,
such as TDD, and using tools that drive repeatable processes, such as
• Portability continuous integration (CI). This will also create a continued drive for
the use of ALM solutions that provide integration across the life cycle.
• Consistency
A great challenge will be dealing with development that happens
• Maintainability outside the traditional IT process. Simplified business process
management (BPM) and mashup tools make it easy for business
• Testability analysts and end users to quickly assemble new solutions. However,
this requires that the underlying components are stable, secure and
• Usability scalable. It also requires that organizations are consistent. These
requirements will continue to drive the market for static analysis tools
• Reliability and service registries and repositories.
• Structure Recommendations:
• Develop testing practices and expertise in security, scalability and
• Efficiency automation.
• Security • Drive practices that drive quality from start to finish on a project.
This includes shoring up weak requirements practices.
The ongoing promise of evolving Web application architectures is to
deliver applications and services that are customizable by business • Establish quality career path and standard definitions to set
analysts and end users. Just as many organizations have moved expectations and drive consistency.
more than 50% of their “development” budgets into packaged
implementations, we believe that this trend will continue with
increased capabilities for non-developer-targeted development.
However, companies that seek to utilize technology to drive business Source: Gartner
innovation will evolve a more holistic view of software quality,
because without it, they will not be able to support the ever-increasing
maintenance burden.
3

5. Software Risk Management
The process by which IT business software is However, as Thomas Murphy points out in managing delivery risk alone only addresses
built and the resulting software product itself the excerpt above, software quality is often a part of the problem. It’s like addressing the
are to some extent intertwined. It’s tempting erroneously equated with mitigating risk symptoms of a disease rather than taking
then to think that when we have reliable, in “practices and scheduling in software aim at curing its cause. To get to the root
repeatable processes for building the projects.” There is much more to software causes, we have to define, analyze, and
software product, the quality of the resulting risk than that. The main, if not only, reason measure software product quality.
product will be equally good. for building and maintaining applications is
for the business value they generate. With 2. Three Kinds of Software Product
Despite that temptation, we have all known this in mind, let’s distinguish three kinds of Quality and the Importance of Structural
first hand that an application delivered on business risk from software applications. Quality
time, on budget, and even on scope cannot Let’s begin by distinguishing three basic
achieve its business goals if it is slow, 1. Delivery Derailment Risk – risks that add types of software product quality.
behaves unpredictably, or compromises IT cost or stop business revenue due to
privacy. Moreover, a poorly built application delayed launch or cancellation. 1. Functional Quality – a measure of
is expensive and slow to respond to what the software does versus what it’s
business, further eroding present and 2. Business Case Risk – risks that affect the supposed to do.
future business value. Nonetheless, most quality of a delivered application; even
discussions of managing software risk though the application works, it doesn’t 2. Non-Functional Quality – a measure of
continue to equate the quality of the process work as well as it should. The number how well it does it versus how well it’s
with the quality of the resulting product. of successful transactions per unit time supposed to do it.
cannot be completed to fulfill the benefits
To truly manage the business risk of articulated in the business case. 3. Structural Quality – a measure of how
applications, we must move beyond the well it will continue to perform as it is
quality of the process to the quality of the 3. Business Opportunity Risk – risks that meant to in the future.
product itself. The main aim of this article make the application hard to maintain
is to distinguish three kinds of software and change in the face of pressing When it comes to the quality of the software
product quality: functional, non-functional, business demand. The resulting loss of product, functional quality alone is not
and structural quality and explain why agility damages future business revenue. enough. If all that matters is having the right
structural quality is essential for managing functionality, then every car that lines up on
the root drivers of IT costs and business Managing delivery derailment risk alone is the NASCAR starting grid would win the race!
risks in your mission-critical applications. insufficient for generating business value. But of course, winning the race takes more
Structural quality metrics enables us to Reliable project management processes than satisfying the functional specification – it
understand, predict, and control the key and the right functionality are nothing if the takes superior performance in the real world.
drivers of software costs and business risks. application works unpredictably, is slow, or
breaks down often. In addition to on-time, Similarly, non-functional quality is not
1. Why Software Quality is Critical on-budget and on-scope delivery, business enough. Non-functional quality focuses
Software is the backbone of the modern value is generated by the functionality on the visible behavior of the software
enterprise. Software animates critical working like it should. When the application – the availability and latency of critical
elements of an enterprise’s value chain. is not performing like it should you cannot transactions. While this is important (in
These statements are obvious enough to achieve the benefits articulated in the addition to the software’s usability), these
be clichés. But current conditions – both business case. performance indicators are skin deep. To
technological and business – make equate them with product quality would be
modern value chains increasingly difficult to Unlike the quality of the process by which to equate, for example, the destruction left
animate without incurring large amounts of software is built, enhanced, and maintained, in the wake of an uncontrolled skid with the
business risk. functional, non-functional, and structural quality of the suspension system that was the
quality have to do with the software product root cause of this destruction.
itself – the asset that generates business
value. Managing software quality by
4

6. 5
Availability and latency are classic examples Analyzing the quality of modern applications in To truly manage the cost and business risks
of “visible” or “above-the-waterline” metrics. the context of the numerous interconnections of your mission critical applications you must
They are rear-view mirror metrics with little with other code, databases, middleware, and move beyond process metrics to product
or no predictive power. They tell you how APIs is monstrously complex. It can only be quality metrics and in particular, measure
the system is doing (symptoms) but not why accomplished with an automated system that the structural quality of applications.
things are going well (or badly). analyzes the inner structure of all components
and evaluates their interactions in the context of Structural quality metrics are forward looking
On the other hand, structural quality measures the entire business application. and actionable – they go beyond functional
how well the application is designed and how and non-functional quality to the root causes
well it is implemented (the quality of the coding Moreover, the component and/or its of application costs and business risks. They
practices and the degree of compliance with environment changes as a result of technology give you the visibility and control you need to
the best practices of software engineering that upgrades, user needs, and business needs. This manage your mission-critical applications.
promote security, reliability, and maintainability. means that any system for measuring product
Structural quality metrics track the root causes quality must have both breadth and depth. The multi-tier, multi-language, and multi-
of application latency and availability. They platform nature of modern applications
are forward-looking metrics that enable us • Breadth: Comprehensive coverage of the make automation essential for measuring
to control how an application performs, how entire system from end to end. In modern structural quality. No human or team has
readily it can be enhanced in response to systems, this means it has to cover a sufficient end-to-end visibility of the entire
urgent business requests, and how much it will multiplicity of languages, technologies, and system. Moreover, because structural
cost to maintain. frameworks all the way from the GUI front quality is contextual, it requires sophisticated
end to the middleware to the database. algorithms to analyze and measure it.
Let’s consider what structural quality would be
for a house. In this analogy, structural quality • Depth: A detailed architectural/logical In addition to the breadth of technology
would not be about the number of rooms or view of the entire system from end to end. coverage and the sophistication of
the way in which the rooms are furnished. The quality measurement system must contextual quality analysis, automated
Rather, structural quality would be about the be able to create detailed architectural systems for analyzing and measuring
engineering design (e.g. where the load- maps of the entire system -- views of all software quality must also be able to
bearing walls are placed, the strength and the components and how they are inter- provide detailed information on the root
pliability of materials used) and how well the connected. It must be able to capture the cause of quality problems, and provide
materials come together (e.g. the soundness logical aspects of the system, not simply practical guidance on how these root causes
of joints, the organization of the electrical and the physical representation of it. It must can be fixed once and for all.
plumbing lines). A house with high structural be able to use this detailed logical view to
quality is typically easy to maintain and extend. evaluate the product quality of the system When these quality metrics are measured
in the context of the entire system. over time, they provide valuable information
But what do structural quality metrics look on quality trends – actionable information
like and what does it take to measure them? Conclusion for prioritizing focus areas and allocating
Let’s consider those questions next. Application quality is often equated with the resources for improvement. And these very
results of testing or with being able to manage same quality metrics (the change in their
3. How to Measure Structural Quality delivery derailment risks. This is dangerous values) serve to measure the effectiveness of
Structural Quality is Contextual because it completely misses the key reason for these improvement efforts.
building and operating business applications:
Source: CAST Inc.
The fundamental challenge of software product the creation of business value.
quality is that it is contextual. The quality of a
single component depends on its local and
global environment. The quality of a single
component cannot be evaluated independently
from its environment.1
1
Olivier Bonsignour and Bill Curtis, “Why Application Quality Is Different From and More Important Than Code Quality”
(http://www.slideshare.net/jsub/application-quality-is-not-code-quality)
5

7. About CAST
The CAST Application Intelligence Platform is the only enterprise-grade
software quality assessment and performance measurement solution
available in the market today. The CAST solution inspects the source
code, identifies and tracks quality issues, and provides the data to
www.castsoftware.com
monitor development performance.
CAST Headquarters
North America: +1 212-871-8330
CAST can read, analyze and semantically understand most kinds
Europe: +33 1 46 90 21 00
of source code, including scripting and interface languages, 3GLs,
4GLs, web and mainframe technologies, across all layers of an
application (UI, logic and data). By analyzing all tiers of a complex
application, CAST measures quality and adherence to architectural
and coding standards, while providing visual specification
models. Managers get real time access to this information via a
web interface by which they can proactively monitor, measure and
improve application health and development team performance.
CAST’s unique technology is the result of more than $80 million in
R&D investment. Top engineering talent, dedicated to building the
best technology for assessing the structural quality of mission-
critical applications, has made CAST the leader in Automated
Application Intelligence. CAST’s mission is to use software
measurement to transform application development into a
management discipline.
Founded in 1990, CAST has helped more than 650 organizations
worldwide speed IT delivery to the business, mitigate risks in
production, improve customer experience, and reduce the total
cost of application ownership. CAST is listed on NYSE-Euronext
(Euronext: CAS) and serves Global 2000 organizations worldwide
with a global network of locations in the US and Europe.
6

8. L a nmo ea o t A T
er r bu C S
w w c ss f aec m
w .a tot r.o
w
bo .a tot aec m
lgc ss f r.o
w
w w fc b o .o c so q a t
w . e o kc m/a tn u ly
a i
w w sd s aen t a tot ae
w . ie h r.e/ ss f r
l c w
w w t ie.o O Q a t
w . t r m/ n u ly
wt c i