Integrate model-based testing to find quality problems early in development

Integrate model-based testing to find quality
problems early in development
Advantages of using the Rational Rhapsody TestConductor
Add On with Rational Quality Manager throughout the software
development process

Skill Level: Introductory

Marc Lettrari, PhD (marc.lettrari@btc-es.de)
Vice President, Rhapsody and Statemate Testing Products
BTC Embedded Systems AG

05 Apr 2011

Take advantage of the integration of the Rational Rhapsody TestConductor Add On
with Rational Quality Manager to enhance your model-based testing inside a
powerful generic framework. This article shows how the integration helps enable
risk-based testing to assist QA teams as they prioritize testing of specific features
and functions, based on their importance in the overall project and likelihood or
impact of failure. Combining this with new reporting dashboards gives product
managers a more realistic view of product performance according to set business
objectives to better ensure that software development projects stay on track.

Managing model-based tests with Rational Quality Manager
Quality assurance teams are often brought into the development lifecycle too late,
after requirements are set and errors introduced into the design. To remedy this,
IBM has integrated the Rational® Rhapsody TestConductor Add On with Rational®
Quality Manager software to provide a live test that span the entire product life cycle.
The result provides a consolidated view from requirements to product delivery. This
article describes how to leverage this integration by bringing model-based tests into
your overall testing process.

Integrate model-based testing to find quality problems early in development Trademarks
© Copyright IBM Corporation 2011 Page 1 of 13

developerWorks® ibm.com/developerWorks

This integration works by using the Rational Quality Manager solution to manage
different kinds of tests cases, their execution and results executions. This tool
displays requirements, test cases, and other resources in one server-based view,
which helps geographically dispersed team members exchange information in real
time. The Rhapsody TestConductor Add On uses the UML testing profile to
automatically specify the test architectures, to visualize test cases for improved
understanding, and to execute the tests to pinpoint design model failures and
implementation errors.

The article also shows how Rhapsody TestConductor test cases can be managed
and incorporated into the overall quality management process within Rational
Quality Manager. It explains how the integration helps enable risk-based testing,
thus assisting QA teams as they prioritize testing of specific features and functions
based on their importance in the overall project and likelihood or impact of failure.
The ability to prioritize, combined with new reporting dashboards in the software,
offers product managers a more realistic view into product performance against set
business objectives to better ensure that your project stays successfully on track.

Quality management throughout the software development lifecycle

Today's tech-savvy consumers expect high quality from products they use.
Delivering anything less than that can result a company reeling from a perception of
poor quality for years to come. Consumers also expect advanced features, which
drives complexity. The amount of software within embedded products is increasing
and driving key product differentiation.

With these challenges of quality, complexity, and size, companies need to optimize
their software testing to achieve desired business outcomes. Often, software testing
is done late in the development cycle, which leads to finding the majority of defects
near the end, when they are the most costly and riskiest to address. However, these
defects were introduced into the design early in the development process. Software
developers perform tests while developing software, such as unit tests, but these
tests tend to be very isolated to the developer or inconsistent across developers.
Software developers need to focus on developing, so testing becomes an area they
take shortcuts on when a deadline looms.

The advantage of model-driven testing

Fortunately, emergence of the Unified Modeling Language (UML) and Systems
Modeling Language (SysML) standards has provided the opportunity for
breakthrough gains in streamlining the development process. These standards
enable engineers and software designers to significantly improve their productivity
by transitioning from a code-based development process to model-driven
development (MDD). Using models, systems engineers and software developers
can more clearly understand and analyze complex requirements, automate repetitive
tasks, maintain consistency across design activities, test systems concepts earlier


ibm.com/developerWorks developerWorks®

by using simulation, and automatically generate code for direct deployment on the
target hardware.

Model-based testing extends model-based development beyond design to testing.
The Rational Rhapsody TestConductor Add On adds model-based testing
capabilities based on the UML Testing Profile to the Rational Rhapsody Developer,
Rational Rhapsody Designer for Systems Engineers, or Rational Rhapsody Architect
for Software editions. The testing profile tailors the development environment for
testing by adding concepts such as test architectures and test behaviors into UML.
Test architectures extend the existing UML 2.0 structural concepts to describe the
elements involved in a test and their relationships. Similarly, the test behavior
extends the existing UML 2.0 behavioral concepts to encompass all observations
and activities during the test.

The TestConductor Add On automates the testing activities by creating a test
architecture and enabling tests to be graphically captured by using UML sequence
diagrams, state charts, or flowcharts. Capturing tests graphically allows better
communication of the tests and further elicits the behavior of the design. The tests
can be executed and results monitored to automate unit and regression testing.

By extending testing forward in the process to include the design model, then quality
managers, and software engineers can check that each feature meets requirements
early in development so they can assess the overall quality of the design early.

Overview of Rational Quality Manager

IBM® Rational® Quality Manager is a collaborative, role-based, business-driven
environment for test planning and execution, workflow control, tracking, and metrics
reporting. It provides a web-based centralized test and quality management
environment for project managers, quality engineers, and other project stakeholders
to collaborate and share information about the current quality of the project.

Rational Quality Manager quantifies how project decisions and deliverables affect
and align with business objectives and outcomes. It helps the team reduce defects
and accelerate product delivery schedules.

Using this software, you can create a customizable dashboard for quick and easy
review of systems quality metrics and project status, such as builds ready to be
tested. It also provides traceability features that can help test teams ensure that test
plans are covering all requirements. The dashboard helps highlight areas where
quality is at risk in the project and where priority should be focused.

Analyzing and elaborating requirements using executable
models



The starting point of a development project is the requirements analysis phase. The
objective of this phase is to analyze the process inputs. Stakeholder requirements
are translated into system requirements that define what the system must do
(functional requirements) and how well it must perform (quality of service
requirements). In a model-based development process, using IBM Rational
Rhapsody as the underlying UML modeling tool, this means that the initial
stakeholder requirements (which are stored in IBM® Rational® DOORS®
requirements management software, for example) are imported into the
requirements model, which is depicted in Figure 1.

After the stakeholder requirements are imported, a model-based requirements
analysis can be performed. This means that a UML model is created that captures
the given requirements and can be used for analyzing them. With Rational
Rhapsody, it is possible to create executable models, allowing analysis of the
requirements by executing the model derived from these requirements for validation
early in the development lifecycle.

Figure 1. Requirements from Rational DOORS are imported and further
analyzed and validated in a Rhapsody model

Validating requirements with model-based testing

As described in the previous section, based on the initial stakeholder requirements,
an executable UML requirements model has been created that serves as a
reference model for the intended behavior of the system to be developed. However,
if the requirements model should serve in later development stages as a reference
model, it must be verified that the behavior of the model indeed fulfills the given
requirements. This verification step can be done and automated with model-based
testing.



Model-based testing enables dynamic testing of the behavior of the model, meaning
that test cases can be specified (for example, using UML sequence diagrams) and
automatically executed on the model level. The UML Testing profile (see the second
reference in Resources) provides definitions of all concepts and terms needed to
model test architecture, test behavior, and test data within the UML modeling
language. For UML models created with Rational Rhapsody, the TestConductor Add
On provides all needed capabilities and a high degree of automation to help you
create and execute model-based test cases.

Similar to dynamic software testing, dynamically testing the model ensures that the
behavior of the model is indeed correct with regard to the given requirements.
Furthermore, given that both requirements and test cases are part of the model, they
can be linked directly, which ensures complete traceability from requirements to test
cases and test results. Figure 2 shows an example of a test case that is described
with a sequence diagram.

Figure 2. A sequence diagram can specify inputs and expected results of test
cases

Integrating model-based testing into the quality management
process
It is widely accepted that quality management throughout the whole development
lifecycle is necessary to get a high-quality product. Rational Quality Manager helps
optimizing project quality with a single, shared test management hub that provides



integrated lifecycle support across many types of testing. It provides a customizable,
role-driven solution for test planning, creation, and execution. as well as workflow
control, tracking, and end-to-end traceability.

One important aspect of integrated quality management is proactively managing
projects and quality risks with real-time, fact-based insights and reporting to ensure
that project decisions and deliverables remain aligned with your business objectives.
At any time during the development lifecycle, quality management should be able to
answer the question: "Are we still on track?"

Especially in early development phases, such as during the functional system
specification phase, answering this question can sometimes be hard, because
dynamic software testing cannot be performed because there is no software
implementation available at this stage. However, if a model-based approach is used
for system specification, so that an executable model is already available,
model-based testing can already be applied at this early stage to assess to what
extent the original stakeholder requirements are met by the current system
specification. This means that, despite being quite early in the development phase,
you can already measure the quality status of the current project.

How to integrate model-based testing into the quality management process

For Rational Quality Manager and the Rhapsody TestConductor Add On, there is a
specific adapter that you can use to access model-based tests directly from Rational
Quality Manager. After the adapter is started, it automatically connects to Rational
Quality Manager.

Thereafter, when you create a test script in Rational Quality Manager, you can
connect it to an executable model-based test in Rhapsody by following these steps:

1. Create a test script in Rational Quality Manager (see Figure 3). In
Rational Quality Manager, each test case needs a test script in order to
execute the test case.

Figure 3. Create a test script in Rational Quality Manager to execute the test
case



Larger view of Figure 3.

2. After specifying details such as the test script name and so forth, select
Rhapsody TestConductor in the Type field of the test script (see Figure
4). This tells Rational Quality Manager that the test script is actually
implemented as a model-based test in Rational Rhapsody.

Figure 4. Specify Rational TestConductor as the type of test script




3. After specifying that the test script is based on a model-based test, you
need to define which model-based test actually implements the test script.
To do this, from the Create Test Script dialog window, click Select
adapter.

4. When a dialog window opens that shows all available adapters, simply
select the adapter that you have started (see Figure 5).

Figure 5. Select the adapter for the test case


5. After you have selected the adapter, the adapter lets you choose the
Rhapsody model (Figure 6) that contains the model-based test.

6. After selecting the model, select the model-based test case that should be
connected to the test script (see Figure 7).

Figure 6. Specify the Rhapsody model where the test case is located




Figure 7. Specify test case to associate with test script


That's it. Now Rational Quality Manager recognizes that this test script is actually
implemented by a model-based test that can be executed with Rhapsody
TestConductor.



When you run the test script, Rational Quality Manager automatically detects that
this is a model-based test and uses the TestConductor adapter to execute it on the
referenced executable Rhapsody model, as depicted in Figure 8.

Figure 8. Model-based test cases can be run from Rational Quality Manager


After executing the test script in Rational Quality Manager, all of the test results,
such as status (passed or failed) and log files, are directly accessible in Rational
Quality Manager (see Figure 9). This means that you can use all of the reporting
capabilities of Rational Quality Manager to assess the quality status of the project,
even in early development phases.

Figure 9. Results of TestConductor tests can be stored and managed in
Rational Quality Manager




Reusing model-based tests for software verification
The previous section described how model-based tests can be integrated into an
overall quality management process that is based on Rational Quality Manager so
that you can measure the quality status in early development phases. Model-based
tests can also be used to measure the quality status of the software implementations
as soon as they are available. The model-based tests can be adjusted easily so that,
rather than testing the underlying model, they test the real system and software
implementations, respectively. If these adjustments are made, the model-based
tests can be executed from Rational Quality Manager in exactly the same way as
before, but not to test the real implementation.

And again, Rational Quality Manager can use the results of these tests along with
the metrics measuring reporting capabilities included to assess the quality status of
the implementations. Thus, with Rational Quality Manager, model-based tests
specified early can be used throughout the software development lifecycle to help
assess the project's quality.

Summary

Integrating model-based testing with the Rational Rhapsody TestConductor Add On
and using with Rational Quality Manager for quality management helps bring make
the quality status of the project visible earlier in the development phases. Teams can
then identify quality risks in the project earlier, respond to changes quickly, and
assess the impact from a quality perspective. By seeing the tests that fail, teams can
recognize critical areas of concern and work to mitigate those risks earlier in the
development lifecycle.



Resources
Learn
• To learn more about the Rational Rhapsody collaborative, model-driven
development tool for embedded systems, start with these overviews:
Introduction to Rational Rhapsody, the Rational Rhapsody Developer product
page, and the Rational Rhapsody page on IBM developerWorks.
• Explore the various versions, too:
• IBM® Rational® Rhapsody® Architect for Software, a visual
development environment for embedded systems and software
• IBM® Rational® Rhapsody® Architect for Systems Engineers
• IBM® Rational® Rhapsody® Designer for Systems Engineers
• IBM® Rational® Rhapsody® Developer edition. This edition is
required for Eclipse users and editions are available to create
specialized projects in C, C++, Java, and Ada languages.
• Read the PDF file titled Rhapsody TestConductor Add On Adapter for
Rational Quality Manager Overview, which is available in the Rational
Rhapsody installation when the TestConductor Add On is installed.

• Check the Rational Quality Manager page on IBM® developerWorks® for links
to product documentation, articles, tutorials, courses, downloads, and other
useful areas. Also, explore the Rational Quality Manager Information Center for
technical details and browse the IBM Quality Management page to learn more
about what is available. For tips, read these developerWorks articles by Michael
Kelly:
• Getting started with IBM Rational Quality Manager (December 2008)
• For Managing your first project with IBM Rational Quality Manager
(October 2008)

• Visit the Rational software area on developerWorksfor technical resources and
best practices for Rational Software Delivery Platform products.
• Stay current with developerWorks technical events and webcasts focused on a
variety of IBM products and IT industry topics.
• Attend a free developerWorks Live! briefing to get up-to-speed quickly on
IBM products and tools, as well as IT industry trends.
• Watch developerWorks on-demand demos, ranging from product
installation and setup demos for beginners to advanced functionality for
experienced developers.



• Improve your skills. Check the Rational training and certification catalog, which
includes many types of courses on a wide range of topics. You can take some
of them anywhere, any time, and many of the "Getting Started" ones are free.
Get products and technologies
• Download Rational Rhapsody Developer and try it free for 30 days.
• Download the free trial version of Rational Quality Manager.
• Evaluate IBM software in the way that suits you best: Download it for a trial, try
it online, use it in a cloud environment, or spend a few hours in the SOA
Sandbox learning how to implement service-oriented architecture efficiently.
Discuss
• Join the discussion in the Rational Rhapsody forum.
• Join the Rational Quality Manager forum, which also includes discussions about
Rational Test Lab Manager.
• Share your knowledge and help others who use Rational software by writing a
developerWorks article. You'll get worldwide exposure, RSS syndication, a
byline and a bio, and the benefit of professional editing and production on the
developerWorks Rational website.
• Follow Rational software on Facebook and Twitter (@ibmrational), and add your
comments and requests.
• Ask and answer questions and increase your expertise when you get involved
in the Rational forums, cafés, and wikis.
• Connect with others who share your interests by joining the developerWorks
community and responding to the developer-driven blogs.

About the author
Marc Lettrari, PhD
Dr. Marc Lettrari received his master’s degree in computer science from
the University Oldenburg, in Germany, in 2000 and his PhD in computer
science in 2005. He previously worked at the OFFIS computer science
institute in Oldenburg and conducted research in the field’s formal
methods, model-based development, and testing. Since then, he has
been working at BTC Embedded Systems AG, in Germany, where he is
Vice President for Rhapsody and Statemate Testing Products.


Integrate model-based testing to find quality problems early in development

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (17)

Mehr von Bill Duncan

Mehr von Bill Duncan (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Integrate model-based testing to find quality problems early in development