2. References
Automating regression testing for evolving
GUI software
– Atif Memon, Adithya Nagarajan and Qing Xie
– JOURNAL OF SOFTWARE MAINTENANCE
AND EVOLUTION: RESEARCH AND
PRACTICE 2005; 17:27–64.
2
3. Outline
1. Introduction
2. The DART process
3. Design of DART
– 3.1. GUI Representation
– 3.2 Modules of DART
4. Experiments
5. Conclusions
3
4. 1. Introduction
Today’s competitive software development
market demands that several developers,
who are perhaps geographically
distributed, work simultaneously on large
parts of the code during maintenance.
Rapid quality assurance feedback
– nightly/daily building and smoke testing.
– To build the product and test it every day
4
5. DART
(Daily Automated Regression Tester)
A new framework that address the needs of re-
testing frequent builds of GUI software.
Automation!!
– structural GUI analysis
– test-case generation
– test-oracle creation
– code instrumentation to
test execution
coverage evaluation,
regeneration of test cases
Test cases re-execution.
5
10. 3. Design of DART
Driving philosophy behind the design
1. Automated
2. Efficient
3. Robust
4. Portable
5. General
10
11. Runs all
test cases
Expected
results
How much
testing
“glue”
Coverage
information Reverse
Engineering
Test Cases
11
12. 3.1 GUI Representation
A formal model of the AUT’s GUI
GUI Representation
1 Objects & Properties
2 Events 3 Components
4 Event-flow Graphs 5 Integration Tree
12
13. 3.1.1 Objects and properties
objects O = {o1, o2, . . . ,om}
properties P = {p1, p2, . . . ,pl }
A GUI = O + P
The state of a GUI = a set P at time t.
valid initial state set = iff the GUI may be in
any state Si ∈ SI when it is first invoked.
13
14. 3.1.2 Events
The events E = {e1, e2, . . . , en} associated
with a GUI are functions from one state to
another state of the GUI.
legal event sequence = e1;e2;e3; . . . ; en
14
15. 3.1.3 Component
A GUI component C is an ordered pair (RF, UF)
– A modal window + a set of modeless windows
RF represents a modal window in terms of its
events.
UF is a set whose elements represent modeless
windows also in terms of their events.
Each element of UF is invoked either by an
event in UF or RF.
15
16. 3.1.4 Event-flow graphs
All possible interactions among the
events in a component.
An event-flow graph for a component C is
a 4-tuple <V, E, B, I> where:
– V : all the events in the component
– E ⊆ V × V : directed edges
– B ⊆ V : available to the user when the
component is first invoked
– I ⊆ V : invoke other components.
16
18. 3.1.5 Integration Tree (IT)
To identify interactions among
components.
Invocations
– Component Cx invokes component Cy iff
Cx contains an event ex that invokes Cy .
18
19. Definition of IT
An IT is a triple <N, R, B>
– N : components in the GUI.
– R ∈ N: a designated component called the
Main component.
– B : directed edges , the invokes relation
between components
19
27. 3.2 Modules of DART
2
1 Test Case Generator
3
GUI Ripper Test Oracle Generator
GUI Representation
4 6
5 Test Executor
Coverage Evaluator
Code/Event Instrumenter
27
28. 3.2.1 GUI Ripper
Step1: GUI traversal and extraction (DFS)
Step2: Manual inspection by Tester
Step3: Generating the event-flow graph and
Integration Tree
28
29. 3.2.2 Test-case generator
A GUI test case T is a pair <S0, e1; e2; . . . ;
e n>
Generate test case by traversing the
event-flow graphs
29
30. 3.2.3 Test-oracle generator
Test oracles are used to determine
whether or not the software executed
correctly during testing.
<S0, e1; e2; . . . ; en> S1;S2;…;Sn
expected state or not
30
31. Approach: execution extraction
A test case is executed on an existing,
presumably correct version of the software
and its state is extracted and stored as
oracle information.
31
32. Level of Oracle Information
(LOI)
Complete (LO1)
– {(w, p, o), ∀w ∈ Windows, ∀o = objects ∈ w, ∀p =
properties ∈ o}
Complete visible (LO2)
– {(w, p, o), ∀(w ∈ Windows)&(w is visible), ∀o = objects
∈ w, ∀p = properties ∈ o}
Active window (LO3)
– {(w, p, o), (w = active Window), ∀o = objects ∈ w, ∀p =
properties ∈ o},
Widget (LO4)
– {(w, p, o), (w = active Window), o = current object, ∀p
= properties ∈ o}
32
34. 3.2.5 Event instrumenter
All event sequences that are executed on
the GUI be collected.
The key Idea: To detect the existing
listeners and attach our own listeners.
34
36. 3.2.6 Test executor
Executing an entire test suite
automatically on the AUT.
– perform all the events in each test case
What properties should be compared?
– level of testing (LOT1 - LOT4)
– The test designer may choose to employ
partial oracle information.
36
37. Possibilities available to the test designer for level
of detail of oracle information.
10 different combinations
37
38. 4. Experiments
The subject applications: TerpOffice
http://www.cs.umd.edu/~atif/newsite/terpoffice.htm
38
47. Time requirements
All times are reported for a 2.2 GHz
Pentium 4 machine with 256 MB of RAM.
The results show that the smoke testing
process is practical, in that it can be
performed in one night.
47
54. Code coverage
We instrumented the application before
running all of the smoke test cases.
We recorded the statements that were
executed for each user-implemented class
during test-case execution.
54
61. 5. Conclusions
We define a formal model of a GUI
derived from specifications that is
useful for smoke testing.
We develop a new process for re-
testing nightly builds of GUI software.
Our regression testing process
cannot only be used for nightly builds,
but also for general GUI re-testing.
61
62. In the future
We will study the effectiveness of the DART
process by analyzing the number of faults
detected.
We will also integrate DART in a higher-level
process that involves executing other types non-
GUI) of smoke tests on the software.
We will also investigate the application of DART
to other software systems that take events as
input. (e.g. Web applications)
62
Hinweis der Redaktion
對 GUI 的一種 modeling ,是不是一種 specification ??
TerpCalc - a calculator program TerpManager- an explorer program for TerpOffice Suite, customizable to any set of programs TerpPaint - a paint/drawing program TerpPresent - a presentation program TerpSpreadsheet - a spreadsheet program TerpWord - a word processor/html editor