1. Leveraging Selenium
recorded tests to generate
more effective tests
Author: Cu D. Nguyen, PhD
http://selab.fbk.eu/dnguyen/
Tuesday, January 8, 13

2. Selenium tests
• A Selenium test consists of a sequence of
events and their input parameters.
• Example:
Event
open /CuteNews/
clickAndWait link=Add News link=Add News
type name=title afternoon news
select id=selecsinglecat label=Hot
clickAndWait css=input[type="submit"]
clickAndWait link=Logout
Tuesday, January 8, 13

3. Selenium tests (contd.)
• Often capture/replay specific test scenarios
(or paths)
• Stick with specific input values
• The user might have to copy/paste/
modify recorded tests and specify new
inputs
➜ often focus on some specific functionality
➜ doesn’t scale, costly !
Tuesday, January 8, 13

4. We can do better with Selenium recored
tests!
• How:
1. combining test paths (scenarios) to
create a test machine (a map, as an
analogy) to generate more paths
2. fusing available test inputs to infer input
partitions, and generate new inputs from
that
3. combining new paths + new inputs to
create new tests
Tuesday, January 8, 13

5. I. Inferring test machine
Tuesday, January 8, 13

6. Inferring test machine from available
paths
• Simple example illustrating the value of
combining test paths
Available paths: 1. add, add
2. add, remove, checkout
1. add, add
Results by 2. add, remove, checkout
combining paths: 3. add, checkout
4. add, add, checkout
- + Buy +
that's easy!
Here we are testing a web page that has
want to buy!
only three buttons: add, remove, and
checkout add remove checkout
Tuesday, January 8, 13

7. Inferring test machine (contd.)
• We can do even better: from existing test
paths, we infer a test machine (or in a
technical terminology: a finite state
machine, FSM)
• Example:
Available paths: Inferred machine:
1. add, checkout add
2. add, remove, checkout remove
add
3. add, add, checkout
checkout
Tuesday, January 8, 13

8. Inferring test machine (contd.)
• From the test machine, we can generate
more test paths
• Example:
Inferred machine: Generated test paths
add 1. add, checkout
add remove
2. add, remove, checkout
3. add, add, checkout
checkout
4. add, add, add, checkout
5. add, remove, remove, checkout
6. add, remove, add, remove, checkout
7. …...
➜ many more test paths are generated!
Tuesday, January 8, 13

9. 2. Inferring test input
partitioning
Tuesday, January 8, 13

10. Test input partitioning
• Input space for even a single variable/parameter is
huge
• Example: price of an object can be any non negative
number.
• Input space of a set of variables/parameters is even
much more larger
• In software testing, we often partition input space of
each input into equivalent classes
• Example: input of x is
classified into 2 classes:
Tuesday, January 8, 13

11. Inferring test input partitions
• Test input partitions can be specified by domain experts
• Or inferred automatically (when recorded/logged traces
are available)
• concrete data are recored with events
• machine learning techniques can classify such data into
clusters, input partitions can then be inferred from
them
• Example: from all traces, at the event add(#item), the
parameter #item receives the following values:
{1,2,3,4,1,2,4,3,1}, we can infer that 1 ≤ #item ≤ 4.
Therefore, we can partition the domain of item into 3
classes: #item < 1; 1 ≤ #item ≤ 4; #item > 4
Tuesday, January 8, 13

12. 3. Putting them together
to generate more tests
Tuesday, January 8, 13

13. Test generation
• Test machine can generate test paths, each
path is a sequence of events
• The input of each event parameter is
partitioned into classes
• Now, we can combine the two to generate
more test cases
• Use pairwise criteria to reduce the
number of input combination
Tuesday, January 8, 13

14. Example
• Test sequence: add(#item), remove(#item)
• #item has 3 partitions: p1: #item < 1; p2:1
≤ #item ≤ 4; and p3: #item > 4
add remove
p1 p2 p3 p1 p2 p3
• Generated new tests: add(p1), remove(p1)
add(p1), remove(p2)
add(p1), remove(p3)
add(p2), remove(p1)
……………...
Concrete values of #item can be randomly generated based
on the partition information, example #item = 5 in p3
Tuesday, January 8, 13

15. Automated tools
Tuesday, January 8, 13

16. Selen2FSM & M[agi]C
• Available at http://selab.fbk.eu/magic
• Selen2FSM can infer test machines (or
FSMs), and input partitions from existing
Selenium tests
• M[agi]C takes the FSMs and input partitions
and generate executable test cases
• In Selenium Webdriver
• And Robotium for Android applications
Tuesday, January 8, 13

17. More info & support
• Selen2FSM & M[agi]C are developed under the
support of the FITTEST project
• https://www.facebook.com/FITTESTproject
• More info
• Currently we support Selenium for web
testing, and Robotium for Android application
• However, the general idea can be applied for
other type of functional testing as well.
• For support, contact cunduy AT fbk DOT eu
Tuesday, January 8, 13