Agile Testing Practices

Leveraging Emerging
Technologies in Agile
Teams
… and some testing tools
you should try at home
Paul King
Director, ASERT
@paulk_asert
paulk@asert.com.au

Topics
 Introduction
• Using testing DSLs
• All combinations and all pairs
• Auto generated tests
© ASERT 2006-2011

• Model-based testing
• Business/logic rules
• Example driven testing
• Further information
AgileAust 2011 - 2

Emerging Technology Areas
• Development • Deployment & Mgmt
– Continuous integration – Continuous deployment
– Convention over – Build optimisation
configuration – Cloud computing
– Language evolution
– Framework Support Agile
© ASERT 2006-2011

evolution via improved
– Concurrency Management of
– Configuration Complexity
management
• A few favourites
• Testing
– Grails
– ATDD/BDD
– Gpars
– Testing DSLs
– Gradle
– Scripting
– Tableaux AgileAust 2011 - 3

Advances in the Testing Arena

Unit Testing
Mock/interaction testing Techniques
State-based testing Testing DSLs
ATDD/BDD
Data-driven
Logic-driven
© ASERT 2006-2011

Integration Testing
Model-driven
Performance
testing
Acceptance Testing
All-pairs &
Web drivers
combinations
Non-web drivers
Gpars
Test runners

AgileAust 2011 - 4

Groovy and Testing Tool Spectrum*
Utilities Runners
AllPairs, Combinations Native Groovy, JUnit, TestNG, Spock, EasyB,
Polyglot languages JBehave, Cucumber, Robot Framework, SLIM
Logic programming
Threads, Parallel /
Web Database SOAP / Other
Concurrency libraries
Drivers Drivers REST Drivers
Data-driven libraries
Drivers
Networking libraries WebTest DbUnit FEST
© ASERT 2006-2011

XML Processing GroovyWS
WebDriver DataSets Email
Read/write files /
JWebUnit SqlUnit XML-RPC FTP
Excel / Word / CSV
Reporting, Logging Tellurium groovy.sql CXF AntUnit
Selenium JPA Axis2 Telnet
HtmlUnit JDO JAX-WS SSH
Tools Watij BigTable JAX-RS Exec
iTest2, SoapUI, Twist, HttpBuilder JDBC WindowLicker
IDEs, JMeter, Text Cyberneko
editors, Recorders,
Sahi, Build Tools, CI

* Tools/libraries/frameworks don't always neatly fall into one category – still useful conceptually AgileAust 2011 - 5

Application under Test
© ASERT 2006-2011

AgileAust 2011 - 6

Concept
Manual

HTTP Request / Response

Web Server
© ASERT 2006-2011

Automated Driver

Runner
<webtest name="myTest">
<steps>
<invoke
description="get Login Page"
url="login" />
<verifyTitle
description="we should see the login title"
text="Login Page" />
</steps>
</webtest>
Read
Script HTTP Request / Response

AgileAust 2011 - 7

Driver Category
• Real browser invoker • Browser Emulators
– Runs on platform – Can simulate multiple
supported by real browsers
browser – Less platform
– May need multiple restrictions
platforms, e.g. IE6/IE7 – Good for CI
© ASERT 2006-2011

– Uses actual JavaScript – Easier to not download
engine images, resources
– Can be easier to use – Ability to optimise
with test recorders JavaScript interactions
– Automation – More extensible
capabilities differ – Ability to disable
across browsers JavaScript
– Can typically get to all – Scope for parallelism
aspects of browser
AgileAust 2011 - 8

What is Groovy?
• “Groovy is like a super version
of Java. It can leverage Java's
enterprise capabilities but also
has cool productivity features like closures,
DSL support, builders and dynamic typing.”
© ASERT 2006-2011

Groovy = Java – boiler plate code
+ mostly dynamic typing
+ closures
+ domain specific languages
+ builders
+ metaprogramming
+ GDK library
AgileAust 2011 - 9

Topics
• Introduction
 Using testing DSLs
© ASERT 2006-2011

AgileAust 2011 - 10

Prefer scripted over tool centric/recorders
• Scripting • Tools/Recorders
– Choose open – Lock-in to vendor tools
languages – Testing often done
– Developer friendly after development
– Test-first friendly – Tests separate from
– Test using whatever code
© ASERT 2006-2011

levels make sense – Proprietary scripting
– Tests with code languages
– Reduce testing – User-interface
documentation focussed
• Hard-coded details
– Refactor friendly
leads to fragile tests
– Run with every change
– No or hard refactoring
as part of continuous
integration
AgileAust 2011 - 11

Prefer business talk over tech talk
• Avoid • Prefer
import org.openqa.selenium.By
import org.openqa.selenium.htmlunit.HtmlUnitDriver

def driver = new HtmlUnitDriver()
driver.get('http://localhost:8080/postForm')
assert driver.title == 'Welcome to SimpBlog'

// fill in query form and submit it

Source: Evolving Web-Based Test Automation into
Agile Business Specifications, Mugridge et al
driver.findElement(By.name('title')).
sendKeys('Bart was here')
driver.findElement(By.name('content')).
© ASERT 2006-2011

sendKeys('Cowabunga dude!')
def select = driver.
findElement(By.name('category'))
select.
findElements(By.tagName("option")).
find{ it.text == 'Home' }.setSelected()
driver.findElement(By.name('btnPost')).click()

...

given we are on the blog entry page
when entering 'Bart was here' as the title
and entering 'Home' as the category
then posting 5 appears

AgileAust 2011 - 12

Topics
• Introduction
 All combinations and all pairs
© ASERT 2006-2011

AgileAust 2011 - 13

Workshop
• Our blog application sometimes
has errors but only for certain
combinations of Author,
Category and Content.
• How should we test it?
© ASERT 2006-2011

AgileAust 2011 - 14

Workshop
• Our blog application sometimes
has errors but only for certain
combinations of Author,
Category and Content.
• How should we test it?
© ASERT 2006-2011

What about:
* When a new
Category or
Author is added
in the future?
* Tests take too
long to run?

AgileAust 2011 - 15

All Combinations
• Description
test('MacOS', '4G', '250G')
– Don't have a bunch test('Linux', '4G', '250G')
of hard-coded, hard test('Vista', '4G', '250G')
to maintain manual test('MacOS', '8G', '500G')
test data or even test('Linux', '8G', '500G')
manually generated test('Vista', '8G', '500G')
// 30 more rows
CSV file
© ASERT 2006-2011

– Much better to
generate test cases [
from succinct ['MacOS', 'Linux', 'Vista'],
['2G', '4G', '6G', '8G'],
expressions of ['250G', '350G', '500G']
what you are trying ].combinations().each{
to achieve os, mem, disk ->
test(os, mem, disk)
}

AgileAust 2011 - 16

All Combinations Case Study
def combos = [
["Bart", "Homer", "Marge", "Lisa", "Maggie"],
["Work", "School", "Home", "Travel", "Food"],
["foo", "bar", "baz"]
].combinations()

println "Found ${combos.size()} combos"

combos.each { author, category, content ->
© ASERT 2006-2011

postAndCheck author, category, content
}

def postAndCheck(author, category, content) {
// ... details not shown ...
}

Found 75 combos

AgileAust 2011 - 17

All Pairs
• Description
– Sometimes
called
pairwise
testing or
orthogonal
array testing
© ASERT 2006-2011

– Technique
to limit the
explosion of test cases by identifying samples of
important classes of test cases (equivalence classes)
• providing maximum coverage with minimum testing
– Instead of all combinations, systematically use pair-
wise combinations of interactions between objects
• as most faults result from adverse two-way interactions
AgileAust 2011 - 18

SimpBlog Case Study...
def cases = new AllPairs().generate(
author: ["Bart", "Homer", "Marge", "Lisa", "Maggie"],
category: ["Work", "School", "Home", "Travel", "Food"],
content: ["foo", "bar", "baz"])

println "Found ${cases.size()} cases"

cases.each { next ->
© ASERT 2006-2011

println next // just for debugging purposes
postAndCheck next.author, next.category, next.content
}

def postAndCheck(author, category, content) {
// ... details not shown ...
}

AgileAust 2011 - 19

...SimpBlog Case Study
Found 18 cases
[content:bar, category:Food, author:Bart]
[content:bar, category:School, author:Homer]
[content:foo, category:Work, author:Bart]
[content:baz, category:School, author:Homer]
[content:bar, category:Home, author:Maggie]
[content:foo, category:School, author:Marge]
[content:bar, category:Work, author:Bart]
© ASERT 2006-2011

[content:baz, category:Travel, author:Bart]
[content:foo, category:Home, author:Homer]
[content:bar, category:Travel, author:Marge]
[content:baz, category:Work, author:Homer]
[content:bar, category:Travel, author:Lisa]
[content:baz, category:Travel, author:Maggie]
[content:baz, category:Home, author:Marge]
[content:baz, category:Food, author:Homer]
[content:baz, category:Travel, author:Lisa]
[content:foo, category:Food, author:Maggie]
[content:foo, category:Travel, author:Lisa]
AgileAust 2011 - 20

All Combinations/Pairs: Going Further
• Advanced options
– N-Wise
– Compulsory combinations
– Excluding combinations
– Prioritising combinations
– Shuffling
© ASERT 2006-2011

• Sites
– http://en.wikipedia.org/wiki/All-pairs_testing
– http://www.pairwise.org/
• (content starting to age but still useful)

• Tools
– SpecExplorer
– http://code.google.com/p/jwise/
AgileAust 2011 - 21

Topics
• Introduction
 Auto generated tests
© ASERT 2006-2011

AgileAust 2011 - 22

Workshop
• Creating good test data
is taking a long time
• Errors seem to occur
when the system is used
with data that we haven’t
© ASERT 2006-2011

tried before
• What else can we do?

AgileAust 2011 - 23

Workshop
• Creating good test data
is taking a long time
• Errors seem to occur
when the system is used
with data that we haven’t
© ASERT 2006-2011

What about:
tried before * When fields
change in
• What else can we do? the future?
* Measuring
coverage?
* Repeatable
tests?

AgileAust 2011 - 24

QuickCheck
• Test Java/Groovy programs more declaratively
using automatic random test data generation
• Replace manually selected scenario-based tests
with specification-based testing
• Generators available:
– primitive types,
© ASERT 2006-2011

collections, POJOs
– value ranges
– lists, sets and array
– distinct values
– distributions
– determinism (random,
deterministic )
– generator strategies (composition of generator values (oneOf,
frequency, list, array, nullsAnd), transformation, mutation)
– value frequencies (frequency, oneOf) AgileAust 2011 - 25

QuickCheck Samples…
682
Ant
Bee for (words in someNonEmptyLists(strings())) {
141 assert words*.size().sum() == words.sum().size()
Jul }
Dog
Dog
def pets = fixedValues(['Ant', 'Bee', 'Cat', 'Dog'])
Dog def nums = excludeValues(integers(100, 999, INVERTED_NORMAL), 500..599)
© ASERT 2006-2011

801 def months = new Generator<String>() {
177 Generator<Date> genDate = dates()
String next() { genDate.next().format("MMM") }
Ant
}
Dog def gen = new DefaultFrequencyGenerator(pets, 50)
951 gen.add(nums, 30)
Cat gen.add(months, 20)
Aug
20.times {
Bee def next = gen.next().toString()
Bee println next
Oct assert next.size() == 3
}
Dog
241
AgileAust 2011 - 26

…QuickCheck Samples…
682
Ant
Jul }
Dog
Dog
© ASERT 2006-2011

Ant
}
Aug
20.times {
Bee println next
}
Dog
241
AgileAust 2011 - 27

…QuickCheck Samples…
682
Ant
Jul }
Dog
Dog
© ASERT 2006-2011

Ant
}
Aug
20.times {
Bee println next
}
Dog
241
AgileAust 2011 - 28

…QuickCheck Samples
682
Ant
Jul }
Dog
Dog
© ASERT 2006-2011

Ant
}
Aug
20.times {
Bee println next
}
Dog
241
AgileAust 2011 - 29

QuickCheck: SimpBlog Case Study…
• Approach
– Auto-selected values for author, category
– Auto-generated values for title, content

def authors = ["Bart", "Homer", "Lisa", "Marge", "Maggie"]
def categories = ["Home", "Work", "Food", "Travel"]
© ASERT 2006-2011

10.times {
postAndCheck anyString(),
anyFixedValue(categories),
anyFixedValue(authors),
anyString()
}

def postAndCheck(title, category, author, content) {
...
}

AgileAust 2011 - 30

…QuickCheck: SimpBlog Case Study
© ASERT 2006-2011

Screenshot
of fixed
titles version

AgileAust 2011 - 31

Test Generation: Going Further
• AutoMocks
• Auto domain classes
• Assertions
• http://en.wikipedia.org/wiki/QuickCheck
– Erlang, Scheme, Common Lisp, Perl, Python, Clojure,
© ASERT 2006-2011

Scala, Ruby, Java, F#, Standard ML, JavaScript, C++

AgileAust 2011 - 32

Topics
• Introduction
© ASERT 2006-2011

 Model-based testing
AgileAust 2011 - 33

Workshop
• We recently added AJAX to the
SimpBlog application and some
of our users are reporting issues
• Depending on the order in which
the blog form is filled out there
can sometimes be strange error
© ASERT 2006-2011

messages displayed
• What testing strategies can we
use in this scenario?

AgileAust 2011 - 34

Workshop
• We recently added AJAX to the
SimpBlog application and some
of our users are reporting issues
• Depending on the order in which
the blog form is filled out there
can sometimes be strange error
© ASERT 2006-2011

messages displayed What about:
* When a new
• What testing strategies can we
Category or
use in this scenario? Author is added
in the future?
* What does
coverage mean?

AgileAust 2011 - 35

Model-based testing
• Deriving test suites from source code is
usually deemed impractical
• Instead develop models which describe
certain characteristics of the desired
system behaviour
© ASERT 2006-2011

• May involve:
– theorem proving
– constraint logic programming
– model checking
– event-flow models
– markov chain models

AgileAust 2011 - 36

ModelJUnit...
• Description
– Supports model-based testing
– Allows you to write simple finite
state machine (FSM) models or
extended finite state machine
(EFSM) models in Java or Groovy
© ASERT 2006-2011

– You can then generate tests from
those models and measure various
model coverage metrics

AgileAust 2011 - 37

...ModelJUnit...
// require modeljunit.jar
import nz.ac.waikato.modeljunit.coverage.*
import nz.ac.waikato.modeljunit.*

class VendingMachineModel implements FsmModel {
def state = 0 // 0,25,50,75,100
void reset(boolean testing) {state = 0}

boolean vendGuard() {state == 100}
@Action void vend() {state = 0}

boolean coin25Guard() {state <= 75}
© ASERT 2006-2011

@Action void coin25() {state += 25}

boolean coin50Guard() {state <= 50}
@Action void coin50() {state += 50}
}

def tester = new RandomTester(new VendingMachineModel())
tester.buildGraph()
def metrics = [new ActionCoverage(), new StateCoverage(),
new TransitionCoverage(), new TransitionPairCoverage()]
metrics.each { tester.addCoverageMetric it }

tester.addListener "verbose"
tester.generate 20

println 'nMetrics Summary:'
tester.printCoverage()
AgileAust 2011 - 38

...ModelJUnit
...
done (0, coin50, 50)
done (50, coin25, 75)
done (75, coin25, 100)
done Random reset(true)
done (50, coin25, 75)
done (75, coin25, 100)
done (100, vend, 0)
...
© ASERT 2006-2011

done (50, coin50, 100) Metrics Summary:
done (100, vend, 0) action coverage: 3/3
done (0, coin25, 25) state coverage: 5/5
done (25, coin25, 50) transition coverage: 7/8
done Random reset(true) transition-pair coverage: 8/12
...
done (50, coin25, 75)
done (75, coin25, 100)
done (100, vend, 0)
done (50, coin25, 75)
...
AgileAust 2011 - 39

Interlude: Seven Bridges of Königsberg
• Königsberg, Prussia (now Kaliningrad, Russia)
spans the Pregel River including two large
islands connected by 7 bridges
• Find a walk through the city crossing each
bridge once and only once. The islands can not
be reached by any route other than the bridges,
© ASERT 2006-2011

and every bridge must be crossed completely
every time.

Source: http://en.wikipedia.org/wiki/Seven_Bridges_of_K%C3%B6nigsberg AgileAust 2011 - 40

Interlude: Seven Bridges of Königsberg
• Königsberg, Prussia (now Kaliningrad, Russia)
spans the Pregel River including two large
islands connected by 7 bridges
• Find a walk through the city crossing each
bridge once and only once. The islands can not
be reached by any route other than the bridges,
© ASERT 2006-2011

and every bridge must be crossed completely
every time. Euler showed there is no solution.


Interlude: Königsberg Bridges Variation
8. The Blue Prince contrives a stealthy plan to build an
eighth bridge so that he can begin in the evening at
his Schloß, walk the bridges, and end at the Gasthaus
to brag of his victory. Of course, he wants the Red
Prince to be unable to duplicate the feat from the
Red Castle. Where does the Blue Prince build the
eighth bridge?
9. The Red Prince, infuriated by his brother's Gordian
solution to the problem, wants to build a ninth bridge,
© ASERT 2006-2011

enabling him to begin at his Schloß, walk the bridges,
and end at the Gasthaus to rub dirt in his brother's
face. As an extra bit of revenge, his brother should
then no longer be able to walk the bridges starting and
ending at his Schloß as before. Where does the Red
Prince build the ninth bridge?
10. The Bishop has watched this furious bridge building
with dismay. It upsets the town's Weltanschauung
and, worse, contributes to excessive drunkenness.
He wants to build a tenth bridge that allows all the
inhabitants to walk the bridges and return to their own
beds. Where does the Bishop build the tenth bridge?

ModelJUnit: SimpBlog Case Study...
• Does the order in which form
information is entered affect
the application?
...
void reset(boolean testing) {
authorSelected = false
// require modeljunit.jar, htmlunit.jar categorySelected = false
© ASERT 2006-2011

import nz.ac.waikato.modeljunit.coverage.* titleEntered = false
import nz.ac.waikato.modeljunit.* contentEntered = false
import com.gargoylesoftware.htmlunit.WebClient client = new WebClient()
url = 'http://localhost:8080/postForm'
class SimpBlogModel implements FsmModel { page = client.getPage(url)
boolean authorSelected = false assert 'Welcome to SimpBlog' == page.titleText
boolean categorySelected = false form = page.getFormByName('post')
boolean titleEntered = false }
...
boolean contentEntered = false
int count = 0
def client, page, form

// Special known method, allows equivalence class definition
// example states: __ __ __ __, AU __ __ __, AU CA TI CO
def getState() {
"${authorSelected ? ' AU ' : ' __ '}${categorySelected ? ' CA ' : ' __ '}" +
"${titleEntered ? ' TI ' : ' __ '}${contentEntered ? ' CO ' : ' __ '}"
}
...
AgileAust 2011 - 43

...ModelJUnit: SimpBlog Case Study...
...
boolean "enter title Guard"() { !titleEntered }
@Action void "enter title "() { titleEntered = true
form.getInputByName('title').setValueAttribute("Title ${count++}")
}

boolean enterContentGuard() { !contentEntered }
@Action void enterContent() { contentEntered = true
form.getTextAreaByName('content').setText("Content ${count++}")
}

boolean chooseAuthorGuard() { !authorSelected }
@Action void chooseAuthor() { authorSelected = true // simple version just Lisa
© ASERT 2006-2011

form.getSelectByName('author').getOptions().find{ it.text == 'Lisa' }.setSelected(true)
}

boolean pickCategoryGuard() { !categorySelected }
@Action void pickCategory() { categorySelected = true // simple version just Home
form.getSelectByName('category').getOptions().find{ it.text == 'Home' }.setSelected(true)
}

boolean "submit post Guard"() { categorySelected && authorSelected &&
titleEntered && contentEntered }
@Action void "submit post "() {
def result = form.getInputByName('btnPost').click()
assert result.getElementsByTagName('h1').item(0).textContent.matches('Post.*: Title .*')
// could do more asserts here
reset(true)
}
} // end of SimpBlogModel class definition
...

AgileAust 2011 - 44

def tester = new RandomTester(new SimpBlogModel())
tester.buildGraph()
def metrics = [
new ActionCoverage(),
new StateCoverage(),
new TransitionCoverage(),
new TransitionPairCoverage()
]
metrics.each {
© ASERT 2006-2011

tester.addCoverageMetric it
}

tester.addListener "verbose"
tester.generate 50

println 'nMetrics Summary:'
tester.printCoverage()

def graphListener = tester.model.getListener("graph")
graphListener.printGraphDot "simpblog.dot"
println "nGraph contains " + graphListener.graph.numVertices() +
" states and " + graphListener.graph.numEdges() + " transitions."

AgileAust 2011 - 45

done ( __ __ __ __ , pickCategory, __ CA __ __ ) ...
done ( __ CA __ __ , enterContent, __ CA __ CO ) done ( __ __ __ __ , pickCategory, __ CA __ __ )
done ( __ CA __ CO , enter title , __ CA TI CO ) done ( __ CA __ __ , enterContent, __ CA __ CO )
done ( __ CA TI CO , chooseAuthor, AU CA TI CO ) done ( __ CA __ CO , chooseAuthor, AU CA __ CO )
done ( AU CA TI CO , submit post , __ __ __ __ ) done ( AU CA __ CO , enter title , AU CA TI CO )
done ( __ __ __ __ , pickCategory, __ CA __ __ ) done ( AU CA TI CO , submit post , __ __ __ __ )
done ( __ CA __ __ , chooseAuthor, AU CA __ __ ) done ( __ __ __ __ , chooseAuthor, AU __ __ __ )
done ( AU CA __ __ , enter title , AU CA TI __ ) done ( AU __ __ __ , pickCategory, AU CA __ __ )
done ( AU CA TI __ , enterContent, AU CA TI CO ) done ( AU CA __ __ , enterContent, AU CA __ CO )
done ( AU CA TI CO , submit post , __ __ __ __ ) done ( AU CA __ CO , enter title , AU CA TI CO )
done ( __ __ __ __ , chooseAuthor, AU __ __ __ ) done ( AU CA TI CO , submit post , __ __ __ __ )
done ( AU __ __ __ , pickCategory, AU CA __ __ ) done ( __ __ __ __ , chooseAuthor, AU __ __ __ )
done ( AU CA __ __ , enter title , AU CA TI __ ) done ( AU __ __ __ , enter title , AU __ TI __ )
done ( AU CA TI __ , enterContent, AU CA TI CO ) done ( AU __ TI __ , pickCategory, AU CA TI __ )
done ( AU CA TI CO , submit post , __ __ __ __ ) done ( AU CA TI __ , enterContent, AU CA TI CO )
© ASERT 2006-2011

done ( __ __ __ __ , enterContent, __ __ __ CO ) done ( AU CA TI CO , submit post , __ __ __ __ )
done ( __ __ __ CO , pickCategory, __ CA __ CO ) done Random reset(true)
done ( __ CA __ CO , chooseAuthor, AU CA __ CO ) done ( __ __ __ __ , pickCategory, __ CA __ __ )
done ( AU CA __ CO , enter title , AU CA TI CO ) done ( __ CA __ __ , enterContent, __ CA __ CO )
done ( AU CA TI CO , submit post , __ __ __ __ ) done ( __ CA __ CO , enter title , __ CA TI CO )
done ( __ __ __ __ , pickCategory, __ CA __ __ ) done ( __ CA TI CO , chooseAuthor, AU CA TI CO )
done ( __ CA __ __ , enter title , __ CA TI __ )
done ( __ CA TI __ , chooseAuthor, AU CA TI __ ) Metrics Summary:
done ( AU CA TI __ , enterContent, AU CA TI CO ) action coverage: 5/5
done ( AU CA TI CO , submit post , __ __ __ __ ) state coverage: 12/16
done ( __ __ __ __ , chooseAuthor, AU __ __ __ ) transition coverage: 19/33
done ( AU __ __ __ , pickCategory, AU CA __ __ ) transition-pair coverage: 26/56
done ( AU CA __ __ , enter title , AU CA TI __ )
done ( AU CA TI __ , enterContent, AU CA TI CO ) Graph contains 16 states and 33 transitions.
done ( AU CA TI CO , submit post , __ __ __ __ )
...

AgileAust 2011 - 46

© ASERT 2006-2011

Advanced
Simplified version
version
(just Lisa)

AgileAust 2011 - 47

...ModelJUnit: SimpBlog Case Study
© ASERT 2006-2011

AgileAust 2011 - 48

Model-based testing: Going further
– http://www.cs.waikato.ac.nz/~marku/mbt/modeljunit/
– http://en.wikipedia.org/wiki/Model_based_testing
– http://home.mit.bme.hu/~micskeiz/pages/modelbased_testing.html
– A Taxonomy of Model-Based Testing
http://www.cs.waikato.ac.nz/pubs/wp/2006/uow-cs-wp-2006-04.pdf

• Related algorithms
© ASERT 2006-2011

– Job scheduling problem
– Travelling salesman problem
– Chinese postman problem
– Vehicle routing problem
– Spanning trees and Hamiltonian paths
– http://www.nada.kth.se/~viggo/problemlist/compendium.html
– http://en.wikipedia.org/wiki/List_of_terms_relating_to_algorithm
s_and_data_structures
AgileAust 2011 - 49

Topics
• Introduction
© ASERT 2006-2011

 Business/logic rules
AgileAust 2011 - 50

Workshop
• Set up some test cases
representing the Simpsons
weekly blogging habits:
– They never blog on the same day
– Marge blogs only on a Saturday or
Sunday
© ASERT 2006-2011

– Maggie blogs only on a Tuesday or
Thursday
– Lisa blogs only on a Monday,
Wednesday or Friday
– Bart blogs only on the day after Lisa
– Homer only blogs if noone else
blogged the previous day and
doesn't allow anyone to blog the
next day AgileAust 2011 - 51

Workshop
• Set up some test cases
representing the Simpsons
weekly blogging habits:
– They never blog on the same day
– Marge blogs only on a Saturday or
Sunday
© ASERT 2006-2011

– Maggie blogs only on a Tuesday or What about:
Thursday * When the
– Lisa blogs only on a Monday, habits change?
Wednesday or Friday
– Bart blogs only on the day after Lisa
– Homer only blogs if noone else
blogged the previous day and
doesn't allow anyone to blog the
next day AgileAust 2011 - 52

Constraint/Logic Programming...
• Description
– Style of programming where relations between
variables are stated in the form of constraints
– First made popular by logic programming languages
such as Prolog but the style is now also used outside
logic programming specific languages
© ASERT 2006-2011

– Constraints differ from the common primitives of
other programming languages in that they do not
specify one or more steps to execute but rather the
properties of a solution to be found
– Popular libraries used with Groovy supporting
constraint programming include Gecode/J, Choco
and tuProlog
– We'll look at Choco as an example

AgileAust 2011 - 53

...Constraint/Logic Programming...
© ASERT 2006-2011

Source: http://xkcd.com/287/
AgileAust 2011 - 54

...Constraint/Logic Programming...
// requires choco 2.1.0-basic.jar from http://choco.emn.fr/
import static choco.Choco.*
import choco.kernel.model.variables.integer.IntegerVariable

def m = new choco.cp.model.CPModel()
def s = new choco.cp.solver.CPSolver()

def menu = [ Found a solution:
© ASERT 2006-2011

'Mixed fruit' : 215, 7 * Mixed fruit
'French fries' : 275, Found a solution:
'Side salad' : 335, 1 * Mixed fruit
'Hot wings' : 355, 2 * Hot wings
'Mozzarella sticks' : 420, 1 * Sampler plate
'Sampler plate' : 580
]
def numOrdered = new IntegerVariable[menu.size()]
def priceEach = new int[menu.size()]
def sum = 1505
...
AgileAust 2011 - 55

...Constraint/Logic Programming
...
menu.eachWithIndex { name, price, i ->
// number ordered >= 0
// number ordered * price <= sum
numOrdered[i] = makeIntVar(name, 0, sum.intdiv(price))
priceEach[i] = price
}
m.addConstraint(eq(scalar(numOrdered, priceEach), sum))
© ASERT 2006-2011

s.read(m)

def more = s.solve()
while (more) {
println "Found a solution:"
numOrdered.each {
def v = s.getVar(it)
if (v.val) println " $v.val * $v.name"
}
more = s.nextSolution()
}
AgileAust 2011 - 56

SimpBlog Case Study...
def m = new choco.cp.model.CPModel()
def s = new choco.cp.solver.CPSolver()

daysOfWeek = ["Sunday", "Monday", "Tuesday",
"Wednesday", "Thursday", "Friday", "Saturday"]

def bart = makeIntVar('Bart', 0, 6)
© ASERT 2006-2011

def homer = makeIntVar('Homer', 0, 6)
def marge = makeIntVar('Marge', 0, 6)
def lisa = makeIntVar('Lisa', 0, 6)
def maggie = makeIntVar('Maggie', 0, 6)
def simpsons = [bart, homer, marge, lisa, maggie]
...

AgileAust 2011 - 57

...SimpBlog Case Study...
...
// They never blog on the same day
for (i in 0..<simpsons.size())
for (j in 0..<i) m.addConstraint(neq(simpsons[i], simpsons[j]))

// Marge blogs only on a Saturday or Sunday
m.addConstraint(or(eq(marge, 0), eq(marge, 6)))

// Maggie blogs only on a Tuesday or Thursday
m.addConstraint(or(eq(maggie, 2), eq(maggie, 4)))
© ASERT 2006-2011

// Lisa blogs only on a Monday, Wednesday or Friday
m.addConstraint(or(eq(lisa, 1), eq(lisa, 3), eq(lisa, 5)))

// Bart blogs only on the day after Lisa
m.addConstraint(eq(plus(lisa, 1), bart))
// Homer only blogs if noone else blogged the previous
// day and doesn't allow anyone to blog the next day
m.addConstraint(and(distanceNEQ(homer, marge, 1),
distanceNEQ(homer, bart, 1),
distanceNEQ(homer, maggie, 1),
distanceNEQ(homer, lisa, 1)))
...
AgileAust 2011 - 58

...SimpBlog Case Study
...
s.read(m)
def more = s.solve()
if (!more) println "No Solutions Found"
else println pad("Solutions:") +
simpsons.collect{ pad(it.name) }.join()
while (more) {
print pad("")
println simpsons.collect {
© ASERT 2006-2011

def v = s.getVar(it)
pad(daysOfWeek[v.val])
}.join()
more = s.nextSolution()
}

def pad(s) { s.padRight(12) }

Solutions: Bart Homer Marge Lisa Maggie
Thursday Saturday Sunday Wednesday Tuesday
Tuesday Saturday Sunday Monday Thursday
Saturday Tuesday Sunday Friday Thursday
Thursday Sunday Saturday Wednesday Tuesday
AgileAust 2011 - 59

Logic solvers: Going further
• Available tools
– JSR-331: Java Constraint Programming API
http://jcp.org/en/jsr/detail?id=331
– JaCoP: Java Constraint Programming solver
http://jacop.osolpro.com/
– ChocoSolver
© ASERT 2006-2011

http://www.emn.fr/z-info/choco-solver/

AgileAust 2011 - 60

Canoo WebTest
• Description
– Open source tool for automated testing of web applications
– Declarative approach in XML or testing DSL in Groovy
– Has Test Recorder
– Excellent reporting options
– Ant-based under the covers
<target name="login" >
<testSpec name="normal" >
© ASERT 2006-2011

&config;
<steps>
<invoke stepid="get Login Page"
url="login.jsp" />
<verifytitle stepid="we should see the login title"
text="Login Page" />
<setinputfield stepid="set user name"
name="username"
value="scott" />
<setinputfield stepid="set password"
name="password"
value="tiger" />
<clickbutton stepid="Click the submit button"
label="let me in" />
<verifytitle stepid="Home Page follows if login ok"
text="Home Page" />
</steps>
</testSpec>
</target>
AgileAust 2011 - 63

EasyB Example ...
• When run will be marked as pending
– perfect for ATDD

scenario "Bart posts a new blog entry", {
given "we are on the create blog entry page"
when "I have entered 'Bart was here' as the title"
and "I have entered 'Cowabunga Dude!' into the content"
and "I have selected 'Home' as the category"
© ASERT 2006-2011

and "I have selected 'Bart' as the author"
and "I click the 'Create Post' button"
then "I expect the entry to be posted"
}

AgileAust 2011 - 65

...EasyB Example...
description "Post Blog Entry Feature"

narrative "for feature", {
as_a "Blogger"
i_want "to be able to post a blog"
so_that "I can keep others informed"
}

before "posting blog", {
given "we are on the create blog entry page", {
© ASERT 2006-2011

webClient = new com.gargoylesoftware.htmlunit.WebClient()
page = webClient.getPage('http://localhost:8080/postForm')
}
}

scenario "Bart was here blog", {

when "I have entered 'Bart was here' as the title", {
form = page.getFormByName('post')
form.getInputByName('title').setValueAttribute(
'Bart was here (and so was EasyB)')
}

...
AgileAust 2011 - 66

...EasyB Example...
...
and "I have entered 'Cowabunga Dude!' into the content", {
form.getTextAreaByName('content').setText('Cowabunga Dude!')
}

and "I have selected 'Home' as the category", {
form.getSelectByName('category').getOptions().find { it.text == 'Home' }.setSelected(
}

and "I click the 'Create Post' button", {
result = form.getInputByName('btnPost').click()
}
© ASERT 2006-2011

then "I expect the entry to be posted", {
// check blog post details
assert result.getElementsByTagName('h1').item(0).textContent.matches('Post.*: Bart wa
def h3headings = result.getElementsByTagName('h3')
assert h3headings.item(1).textContent == 'Category: Home' // traditional style
h3headings.item(2).textContent.shouldBe 'Author: Bart' // BDD style

// expecting: <table><tr><td><p>Cowabunga Dude!</p></td></tr></table>
def cell = result.getByXPath('//TABLE//TR/TD')[0]
def para = cell.firstChild
assert para.textContent == 'Cowabunga Dude!'
// para.shouldHave textContent: 'Cowabunga Dude!'
}
}

AgileAust 2011 - 67

JBehave Example...
Given we are on the create blog entry page
When I have entered "Bart was here" as the title
And I have entered "Cowabunga Dude!" as the content
And I have selected "Home" from the "category" dropdown
And I have selected "Bart" from the "author" dropdown
And I click the 'Create Post' button
Then I should see a heading message matching "Post.*: Bart was here.*"

import org.jbehave.scenario.Scenario
© ASERT 2006-2011

import org.jbehave.scenario.steps.Steps

class NewPostScenario extends Scenario {
NewPostScenario() {
super([new CreateBlogSteps()] as Steps[])
}
} Scenario:

Given we are on the create blog entry page
When I have entered "Bart was here" as the title
And I have entered "Cowabunga Dude!" as the content
And I have selected "Home" from the "category" dropdown
And I have selected "Bart" from the "author" dropdown
And I click the 'Create Post' button
Then I should see a heading message matching "Post.*: Bart was here.*"

AgileAust 2011 - 69

...JBehave Example...
import org.jbehave.scenario.steps.Steps
import org.jbehave.scenario.annotations.*
import com.gargoylesoftware.htmlunit.WebClient

class CreateBlogSteps extends Steps {
def page, form, result

@Given("we are on the create blog entry page")
void gotoEntryPage() {
page = new WebClient().getPage('http://localhost:8080/postForm')
}
© ASERT 2006-2011

@When('I have entered "$title" as the title')
void enterTitle(String title) {
form = page.getFormByName('post')
form.getInputByName('title').
setValueAttribute(title + ' (and so was JBehave)')
}

@When('I have entered "$content" as the content')
void enterContent(String content) {
form.getTextAreaByName('content').setText(content)
}
...
AgileAust 2011 - 70

Interlude: Einstein’s Riddle…
• Wikipedia: The zebra puzzle is a well-
known logic puzzle
– Often called Einstein's Puzzle or
Einstein's Riddle
– Said to have been invented by Albert
© ASERT 2006-2011

Einstein as a boy
– Claim is that Einstein said “Only 2
percent of the world's population can
solve it.”

AgileAust 2011 - 72

Interlude: …Einstein’s Riddle
• Some premises:
– The British person lives in the red house
– The Swede keeps dogs as pets
– The Dane drinks tea
– The green house is on the left of the white house
– The green homeowner drinks coffee
– The man who smokes Pall Mall keeps birds
–
© ASERT 2006-2011

The owner of the yellow house smokes Dunhill
– The man living in the center house drinks milk
– The Norwegian lives in the first house
– The man who smokes Blend lives next to the one who keeps cats
– The man who keeps the horse lives next to the man who smokes Dunhill
– The man who smokes Bluemaster drinks beer
– The German smokes Prince
– The Norwegian lives next to the blue house
– The man who smokes Blend has a neighbor who drinks water

• And a question: Who owns the fish?
AgileAust 2011 - 73

Einstein’s Riddle : Prolog
% from http://www.baptiste-wicht.com/2011/09/solve-einsteins-riddle-using-prolog

% Preliminary definitions
persons(0, []) :- !.
persons(N, [(_Men,_Color,_Drink,_Smoke,_Animal)|T]) :- N1 is N-1, persons(N1,T).
person(1, [H|_], H) :- !.
person(N, [_|T], R) :- N1 is N-1, person(N1, T, R).

% The Brit lives in a red house
hint1([(brit,red,_, _, _)|_]).
hint1([_|T]) :- hint1(T).
© ASERT 2006-2011

% The Swede keeps dogs as pets
hint2([(swede,_,_,_,dog)|_]).

% The Dane drinks tea
hint3([(dane,_,tea,_,_)|_]).

% The Green house is on the left of the White house
hint4([(_,green,_,_,_),(_,white,_,_,_)|_]).

% The owner of the Green house drinks coffee.
hint5([(_,green,coffee,_,_)|_]).
...
AgileAust 2011 - 74

Einstein’s Riddle : Groovy testing DSL

the Dane drinks tea
the German smokes prince
the Swede keeps dogs
© ASERT 2006-2011

the Brit has a red house
the Norwegian owns the first house
the man from the centre house drinks milk
the owner of the green house drinks coffee
the owner of the yellow house smokes dunhill
the person known to smoke pallmall rears birds
the man known to smoke bluemaster drinks beer
the green house is on the left side of the white house
the man known to smoke blends lives next to the one who keeps cats
the man known to keep horses lives next to the man who smokes dunhill
the man known to smoke blends lives next to the one who drinks water
the Norwegian lives next to the blue house

AgileAust 2011 - 75

© ASERT 2006-2011

SpiderFixture from FitLibrary (FitNesse)
ZiBreve…

ZiBreve

Source: Evolving Web-Based Test Automation into
Agile Business Specifications, Mugridge et al
AgileAust 2011 - 77

…ZiBreve
© ASERT 2006-2011

Source: Evolving Web-Based Test Automation into Agile Business Specifications, Mugridge et al AgileAust 2011 - 78

Example driven testing
• Bridging the communication Gap
– Specification by example and agile acceptance testing by
Gojko Adzic
• Specification by Example
– How successful teams deliver the right software by Gojko
Adzic: http://manning.com/adzic/
© ASERT 2006-2011

• Exploration Through Example
– http://www.exampler.com/blog/ (Brian Marick)
• Evolving Web-Based Test Automation into
Agile Business Specifications
Rick Mugridge et al
– http://www.mdpi.com/1999-5903/3/2/159/pdf
AgileAust 2011 - 79

Conclusion
• Don’t be afraid to use technology
– For development, testing, deployment
• Embrace continuous learning
• There are many ways to capture business
requirements
© ASERT 2006-2011

– “Deciding what to build”
– Find the best way to represent your
requirements
– Prefer machine understandable,
executable solutions

AgileAust 2011 - 81

About the Author
Dr Paul King leads ASERT, an organisation based in
Brisbane, Australia which provides software development,
training and mentoring services to customers wanting to
embrace new technologies, harness best practices and
innovate. He has been contributing to open source projects
for nearly 20 years and is an active committer on numerous
projects including Groovy. Paul speaks at international
© ASERT 2006-2011

conferences, publishes in software magazines and journals,
and is a co-author of Manning's best-seller: Groovy in Action.

Paul King
Director, ASERT
@paulk_asert
paulk@asert.com.au
83

Agile Testing Practices

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Agile Testing Practices

Ähnlich wie Agile Testing Practices (20)

Mehr von Paul King

Mehr von Paul King (20)

Agile Testing Practices