My presentation to support my paper (same title) from PNSQC 2012.

1. ATDD GUI
ET
GET THE BALANCE RIGHT
ACCEPTANCE TEST DRIVEN
DEVELOPMENT, GUI AUTOMATION AND
EXPLORATORY TESTING

2. MICHAEL LARSEN
Senior Tester, Sidereel.com
Chair, Education Special Interest
Group, Association For Software Testing
Black Belt, Miagi-do School of Software Testing
Founder/Facilitator, Weekend Testing Americas
TESTHEAD: http://mkltesthead.com
Twitter: @mkltesthead

3. WHAT’S HAPPENED
SINCE 2001?
Eleven years since the Agile Manifesto
Test Driven Development (TDD) has emerged as a
standard to guide code development driven by tests.
Acceptance Test Driven Development (ATDD)
extends model.
Proliferation of automated testing tools for a variety
of platforms and needs.

4. DO WE STILL NEED
TESTERS?
NO: Testing is dead. With the development
of these tools, developers now do the
testing earlier and up front [Savioa, 2011].
YES: Apply reason and dynamic thought to
problems. Thinking, actively engaged
testers are needed now more than ever
[Tomlinson, 2011].

5. TDD, WHAT IS IT?

6. ATDD, WHAT IS IT?
Places the focus on User Stories from the
expectations of the customers.
Is geared towards the members of the team
that are not necessarily programmers.
Uses tools that allow a more natural
language to present the requirements.

7. ATDD
Scenario: Twitter user signs in
Given I connect to twitter from the sign in page
When I log into twitter with good credentials
Then I go to the user private profile page
And I can confirm my twitter credentials
Expressed in Gherkin, the language used
with Cucumber.
Tied to actual coded methods to make the
process work.

8. TDD AND ATDD ARE
NOT TESTING!
TDD and ATDD are design processes.
Help develop clean, well functioning code.
Help ensure software being developed
meets needs of the project

9. WHY ISN’T THIS
TESTING?
Testing:
“The art and the act of asking questions of
a product, then developing & devising new
and more inventive questions based on the
answers we receive.”

10. WHERE DO TDD AND
ATDD EXCEL?
Disciplined approach to developing cleaner
software.
Act as a brake on "cowboy code".
New programmers can see code in context
with the tests that have been written.

11. WHERE DO TDD AND
ATDD EXCEL?
TDD and ATDD work well with Continuous Integration.
Keeps issues based on dependencies to a minimum.
Programmers focus on "just enough design" and "just
enough development".
Entire product team does design and development
work.

12. WHERE DO TDD AND
ATDD FALL SHORT?
Unrealistic expectations.
Only be as good as the overall skill of the
developer(s).
Can be over applied.

13. WHAT IS FRONT END
GUI AUTOMATION?
Proliferation of tools that help simulate the actions
of users.
Follow standard workflows
• (logging in, navigating to pages, clicking on
links, filling in forms, etc.).
Tools range from lightweight apps like Texter up to
full feature development suites
(Selenium/WebDriver, FitNesse, TestComplete, etc
.).

14. HOW DOES GUI
AUTOMATION DIFFER
FROM TDD/ATDD?
Helps with constructing
acceptance test cases
Helps to check and
demonstrate the acceptance
criteria has been met.

15. GUI AUTOMATION
Remember our original example:

16. GUI AUTOMATION
Cucumber statements = abstraction of real
code.
Map to appropriate tools that make statements
work on the application or browser.

17. GUI AUTOMATION
The code below is Ruby, using Capybara to
make calls to the browser:

18. GUI AUTOMATION
Step Definitions -> Capybara and Ruby

19. WHERE DOES GUI
AUTOMATION EXCEL?
Tremendous blessing when dealing
with repetitious steps (set up and take
down).
Important part of Acceptance Test
verification.
Semi proxy for human interaction.

20. GUI AUTOMATION
DEFICIENCIES
GUI tends to be the most fragile
layer of an application.
Tremendous overhead of script
maintenance.
Not good at checking context.

21. WHAT IS EXPLORATORY
TESTING?

22. WHAT IS EXPLORATORY
TESTING?
“Scientific thinking in real-time” [Bach, 2012]
Puts test design and test execution
together.
The tests that we did before inform the tests
that we will do now. Those tests will inform
the tests we will perform later.

23. EXPLORATORY
TESTING
Concepts are defined, but they are not completely
predefined or run in a rigid sequence.
Allows for a development of test ideas that are
interesting.
Example: Automated tests can be made
exploratory by randomizing order.
Purpose: Discover where dependencies or state
conditions do exist.

24. EXPLORATORY TESTING
VS. AUTOMATED
TESTING
Instead of saying "automated testing", I
prefer to use the term "computer aided
testing".
Exploratory Testing can, and often
does, use elements of computer aided
testing to help accomplish its goals.

25. AUTOMATION CAN’S
AND CANNOT’S
Automation Can: Automation Cannot:
Generate test data to be used in Create curiosity.
forms or as variable values.
Parse the output of a program and Make sapient (actively thinking)
use it as input for another program. decisions.
Create a log of actions and Invent a new idea or an approach
transactions. based on the output of a sequence
of tests.
Alert if an assertion is met or not Notice something unexpected
met. (unless we have already defined
what is unexpected).
Search for patterns in output, or Make a judgment call as to the
help to reveal patterns we did not value or importance of a piece of
know about. functionality.

26. WHERE DOES
EXPLORATORY TESTING
EXCEL?
Allows the tester to choose the sequence of
steps that can provide answers about the
state or condition of a product.
Imagination and creativity open up
possibilities.
When there is little in the way of formalized
documentation.

27. EXPLORING
REQUIREMENTS
Remember this acceptance test?
Scenario: Twitter user signs in through Twitter
Given I connect to twitter from the sign in page
When I log into twitter with good credentials
Then I go to the user private profile page
And I can confirm my twitter credentials

28. SAMPLE EXPLORATION
QUESTIONS
Is there a login interface?
Does it provide for proper error handling if I can't log
in?
Does it give me feedback to let me know that I have
successfully logged in?
Can I trick the system into letting me log in with
improper credentials?
Does it present me with information that could help me
guess a login without actually having one?
Where else could I use this functionality?

29. EXPLORATORY TESTING
IS PART OF TDD/ATDD
Proposing a failing test first, and then
creating code that meets the acceptance
criteria
Programmer must consider what the
application needs to do
Determine which avenue(s) to implement
the feature(s).

30. BENEFITS OF
EXPLORATORY TESTING
Walking down various paths
Jumping off point to look in other places
Simple Cucumber statement "And let me see that":
And /^let me see that$/ do
puts "PAUSED - Press Enter to continue: #{$1}aaa"
puts ""
$stdin.gets
End

31. EXPLORATORY
TESTING’S DEFICIENCIES
Less likely to find new revelations by going
over the same ground (pesticide paradox)
Not enough time to go through all
possibilities.
Sentient humans do it. Sentient humans get
bored.
• SBTM techniques can help

32. CAN WE CREATE A
BALANCE?
TDD, ATDD, and front end GUI automation
are processes.
Exploratory Testing is a process and a
mindset.
Developing and defining acceptance is
exploratory.
Front End GUI Automation starts from
capturing discoveries made while
examining requirements.

33. THE ROLE OF
TESTERS
”Testing is dead" == testing needed to be a
part of an entire process of quality and
improvement, starting at the earliest stages
of design and development.
Exploration, automation, and testing must
come earlier in the process, and involve not
just testers, but everyone on the product
team.

34. STRIKING A BALANCE
Can we leverage the "testing
everywhere" idea, and help develop
a balance for all of these goals?
Can we see them as
interdependent, and not as separate
and standalone activities?

35. TDD AND ATDD TAKE AN
EXPLORATORY MINDSET
• Testing is part of the initial development effort.
• Testing comes first.
• Putting the emphasis on TDD/ATDD makes sure that it
meets the criteria it is being designed for.
Testers can also help guide and provide consideration for
testability in areas the programmers might not consider.
Asking "what if" questions vital at this stage

36. FOCUS ON DELIVERING
BUSINESS VALUE
Question 1: "is what I am doing helping
to deliver real value to my customers"?
If we put the business value first, that
will guide us in our testing efforts and
the approaches that we use
[Crispin, 2008].

37. AUTOMATION DOES NOT
HAVE TO BE PERMANENT
Make room for "throwaway automation”; steps
that can be used to help explore different
avenues.
Freight or passenger trains vs. taxi cabs.
• Freight and passenger trains limited to the
rails the trains can ride on.
• Taxi cab can be called to appear anywhere
and take us anywhere.

38. EXPLORATION IS A
MINDSET
Exploratory Testing is a way of thinking
about questions and answers.
Exploratory Testing does not live outside
of automated testing.
Automation and Computer Aided Testing
can be central to exploration.

39. ZEN MIND,
BEGINNERS MIND
Domain Expert Level Exploratory Testing
Novice Level Exploratory Testing
They are different.
Source of mental friction is knowing the
eccentricities of your application and when
they can get in the way of effective testing and
exploration.

40. USE PERSONAS TO
VISUALIZE GOALS
Put yourself into the shoes of as many potential
customers of our product as I can.
That means understanding the interests, desires
and ways of interaction of many different groups of
people.
The ways that they interact with the devices they
use to discover content/interact is very different.

41. SESSION BASED
TECHNIQUES
• Developing small charters and executing
on them in set periods of time.
• Focus testing areas that are most
important.
• Focus on a small sub system.
Keep our testing fresh, our eyes and minds
focused, work on what's most important.

42. CONCLUSION
TDD focuses on writing correct and clean code.
ATDD focuses on making sure the functionality
being delivered makes good on the goals and
promises made to our customers.
Automation happens at many levels.
Exploration happens at all of these levels, and is
an approach and a mindset, not a methodology.
Working together, the odds of quality software are
much more heavily in our favor.

43. QUESTIONS
???