Exploratory testing is a systematic approach to discovering risks by rigorously analyzing and testing a system through heuristics and testing techniques. It involves simultaneously learning about the target system while designing and immediately executing tests, using feedback from previous experiments to inform new tests. Exploratory testing focuses on discovering things you can vary and uses models, heuristics, time-boxed sessions, and charters to structure the exploration and focus on important areas.

1. Exploratory Testing
in Practice
Elisabeth Hendrickson
Blog: testobsessed.com
Twitter: @testobsessed
Slides Last updated June 22, 2012

2. Two Sides of Testing
Tested =

Checked
+
Explored

3. That’s Nice…
…But we
Believe in our
comprehensive,
repeatable
regression
tests.

4. Incidental Correctness
Given a series of 1’s and 0’s,
Compute the corresponding
decimal number
Input Expected Result
000 0 0

111 7 7

101 5 5

x 011 3 6

5. Missed Requirements
Hey, Where’s the Motor?!?
motor for this You never
remote control said anything
car? about a
motor!

6. Real World Users Happen
You assume they’re
reasonable people
who will do
reasonable things.
They aren’t and
they won’t.

7. So What Is Exploratory Testing?
a systematic approach for
discovering risks
using rigorous
analysis techniques
coupled with
testing heuristics.

8. OK…But What Do You Actually Do?
Simultaneously…
…learning about the Target
…While designing tests
…And executing Them Immediately
using feedback from the last Little
Experiment to inform the next
(Term coined by Cem Kaner. Definition refined by
James Bach. Others have contributed to ET including
James Lyndsay, Jonathan Kohl, Michael Bolton.)

9. Experiment
So it handled all Next, let’s try
the variations interrupting the
on disconnected transaction
networks we here.
could think of…

10. Discover Things You Can Vary

11. Use State Models
Idle
Time to
No
connect
Network
System
Updated Check
Network
No Updates Network
Avail
Update Get
System Updates
Content
Downloaded 11

12. Use Data Models
Customer
Vendor
Has 0 or More
Orders
Has 0 or More
Has 1 Or More
Products

13. Use Deployment Diagrams
Server Farm
Local
Agent
AWS

14. Use Heuristics
See the Test Heuristics Cheatsheet for more
http://bit.ly/testcheatsheet
Goldilocks: too big, too small,
just right Counts: 0, 1, Many
Selection: Some, None, All
Position: Beginning, Middle, End
Violate Domain-Specific Rules:
Age -1 IP address 999.999.999.999
Interrupt States
Delete Dependent Data

15. In Flight Feedback at #AU2H
Matt Barcomb
(Coding)
Jon Bach
(Exploring)

16. Result: Feedback on Process AND Product
We’ve automated I’ll add things
the regression like that to the
test when we fixed acceptance
the bug. criteria.
We need to think
about that kind of
thing in future
designs too.

17. Pitfall: Going Off in the Weeds
it doesn’t work on
15 year old laptops
running Windows 3.1!
Uh…
Why
Would We
Care?

18. Focus with Charters
Explore (Area/feature)
With (Resources,
conditions, Dependency,
etc.)
to discover (information)

19. Structure with Time Boxed Sessions
Charters Sessions
Explore login in combination Session
with bookmarks & history to
discover information about Session
interactions.
Session
Explore the site with various
browsers and settings to
Session
discover risks related to
unsupported configurations.
Session
Explore error handling using
data type attacks to find Session
problems related to data
validation.

20. Explore Early, Explore Often

21. Exploratory Testing on Agile Projects
Within an Iteration
Automated
Automated Exploratory
Acceptance
Unit Tests Testing
Tests
Define “done”, Drive design, Discover
represent represent
executable executable risks
requirements specifications EARLY