On agile teams, testers often struggle to “keep up” with the pace of development if they continue employing a waterfall-based verification process—finding bugs after development. Nate Oster challenges you to question waterfall assumptions and replace this legacy verification testing with Acceptance Test-driven Development (ATDD). With ATDD, you “test first” by writing executable specifications for a new feature before development begins. Learn to switch from “tests as verification” to “tests as specification” and to guide development with acceptance tests written in the language of your business. Get started by joining a team for a simulation and experience how ATDD helps build in quality instead of trying to test out defects. Then progress to increasingly more realistic scenarios and practice the art of specifying intent with plain-language and table-based formats. These paper-based simulations give you meaningful practice with how ATDD changes the way you think about tests and collaborate as a team. Leave empowered with a kit of exercises to advocate ATDD with your own teams!

1. MN
PM Tutorial
4/29/13 1:00PM
Acceptance Test-driven
Development: Mastering Agile Testing
Presented by:
Nate Oster
CodeSquads, LLC
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Nate Oster
An agile player-coach and founder of CodeSquads, Nate Oster helps clients adopt lean and agile
methods. Nate builds high-performance teams that adapt to change, embrace a pragmatic philosophy of
continuous improvement, measure progress with new features, and deliver high-quality software that
delights customers. As a coach, he inspires adopters with hands-on mentoring and simulations that
provide a safe learning environment for new ideas. Nate promotes testing as a serious technical discipline.
While Nate maintains a speaking schedule at software development conferences, he spends most of his
time as a hands-on advisor to software product teams from startups to multinationals. Nate is easily
defeated at all card games. Contact him at NateOster@CodeSquads.com.

3. ACCEPTANCE TEST-DRIVEN DEVELOPMENT:
MASTERING AGILE TESTING
Nate Oster
NATE OSTER
Agile Player-Coach &
Founder, CodeSquads
Focused on adopting
Lean & Agile methods
Mail: Nate.Oster@codesquads.com
Twitter: nateoster
Voice: 703.930.4100

4. LEARNING OBJECTIVES
Stop finding bugs after development!
Shift from tests as verification to
tests as specification by learning to test first.
Simulate the feel of ATDD in a non-threatening way
Gain meaningful practice with table-based and
plain-language formats for specifying intent.
Leave empowered to advocate ATDD
with your own teams and managers!
AGENDA
Agile Testing has limits
Simulation: Tests as Verification
Intro to ATDD
Simulation: Tests as Specification

5. AGENDA
Specification by Example
Simulation: Examples using Tables
Break!
Simulation: Examples using Given-When-Then
Wrap-up: Action Planning, Q&A
THE PROBLEM
Finding defects is expensive
Rework, Scrap, Delays
Feedback speed is limited

6. SIMULATION:
TESTS AS VERIFICATION
SIMULATION:
TESTS AS VERIFICATION
Teams of 3
“Develop” an image by
punching graph paper
“Test” your solution
by fitting it over pegs
Satisfy your Customer!

7. RETROSPECTIVE:
TESTS AS VERIFICATION
ACCEPTANCE TESTDRIVEN DEVELOPMENT
Stop testing bugs out.
Start building quality in.

8. Agile&Tes)ng&Quadrants&
Business-facing Tests
Acceptance Tests
Unit & Component Tests
Automated
Frameworks
Mostly
Manual
Exploratory Tests
Usability Testing
UAT
Performance Testing
Security analysis
“-ility” tests
Tests that critique the product
Tests that support the Team
Mostly
Automated
Specialized
Tools
Technology-facing Tests
“Agile Testing,” Lisa Crispin & Janet Gregory, used with permission (originally Brian Marick, 2006)
The$Test$Automa,on$Pyramid$
Exploratory
/Manual
GUI
More business
facing, realistic
Sn
Acceptance Tests
(API Layer)
FitNesse/
Cuke
Lower-cost
Easier maintenance
Faster feedback
xUnit
Unit & Component Tests
Adapted from Mike Cohn (Automated Test Pyramid) & “Agile Testing”, Lisa Crispin & Janet Gregory..

9. THE TEST AUTOMATION PYRAMID
Push tests as low as
possible in the pyramid
Have tests focus on
intent, not implementation
Team decides appropriate
level for each test
Anatomy(of(an(Itera-on(
2(–(4(week(-mebox(
Define(
Define(
Build(
Define(
Build(
Build(
Test(
Define(
Test(
Build(
Test(
Define(
Define(
Build(
Define(
Test(
Build(
Test(
Define(
Test(
Build(
Test(
Itera-on(Planning…(
Retrospec-ve(
Itera-on(Demo(
Itera-on(Planning(
Feature(
Team(
Build(
Test(
a(f
(
y…
da
xt(
ne rs(
ou (
2(h ours
(h
2@4
urs
(
urs
ys(
(d a
ew
(h o
ew
a(f
(ho
2@4
(
Nate%Oster,%“The%Agile%Challenge”%Workshop,%CodeSquads%LLC.%

10. SIMULATION:
TESTS AS SPECIFICATION
SIMULATION:
TESTS AS SPECIFICATION
Start each story by outlining the acceptance criteria
Specify acceptance tests as concrete examples
If we don’t know when we’re done,
then why are we starting?
Hypothesize the top 3 effects of this change

11. RETROSPECTIVE:
TESTS AS SPECIFICATION
Reflect:
Where your
hypotheses correct?
Why or why not?
Prepare to Share:
Appoint a scribe
Share your results
at your tables
List the top
improvements
SPECIFICATION BY EXAMPLE
“An example would be helpful right about now.”
- Brian Marick

12. Essential
Incidental
Hat tip: Dale Emery
Agile&Tes)ng&Quadrants&
Business-facing Tests
Acceptance Tests
Unit & Component Tests
Automated
Frameworks
Exploratory Tests
Usability Testing
UAT
Performance Testing
Security analysis
“-ility” tests
Technology-facing Tests
Mostly
Manual
Tests that critique the product
Tests that support the Team
Mostly
Automated
Specialized
Tools
“Agile Testing,” Lisa Crispin & Janet Gregory, used with permission (originally Brian Marick, 2006)

13. We'refine'the'Product'Backlog'just2in2*me'for'
immediate'goals.'
Vision'&'Roadmap'
Release'Planning'
Itera*on'
Planning'
Next'
Release'
Future'Roadmap…'
Future'Releases'
Current'' (forecast)'
Release'
Current'
Itera*on'
?'
Itera*ons'
Es*mate'
Accuracy'
+/2'
+/2'
+/2'
+/2'
Near'Term:'Fine2Grained'
?'
Long2Term:'Coarse2Grained'
Nate%Oster,%“The%Agile%Challenge”%Workshop,%CodeSquads%LLC.%
The$Test$Automa,on$Pyramid$
Exploratory
/Manual
GUI
More business
facing, realistic
Sn
Acceptance Tests
(API Layer)
FitNesse/
Cuke
Lower-cost
Easier maintenance
Faster feedback
xUnit
Unit & Component Tests
Adapted from Mike Cohn (Automated Test Pyramid) & “Agile Testing”, Lisa Crispin & Janet Gregory..

14. SOME TOOLS FOR
EXECUTABLE SPECIFICATIONS
TOOL
TECHNOLOGY
SWEET SPOT
FitNesse
Java (native)
.NET (plugin)
Cucumber
Ruby (native)
Java (CucumberJVM)
.NET (plugin)
SpecFlow
.NET (native)
SIMULATION:
SPECIFICATION BY EXAMPLE
USING TABLES

15. TABLE-BASED
SPECIFICATIONS
(Re)-build a brokerage
pricing engine
Start with the simplest
scenarios using
realistic examples
Be customer-focused!
Pricing.Stocks
Trade Source
Internet
Internet
IVR
Broker
Ticker Buy/Sell
LXK
Buy
LXK
Sell
Order Type
Limit
Limit
Commission?
$9.99
$9.99

16. Pricing.Stocks
Trade Source
Internet
Internet
IVR
Ticker Buy/Sell
LXK
Buy
LXK
Sell
LXK
Buy
Order Type
Limit
Limit
Market
Commission?
$9.99
$9.99
$34.99
[expected “$34.99”, actual “$9.99”]
Pricing.MutualFunds.Buy
Test Funds
Ticker
MNTF
MNLD
MFEE
Fund Type
NTF
No-Load
Load
Load %
N/A
N/A
4% on Buy

17. Pricing. MutualFunds. NTF. ShortTermRedemption
No Transaction Fee
Mutual Funds
180 days or less
No commission
Ticker
MNTF
MNLD
MFEE
Fund Type
NTF
No-Load
Load
Load %
N/A
N/A
4% on Buy
$49.99 fee
WHAT IF I BUY TWICE
IN 180 DAYS?

18. TABLES MAY NOT BE EXPRESSIVE
ENOUGH FOR SOME SCENARIOS
What if I buy the same fund
more than once in 180 days?
What if I sell when only part
of the balance is 180 days old?
“An example would be helpful right about now.”
- Brian Marick
SIMULATION:
SPECIFICATION BY EXAMPLE
USING TEST LINGOS*
*Credit to Patrick Wilson-Welsh and Dawn Cannon
for coining the term!

19. LINGO-BASED
SPECIFICATIONS
Given - When - Then
GIVEN - Preconditions
WHEN - Action
THEN - Result / Postcondition
THINKING WITH
GIVEN-WHEN-THEN
Pricing. MutualFunds. Buy
Scenario: Buy a no-load fund.
Given I do not own fund MNLD
When I buy $10,000 in fund MNLD
then the commission is $49.99
and the fee is $0.00.
When I buy fund MNLD,
then the commission is $49.99.

20. Pricing. MutualFunds.Buy
Scenario: Buy a mutual fund.
Test Funds
Ticker
MNTF
MNLD
MFEE
Fund Type
NTF
No-Load
Load
Load %
N/A
N/A
4% on Buy
GIVEN-WHEN-THEN CAN BE MORE
EXPRESSIVE THAN TABLES
What if I buy the same fund
more than once in 180 days?
What if I sell when only part
of the balance is 180 days old?
“An example would be helpful right about now.”
- Brian Marick

21. Pricing. MutualFunds. NTF. ShortTermRedemption
No Transaction Fee
Mutual Funds
180 days or less
No commission
Ticker
MNTF
MNLD
MFEE
Fund Type
NTF
No-Load
Load
Load %
N/A
N/A
4% on Buy
$49.99 fee
Mutual Fund Short Term Redemptions
No transaction fee (NTF) funds held 180 days or less are
subject to a Short Term Redemption Fee of $49.99.
Scenario: Part of sale held less than 180 days charges
short term redemption fee.
Given I own 100 shares of MNTF for 181 days
and I own 100 shares of MNTF for 179 days
When I sell 101 shares of MNTF
Then I am charged a short term redemption fee of $49.99.
Scenario: ...

22. Scenario: Part of sale held less than 180 days charges
short term redemption fee.
Given I own MNTF
|shares| days held |
| 100 | 181 |
| 100 | 179 |
When I sell 101 shares
Then I am charged a short term redemption fee.
NEW FEATURE:
CONCIERGE LEVEL PRICING
Concierge Level pricing is a trial promotion.
It offers selected customers free internet trades if their
total assets exceed $250,000 or if a trade is for more
than $10,000.
Brainstorm acceptance tests for this feature using
tables, given-when-then, or both. Your choice!

23. Feature: Concierge pricing promotion.
Background:
Given the concierge pricing promotion is active.
Scenario Outline: Buying in a selected account.
Given a selected account with a balance of <total assets>,
when I buy <trade amount> of <ticker>
using source <trade source>,
then the commission is <fee>.
Examples:
| total assets | trade amount | ticker | trade source | fee |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
REVIEW

24. WRAP-UP: CREATE AN
ACTION PLAN
What outcomes could ATDD improve
on your Feature Team?
What impediments to adopting ATDD do you
anticipate on your team?
Is there evidence these impediments can be
overcome?
What small experiments will confirm or disprove these
opportunities with the least possible risk?
EXPLORING FURTHER
Learn More:
Experiment:
Specification by Example
(Gojko Adzic)
“Pull” in support through
your Team’s retrospectives
The Cucumber Book
(Aslak Hellesoy)
People & improvement
Feedback Interactions ideas
over Processes & Tools!
Agile Testing
(Lisa Crispin & Janet Gregory)

25. NATE OSTER
Agile Player-Coach &
Founder, CodeSquads
Focused on adopting
Lean & Agile methods
Mail: nate.oster@codesquads.com
Twitter: nateoster
Voice: 703.930.4100