FROM STAKEHOLDER EXAMPLES TO LIVING DOCUMENTATION
Christian Hassa - ch@techtalk.at - @chr99ha
.NET User Group Bern, 5th of...
TechTalk at a glance
• Agile consulting and delivery
• Offices in: Vienna, Budapest, Zurich
• Established in 1993
Vienna/A...
3
How do we specify
requirements?
4
The purpose of user stories …
• Describe user needs
• Describe system features
• Unit of planning
• Reminder for a discu...
5
Impact Mapping
Story Mapping
Specification-By-Example
Establishing a shared understanding
Why?
How?
Code
Acceptance
Crit...
6
Collecting Acceptance Criteria
“I would try to put a book into the
shopping cart …”
“I would try to remove a book
from t...
7
Why
do we need
examples?
8
UI wire frames,
existing UI
rules, key examples
existing artifacts,
samples
Examples for user stories
10
Specification-by-Example
Examples …
• make abstract descriptions
better understandable
However …
• examples are usually...
11
Discussion of acceptance criteria
public void TestInitialOrderDiscount()
{
Customer newCustomer = new Customer();
Order...
12
… illustrated with formalized examples
Given the user has not ordered yet
When the user adds a book with the price of E...
13
Discover hidden assumptions
Actually, this not quite right:
Books on sale should be excluded.
Original idea for the ill...
14
Collaboration: 3 amigos
“Happy
Path”
Technical
feasability
Exceptions,
border cases
Original idea for the illustration:...
15
Abstract acceptance criteria
As a shop visitor
I want to collect books in my shopping basket
so that I can purchase mul...
16
Examples for acceptance criteria
As a shop visitor
I want to collect books in my shopping basket
so that I can purchase...
17
As a shop visitor
I want to collect books in my shopping basket
so that I can purchase multiple books at once.
Books ca...
18
The same book can be added multiple times to the shopping basket
Structure of examples
Given my shopping basket contain...
20
How long do we
care
about the
examples?
21
Purpose of the examples
• Shared understanding:
acceptance criteria
• Documentation:
Look-up detail aspects
of the syst...
22
Continuous validation with automation
Given my shopping basket contains 1 copy of “Harry Potter”
When I add the book “H...
23
Demo
http://www.specflow.org
Gherkin automation for .NET
• Visual Studio plugin (VS-Gallery)
• NuGet Package
24
What does that
mean for
testing?
25
UI automation becomes expensive
when …
• trying to automate
manual tests
• making tests
unreadable when
automating them...
26
Structure
Manual tests
Asserts Multiple combined
features
Structure ACT-ASSERT-
ACT-ASSERT-
ACT-ASSERT-
…
Dependent on ...
27
Test automation pyramid
User
journeys
Acceptance-
criteria
Units
exploratory
testing
Source: Mike Cohn
many
few
hard
ea...
28
// Go to web page 'http://localhost:40001/' using new browser instance
BrowserWindow localhostBrowser = BrowserWindow.L...
29
A readable test case
Scenario: New found items should receive a
consecutive number for the current year
Given the previ...
30
When to test (point in time)
Acceptance criteria
(ATDD, BDD)
Unit Tests
(TDD)
business view
technical view
Exploratory ...
31
Cross-functional teams
Co-creation
Fast feedback
32
Cross-functional work
Sprint 1 Sprint 2 Sprint 3
short iteration
US4
Plan
Implement &
autom. test
US5
Plan
Implement &
...
33
Manual Testing is always necessary!
User
journeys
Acceptance-
criteria
Units
exploratory
testing
Source: Mike Cohn
many...
34
What does that
mean for
development?
35
Workflow - TDD
36
Extending TDD for business
37
Transparency for stakeholders
In Progress
38
Current sprint report: all sprint scenarios
39
Starting with first scenario (AC)
40
Finishing the first scenario (AC)
41
Progressing scenario after scenario
42
Progressing scenario after scenario
43
Progressing scenario after scenario
44
Implementing user stories in parallel
45
First user story ready for testing
46
Manual testing can start even earlier
47
Already done work can break again
48
Already done work can break again
49
See what is temporarily not working
51
Living
documentation
52
Story Mapping
Optimizing and refining scope
Why?
How?
Code
Acceptance
Criterion
Epic
Capability
Impact, Goal
Easier to ...
53
Story Maps
• Original concept by Jeff Patton
• Supports iterative product design
• Optimized for desired outcome or
del...
56
Building story maps
Get mainstream book
fast and conveniently
delivered
Find book
I want
Collect
books
Complete
order
W...
57
Walking
skeleton
Enabling build – measure - learn
Find book
I want
Collect
books
Complete
order
Wait for
book
Receive
b...
60
Story Maps in ALM with TFS
TFS work items in story maps
http://speclog.net
61
Story Map in SpecLog
62
Sprint 1
63
Sprint 2
64
Sprint 3
65
Sprint 4
66
Dropped Scope
67
Added Scope
69
User Stories vs. Features
Product/Sprint Backlog
User Story 1
AccCrit 1
AccCrit 2
User Story 2
AccCrit 3
AccCrit 4
Livi...
71
Living documentation
• Link business readable automation from source code to story maps
• Feed execution results into s...
72
Additional
thoughts
73
Level of automation
Controller
Business Layer
Data Layer
Model
View
Browser
automation
Trigger behaviour
through contro...
74
Performance
• Grouping tests
• Current WIP
• Completed work
• Database
• In-memory
• Templates for setup
• Parallel exe...
75
Internal vs. external DSL
Example Source:
Liz Keogh
https://github.com/lunivore/tictactoe-java/blob/master/
scenarios/c...
76
Non-functional acceptance criteria
Given there are 100,000 users registered on the system
When I create a new account
T...
77
Tools
78
Gherkin automation tools
www.cukes.info
www.behat.org
Ruby, Java, JavaScript, C++
www.specflow.org
.NET, Mono, Silverli...
79
80
Books
Gojko Adzic
Bridiging the
Communication Gap
Gojko Adzic
Specification by
Example
Elisabeth Hendrickson
Explore It!
81
Christian Hassa: ch@techtalk.at - @chr99ha
Nächste SlideShare
Wird geladen in …5
×

Specification-By-Example with Gherkin

8.273 Aufrufe

Veröffentlicht am

Veröffentlicht in: Technologie
0 Kommentare
17 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

Keine Downloads
Aufrufe
Aufrufe insgesamt
8.273
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
64
Aktionen
Geteilt
0
Downloads
154
Kommentare
0
Gefällt mir
17
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

Specification-By-Example with Gherkin

  1. 1. FROM STAKEHOLDER EXAMPLES TO LIVING DOCUMENTATION Christian Hassa - ch@techtalk.at - @chr99ha .NET User Group Bern, 5th of December 2012 Specification-By-Example with Gherkin
  2. 2. TechTalk at a glance • Agile consulting and delivery • Offices in: Vienna, Budapest, Zurich • Established in 1993 Vienna/Austria
  3. 3. 3 How do we specify requirements?
  4. 4. 4 The purpose of user stories … • Describe user needs • Describe system features • Unit of planning • Reminder for a discussion • Mechanism to defer a discussion
  5. 5. 5 Impact Mapping Story Mapping Specification-By-Example Establishing a shared understanding Why? How? Code Acceptance Criterion Epic Capability Impact, Goal Easier to define upfront Harder to define upfront Reminder for a discussion Bug report Isolated, formalized example User Activity User Story Example
  6. 6. 6 Collecting Acceptance Criteria “I would try to put a book into the shopping cart …” “I would try to remove a book from the shopping cart…” “I’d check whether the shopping cart is empty, when I enter the shop …” “I would try to add the same book again to the shopping cart …” Books can be placed into shopping cart. Books can be removed from shopping cart. Shopping cart should be empty when entering the shop. Adding same book again to shopping cart should increase quantity. As a potential customer I want to collect books in a shopping cart So that I can order several books at once. “Imagine this story is already implemented: how would you verify it?”
  7. 7. 7 Why do we need examples?
  8. 8. 8 UI wire frames, existing UI rules, key examples existing artifacts, samples Examples for user stories
  9. 9. 10 Specification-by-Example Examples … • make abstract descriptions better understandable However … • examples are usually not formally exchanged or documented Brian Marick Examples Tests Requirements consist of describe verify fulfillment of
  10. 10. 11 Discussion of acceptance criteria public void TestInitialOrderDiscount() { Customer newCustomer = new Customer(); Order newOrder = new Order(newCustomer); newOrder.AddBook( Catalog.Find(“ISBN-0955683610”) ); Assert.Equals(33.75, newOrder.Subtotal); } Register as “bart_bookworm” Go to “/catalog/search” Enter “ISBN-0955683610” Click “Search” Click “Add to Cart” Click “View Cart” Verify “Subtotal” is “$33.75” We would like to encourage new users to buy in our shop. Therefore we offer 10% discount for their first order. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  11. 11. 12 … illustrated with formalized examples Given the user has not ordered yet When the user adds a book with the price of EUR 37.5 into the shopping cart Then the shopping cart sub-total is EUR 33.75. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  12. 12. 13 Discover hidden assumptions Actually, this not quite right: Books on sale should be excluded. Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  13. 13. 14 Collaboration: 3 amigos “Happy Path” Technical feasability Exceptions, border cases Original idea for the illustration: George Dinwiddie http://blog.gdinwidiee.com
  14. 14. 15 Abstract acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Books can be removed from the shopping basket Shopping basket is initially empty The same book can be added multiple times to the shopping basket
  15. 15. 16 Examples for acceptance criteria As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Given my shopping basket is empty When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 1 copy of “Harry Potter”
  16. 16. 17 As a shop visitor I want to collect books in my shopping basket so that I can purchase multiple books at once. Books can be added to the shopping basket Examples for acceptance criteria When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” The same book can be added multiple times to the shopping basket Given my shopping basket contains 1 copy of “Harry Potter”
  17. 17. 18 The same book can be added multiple times to the shopping basket Structure of examples Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” Title: Describes intention/abstract acceptance criterion Arrange: Context, initial state of the system Act: Execution of the feature Assert: Assertion of observable behaviour And I should see the warning: “Book already existed in basket” Triple-A constraint “Checks” Chaining up steps
  18. 18. 20 How long do we care about the examples?
  19. 19. 21 Purpose of the examples • Shared understanding: acceptance criteria • Documentation: Look-up detail aspects of the system • Regression-tests: Understand what assumptions have been violated
  20. 20. 22 Continuous validation with automation Given my shopping basket contains 1 copy of “Harry Potter” When I add the book “Harry Potter” to my shopping basket Then my shopping basket should contain 2 copies of “Harry Potter” System „Step Definitions“ are binding individual steps to an automatable interface of the application. Automatable interface UI Automation Automation does not necessarily have to bind to the UI. Automatability of system is supported/evolving with development.
  21. 21. 23 Demo http://www.specflow.org Gherkin automation for .NET • Visual Studio plugin (VS-Gallery) • NuGet Package
  22. 22. 24 What does that mean for testing?
  23. 23. 25 UI automation becomes expensive when … • trying to automate manual tests • making tests unreadable when automating them • automating after completing development structure readability point in time
  24. 24. 26 Structure Manual tests Asserts Multiple combined features Structure ACT-ASSERT- ACT-ASSERT- ACT-ASSERT- … Dependent on other features Long test path with high chance to break Automated Check Single aspect of a single feature ARRANGE – ACT – ASSERT Independent of other features Short test path with lower chance to break
  25. 25. 27 Test automation pyramid User journeys Acceptance- criteria Units exploratory testing Source: Mike Cohn many few hard easy Automatability
  26. 26. 28 // Go to web page 'http://localhost:40001/' using new browser instance BrowserWindow localhostBrowser = BrowserWindow.Launch( new System.Uri(this.RecordedMethod1Params.Url)); // Click 'Register found item' link Mouse.Click(uIFundstückerfassenHyperlink, new Point(56, 9)); // Click 'Save' button Mouse.Click(uISpeichernButton, new Point(44, 14)); int fundNr1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9)); // Click 'Register found item' link Mouse.Click(uIFundstückerfassenHyperlink, new Point(63, 7)); // Click 'Save' button Mouse.Click(uISpeichernButton, new Point(34, 11)); int fundNr2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9)); Assert.IsTrue(fundNr1 + 1 == fundNr2); // Click 'Close' button Mouse.Click(uICloseButton, new Point(26, 11)); Readability
  27. 27. 29 A readable test case Scenario: New found items should receive a consecutive number for the current year Given the previous found item of the current year had the number 145 When I register a new found item Then the last found item of the current year should have the number 146
  28. 28. 30 When to test (point in time) Acceptance criteria (ATDD, BDD) Unit Tests (TDD) business view technical view Exploratory tests Workflow tests Performance, Scalability, Usability,Security, … definingtheproduct criticizingtheproduct New dimension: defining the product Synergy: Specification of requirements and tests Agile Testing Quadrants: Brian Marick
  29. 29. 31 Cross-functional teams Co-creation Fast feedback
  30. 30. 32 Cross-functional work Sprint 1 Sprint 2 Sprint 3 short iteration US4 Plan Implement & autom. test US5 Plan Implement & autom. test US2 Plan Implement & autom. test US3 Plan Implement & autom. test US6 Plan Implement & autom. test US1 Plan Implement & autom. test US7 Plan Implement & autom. test US8 Plan Implement & autom. test US9 Plan Implement & autom. test ExploratoryTests Specification and test Collaboration for defining acceptance criteria Collaboration for automation Preventing bugs instead of finding them! Extension of “Test Cases” Limit WIP Collaboration in manual testing
  31. 31. 33 Manual Testing is always necessary! User journeys Acceptance- criteria Units exploratory testing Source: Mike Cohn many few harder easier Automatability Manual Check after Story Done Main success pathes Undiscovered acceptance criteria No/(few) manual regression checks Few pathes are enough More time for exploration
  32. 32. 34 What does that mean for development?
  33. 33. 35 Workflow - TDD
  34. 34. 36 Extending TDD for business
  35. 35. 37 Transparency for stakeholders In Progress
  36. 36. 38 Current sprint report: all sprint scenarios
  37. 37. 39 Starting with first scenario (AC)
  38. 38. 40 Finishing the first scenario (AC)
  39. 39. 41 Progressing scenario after scenario
  40. 40. 42 Progressing scenario after scenario
  41. 41. 43 Progressing scenario after scenario
  42. 42. 44 Implementing user stories in parallel
  43. 43. 45 First user story ready for testing
  44. 44. 46 Manual testing can start even earlier
  45. 45. 47 Already done work can break again
  46. 46. 48 Already done work can break again
  47. 47. 49 See what is temporarily not working
  48. 48. 51 Living documentation
  49. 49. 52 Story Mapping Optimizing and refining scope Why? How? Code Acceptance Criterion Epic Capability Impact, Goal Easier to define upfront Harder to define upfront User Activity User Story Example
  50. 50. 53 Story Maps • Original concept by Jeff Patton • Supports iterative product design • Optimized for desired outcome or deliverable the system should support
  51. 51. 56 Building story maps Get mainstream book fast and conveniently delivered Find book I want Collect books Complete order Wait for book Receive book time browse best sellers put into basket enter address receive delivery slip receive delivery notificat. pay with credit card search book by title create wish list inquiry order status desired outcome or deliverable user activities system features necessity
  52. 52. 57 Walking skeleton Enabling build – measure - learn Find book I want Collect books Complete order Wait for book Receive book time browse best sellers enter address receive delivery slip pay with credit card search book by title create wish list inquiry order status put into basket receive delivery notificat. necessity manual workaround omitted steps Get mainstream book fast and conveniently delivered
  53. 53. 60 Story Maps in ALM with TFS TFS work items in story maps http://speclog.net
  54. 54. 61 Story Map in SpecLog
  55. 55. 62 Sprint 1
  56. 56. 63 Sprint 2
  57. 57. 64 Sprint 3
  58. 58. 65 Sprint 4
  59. 59. 66 Dropped Scope
  60. 60. 67 Added Scope
  61. 61. 69 User Stories vs. Features Product/Sprint Backlog User Story 1 AccCrit 1 AccCrit 2 User Story 2 AccCrit 3 AccCrit 4 Living Documentation Feature 1 AccCrit 1 AccCrit 2 Feature n AccCrit 4 AccCrit m User Story n AccCrit 5 AccCrit m AccCrit 3 AccCrit 5 „Done“ • Future options of the system • Organized/refined according to priority, value, effort, risk, ... • Next possible increments of the product (units of work) • Current state of the system • Organized/refined for functional overview • Versioned and maintained together with source code
  62. 62. 71 Living documentation • Link business readable automation from source code to story maps • Feed execution results into story maps
  63. 63. 72 Additional thoughts
  64. 64. 73 Level of automation Controller Business Layer Data Layer Model View Browser automation Trigger behaviour through controller Assert behaviour on model, db, .. Setup pre-conditions through service interfaces Out-of-process In-process
  65. 65. 74 Performance • Grouping tests • Current WIP • Completed work • Database • In-memory • Templates for setup • Parallel execution • Smart execution order
  66. 66. 75 Internal vs. external DSL Example Source: Liz Keogh https://github.com/lunivore/tictactoe-java/blob/master/ scenarios/com/lunivore/tictactoe/scenarios/ Three_in_a_row_wins.java
  67. 67. 76 Non-functional acceptance criteria Given there are 100,000 users registered on the system When I create a new account Then I should be taken to my dashboard within 5ms Given 1000 users are hitting the homepage simultaneously Then each user should get a response within 2ms Matt Wynne http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing
  68. 68. 77 Tools
  69. 69. 78 Gherkin automation tools www.cukes.info www.behat.org Ruby, Java, JavaScript, C++ www.specflow.org .NET, Mono, Silverlight, WP7 PHP
  70. 70. 79
  71. 71. 80 Books Gojko Adzic Bridiging the Communication Gap Gojko Adzic Specification by Example Elisabeth Hendrickson Explore It!
  72. 72. 81 Christian Hassa: ch@techtalk.at - @chr99ha

×