Cucumber is a very good tool for achieving functional testing. However your cuke suite can become unwieldy very soon. Even a little bit of indiscipline can lead to huge build times, with random failures. Over time, maintaining and writing the cukes starts costing more than the benefit derived out of it. On the other hand, you can start ignoring your functional build only to discover some nasty bug later which your poor cuke suite was telling you.
In this presentation we try and look at what can be done to avoid this and derive complete value from cukes. We share experiences and learnings about cuking the right way from real projects
Learnings
1) best practices of writing and maintaining cukes
2) some new things that could be tried out with cukes to make them more reliable and valuable.
4. Cucumber is meant to provide stakeholders a way
to express specifications in plain english. All
team members can be on the same page about the
specifications. Also since this is automated, you
effectively have executable specifications.
Cucumber consists of feature files which has
scenarios in given, when then statements and
corresponding implementation of those in
step_definition files.
5. Scenario: Purchasing books online
Given I have selected following books in my
shopping cart:
|title | price|
|Sherlock Homes |12.35 |
|Little Women |5.00 |
When I proceed to checkout
And I fill out the billing and shipping details
And I give my credit card details
Then I should see the order confirmation message
6.
7.
8.
9.
10.
11. We will look at a sample scenario and
try and analyse what’s wrong in the
way it is written
12. Scenario: User should be able to verify items in his shopping cart
and proceed to checkout by filling in billing and shipping details.
Given I have a user "john“ with last name “doe” and email
“jdoe@gmail.com”
And I have a product “Kindle” in products list
And product “ipad" exists with color “white”
And I have added 2 “Kindle” and 2 “ipad” with color “white” in my
shopping cart.
When I click on view shopping cart within “//div[@id=header]”
And I wait “10"
Then I should see "Kindle" and “ipad" in the view shopping cart
overlay
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page“
When I fill in “First Name” with “John”
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com”
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
Then I should see my shipping details on the order confirmation
page
13. Scenario: User should be able to verify items in his shopping cart and
proceed to checkout by filling in billing and shipping details.
Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”
And I have a product “Kindle” with price “50$” in
products list
And product “ipad" exists with color “white”
And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping
cart.
When I click on view shopping cart within “//div[@id=header]”
And I wait “10"
Then I should see "Kindle" and “ipad" in the view shopping cart overlay
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page"
When I fill in “First Name” with “John”
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com”
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
Then I should see my shipping details on the order confirmation page
14. Step rewritten:
Given following products are available:
|product title| price |
|Kindle | 50.00 |
|ipad | 50.00 |
And product “ipad” is available with following
options:
|product option|product option value|
| colour | white |
And I wait “10"
Then I should see "Kindle" and “ipad" in the view shopping cart overlay
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page"
When I fill in “First Name” with “John”
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com”
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
Then I should see my shipping details on the order confirmation page
15. Scenario: User should be able to verify items in his shopping cart and
proceed to checkout by filling in billing and shipping details.
Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”
And I have a product “Kindle” with price “50$” in products list
And product “ipad" exists with color “white”
And I have added 2 “Kindle” and 2 “ipad” with color
“white” in my shopping cart.
When I click on view shopping cart within “//div[@id=header]”
And I wait “10"
Then I should see "Kindle" and “ipad" in the view shopping cart overlay
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page"
When I fill in “First Name” with “John”
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com”
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
Then I should see my shipping details on the order confirmation page
16. Scenario: User should be able to verify items in his shopping cart and proceed to checkout by
filling in billing and shipping details.
Step rewritten:
And I add the following to my shopping cart:
|product name|product option|option value|quantity|
| kindle | color | white | 2 |
| ipad | | | 2 |
And I have a product “Kindle” in products list
And product “ipad" exists with color “white”
And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping cart.
When I click on view shopping cart within “//div[@id=header]”
And I wait “10"
Then I should see "Kindle" and “ipad" in the view shopping cart overlay
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page"
When I fill in “First Name” with “John”
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com”
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
Then I should see my shipping details on the order confirmation page
17. Scenario: User should be able to verify items in his shopping cart and
proceed to checkout by filling in billing and shipping details.
Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”
And I have a product “Kindle” with price “50$” in products list
And product “ipad" exists with color “white”
And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping
cart.
When I click on view shopping cart within
“//div[@id=header]”
And I wait “10"
Then I should see "Kindle" and “ipad" in the view shopping cart overlay
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page"
When I fill in “First Name” with “John”
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com”
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
Then I should see my shipping details on the order confirmation page
18. filling in billing and shipping details.
Step rewritten:
When I view the shopping cart
When /^I view the shopping cart$/ do
with_scope(Sections::MAP[“header”]) do
click_link(“Shopping Cart”)
end
end
And I have a product “Kindle” in products list
And product “ipad" exists with color “white”
e.g. sections. rb
And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping cart.
module Sections
When I click on view shopping cart within “//div[@id=header]”
And I wait “10"
MAP =
Then I should see "Kindle" and “ipad" in the view shopping cart overlay
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page"
{
When I fill in “First Name” with “John” „header' =>
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com” '//div[@id=„header‟]„
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all countries
}
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
end
And I press “Next”
Then I should see my shipping details on the order confirmation page
19. Scenario: User should be able to verify items in his shopping cart and
proceed to checkout by filling in billing and shipping details.
Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”
And I have a product “Kindle” with price “50$” in products list
And product “ipad" exists with color “white”
And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping
cart.
When I click on view shopping cart within “//div[@id=header]”
And I wait “10"
Then I should see "Kindle" and “ipad" in the view shopping cart overlay
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page"
When I fill in “First Name” with “John”
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com”
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
Then I should see my shipping details on the order confirmation page
20. Scenario: User should be able to verify items in his shopping cart and proceed to checkout by
When I view the shopping cart
When /^I view the shopping cart$/ do
with_scope(sections[“header”]) do
click_link(“Shopping Cart”)
end
And %{wait for element to appear
sections[„cart‟]”}
End
When /^wait for element to appear "([^"]*)"$/ do
|selector|
loop_until { find(:xpath, selector).visible? }
end
And I fill in “Email” with “jdoe@gmail.com”
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
Then I should see my shipping details on the order confirmation page
21. Scenario: User should be able to verify items in his shopping cart and
proceed to checkout by filling in billing and shipping details.
And product “ipad" exists with color “white”
And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping
cart.
When I click on view shopping cart within “//div[@id=header]”
And I wait “10"
Then I should see "Kindle" and “ipad" in the view shopping cart overlay
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page"
When I fill in “First Name” with “John”
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com”
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all
countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
Then I should see my shipping details on the order confirmation page
22. Scenario: User should be able to verify items in his shopping cart and
proceed to checkout by filling in billing and shipping details.
And product “ipad" exists with color “white”
And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping
cart.
When I click on view shopping cart within “//div[@id=header]”
And I wait “10"
Then I should see "Kindle" and “ipad" in the view shopping cart overlay
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page"
When I fill in “First Name” with “John”
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com”
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all
countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
Then I should see my shipping details on the order confirmation page
23. Step rewritten:
When I fill in my billing details
And I choose to keep shipping address same as
billing
When /^I fill in my billing details$/ do
fill_in(“first_name”, :with => “John”)
fill_in(“last_name”, :with => “Doe”)
fill_in(“email”, :with => “jdoe@gmail.com”)
fill_in(“address”, :with => “Chicago, IL”)
fill_in(“zipcode”, :with => “John”)
select(“US”, :from => “Country”)
End
When /^I choose to keep shipping address same as
billing$/ do
check(“shipping_same_as_billing”)
End
And I fill in “Address” with “Chicago, IL”
And I press “Next”
24. Step rewritten:
When I fill in my billing details
And I choose to keep shipping address same as
billing
When /^I fill in my billing details$/ do
fill_in(“first_name”, :with => “John”)
fill_in(“last_name”, :with => “Doe”)
fill_in(“email”, :with => “jdoe@gmail.com”)
fill_in(“address”, :with => “Chicago, IL”)
fill_in(“zipcode”, :with => “John”)
select(“US”, :from => “Country”)
End
When /^I choose to keep shipping address same as
billing$/ do
check(“shipping_same_as_billing”)
End
And I fill in “Address” with “Chicago, IL”
And I press “Next”
25. Scenario: User should be able to verify items in his shopping cart and
proceed to checkout by filling in billing and shipping details.
Given I have a user "john“ with last name “doe” and email “jdoe@gmail.com”
And I have a product “Kindle” with price “50$” in products list
And product “ipad" exists with color “white”
And I have added 2 “Kindle” and 2 “ipad” with color “white” in my shopping
cart.
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page"
When I fill in “First Name” with “John”
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com”
And I fill in “Address” with “Chicago, IL”
And I select “US” from “Country”
Then the countries list should show all
countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
Then I should see my shipping details on the order confirmation page
26. Then I should see my shipping details on the order
confirmation page
When I press "//span[@id='proceed_to_checkout']" button
Then /^I should see my shipping details on the
order confirmation page$/ do
shipping_details = @user.shipping_address
[:fist_name, :email, :zipcode].each do |field|
Then %{I should see “#{shipping_details[field]}”}
end
om “Country”
Then the countries list should show all countries
This step depends on
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
an instance variable
And I press “Next”
being populated from
some previous step
27. Scenario: User should be able to verify items in his shopping cart and proceed to checkout by filling in billing
and shipping details.
Given I have a user "john“ with last name “doe” and email jdoe@gmail.com
When I click on view shopping cart within “//div[@id=header]”
And I wait “10"
Then I should see "Kindle" and “ipad" in the view shopping cart overlay
Then I should see my shipping details on the order
confirmation page
When I press "//span[@id='proceed_to_checkout']" button
Then I see the page title as "Billing Page"
When I fill in “First Name” with “John”
And I fill in “Last Name” with “Doe”
And I fill in “Email” with “jdoe@gmail.com
Then /^I should see my shipping details on the
order confirmation page$/ do
shipping_details = @user.shipping_address
[:fist_name, :email, :zipcode].each do |field|
Then %{I should see “#{shipping_details[field]}”}
end
om “Country”
Then the countries list should show all countries
And I fill in “Zipcode” with “60601”
When I check “shipping_same_as_billing”
And I press “Next”
28. Scenario: User should be able to verify items in his shopping cart and proceed to checkout by
filling in billing and shipping details.
Then I should see shipping details for user “john”
on the order confirmation page
Then /^I should see shipping details for user
“(.*)” on the order confirmation page$/ do
|identifier|
user = User.find_by_identifier(identifier)
shipping_details = user.shipping_address
[:fist_name, :email, :zipcode].each do |field|
Then %{I should see “#{shipping_details[field]}”}
end
Send in unique
Then the countries list should show socountries the
identifiers all that
And I select “US” from “Country”
And I fill in “Zipcode” with “60601”
specific record can be
When I check “shipping_same_as_billing”
And I press “Next”
retrieved and testing can
Then I should see my shipping details on the order confirmation page
be done
29. Given I have a user with details as:
|first name| John |
|last name | Doe |
|email | jdoe@gmail.com|
Given following products are available:
|product title| price |
|Kindle | 50.00 |
|ipad | 50.00 |
And product “ipad” is available with following options:
|product option|product option value|
| colour | white |
And I add the following to my shopping cart:
|product title|product option|option value| quantity |
| kindle | color | white | 2 |
| ipad | | | 2 |
When I view the shopping cart
Then I should see the following items in “Cart” section:
| product |quantity|
|kindle |2 |
| ipad |1 |
When I proceed to checkout
When I fill in my billing details
And I choose to keep shipping address same as billing
And I submit the billing form
Then I should see shipping details for user “john” on the order
confirmation page
33. Backdoor entry for login:
Quite often end to end scenarios consist of a user
getting logged in and then doing things on the
app. You need not repeat the complicated login
process (via the ui). Simply get the user logged in
the fastest way via a backdoor entry. Saves a lot of
time !! Cut down 5 mins in a feature file.
34. Backdoor entry for login:
Without backdoor With backdoor
When /^I log in as "([^"]*)" Given /^I am logged in as
with password "([^"]*)" without "(.*)"$/ do |screen_name|
overlay$/ do |email, password| url =
When %{I go to the login page} "/backdoor/#{screen_name}“
When %{authenticate user call to visit url
cis is stubbed to return end
"<user><id>1</id></user>"}
When %{search user call to cis is
stubbed to return
"<user><id>1</id></user>"}
When %{I fill in "email" with
"#{email}"}
When %{I fill in "password" with
"#{password}"}
When %{I press "login"}
And %{I wait "1"}
@logged_in_user =
User.find_by_email(email)
end
35. if Rails.env.test?
get "/backdoor/:screen_name", :to =>
"sessions#backdoor", :as => "backdoor“
end
def backdoor
user =
User.find_by_screen_name(params[:screen_na
me])
set_current_user(user)
head :ok
end
36. Do not use page.has_xpath? should be false. Go for
page.should_not have_xpath.
Has_xpath? would try repeatedly (till capybara timeout)
to find the element. When it doesn’t find it, the value
returned would be false which would match with the
assert. Rather than that use has_no_xpath? So then
capybara exits the first time itself when it does not find
the element. Will save a lot of time!!! Cut down build
time by 7 mins for us!
37. Use scenario outlines with caution
Scenario outlines should be used only if
you actually need the whole scenario to be
repeated for your examples not otherwise.
38.
39.
40. When /^I fill in my billing details$/ do
And %{I fill in “first_name” with “John”}
And %{I fill in “zipcode” with “John”}
And %{I select “US” from “Country”}
End
When /^I fill in my billing details$/ do
fill_in(“first_name”, :with => “John”)
fill_in(“zipcode”, :with => “John”)
select(“US”, :from => “Country”)
End
Rather than dealing with regex matches deal with
capybara methods which are equally readable.
Introduce methods, modules, and keep your code dry
and clean.
When I started the Cucumber project in 2008 I had three major goals:Help stakeholders express what they want via executable specificationsHelp programmers write better software by making TDD/BDD easyReduce misinterpretations through a ubiquitous language
When I started the Cucumber project in 2008 I had three major goals:Help stakeholders express what they want via executable specificationsHelp programmers write better software by making TDD/BDD easyReduce misinterpretations through a ubiquitous language
small changes causing lot of cukes to failfailures causing long build timesSo much so that you seem to land up spending more time on the cukes rather than on the implementation
small changes causing lot of cukes to failfailures causing long build timesSo much so that you seem to land up spending more time on the cukes rather than on the implementation
Steps in different filesRedundant codeLess predictability
Steps in wrong filesOrganized readableFlexible
Dependencyoverriding
Dependencyoverriding
you can then specify the output location for your failures. And then run the failures in rerun profile. “A test is non-deterministic when it passes sometimes and fails sometimes, without any noticeable change in the code, tests, or environment. Such tests fail, then you re-run them and they pass. Test failures for such tests are seemingly random.” - Martin Fowler
you can then specify the output location for your failures. And then run the failures in rerun profile. “A test is non-deterministic when it passes sometimes and fails sometimes, without any noticeable change in the code, tests, or environment. Such tests fail, then you re-run them and they pass. Test failures for such tests are seemingly random.” - Martin Fowler
21 mins to 17 mins
21 mins to 17 mins
Reduced time for one feature from 7.5 minutes to 2.4 minutes
Reduced time for one feature from 7.5 minutes to 2.4 minutes
Parallel_tests, hydra, tlb
Use more of Capybara DSL than web_stepsUse of CSS locator strategy than xpathConsistency set of guidelines for teamUse proper taggingStub out third party services calls and have seperate integration testsTreat cucumber scenarios like your unit tests code.