The document discusses advanced testing techniques, including:
1. The speaker shares experiences with testing and encourages thinking critically about testing like an advanced tester. Unit testing with RSpec and integration testing with Cucumber are covered.
2. Examples of unit testing, boundary value analysis, isolation, mocks/stubs, and fixtures are provided. The benefits of testing, such as isolating units of code and finding edge cases, are discussed.
3. Integration testing with Cucumber is recommended to test the full stack of models, views, and controllers in a behavior-driven manner. An example Cucumber scenario tests account creation on a homepage.
2. Goals
• Share our experiences about testing
• Make you start thinking critical about testing
like an advanced tester
• Unit testing with RSpec
• Integration testing with Cucumber
6. • Web application written in Ruby on Rails
• Create and send invoices online
• More than 16.000 invoices sent
• Short development cycle: 2 months
• API for web developers
• More info: www.moneybird.com
7.
8. “A donkey does not knock
himself to the same stone twice”
9. It is okay to write code that doesn’t work...
But never ‘cap deploy’ it!
12. Software testing
... is a technical process performed by executing
a product in a controlled environment, following
a specified procedure, with the intent of
measuring the quality of the software product by
demonstrating deviations of the requirements.
13. Unit testing
• Test individual units of code
• A unit is the smallest testable piece of
code: methods of classes
• E.g. methods of controllers and models
15. TDD / BDD
Test before you write a single piece of code
Can be a nice approach, but requires a lot of
discipline and might not fit your needs,
but know how it can help you!
16. Example
Recurring invoices
Calculate the next occurence of a recurring invoice.
Start date Current occurence Next occurence
Jan Febr Mar Apr May
18. RSpec-t
describe "Date calculation" do
it "should return the next month" do
next_occurence(Date.parse('2009-01-01')).should ==
Date.parse('2009-02-01')
end
end
You ain’t testing this,
because it looks too easy
20. Customer: “I want to bill every
last day of the month”
‘2009-01-31’ + 1.month = ‘2009-02-28’
‘2009-02-28’ + 1.month = ‘2009-03-28’
‘2009-01-31’ + 2.months = ‘2009-03-31’
22. Boundary Value Analysis
Find the boundaries of the input and test them
Start Current Next
2009-01-31 2009-01-31 2009-02-28
2009-01-31 2009-02-28 2009-03-31
2009-01-31 2009-03-31 2009-04-30
2009-01-30 2009-01-30 2009-02-28
2009-01-30 2009-02-28 2009-03-30
2009-01-30 2009-03-30 2009-04-30
2009-01-29 2009-01-29 2009-02-28
2009-01-29 2009-02-28 2009-03-29
2009-01-29 2009-03-29 2009-04-29
2009-01-28 2009-01-28 2009-02-28
2009-01-28 2009-02-28 2009-03-28
2009-01-28 2009-03-28 2009-04-28
23. Boundary Value Analysis
Find the boundaries of the input and test them
Start Current Next
2009-01-31 2009-01-31 2009-02-28
2009-01-31 2009-02-28 2009-03-31
2009-01-31 2009-03-31 2009-04-30
2009-01-30 2009-01-30 2009-02-28
2009-01-30 2009-02-28 2009-03-30
2009-01-30 2009-03-30 2009-04-30
2009-01-29 2009-01-29 2009-02-28
2009-01-29 2009-02-28 2009-03-29
2009-01-29 2009-03-29 2009-04-29
2009-01-28 2009-01-28 2009-02-28
2009-01-28 2009-02-28 2009-03-28
2009-01-28 2009-03-28 2009-04-28
24. Boundary Value Analysis
Find the boundaries of the input and test them
Start Current Next
2009-01-31 2009-01-31 2009-02-28
2009-01-31 2009-02-28 2009-03-31
2009-01-31 2009-03-31 2009-04-30
2009-01-30 2009-01-30 2009-02-28
2009-01-30 2009-02-28 2009-03-30
2009-01-30 2009-03-30 2009-04-30
2009-01-29 2009-01-29 2009-02-28
2009-01-29 2009-02-28 2009-03-29
2009-01-29 2009-03-29 2009-04-29
2009-01-28 2009-01-28 2009-02-28
2009-01-28 2009-02-28 2009-03-28
2009-01-28 2009-03-28 2009-04-28
25. Boundary Value Analysis
Find the boundaries of the input and test them
Start Current Next
2009-01-31 2009-01-31 2009-02-28
2009-01-31 2009-02-28 2009-03-31
2009-01-31 2009-03-31 2009-04-30
2009-01-30 2009-01-30 2009-02-28
2009-01-30 2009-02-28 2009-03-30
2009-01-30 2009-03-30 2009-04-30
2009-01-29 2009-01-29 2009-02-28
2009-01-29 2009-02-28 2009-03-29
2009-01-29 2009-03-29 2009-04-29
2009-01-28 2009-01-28 2009-02-28
2009-01-28 2009-02-28 2009-03-28
2009-01-28 2009-03-28 2009-04-28
26. Create tests for all boundary values...
...and improve your code!
27. The solution
def self.calculate_next_occurence(start_date, current_date)
next_date = current_date + 1.month
while start_date.day != next_date.day and
next_date != next_date.end_of_month
next_date = next_date.tomorrow
end
next_date
end
32. Isolate for productivity
• Test isolated units of code, assuring they are working
• No need to bother about it later
• No need to find all edge cases in the user interface:
F5 syndrom
33. Isolate your tests
• Don’t test code that is tested elsewhere
• Only make sure it is used well in the code you’re testing
• The solution: mocks and stubs
34. Mock objects are simulated objects that mimic
the behavior of real objects in controlled ways
A stub is a piece of code used to stand in
for some other programming functionality
41. How to isolate?
def create_invoice(contact)
invoice = Invoice.new
invoice.contact_id = contact.id
invoice.details_attributes =
[{ :description => "RER09", :price => 79 }]
invoice.save
end
class Invoice < ActiveResource::Base
self.site = "https://account.moneybird.com"
end
42. How to isolate?
def create_invoice(contact)
invoice = Invoice.new
invoice.contact_id = contact.id
invoice.details_attributes =
[{ :description => "RER09", :price => 79 }]
invoice.save
end
class Invoice < ActiveResource::Base
self.site = "https://account.moneybird.com"
end
ActiveResource is tested elsewhere +
very slow to test
43. describe "MoneyBird API" do
it "should create the invoice" do
create_invoice(contact_mock).should be_true
end
end
44. describe "MoneyBird API" do
it "should create the invoice" do
contact_mock = mock(:contact)
contact_mock.should_receive(:id).and_return(3)
create_invoice(contact_mock).should be_true
end
end
45. describe "MoneyBird API" do
it "should create the invoice" do
Invoice.should_receive(:new).and_return(invoice_mock)
contact_mock = mock(:contact)
contact_mock.should_receive(:id).and_return(3)
create_invoice(contact_mock).should be_true
end
end
46. describe "MoneyBird API" do
it "should create the invoice" do
invoice_mock = mock(:invoice)
invoice_mock.stub!(:contact_id=)
invoice_mock.stub!(:details_attributes=)
invoice_mock.should_receive(:save).and_return(true)
Invoice.should_receive(:new).and_return(invoice_mock)
contact_mock = mock(:contact)
contact_mock.should_receive(:id).and_return(3)
create_invoice(contact_mock).should be_true
end
end
49. Isolation
Isolated A B
Not isolated A B
Double
=
Extra test work +
maintenance work!
50. What to test?
Everything!
But never too much!! Be critical
51. Downside of testing
• Every test gives overhead
• Testing the obvious takes time
• Ruby library, CRUD controllers
52. Downside of testing
• Every test gives overhead
• Testing the obvious takes time
• Ruby library, CRUD controllers
But what if suddenly:
BigDecimal("10.03").to_f != 10.03
54. RSpec
• All business logic in models
• Models tested with RSpec
• Basic data model in fixtures
55. Fixtures?
• Models make use of the database
• Sometimes data in de database is needed
for the model to work
• Fixtures are easy to manage on the
filesystem (+ version control)
• Fixtures can be used for bootstrapping
application: rake db:fixtures:load
56. first_contact:
company: bluetools
name: Edwins company
contact_name: Edwin Vlieg
address1: Street 82
zipcode: 7541 XA
city: Enschede
country: NL
send_method: mail
created_at: <%= 60.days.ago.to_s :db %>
contact_hash: 1
second_contact:
company: bluetools
name: BlueTools B.V.
address1: Postbus 123
zipcode: 7500EA
city: Enschede
country: NL
send_method: post
contact_hash: 2
57. first_contact:
company: bluetools
Unique identifier for
name: Edwins company ‘row’ in database.
contact_name: Edwin Vlieg Don’t set the id column!!
address1: Street 82
zipcode: 7541 XA
city: Enschede
country: NL
send_method: mail
created_at: <%= 60.days.ago.to_s :db %>
contact_hash: 1
second_contact:
company: bluetools
name: BlueTools B.V.
address1: Postbus 123
zipcode: 7500EA
city: Enschede
country: NL
send_method: post
contact_hash: 2
58. first_contact:
company: bluetools
Unique identifier for
name: Edwins company ‘row’ in database.
contact_name: Edwin Vlieg Don’t set the id column!!
address1: Street 82
zipcode: 7541 XA
city: Enschede
country: NL
send_method: mail
created_at: <%= 60.days.ago.to_s :db %>
contact_hash: 1
second_contact:
company: bluetools
name: BlueTools B.V. Yes, you can use Ruby!
address1: Postbus 123
zipcode: 7500EA
city: Enschede
country: NL
send_method: post
contact_hash: 2
59. first_contact:
company: bluetools
Unique identifier for
name: Edwins company ‘row’ in database.
contact_name: Edwin Vlieg Don’t set the id column!!
address1: Street 82
zipcode: 7541 XA
city: Enschede
country: NL
send_method: mail
created_at: <%= 60.days.ago.to_s :db %>
contact_hash: 1
second_contact:
company: bluetools
name: BlueTools B.V. Yes, you can use Ruby!
address1: Postbus 123
zipcode: 7500EA
city: Enschede
country: NL
send_method: post
contact_hash: 2 Reference to unique identifier
in companies table
67. Feature: Homepage
Visitors at the homepage should get clear information about
our product and be able to create an account
Scenario: Create a new free account
When I visit the homepage
And I follow "pricing"
And I follow "Signup"
And I fill in the following:
| Company name | Test company |
| Your fullname | Edwin Vlieg |
| E-mail | test@test.com |
| company_domain | testcompany |
| Username | Edwin |
| Password | testtest |
| Password confirmation | testtest |
And I press "Create your account"
Then a company with name "Test company" should exist
And an e-mail with subject "Welcome to MoneyBird" should have been sent
And I should see "Thanks for signing up!"
68. Cucumber stories
• Each line is parsed and matched on a step
• Step contains actual execution of code
• 3 types of steps: Given, When and Then.
69. homepage.feature
When I follow "Signup"
webrat_steps.rb
When /^I follow "([^"]*)"$/ do |link|
click_link(link)
end
72. Webrat
• Webrat gives you control over the user
interface (almost) like an end user has
• But doesn’t emulate a browser like
Selenium or Watir (= no JavaScript)
73. Web-what?
Web browser
Apache
Mongrel / Passenger Webrat
ActionController
74. When I follow "Signup"
1. Locate the link on the page
<a href="/signup">Signup</a>
<a href="/signup" title="Signup"><img src="..." /></a>
<a href="/signup" id="Signup" title="Click to signup"><img src="..."></a>
75. When I follow "Signup"
1. Locate the link on the page
<a href="/signup">Signup</a>
<a href="/signup" title="Signup"><img src="..." /></a>
<a href="/signup" id="Signup" title="Click to signup"><img src="..."></a>
76. When I follow "Signup"
1. Locate the link on the page
<a href="/signup">Signup</a>
<a href="/signup" title="Signup"><img src="..." /></a>
<a href="/signup" id="Signup" title="Click to signup"><img src="..."></a>
2. ‘Click’ the link
Grab the ‘href’ attribute from the element and feed it to the
Rails application. HTML is replaced: next search of element
will be in new page.
77. Cucumber & Webrat
Cucumber comes with default Webrat steps to:
• Visit pages
• Click links
• Fill in forms
• Submit forms
• Assert the content of the page
79. Testability
Webrat doesn’t execute JavaScript, so write
unobtrusive JavaScript to keep it testable.
HTML
<a href="/popup.html" id="open">Open popup</a>
Link opens popup, even without JavaScript
80. Testability
Webrat doesn’t execute JavaScript, so write
unobtrusive JavaScript to keep it testable.
HTML
<a href="/popup.html" id="open">Open popup</a>
Link opens popup, even without JavaScript
JavaScript
$('open').click(...);
JavaScript adds extra behaviour to link
81. Testability
Webrat doesn’t execute JavaScript, so write
unobtrusive JavaScript to keep it testable.
HTML
<a href="/popup.html" id="open">Open popup</a>
Link opens popup, even without JavaScript
JavaScript
$('open').click(...);
Rails 3
JavaScript adds extra behaviour to link
82. Common steps
Database steps:
Given I've created an invoice with invoice id "2008-0100"
Then a contact with name "Test company" should exist
Mailer steps:
Then an e-mail with subject "Invoice 2008-0100 from
Bluetools" should have been sent
Full source: http://pastie.org/667777
83. Database steps
Given I've created an invoice with invoice id "2008-0100"
Should create a new invoice with invoice id
“2008-0100”, but what about the rest of the attributes?
84. FactoryGirl
• Makes it easy to create records in the
database
• Define default attributes for a record:
Factory.define :contact do |c|
c.company_id { |c| Company.find_by_domain($subdomain).id }
c.name "Test company"
c.contact_name "Edwin Vlieg"
c.address1 "Hengelosestraat 538"
c.zipcode "7500AG"
c.city "Enschede"
c.country "NLD"
c.email "test@moneybird.nl"
end
85. FactoryGirl
• Easy instantiation of records in database:
Factory(:contact, :name => "Foobar")
Factory name Override default
attributes
More information:
http://github.com/thoughtbot/factory_girl
86. Downside
When I follow "Signup"
What if we want to change “Signup”
to “Create account”?
Test breaks, but application is functionally correct.
89. Don’t stare blindly at TDD or BDD
Creating mesmerizing products needs creativity,
this doesn’t always fit into the ‘test-first’ approach,
but know the ideas behind the approaches,
so you can use them when needed!
90. Creating a good test environment takes time:
it just doesn’t end with RSpec or Cucumber
Mocks, stubs, fixtures and factories are your friends!