Your first win is to fully rely on test automation. But then it becomes a problem by itself.

1. Taming the Testing Beast
Tuesday, October 23rd 2012
Daniel Doubrovkine (dB.)
@dblockdotorg
#agiledc 2012

2. State of Testing: Scary!
http://art.sy/artwork/francisco-de-goya-saturn-devouring-one-of-his-sons-from-the-series-of-black-paintings-dot

3. Test Automation @
3×
Independent Teams
Hardware Platforms
Vendors
QA
automated voting
user acceptance tests

4. Test Automation @
1124× Test Code vs. Code
http://www.sqlite.org/testing.html

5. Test Automation @ Your Company
1:1 Dev : Test
some automation
QA some people test
some software often
doesn’t install

6. Test Automation @ My Company
0
http://art.sy/artwork/conrad-shawcross-chord
Testers
Formalities
100% automation
QA continuous everything
8 developers
8 hours of sleep / day

7. It’s [Relatively] Big
4678 Automated Tests
100% user scenario
coverage ~ 800 tests
1h to Run http://art.sy/artwork/liu-di-animal-regulation-no-2

8. Technology and Tools
 RSpec BDD : it “does X”
 Mongo Data Models : ODM with “real” data
 Ruby API : HTTP request – response
 Integration : Capybara

9. Technology and Tools : Mongo ODM + Fabricators
context “spline” do
let(:spline) { Fabricate :spline }
it “reticulate! updates reticulated count” do
expect {
spline.reticulate!
}.to change(Spline.reticulated, :count).by(1)
end
end

10. Technology and Tools : Ruby API
context “spline” do
it “create” do
expect {
post “/api/spline?name=test”
response.status.should == 201
}.to change(Spline, :count).by(1)
end
end

11. Technology and Tools : Integration
context “spline” do
let(:spline) { Fabricate :spline }
it “displays a spline” do
visit “/spline/#{spline.id}”
page.should have_css “.spline[id=#{spline.id}]”
end
end

12. Mixed Bag
 High Confidence
 Size
 Duration
http://art.sy/artwork/liu-di-animal-regulation-no-4

13. It Quickly Became a Big Problem
 Intermittent Failures
 Size
 Duration
http://art.sy/artwork/liu-di-animal-regulation-no-6

14. Break Things Up
 Multiple Applications

15. Run Tests in Parallel
 Multiple Test Suites
http://artsy.github.com/blog/2012/10/09/how-to-run-rspec-test-suites-in-parallel-with-jenkins-ci-build-flow/

16. http://artsy.github.com/blog/2012/02/03/reliably-testing-asynchronous-ui-w-slash-rspec-and-capybara/
Understand
 Disable Failing Tests
 One Example at a Time
 No Hacks Allowed
 Wait for Explicit Events
 Combine Techniques
 Track Failures
http://www.friendshipismagic.org/forum/members/captain-falcon/albums/miscellaneous/9452-wat/

17. Test Anxiety
 High Expectations
 Fear of Disappointment
 Lack of Control
 Inability to Change
http://www.sodahead.com/fun/is-your-school-strict/question-3158781/

18. Test Automation is Not a ______ Problem
 Not a Management Problem
 Not a Money Problem
 Not an External Problem
 Not a Consultant’s Problem

19. Test Automation is a Team Culture Problem
 Remove Those that Talk About Testing
 Hire From Open-Source
 Remove the Safety Net

20. Daniel Doubrovkine (dB.)
@dblockdotorg