Budapest Ruby Meetup - 11/09/2013

1. 7 Years In Testing
Tamás Tompa (Digital Natives)
tamas.tompa@digitalnatives.hu
Budapest Ruby Meetup 2013
11/09/2013
Thursday, September 12, 13

2. § 7 years in ruby, rails
§ 7 years in webapp testing
§ 5 years in agile development
§ 5 years in automated testing
Thursday, September 12, 13

3. Looking for testing
best practices
Thursday, September 12, 13

4. Short version
INTEGRATE
AUTOMATE
Thursday, September 12, 13

5. Short version
INTEGRATE
AUTOMATE
(everything)
Thursday, September 12, 13

6. Human VS Machine
§ Use humans for thinking
§ Use machines for executing
§ Humans can forgive, forget things
§ Machines are strict, consequent
Thursday, September 12, 13

7. Theory
Thursday, September 12, 13

8. Development Workflow
Resolved Deployed
Testing /
Working
Testing /
Not
working
In
progress Stable
for feature based development
Thursday, September 12, 13

9. Automating steps
Resolved Deployed
Testing /
Working
Testing /
Not
working
In
progress Stable
Manual Automatic Manual Automatic
Thursday, September 12, 13

10. Automated Testing
§ rspec
§ turnip
§ cucumber
§ capybara
§ selenium -> phantomjs
§ factory girl
§ jenkins / cloud CI
TOOLS
Thursday, September 12, 13

11. Working, bug free software is
priceless...
Thursday, September 12, 13

12. Working, bug free software is
more expensive...
Thursday, September 12, 13

13. ... but only if you don’t know
how to test
and maintain your code
Thursday, September 12, 13

14. Write tests for those parts of the
code which are...
§ the most often used
§ changing frequently
§ changing data and working with
financial data
§ more likely to fail
Thursday, September 12, 13

15. Instrumentation
§ Instrumentation means that the code gets enhanced with
flags, and when the execution passes a certain flag, it
gets set. When a test code coverage measurement is
made then the tool:
§ instruments the code - places these flags
§ runs the test cases - flags get set
§ and finally prints out the result - how many flags have
been set
§ During usage coverage measurement, test execution is
replaced by customer interaction.
Thursday, September 12, 13

16. Most often used code
§ Instrument the code base and deliver
it to the testers / customer
§ Turn on coverage measurement in your
(staging) application
§ Deploy it
§ Do manual testing of critical functions
§ Summarize and analyze metrics
§ Write tests for highly covered classes
Thursday, September 12, 13

17. Frequently changing code
§ Use version control system logs
§ Gitinspector
§ StatSVN
§ Find out which files change often!
Thursday, September 12, 13

18. Working with data
§ Make code reviews
§ Re-prioritize classes and methods
§ Add more priority when data is critical
§ Move to top when working with money
Thursday, September 12, 13

19. Failing, problematic code
§ Find and analyze bugs in your Issue
Tracker System
§ Write test before fixing the issue
§ Review CI server logs
§ collect often failing tests
§ Refactor unstable code
§ refactor your tests also
Thursday, September 12, 13

20. Git branching
§ Work only with feature branches
§ Test the branches separately
§ Use Pull Requests to integrate
feature branches to a development
branch
§ Do more (manual) testing on
development branch
§ Merge easily to stable,
than finally to production
Thursday, September 12, 13

21. Methods
§ TDD - it’s up to you :)
§ BDD - based on TDD + domain
specific acceptance criterias
§ CDD - for prototypes, spikes
Thursday, September 12, 13

22. In practice...
Thursday, September 12, 13

23. Manual Testing
§ Plan your test cases
§ at Sprint Planning Meetings
§ Allocate a good Tester to the team
§ If you can’t, test all User Story after
completion, within team, but between
team members
§ Try to measure coverage during
manual testing
Thursday, September 12, 13

24. Continuous Integration
§ Jenkins server
§ Cloud alternatives: travis-ci, circleci, drone.io
§ Parallel job execution (queues / executors)
§ Main branches VS Feature branches
§ Instant builds VS Daily builds
§ Conflicting databases and shared resources
§ CI rake task in the project
§ Re-run logic for Cucumber tests
§ Long running jobs
Thursday, September 12, 13

25. CI - Ruby, Rails
§ Jenkins / RVM Plugin
§ RVM-managed environment:
ruby-1.9.3-p392@PROJECTNAME
§ Jenkins / Ruby Metrics Plugin
§ Rcov coverage report
§ Other stats
§ Bundler integration
Thursday, September 12, 13

26. Thursday, September 12, 13

27. CI - Feature branches
§ Jenkins / Git Plugin
§ Branches to build:
^(?!.*master|stable|development|origin/HEAD).*$
§ Jenkins / Version Number Plugin
§ Version Number Format String:
#${BUILD_NUMBER}-${GIT_BRANCH}
Thursday, September 12, 13

28. CI - Build types
§ Instant builds
§ Higher priority
§ Immediately after commit
§ Build trigger: Poll SCM
§ Daily builds
§ Once a day
§ Long running tests
§ Build trigger: Build periodically
Thursday, September 12, 13

29. CI - Rake
task :commit => :clean do
Rake::Task['db:migrate'].invoke
Rake::Task['db:test:load'].invoke
Rake::Task['db:test:purge'].invoke
Rake::Task['ci:build:rspec_internal'].invoke
Rake::Task['ci:build:cucumber_internal'].invoke
end
rake ci:build:commit
> Example: https://gist.github.com/tompata/6513021
Thursday, September 12, 13

30. CI - Rerun
§ Re-run only previously failing tests
§ Re-try the randomly failing tests
§ Built-in support in Cucumber
§ Easy integration with Jenkins
§ Save resources and time
> Example: https://gist.github.com/tompata/6513060
Thursday, September 12, 13

31. CI - Integration
§ Bitbucket
§ Redmine
§ Notifiers (HipChat, Email, Desktop)
§ Deployment (Capistrano)
§ Monitoring (Airbrake, New Relic)
Thursday, September 12, 13

32. Thursday, September 12, 13

33. Thursday, September 12, 13

34. Thursday, September 12, 13

35. Thursday, September 12, 13

36. Thursday, September 12, 13

37. Browser tests
§ integration tests
§ cucumber / capybara
§ responsive design
§ phantomjs / screen sizes
§ cross browser (design)
§ mtest (npm) - multiplatform
§ saucelabs / cloud
§ 162 device/os/browser platforms
Thursday, September 12, 13

38. Responsive design
§ PhantomJS
§ Screen resolutions (window_size)
§ Cucumber profiles
> Example: https://gist.github.com/tompata/6512905
Thursday, September 12, 13

39. Stress tests
§ penetration / load testing before
releases
§ use separated, dedicated
environments (staging test)
§ use virtual instances (ec2)
§ monitor instances (new relic)
§ use online tools (loadimpact,
loader.io) or local (jmeter)
Thursday, September 12, 13

40. Coverage, Quality
§ rcov
§ simple cov
§ the magical 80%
§ code climate - code analysis, quality
Thursday, September 12, 13

41. INTEGRATE
AUTOMATE
Thursday, September 12, 13

42. Resources
§ http://zsoltfabok.com/blog/2011/05/narrow-down-what-to-test/
§ http://nvie.com/posts/a-successful-git-branching-model/
§ http://jenkins-ci.org/
§ https://wiki.jenkins-ci.org/display/JENKINS/Ruby+Metrics+Plugin
§ https://saucelabs.com/
§ https://github.com/digitalnatives/
§ https://github.com/digitalnatives/mtest
§ https://gist.github.com/tompata/6513021
§ https://gist.github.com/tompata/6513060
§ https://gist.github.com/tompata/6512905
§ https://codeclimate.com/
§ http://loadimpact.com/
Thursday, September 12, 13

43. We’re hiring!
http://digitalnatives.hu/jobs
Thursday, September 12, 13