Some thoughts about acceptance tests and the tools Pyccuracy and Splinter

1. Fernando Sandes
How to
improve our
acceptance tests !

2. What is
better?
Pyccuracy
vs

4. It`s not (only)
about
the technology

5. It`s an
ideologica
difference

6. @gabrielfalcao
Lettuce`s creator
The problem is that differently of
cucumber and lettuce, pyccuracy is
aimed to web testing, with pre-defined
"step definitions" (pyccuracy uses a
different nomenclature), and when I was
using it I found extremely hard to
maintain the tests as the test suite
grows, it's a hell!
lettuce

7. @gabrielfalcao
Lettuce`s creator
“So I wrote lettuce to make it super flexible
so that I could write my own definitions in a super
easy way, and never couple my tests to robotic
step definitions.”
lettuce

8. Pyccuracy
ROBOTIC
STEP
DEFINITIONS

9. ROBOTIC STEP DEFINITIONS
Given I fill the field "#username" with "test"
And I fill the field "#password" with "1234"
When I click on the button 'input[type="submit"]'
Then I see the page contains the text "Hello Test User!"
Given I am at the login page
And I fill in the login information with "test" and password "1234"
When I click on the login button
Then I see I'm logged in as a normal user

10. ROBOTIC STEP DEFINITIONS
Given I fill the field "#username" with "test"
And I fill the field "#password" with "1234"
When I click on the button 'input[type="submit"]'
Then I see the page contains the text "Hello Test User!"
“You can see how your
business it better defined here”
Given I am at the login page
And I fill in the login information with "test" and password "1234"
When I click on the login button
Then I see I'm logged in as a normal user

11. DONT`T BE
THE
BOILING
FROG !
The presence of step definitions leads people down a very
bad path.
While it gives people a head start in writing scenarios it
also tricks people into writing scenarios are extremely
verbose and fragile.
People don't always realize this, so they continue cranking
out lots of crap scenarios based on pre-steps. This is like
the boiling frog story

12. IT`S ABOUT THE DOMAIN !
The “domain” is defined by the value
the stakeholders
and users hope to achieve with the software.
This can be booking a ticket or sharing pictures with friends or
an infinite number of activities.
Clicking links and buttons or filling in text fields has nothing
(directly) to do with the domain.

13. Pyccuracy
IS BUROCRATIC !

14. a little more…

15. Imperative
VS
Declarative
Scenarios in User Stories

16. The imperative style uses highly reusable
Imperative granular steps which outlines
user interface.
much of the
This binds the scenario to that interface.
Scenario: Successful login
Given a user "Aslak" with password "xyz"
And I am on the login page
And I fill in "User name" with "Aslak"
And I fill in "Password" with "xyz"
When I press "Log in"
Then I should see "Welcome, Aslak“

17. The imperative style uses highly reusable
Imperative granular steps which outlines
user interface.
much of the
This binds the scenario to that interface.
Scenario: Successful login Scenario: User is greeted upon login
Given a user "Aslak" with password "xyz" Given the user "Aslak" has an account
And I am on the login page When he logs in
And I fill in "User name" with "Aslak" Then he should see "Welcome, Aslak"
And I fill in "Password" with "xyz"
When I press "Log in"
Then I should see "Welcome, Aslak“
Declarative

18. The imperative style uses highly reusable
Imperative granular steps which outlines
user interface.
much of the
This binds the scenario to that interface.
Scenario: Successful login Scenario: User is greeted upon login
Given a user "Aslak" with password "xyz" Given the user "Aslak" has an account
And I am on the login page When he logs in
And I fill in "User name" with "Aslak" Then he should see "Welcome, Aslak"
And I fill in "Password" with "xyz"
When I press "Log in"
Then I should see "Welcome, Aslak“
Declarative

19. Imperative Declarative
When we’re describing other parts of the system that
require login, the login details like user name, password
and how to log in are only distracting. That is why
we make it a one-liner.

20. Imperative Declarative

21. Imperative Declarative

22. COMMUNICATION
IS THE
GOAL!
To use a BDD tool "right", one has to understand
that the main goal is primarily about
communication and secondarily
about testing.

23. COMING BACK…
And about
technology

24. SPLINTER
Splinter was not created to be another acceptance
tool, but an abstract layer over other
tools, its goal is provide a unique API that make
acceptance testing easier and funnier :)

25. SPLINTER
A LOT OF OPTIONS !

26. HEADLESS OR NOT HEADLESS ?

27. HEADLESS
FAST
CHEAP
HARD TO DEBUG
IT`S NOT A BROWSER!

28. NOT HEADLESS
SLOW
EASY TO DEBUG
USE THE BROWSER

29. HEADLESS
NOT HEADLESS
SPLINTER

30. SERVER
LOCAL

31. SERVER
HEADLESS
LOCAL

32. NOT HEADLESS
SERVER
HEADLESS
LOCAL

33. SPLINTER
and more…
http status code
multi webdrivers
css ,xpath, tag, name selectors
support to iframe and alert
execute javascript
work's with ajax and async javascript

34. SPLINTER
Browser
• visit(‘url’)
• reload()
• quit()
• html
• url
• title

35. SPLINTER
Searching elements
• find_by_css(‘.classe’)
• find_by_id(‘meuid’)
• find_by_xpath(‘//body’)
• find_by_tag(‘a’)
• find_by_name(‘search’)

36. SPLINTER
Forms
• fill(‘name’, ‘value’)
• attach_file(‘name’, ‘/path/to/file.txt’)
• check(‘name’) #checkbox
• choose(‘name’, ‘value’) #radio button
• select(‘name’, ‘option’)

37. MORE
TESTS

38. SIKULI
http://sikuli.org/index.shtml

39. NEEDLE
http://needle.readthedocs.org
#AUTOMATED TESTS FOR CSS
from needle.cases import NeedleTestCase
class GoogleTest(NeedleTestCase):
def test_footer(self):
self.driver.get('http://www.google.com')
e = self.driver.find_element_by_id('fctr')
self.assertScreenshot(e, 'google-footer')
self.assertEqual(e.get_computed_property('font-size'), '13px')