A talk about Robot Framework, an automated acceptance test framework. Held at SAST Väst Q3 meetup, Lindhomen Sicend park, Göteborg.

1. Robot Framework
A tool for automated acceptance testing

2. Me, My self and I
Anders Arnholm
• Umeå Universitet
• Open Software ▶ HiQ
• Mecel ▶ Delphi
• HiQ

3. Acceptance testing
formal testing with respect to user needs, requirements, and business
processes conducted to determine whether or not a system satisfies the
acceptance criteria and to enable the user, customers or other authorized
entity to determine whether or not to accept the system.
Standard glossary of terms used in Software Testing, Version 2.1. ISTQB. 2010.

4. Automated
the technique, method, or system of operating or controlling a process by
highly automatic means, as by electronic devices, reducing human
intervention to a minimum
dictionary.com

5. Acceptance test driven development
• A way of running Agile Development Project
• Inspired by Test Driven Development
• Product Owner + Tester + Developer refines a story
into an automated acceptance test
• Development until the test passes

6. What is Robot Framework
• Test automation framework
• Implemented in python
• Open source
• Keyword-driven syntax

7. Architecture
Test Data
Robot Framework
Test Libraries
Test Tools
System under test

8. Demo time

9. Keyword driven test
*** Test Cases ***
User can log in with a valid password
Attempt to Login with Credentials test@example.com P4ssw0rd
Should Be Logged In test@example.com
User cannot log in with bad password
Attempt to Login with Credentials test@example.com wrong
Error Should Be Incorrect password

10. Gherkin Style
*** Test Cases ***
The count should increase
Given a logged in user
And the click counter is 0
When the user clicks the button
Then the click counter should be 1

11. Data driven tests
*** Test Cases ***
Invalid email
[Template] Creating a user with invalid email should fail
test_user Invalid email
a+b@:w Invalid email
*** Keywords ***
Creating a user with invalid email should fail
[Arguments] ${username} ${errormessage}
Attempt to create user with Credentials ${username} 12345
Error should be ${errormessage}

12. Custom Keywords
*** Keywords ***
Should Be Logged In
[Arguments] ${username}
Element Should Contain id=login-name-link ${username}
Error Should Be
[Arguments] ${errormessage}
[Timeout] 2 minutes
Element Should Contain id=login-dropdown-list ${errormessage}

13. Extensible libraries
#!/usr/bin/env python
import json
import time
def should_have_error_message(self, jsondata):
print '*INFO:%d* JSON: %s' % (time.time()*1000, jsondata)
questions = json.loads(jsondata)
if not u'errorMessage' in questions:
raise Exception(u'No error message in answer')

14. Variables
*** Variables ***
${SERVER} localhost:3000
${BROWSER} chrome
${DELAY} 0.1
${pageurl} http://${SERVER}/
*** Test Case ***
My variable example
${result}= Get page counter
Greater Than 30 ${pagecounter}
pybot --variable BROWSER:safari mytest.robot

15. Test tagging
*** Settings ***
Force Tags sast
Default Tags HiQ Gothenburg
*** Test Cases ***
Page have greeting
[Tags] smoke critical
Page should contain Welcome
Page have copyright
[Tags] smoke optional
Page should contain ©2015
Page have last
Page should contain SAST

16. Reports

17. Logs

18. Existing libraries
• Standard Libraries
• OperatingSystem, Screenshot, String, Dialogs, XML, …
• External Libraries
• Selenium, Appium, SSH, Database,
• Project/Team specific
• Easy to make your own

19. Test data formats
Also known as test file formats
• HTML (.html, .htm, .xhtml)
• TSV - tab separated values (.tsv)
• plain text (.txt, .robot)
• reStructured text (.rst, .rest)

20. Editors
• RIDE
• Plugins for IntelliJ, Eclipse, Vim, Emacs, SubLime,
TextMate…
• Spreadsheet applications, Excel, Numbers or Calc

21. Easy Integration
• Test suits from files and directories
• Supports any versioning system, e.g. use git
• Simple command line interface
• External tools and scripts can start tests
• Output also to XML
• Machine readable
• Easy to combine
• Plugins for common CI tools
• Jenkins, Ant, Maven

22. The end
robotframework.org
anders.arnholm@hiq.se
www.hiq.se

23. • robotframework.org
• anders.arnholm@hiq.se
• www.hiq.se