Acceptance tests are high-level tests to verify the completeness of user stories (end user features) for a developed product. It is said that while lower level unit and integration tests confirm that the product has been made right, only the acceptance tests confirm that the product actually does the right thing.
Robot Framework is a generic test automation framework for acceptance testing and acceptance test-driven development. The key feature of Robot Framework is its extensible test syntax, which allows to write tests in a business-readable domain-specific language, including BDD (behavior driven development) style for executable acceptance criteria.
Selenium is a browser automation framework. It allows to use real browsers in acceptance testing so that all effects of CSS and JavaScript taken into account, including possible differences between different browsers.
This talk walks through alternative ways for using Robot Framework and Selenium to perform acceptance testing, including
* testing live sites (readonly)
* testing sandboxed Plone sites with live data (beforestorage + demostorage)
* testing sandboxed Plone add-ons (plone.app.testing + plone.app.robotframework)
* testing Sphinx documentation (while generating screenshots)
Standard vs Custom Battery Packs - Decoding the Power Play
Acceptance testing plone sites and add ons with robot framework and selenium
1. Acceptance testing!
Plone sites and add-ons
with Robot Framework!
and Selenium
Asko Soukka, asko.soukka@iki.fi
University of Jyväskylä, Finland
2. Hello!
+ Asko Soukka
Twitter: @datakurre
IRC: datakurre
Email: asko.soukka@iki.fi
Web: datakurre.pandala.org
+ Software developer at
University of Jyväskylä
Finland
+ Plone Foundation Member
3. Menu
Terminology!
+ Acceptance testing
+ Robot framework
+ Selenium
!
Acceptance testing Plone!
+ Just do it with a live instance
+ Do it safely with a dedicated instance
+ Do it correctly in a p.a.testing sandbox
+ Do it with while writing your
documentation with screenshots
!
4. Acceptance testing
Functional testing of user stories!
!
!
Scenario: As an editor I can translate a document
Given a site owner
and a document in English
When I translate the document into Catalan
Then I can view the document in Catalan
6. Robot Framework
!
Keyword driven testing in your own
domain specific language!
!
Extend with Python, Java or XML-RPC
$ pip install robotframework
$ pybot —-help
$ pybot my_test_suite.robot
7. Robot Framework
*** Settings ***
!
Library string WITH NAME string
!
*** Test Cases ***
!
Convert string to uppercase
${value} = string.upper twist and shout!
Should be equal ${value} TWIST AND SHOUT!
8. !
*** Settings ***
!
Library ...
Resource ...
!
*** Variables ***
${NAME} value
!
*** Test Cases ***
Scenario title
Some keyword argument ${NAME}
!
*** Keywords ***
!
Some keyword
[Arguments] ${arg1} ${arg2}
Another keyword ${arg1} ${arg2}
!
!
test_suite.robot!
9.
10.
11. Selenium
Browser automation framework!
!
All browser and service logos and trademarks on this slide are property of their respective owners.
13. Selenium2Library
*** Settings ***
!
Library Selenium2Library
!
*** Test Cases ***
!
Front-page should contain search box
Open browser https://www.jyu.fi/
Element should be visible
... css=form[name='searchform'] input
[Teardown] Close all browsers
!
14.
15. Testing Plone with live data
[buildout]
parts += testinstance
!
[testinstance]
recipe = plone.recipe.zope2instance
...
eggs =
...
zc.beforestorage
Products.PrintingMailHost
...
before-storage = now
demo-storage = true
shared-blob = false
16. Testing Plone add-ons
*** Settings ***
!
Resource plone/app/robotframework/selenium.robot
Library Remote ${PLONE_URL}/RobotRemote
!
*** Test Cases ***
!
Scenario: Powered by Plone remains visible
Open test browser
Go to ${PLONE_URL}
Page should contain Powered by Plone
[Teardown] Close all browsers
17. Testing Plone add-ons
import unittest
!
import robotsuite
from myproduct.testing import MYPRODUCT_ROBOT_TESTING
from plone.testing import layered
!
!
def test_suite():
suite = unittest.TestSuite()
suite.addTests([layered(
robotsuite.RobotTestSuite(’test_hello.robot'),
layer=MYPRODUCT_ROBOT_TESTING)])
return suite
18. Testing Plone add-ons
from plone.testing import z2
from plone.app.testing import FunctionalTesting
from plone.app.robotframework.testing import
REMOTE_LIBRARY_BUNDLE_FIXTURE
!
MYPRODUCT_ROBOT_TESTING = FunctionalTesting(
bases=(MYPRODUCT_FIXTURE,
REMOTE_LIBRARY_BUNDLE_FIXTURE,
z2.ZSERVER_FIXTURE),
name="MyProduct:Robot")
19. Testing Plone add-ons
from plone.testing import z2
from plone.app.testing import FunctionalTesting
from plone.app.robotframework.testing import
REMOTE_LIBRARY_BUNDLE_FIXTURE
!
MYPRODUCT_ROBOT_TESTING = FunctionalTesting(
bases=(MYPRODUCT_FIXTURE,
REMOTE_LIBRARY_BUNDLE_FIXTURE,
z2.ZSERVER_FIXTURE),
name="MyProduct:Robot")
21. To BDD or not to…
...
Scenario: As an editor I can translate a document
Given a site owner
and a document in English
When I translate the document into Catalan
Then I can view the document in Catalan
!
*** Keywords ***
!
A site owner
Enable autologin as Site Administrator
...
23. Sphinx integration
extensions = ['sphinxcontrib_robotframework']
.. code:: robotframework
!
*** Settings ***
!
Library string WITH NAME string
!
*** Test Cases ***
!
Convert string to uppercase
...
24.
25. $ pip install Sphinx
$ pip install sphinxcontrib-robotframework
$ pip install robotframework-selenium2library
.. figure:: ploneconf.png
.. code:: robotframework
!
*** Settings ***
!
Library Selenium2Library
!
*** Test Cases ***
!
Show Plone Conference 2014 website
Open browser http://2014.ploneconf.org/
Capture page screenshot ploneconf.png
[Teardown] Close all browsers
26.
27. !
.. figure:: portal-footer.png
.. code:: robotframework
!
*** Settings ***
!
Resource plone/app/robotframework/server.robot
Resource Selenium2Screenshots/keywords.robot
!
*** Test Cases ***
!
Scenario: Powered by Plone remains visible
Setup Plone site
... plone.app.testing.PLONE_ZSERVER
Go to ${PLONE_URL}
Page should contain Powered by Plone
Capture and crop page screenshot
... portal-footer.png id=portal-footer
[Teardown] Run keywords
... Tear down Plone site
... Close all browsers
!
28.
29.
30. More
+ Robot Framework
http://robotframework.org/
+ Selenium2Library (for Robot Framework)
https://github.com/rtomac/robotframework-selenium2library
+ Overview of Plone-related Robot ecosystem
http://datakurre.pandala.org/2013/09/meet-robot-family-
for-plone-developers.html