In this session the current automated testing infrastructure of the Joomla CMS is explained: theory of system- & unit-tests Joomla's unit testing setup setting up your machine how to execute the test suite how to write a test Joomla's system testing setup setting up your machine how to execute the test suite how Joomla automates the automated tests

1. (Unit-)Testing for Joomla
Because breaking stuff sucks.
David Jardin
Automated Testing Working Group
11 April 2015

2. Image: © Raimond Spekking / Wikimedia Commons

5. 3.2.6
3.3.5
2.5.26
3.3.3
2.5.24
2.5.22 3.1.4

7. THEORY OF TESTING
01

8. Image: © Tadekptaku / Wikimedia Commons

9. A unit test is an automated piece of code that
invokes a unit of work in the system and then
checks a single assumption about the
behavior of that unit of work.
UNIT TEST

10. Image: © KMJ / Wikimedia Commons

11. System testing is testing conducted on a
complete, integrated system to evaluate the
system's compliance with its specified
requirements.
SYSTEM TEST

12. Image: © Evo Flash / Wikimedia Commons

13. Joomla’s unit testing setup
02

14. PHPUnit

17. phpunit.xml
(Unit-)Testing for Joomla David Jardin
<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="tests/unit/bootstrap.php" colors=„false">
<!-- These constants help setup environment configurations for running
optional tests.
<php>
<const name="JTEST_DATABASE_MYSQL_DSN"
value="host=localhost;dbname=joomla_ut;user=utuser;pass=ut1234" />
<const name="JTEST_DATABASE_MYSQLI_DSN"
value="host=localhost;dbname=joomla_ut;user=utuser;pass=ut1234" />
<const name="JTEST_DATABASE_PDO_MYSQL_DSN"
value="host=localhost;dbname=joomla_ut;user=utuser;pass=ut1234" />
<const name="JTEST_DATABASE_POSTGRESQL_DSN"
value="host=localhost;port=5432;dbname=joomla_ut;user=utuser;pass=ut1234" />
<const name="JTEST_DATABASE_SQLSRV_DSN"
value="host=localhost;dbname=joomla_ut;user=utuser;pass=ut1234" />
<const name="JTEST_HTTP_STUB" value="http://localhost/joomla-cms/
tests/unit/stubs/jhttp_stub.php" />
</php>
-->

18. phpunit.xml
(Unit-)Testing for Joomla David Jardin
<testsuites>
<testsuite name="libraries-cms">
<directory>tests/unit/suites/libraries/cms</directory>
</testsuite>
<testsuite name="libraries-platform">
<directory>tests/unit/suites/libraries/joomla</directory>
</testsuite>
<testsuite name="libraries-legacy">
<directory>tests/unit/suites/libraries/legacy</directory>
</testsuite>
<testsuite name="database">
<directory>tests/unit/suites/database</directory>
</testsuite>
<testsuite name="administrator">
<directory>tests/unit/suites/administrator</directory>
</testsuite>
<testsuite name="FinderIndexer">
<directory>tests/unit/suites/finderIndexer</directory>
</testsuite>
</testsuites>

19. phpunit.xml
(Unit-)Testing for Joomla David Jardin
<logging>
<log type="coverage-html" target="build/coverage" title="Joomla-CMS"/>
<log type="coverage-clover" target="build/logs/clover.xml" />
<log type="junit" target="build/logs/junit.xml" />
</logging>
<filter>
<whitelist addUncoveredFilesFromWhitelist="true">
<directory suffix=".php">libraries/cms</directory>
<directory suffix=".php">libraries/joomla</directory>
<directory suffix=".php">libraries/legacy</directory>
<directory suffix=".php">administrator/components/com_finder/helpers/
indexer</directory>
<file>administrator/includes/helper.php</file>
<file>libraries/loader.php</file>
<file>libraries/platform.php</file>
</whitelist>
</filter>

21. Setting up your machine
03

22. 03. Setting up your machine
Requirements:
local *AMP-Stack
memory_limit = 512M
php_sqlite
php_curl
PHPUnit:
wget https://phar.phpunit.de/phpunit.phar
chmod +x phpunit.phar
sudo mv phpunit.phar /usr/local/bin/phpunit
phpunit --version
(Unit-)Testing for Joomla David Jardin

23. Running the unit test suite
04

24. 04. Running the unit test suite
cd /path/to/joomla-git-clone
phpunit
(Unit-)Testing for Joomla David Jardin

26. How to write a test : best practices
05

27. 05. How to write a test : best practices
<?php
class Foo
{
public function bar($string)
{
if (is_int($string))
{
return false;
}
$string = strtoupper($string);
return $string;
}
}
(Unit-)Testing for Joomla David Jardin

28. 05. How to write a test : best practices
<?php
class FooTest extends TestCase
{
public function testBar()
{
$object = new foo();
$this->assertEquals('EXAMPLE', $object->bar('example'));
$this->assertEquals(false, $object->bar(4));
}
}
(Unit-)Testing for Joomla David Jardin

29. One assertion per test.

30. 05. How to write a test : best practices
<?php
class FooTest extends TestCase
{
public function testBar1()
{
$object = new foo();
$this->assertEquals('EXAMPLE', $object->bar('example'));
}
public function testBar2()
{
$object = new foo();
$this->assertEquals(false, $object->bar(4));
}
}
(Unit-)Testing for Joomla David Jardin

31. Use meaningful names for test methods.

32. 05. How to write a test : best practices
<?php
class FooTest extends TestCase
{
public function testStringIsConvertedToUppercase()
{
$object = new foo();
$this->assertEquals('EXAMPLE', $object->bar('example'));
}
public function testFalseIsReturnedWhenIntIsUsedAsArgument()
{
$object = new foo();
$this->assertEquals(false, $object->bar(4));
}
}
(Unit-)Testing for Joomla David Jardin

33. Use the most specific assertion possible.

34. 05. How to write a test : best practices
<?php
class FooTest extends TestCase
{
public function testStringIsConvertedToUppercase()
{
$object = new foo();
// assertSame is type safe, so in this case only strings are accepted
$this->assertSame('EXAMPLE', $object->bar('example'));
}
public function testFalseIsReturnedWhenIntIsUsedAsArgument()
{
$object = new foo();
// reduced code
$this->assertFalse($object->bar(4));
}
}
(Unit-)Testing for Joomla David Jardin

35. Run your tests with --strict and —verbose.

36. Use Testdox.

38. Joomla’s system testing setup
06

39. 06. Joomla’s system testing setup
(Unit-)Testing for Joomla David Jardin
PHPUnit
Selenium Client for PHP
Selenium Webdriver API
Browser

41. Setting up your machine
07

42. PHP
Selenium
PHPUnit
Browser
Plugin
Browser

43. 07. Setting up your machine
(Unit-)Testing for Joomla David Jardin

44. 07. Setting up your machine
git clone https://github.com/joomla-projects/joomla-systemtest-env.git
cd joomla-systemtest-env
docker build -t joomlasystest-env .
(Unit-)Testing for Joomla David Jardin

45. Running the system test suite
08

46. 08. Running the system test suite
docker run -i -t
-e REPO=https://github.com/joomla/joomla-cms joomlasystest-env
-e BRANCH=stating
(Unit-)Testing for Joomla David Jardin

47. 08. Running the system test suite
(Unit-)Testing for Joomla David Jardin

48. Automation of automated tests
09

50. 09. Automation of automated tests
(Unit-)Testing for Joomla David Jardin

51. 09. Automation of automated tests
(Unit-)Testing for Joomla David Jardin
Jenkins
Jenkins-Slave Jenkins-Slave Jenkins-Slave
- Install
- Article Manager
- Install
- Weblinks
- Install
- Users
- … - … - …

52. 09. Automation of automated tests
(Unit-)Testing for Joomla David Jardin

54. Conclusion
10

55. (Unit-)Testing for Joomla David Jardin

56. (Unit-)Testing for Joomla David Jardin

57. Image: © https://www.flickr.com/photos/narro/

58. THANK YOU!
David Jardin
@snipersister
david.jardin@community.joomla.org