Automated Testing for WordPress

1. Automated Testing 101
By Tabby Chapman
@tabbymarie
http://www.tabbychapman.com
1

2. About Tabby Chapman
@tabbymarie
https://github.com/tabbymarie
2

3. About Me
• Work for the University of California, Irvine
• Computer Scientist
• Developer Cheerleader
• Education Junkie
3

4. About Me
• WordPress Skillz
– Reaching my 10th anniversary developing
with WordPress
– Started with 2.0.5, building a custom theme.
– Ran my own company for four years
– Joined Zeek for almost two years
– Currently build highly specialized custom
themes and plugins for UCI.
4

5. About This Presentation
• Geared towards intermediate QA testers
and Developers and also for
stakeholders/clients/customers
• Assumptions:
– You know what WordPress is.
– You know what PHP is.
– You have written PHP code several times.
Successfully. Or.
– You test all the things.
5

6. About This Presentation
• We’ll go over
– The Developer Workflow(s)
– Different Types of Tests
– Automated Testing
– Demo
– Last Words
6

7. How it Currently Works
7
• Generic Software Development Lifecycle
– In the dinosaur days of computers, the
majority of its users were very technical
people.
– As computers became more accessible (and
smaller), more and more users began to use it
– Developers needed to create a process to
think outside of their own processes to
accommodate non-technical users

8. How it Currently Works
8
• Generic Software Development Lifecycle
(cont)
– Many various methods were created to
attempt to meet this need
– One of these is the Software Development
Lifecycle (which was a similar process used in
other disciplines)
– It is a top down approach: generic to specific
– Specification, Design, Coding

9. How it Currently Works
9
• Generic Software Development Lifecycle
(cont)
– Waterfall method is created in the ‘70s.
– Structured analysis, Structured design,
Structured programming (sounds a lot like
specification, design, and coding)
– Most developers tend to use Define, Design,
Develop, Deploy, Maintain.

10. Generic Modern Development
10
Image credit: https://www.owasp.org/images/8/84/SDLC.jpg

11. Freelance Development Cycle
11
Image credit: http://mohamadsubhan.files.wordpress.com/2009/06/linear.jpg

12. How it Currently Works
12
• Full Software Development Lifecycle
– Many developers on teams tend to use some
form of the full development lifecycle, as there
tends to need to be some form of
accountability for teams.
– This includes: feasibility study, requirements
definition, software design, implementation
and unit testing, integration and system
testing, operation and maintenance.

13. Team Development Cycle
13
Image credit: http://www.softwaretestinghelp.com/unit-testing/

14. Types of Tests
• Black box
• White box
• Unit
• Integration
• Functional
• System
• End-to-end
• Sanity
• Regression
• Acceptance
• Load
• Stress
• Performance
• Usability
14
Source: http://www.softwaretestinghelp.com/types-of-software-testing/

15. Testing Players
15
• Quality Assurance (QA) Engineers
• Developers
• Product Owners
• Clients
• End-Users
• Other Stakeholders

16. Black Box Testing?
16

17. Black Box Testing
17
• Tests against requirements and
functionality. Is not concerned with the
internal workings.
• “If I put in 2+4, I expect 6. Does that
work?”

18. White Box Testing?
18

19. White Box
19
• Tests against requirements and
functionality with knowledge of the inner
workings.
• “I know that I this system has these
conditionals, so I will test all possible
outcomes.”

20. Unit Testing?
20

21. Unit Testing
21
• Tests specific methods or functions,
created by a programmer
• “Does function
addTwoIntegers($num1,$num2) work as
expected?”

22. Integration Testing?
22

23. Integration Testing
23
• Testing of whole application after new
work has been integrated as they interact
with the client/server
• “Does this site’s individual modules still
work as expected when we put it all
together?”

24. Functional Testing?
24

25. Functional Testing
25
• Black Box testing for specific functional
requirements.
• “If I tell it to add five text inputs and send
an email, does it add five inputs and send
an email as expected?”

26. Systems Testing
26
• Black Box Testing based on overall
requirements.
• “Testing All the Things”. “Does this system
meet all of the requirements as
expected?”

27. End-to-end Testing
27
• Similar to Systems, but also tests
interaction between various elements,
protocols, layers and servers.
• “Testing All the Things and Moar”. “Does
this system meet all of the requirements
as expected and communicate as
expected to its various parts?”

28. Sanity Testing
28
• A pre-test to make sure the new
development is done enough to fully test.
• “Does it at least work a little bit?”

29. Regression Testing
29
• Testing the whole application after
something has been modified to see if
something was broken as a result of the
modification.
• “Does it STILL work as expected?”

30. Acceptance Testing
30
• User or Customer does a round of manual
tests to make sure that it meets their
requirements/needs
• “Does the client think it works as
expected?”

31. Stress Testing
31
• Testing Performance Requirements
• “Can more than one interaction happen at
the same time? How many more than
one?”

32. Usability Testing
32
• Testing for user-friendliness. This can be
done BEFORE and AFTER development
phase.
• “Yeah, it works, but is it easy to use?”

33. Which testing is important for
developers?
33

34. Which testing is important for
developers?
• Black box
• White box
• Unit
• Integration
• Functional
• System
• End-to-end
• Sanity
• Regression
• Acceptance
• Load
• Stress
• Performance
• Usability
34
Source: http://www.softwaretestinghelp.com/types-of-software-testing/

35. For the Developers
35
• Which testing is important for developers?
– All of them (know thy tester and thy tester’s
methods)
– But especially
• Unit
• Functional
• Systems
• Integration

36. Which testing is important for
testers?
36

37. Which testing is important for testers?
• Black box
• White box
• Unit
• Integration
• Functional
• System
• End-to-end
• Sanity
• Regression
• Acceptance
• Load
• Stress
• Performance
• Usability
37
Source: http://www.softwaretestinghelp.com/types-of-software-testing/

38. For the Testers
38
• Which testing is important for testers?
– All of them (uphold they developer to unit
tests)
– But especially
• Black Box
• White Box
• Integration
• Systems
• Sanity

39. Which testing is important for
Clients/Stakeholders?
39

40. Which testing is important for
Clients/Stakeholders?
• Black box
• White box
• Unit
• Integration
• Functional
• System
• End-to-end
• Sanity
• Regression
• Acceptance
• Load
• Stress
• Performance
• Usability
40
Source: http://www.softwaretestinghelp.com/types-of-software-testing/

41. For the Clients/Stakeholders
41
• Which testing is important for
clients/stakeholders?
• Black Box
• Usability
• Regression
• Acceptance

42. For the Clients/Stakeholders
42
• Which testing is important for
clients/stakeholders?
• Black Box
• Usability
• Regression
• Acceptance

43. What does Automation of
Testing mean?
43

44. What does Automation of Testing
mean?
44
• The traditional/easiest way to test what
you develop is to make a change, and
then load the app to see if that change
was effective. Very Manual
• Automated Testing means to write code to
test code and use testing software to test
the front-end. Don’t rely on the old manual
check.

45. How does this apply to WordPress?
45
• WordPress utilizes a ‘test suite’ “which
have multiple test cases and a library and
command line tool that runs the test suite
or suites” according to WordPress’
automated testing core handbook page
• For this, they use phpUnit.
https://phpunit.de

46. How does this apply to WordPress?
46
• PHPUnit has a series of commands that
you can use to run all tests, specific tests,
or specific types of tests.

47. Developer Workflow
47
• When you are ready to write your plugin:
– Think of the outcomes you want for your
plugin
– Think of the individual methods you want to
write. Should have one output.
– Write a test that checks that the output is what
you expect.
– Write your method.
– Lather. Rinse. Repeat.

48. Developer Workflow
48
• Testing Tools
• phpUnit (official wordpress test suite)
• WordPress Developer Branch
– https://make.wordpress.org/core/handbook/automa
• Terminal or Editor with Testing Integrated

49. Developer Workflow
49
Editors (not limited to…)
– TextMate - https://macromates.com/download
• (https://github.com/meritt/phpunit-tmbundle)
– Sublime – http://www.sublimetext.com/
• https://packagecontrol.io/packages/PHPUnit
– phpStorm -
https://www.jetbrains.com/phpstorm/
• Supports all OS’
• Phpunit built in

50. Developer Workflow
50
• Put your tests IN your plugin
• /plugins/[plugindir]/tests/test-
[pluginslug].php
• Add a phpunit.xml file:
• /plugins/[plugindir]/tests/phpunit.xml

51. Developer Workflow
phpunit.xml
51
<?xml version=“1.0” encoding=“UTF-8” ?>
<phpunit bootstrap=“vendor/autoload.php” >
<testsuites>
<testsuite>
<directory prefix="test-”
suffix=".php">
./tests/
</directory>
</testsuite>
</testsuites>
</phpunit>

52. Developer Workflow
phpunit.xml
52
<?xml version=“1.0” encoding=“UTF-8” ?>
<phpunit bootstrap=“vendor/autoload.php” >
<testsuites>
<testsuite>
<directory prefix="test-”
suffix=".php">
./tests/
</directory>
</testsuite>
</testsuites>
</phpunit>

53. Developer Workflow
phpunit.xml
53
<?xml version=“1.0” encoding=“UTF-8” ?>
<phpunit bootstrap=“vendor/autoload.php” >
<testsuites>
<testsuite>
<directory prefix="test-”
suffix=".php">
./tests/
</directory>
</testsuite>
</testsuites>
</phpunit>

54. Developer Workflow
phpunit.xml
54
<?xml version=“1.0” encoding=“UTF-8” ?>
<phpunit bootstrap=“vendor/autoload.php” >
<testsuites>
<testsuite>
<directory prefix="test-”
suffix=".php">
./tests/
</directory>
</testsuite>
</testsuites>
</phpunit>

55. Tester Workflow
55
• When you are ready to test
– Be familiar with the general requirements for
the site as a whole
– Read the requirements of the change request
– Focus on specifically what is supposed to
have changed.
– Write a test while waiting for the change.
– Run the test once you get the change from
the developer

56. Tester Workflow
56
• Selenium IDE, Firefox Plugin
– http://www.seleniumhq.org/download/
• Browser Specific
– Chrome
https://sites.google.com/a/chromium.org/chromed
– Safari
http://selenium-release.storage.googleapis.com/2

57. Tester Workflow
57
• Selenium
– Installationhttp
://geekpad.ca/blog/post/automating-browser-testi
• First you must download the Selenium server and
the Internet Explorer Server from
http://docs.seleniumhq.org/download/
• Then you must download the Chrome server from
http://chromedriver.storage.googleapis.com/index
.html

58. Tester Workflow
58
• Selenium
– Installation (continued)
• You do not need to download a Firefox driver
since it’s builtin the main Selenium file.
• Once all files have been downloaded you must
add your selenium folder to your existing path see
http://www.java.com/en/download/help/path.xml
• Then you can launch the server using java. With
something that looks like this
– Mac - java -jar selenium-server-standalone-[version].jar

59. Tester Workflow
59
Why is Selenium cool?
•Records your actions
•Let’s you test expected outcomes
•Let’s you test them in a series

60. Tester Workflow
60
Example
•The developer was given a task to make
sure that all instances of the word “Orange
County” is replaced with the word “OC” on
the website, for “legal reasons”.
•Demo

61. Jenkins
61

62. Jenkins
62
• Jenkins is an award-winning application
that monitors executions of repeated jobs,
such as building a software project or jobs
run by cron. Among those things, current
Jenkins focuses on the following two jobs:
– Building/testing software projects continuously
– Monitoring executions of externally-run jobs
https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins

63. Jenkins
63

64. Jenkins
64

65. Questions?
65

66. Resources
• http://www.dcs.bbk.ac.uk/~sven/cdm06/cdm3.
• https://unit-tests.trac.wordpress.org/
• http://www.slideshare.net/ptahdunbar/automat
• http://johnregan3.co/2013/08/13/why-you-need
• http://blog.aizatto.com/2010/05/15/beginni
ng-wordpress-automated-testing/
66

67. Resources
• https://www.owasp.org/index.php/Testing_
Guide_Introduction
• https://make.wordpress.org/core/handboo
k/automated-testing/
• http://stackoverflow.com/questions/224141
34/different-types-of-testing-eg-unit-
functional-integration-etc-document
67

68. Resources
• http://www.seleniumhq.org/
• http://geekpad.ca/blog/post/automating-browser-
testing-with-phpunit-and-selenium
• https://wiki.jenkins-
ci.org/display/JENKINS/Meet+Jenkins
• https://wiki.jenkins-
ci.org/display/JENKINS/Installing+Jenkins
68