1. Unit-Testing Bad-Practices
by Example
Benjamin Eberlei
direkt effekt GmbH
August 2009
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 1 / 49
2. About Me
Benjamin Eberlei
direkt effekt GmBH (digital marketing)
Zend Framework contributor
Test-Driven-Development, Legacy Testing
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 2 / 49
4. Why Test Quality Matters
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 4 / 49
5. “We spent 90% of the time
modifying existing tests to
acommodate for a relatively
minor change.“
(G. Meszaros, xUnit Test Patterns)
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 5 / 49
6. “Walking on water and
developing software from a
specification are easy if both
are frozen.”
(Edward V. Berard)
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 6 / 49
7. Safety Net vs Dead Weight
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 7 / 49
8. Test Smells
“Smell you later!”
(Nelson, The Simpsons)
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 8 / 49
10. ZF Controller Action
public function testInitView ()
{
Zen d_ C o n t r o l l e r _ Front :: getInstance () ->
s e t C o n t r o l l er Dir ec to ry ( ’/ _files ’) ;
require_once ’/ _files / ViewController . php ’;
$controller = new ViewController (
new Z e n d _ C o n t r o l l e r _ R e q u e s t _ H t t p () ,
new Z e n d _ C o n t r o l l e r _ R e s p o n s e _ C l i ()
);
$view = $controller - > initView () ;
$this - > assertTrue ( $view instanceof Zend_View ) ;
$scriptPath = $view - > getScriptPaths () ;
$this - > assertTrue ( is_array ( $scriptPath ) ) ;
$this - > assertEquals ( ’/ views / scripts / ’ , $scriptPath [0])
;
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 10 / 49
11. ZF Controller Action 2
public function testRenderByName ()
{
$request = new Z e n d _ C o n t r o l l e r _ R e q u e s t _ H t t p () ;
$request - > set ControllerName ( ’ view ’)
-> setActionName ( ’ test ’) ;
$response = new Z e n d _ C o n t r o l l e r _ R e s p o n s e _ C l i () ;
Zen d_ C o n t r o l l e r _ Front :: getInstance () ->
s e t C o n t r o l l er Dir ec to ry ( ’/ _files ’) ;
require_once ’/ _files / ViewController . php ’;
$controller = new ViewController ( $request , $response ) ;
$controller - > testAction () ;
$this - > assertContains ( ’ In the index action view ’ ,
$response - > getBody () ) ;
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 11 / 49
12. ZF Controller Refactoring
Extract Test Utility Method:
public function c r e ateViewController ( $controllerName = null ,
$actionName = null )
{
$request = new Z e n d _ C o n t r o l l e r _ R e q u e s t _ H t t p () ;
if ( $controllerName !== null ) {
$request - > setControllerName ( $controllerName ) ;
}
if ( $actionName !== null ) {
$request - > setActionName ( $actionName ) ;
}
$response = new Z e n d _ C o n t r o l l e r _ R e s p o n s e _ C l i () ;
Zen d_ C o n t r o l l e r _ Front :: getInstance ()
-> s e t C o n t r o l le rD ire ct or y ( ’/ _files ’) ;
require_once ’/ _files / ViewController . php ’;
return new ViewController ( $request , $response ) ;
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 12 / 49
13. ZF Controller Refactoring 2
public function t e s tI nit Vi ew Ref ac to red ()
{
// fixture setup
$controller = $this - > createViewController () ;
// execution
$view = $controller - > initView () ;
$scriptPath = $view - > getScriptPaths () ;
// assertions
$this - > assertTrue ( $view instanceof Zend_View ) ;
$this - > assertTrue ( is_array ( $scriptPath ) ) ;
$this - > assertEquals (
’/ views / scripts / ’ , $scriptPath [0]
);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 13 / 49
14. ZF Controller Refactoring 3
public function t e s t R e n d e r B y N a m e R e f a c t o r e d ()
{
// fixture setup
$controller =
$this - > c r e a teViewController ( ’ view ’ , ’ test ’) ;
// execution
$controller - > testAction () ;
// assertions
$this - > assertContains (
’ In the index action view ’ ,
$response - > getBody ()
);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 14 / 49
16. Doctrine ResultSetMapping
public function t e s t B a s i c Re s u l t S e t M a p p i n g ()
{
// Fixture Setup
$rsm = new ResultSetMapping () ;
$rsm - > addEntityResult (
’ Doctrine Tests Models CMS CmsUser ’ ,
’u ’
);
$rsm - > addFieldResult ( ’u ’ , ’ id ’ , ’ id ’) ;
$rsm - > addFieldResult ( ’u ’ , ’ status ’ , ’ status ’) ;
$rsm - > addFieldResult ( ’u ’ , ’ user ’ , ’ user ’) ;
$rsm - > addFieldResult ( ’u ’ , ’ name ’ , ’ name ’) ;
// [..]
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 16 / 49
17. Doctrine ResultSetMapping 2
public function t e s t B a s i c Re s u l t S e t M a p p i n g ()
{
// [..]
$this - > assertFalse ( $rsm - > isScalarResult ( ’ id ’) ) ;
$this - > assertFalse ( $rsm - > isScalarResult ( ’ status ’) ) ;
$this - > assertFalse ( $rsm - > isScalarResult ( ’ user ’) ) ;
$this - > assertFalse ( $rsm - > isScalarResult ( ’ name ’) ) ;
$this - > assertTrue (
$rsm - > getClass ( ’u ’) ==
’ Doctrine Tests Models CMS CmsUser ’
);
$class = $rsm - > getOwningClass ( ’ id ’) ;
$this - > assertTrue (
$class == ’ Doctrine Tests Models CMS CmsUser ’
);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 17 / 49
18. Doctrine ResultSetMapping 3
public function t e s t B a s i c Re s u l t S e t M a p p i n g ()
{
// [..]
$this - > assertEquals ( ’u ’ , $rsm - > getAlias ( ’ id ’) ) ;
$this - > assertEquals ( ’u ’ , $rsm - > getAlias ( ’ status ’) ) ;
$this - > assertEquals ( ’u ’ , $rsm - > getAlias ( ’ user ’) ) ;
$this - > assertEquals ( ’u ’ , $rsm - > getAlias ( ’ name ’) ) ;
$this - > assertEquals ( ’ id ’ , $rsm - > getField ( ’ id ’) ) ;
$this - > assertEquals ( ’ status ’ , $rsm - > getField ( ’ status ’) ) ;
$this - > assertEquals ( ’ username ’ , $rsm - > getField ( ’ user ’) ) ;
$this - > assertEquals ( ’ name ’ , $rsm - > getField ( ’ name ’) ) ;
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 18 / 49
20. ezcUrl Test
public function t e s t R e m o v e O r d e r e d P a r a m e t e r ()
{
$urlCfg = new e zcUrlConfiguration () ;
$urlCfg - > a d dO r d eredParameter ( ’ section ’ ) ;
$urlCfg - > a d dO r d eredParameter ( ’ module ’ ) ;
$urlCfg - > a d dO r d eredParameter ( ’ view ’ ) ;
$u = ’ http :// www . example . com / doc / components ’;
$url = new ezcUrl ( $u , $urlCfg ) ;
// [..]
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 20 / 49
21. ezcUrl Test 2
public function t e s t R e m o v e O r d e r e d P a r a m e t e r ()
{
// [..]
// functionality tested in other tests before
$this - > assertEquals (
array ( ’ section ’ = > 0 , ’ module ’ = > 1 , ’ view ’ = > 2) ,
$url - > configuration - > orderedParameters
);
$this - > assertEquals ( ’ doc ’ , $url - > getParam ( ’ section ’) ) ;
$this - > assertEquals (
’ components ’ , $url - > getParam ( ’ module ’)
);
// [..]
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 21 / 49
22. ezcUrl Test 3
public function t e s t R e m o v e O r d e r e d P a r a m e t e r ()
{
// [..]
// Primary Assertion according to test method name
$url - > configuration - > r em ove Or de re dPa ra me ter ( ’ view ’) ;
$this - > assertEquals (
array ( ’ section ’ = > 0 , ’ module ’ = > 1 ) ,
$url - > configuration - > orderedParameters
);
// [..]?
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 22 / 49
23. ezcUrl Test 4
public function t e s t R e m o v e O r d e r e d P a r a m e t e r ()
{
// [..]
try
{
$this - > assertEquals ( null , $url - > getParam ( ’ view ’) ) ;
$this - > fail ( ’ Expected exception was not thrown . ’) ;
} catch ( e z c U r l I n v a l i d P a r a m e t e r E x c e p t i o n $e ) {
$expected = " ... " ;
$this - > assertEquals ( $expected , $e - > getMessage () ) ;
}
// [..]?
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 23 / 49
24. ezcUrl Test 5
public function t e s t R e m o v e O r d e r e d P a r a m e t e r ()
{
// [..]
// try removing again - nothing bad should happen
$url - > configuration - > r em ove Or de re dPa ra me ter ( ’ view ’) ;
try
{
$this - > assertEquals ( null , $url - > getParam ( ’ view ’) ) ;
$this - > fail ( ’ Expected exception was not thrown . ’) ;
} catch ( e z c U r l I n v a l i d P a r a m e t e r E x c e p t i o n $e ) {
$expected = " ... " ;
$this - > assertEquals ( $expected , $e - > getMessage () ) ;
}
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 24 / 49
28. Global State: Zend Framework
// Z e n d _ C o n t r o l l e r _ A c t i o n _ H e l p e r _ V i e w R e n d e r e r T e s t
protected function setUp ()
{
$this - > request = new Z e n d _ C o n t r o l l e r _ R e q u e s t _ H t t p () ;
$this - > response = new Z e n d _ C o n t r o l l e r _ R e s p o n s e _ H t t p () ;
$this - > front = Ze nd_C ontro ller _Fron t :: getInstance ()
;
$this - > front - > resetInstance () ;
$this - > front - > a ddModuleDirectory ( ’/ _files / modules ’)
-> setRequest ( $this - > request )
-> setResponse ( $this - > response ) ;
$this - > helper = new
Z e n d _ C o n t r o l l e r _ A c t i o n _ H e l p e r _ V i e w R e n d e r e r () ;
Z e n d _ C o n t r o l l e r _ A c t i o n _ H e l p e r B r o k e r :: addHelper (
$this - > helper
);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 28 / 49
29. Indirect Tests: ezcMvc
function t e s t I n t e r n alRedirect () {
$config = new s impleConfiguration () ;
$config - > route = ’ IRController ’;
$dispatcher = new e z c M v c C o n f i g u r a b l e D i s p a t c h e r (
$config ) ;
$dispatcher - > run () ;
self :: assertEquals ( " BODY : Name : name , " .
" Vars : array ([ CR ] ’ nonRedirVar ’ = > 4 , " .
" [ CR ] ’ ReqRedirVar ’ = > 4 ,[ CR ]) " , $config - > store ) ;
}
function t e s t E x t e r n alRedirect () {
$config = new s impleConfiguration () ;
$config - > route = ’ IRController ’;
$dispatcher = new e z c M v c C o n f i g u r a b l e D i s p a t c h e r (
$config ) ;
$dispatcher - > run () ;
self :: assertEquals ( " BODY : Name : name , " .
" Vars : array ([ CR ] ’ nonRedirVar ’ = > 4 , " .
" [ CR ] ’ ReqRedirVar ’ = > 4 ,[ CR ]) " , $config - > store ) ;
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 29 / 49
32. Zend Service Amazon
public function setUp ()
{
$this - > _amazon = new Zend_Service_Amazon () ;
$this - > _query = new Z e n d _ S e r v i ce _ A m a z o n _ Q u e r y ()
$this - > _httpClient =
new Z e n d _ H t t p _ C l i e n t _ A d a p t e r _ S o c k e t () ;
$this - > _amazon - > getRestClient ()
-> getHttpClient ()
-> setAdapter ( $this - > _httpClient ) ;
// terms of use compliance :
// no more than one query per second
sleep (1) ;
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 32 / 49
33. Zend Service Amazon 2
public function t e s t I t e m S ea r c h M u s i c M o z a r t ()
{
$resultSet = $this - > _amazon - > itemSearch ( array (
’ SearchIndex ’ = > ’ Music ’ ,
’ Keywords ’ = > ’ Mozart ’ ,
’ ResponseGroup ’ = > ’ Small , Tracks , Offers ’
));
foreach ( $resultSet as $item ) {
$this - > assertTrue (
$item instanceof Z e n d_ S e r vi c e _ Am a z o n_ I t e m
);
}
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 33 / 49
34. Zend Amazon Refactored
public function setUpRefactored ()
{
$this - > _amazon = new Zend_Service_Amazon () ;
$this - > _httpClient =
new Z e n d _ H t t p _ C l i e n t _ A d a p t e r _ T e s t () ;
$this - > _amazon - > getRestClient ()
-> getHttpClient ()
-> setAdapter ( $this - > _httpClient ) ;
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 34 / 49
35. Zend Amazon Refactored 2
public function t e s t I t e m S e a r c h M u s i c M o z a r t R e f a c t o r e d ()
{
$this - > _httpClient - > setResponse (
fil e_get_ contents ( " ExpectedTestResponse . txt " )
);
$resultSet = $this - > _amazon - > itemSearch ( array (
’ SearchIndex ’ = > ’ Music ’ ,
’ Keywords ’ = > ’ Mozart ’ ,
’ ResponseGroup ’ = > ’ Small , Tracks , Offers ’
));
foreach ( $resultSet as $item ) {
$this - > assertTrue (
$item instanceof Z e n d_ S e r vi c e _ Am a z o n_ I t e m
);
// Assert some relevant stuff now !
}
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 35 / 49
36. Conditional Logic
“Everyone knows that debugging is
twice as hard as writing a program in
the first place. So if you’re as clever as
you can be when you write it, how will
you ever debug it?” (Brian Kernighan)
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 36 / 49
37. FLOW3 Cache Frontend
public function t h e C o n s t r u c t o r A c c e p t s V a l i d I d e n t i f i e r s () {
$mockBackend = $this - > createMockBackend () ;
$identifiers = array (
’x ’ , ’ someValue ’ , ’ 123 fivesixseveneight ’ ,
’ some & ’ , ’ ab_cd % ’ ,
rawurlencode ( ’ package :// some / $ &% sadf ’) ,
str_repeat ( ’x ’ , 250)
);
foreach ( $identifiers as $identifier ) {
$abstractCache = $this - > getMock (
’ F3 FLOW3 Cache Frontend StringFrontend ’ ,
array () ,
array ( $identifier , $mockBackend )
);
}
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 37 / 49
38. FLOW3 Cache Refactored
/* *
* @dataProvider d a t a A c c e p t V a l i d I d e n t i f ie r
*/
public function c o n s t r u c t o r A c c e p t s V a l i d I d e n t i f i e r ( $id ) {
$mockBackend = $this - > createMockBackend () ;
$abstractCache = $this - > getMock (
’ F3 FLOW3 Cache Frontend StringFrontend ’ ,
array () ,
array ( $id , $mockBackend )
);
}
static public function d a t a A c c e p tV a l i d I d e n t i f i e r () {
return array (
array ( ’x ’) , array ( ’ someValue ’) ,
array ( ’ 123 fivesixseveneight ’) ,
array ( ’ some & ’) , array ( ’ ab_cd % ’) ,
array (
rawurlencode ( ’ package :// some / $ &% sadf ’)
),
array ( str_repeat ( ’x ’ , 250) )
);
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 38 / 49
39. Zend Server ReflectionClass
public function testGetMethods ()
{
$r = new Z e n d _ S e r v e r _ R e f l e c t i o n _ C l a s s (
new ReflectionClass ( ’ Ze nd_ Se rv er_ Re fl ect io n ’)
);
$methods = $r - > getMethods () ;
$this - > assertTrue ( is_array ( $methods ) ) ;
foreach ( $methods as $m ) {
$this - > assertTrue (
$m instanceof Z e n d _ S e r v e r _ R e f l e c t i o n _ M e t h o d
);
}
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 39 / 49
40. A working implementation
class Z e n d _ S e r v e r _ R e f l e c t i o n _ C l a s s ()
{
public function getMethods ()
{
return array () ;
}
}
Great, all tests pass!
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 40 / 49
41. Zend Server ReflectionClass
Test Refactoring
public function t e s t G e tM e t ho d s R ef a c t or e d ()
{
$r = new Z e n d _ S e r v e r _ R e f l e c t i o n _ C l a s s (
new ReflectionClass ( ’ Ze nd_ Se rv er_ Re fl ect io n ’)
);
$methods = $r - > getMethods () ;
$this - > assertTrue ( is_array ( $methods ) ) ;
$this - > assertEquals (3 , count ( $methods ) ) ; // (!!)
foreach ( $methods as $m ) {
$this - > assertTrue (
$m instanceof Z e n d _ S e r v e r _ R e f l e c t i o n _ M e t h o d
);
}
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 41 / 49
42. Zend Server ReflectionClass
Test Refactoring 2
public function ass ertReflMethods ( $methods , $expected )
{
$this - > assertTye ( ’ array ’ , $methods ) ;
$this - > assertEquals ( $expected , count ( $methods ) ) ;
foreach ( $methods as $m ) {
$this - > assertTrue (
$m instanceof Z e n d _ S e r v e r _ R e f l e c t i o n _ M e t h o d
);
}
}
public function t e s t G e tM e t ho d s R ef a c t or e d ()
{
$r = new Z e n d _ S e r v e r _ R e f l e c t i o n _ C l a s s (
new ReflectionClass ( ’ Ze nd_ Se rv er_ Re fl ect io n ’)
);
$this - > as sertRe flMethods ( $r - > getMethods () , 3) ;
}
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 42 / 49
47. Conclusion
Don’t use global state (Singleton
Anti-Pattern)
Use utility methods for Assertions, Object
and Mock Creation
Use Mocks (but not exclusively)
Give meaningful test-names
Dont test through the UI
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 47 / 49
48. Further Readings
xUnit Test Patterns
Book by Gerald Meszaros, http://xunitpatterns.com/Test%20Smells.html
TDD Anti Patterns
http://blog.james-carr.org/2006/11/03/tdd-anti-patterns/
Eberlei (direkt effekt GmbH) Unit-Testing Bad-Practices August 2009 48 / 49