Automatisiertes Testen ist zum Standard in der PHP-Welt avanciert. Aber ist Ihre Test-Suite immer noch State of the Art? Nutzen Sie die neuesten Features, um sich das Testleben einfacher zu machen? Sind Ihre Tests verlässlich? Lernen Sie, was en vogue ist in der PHP-Testwelt im Jahr 2012.
http://talks.qafoo.com
1. Testen State of the Art
IPC Spring 2012
Manuel Pichler (@manuelp)
Tobias Schlitt (@tobySen)
2012-06-04
Testen State of the Art 1 / 40
2. About us
Manuel & Toby
Degree in computer sience
More than 10 years of
Qafoo
passion for software quality
professional PHP
Open source enthusiasts
Contributing to various
FLOSS projects
Testen State of the Art 2 / 40
3. About us
Manuel & Toby
Degree in computer sience
More than 10 years of
Qafoo
passion for software quality
professional PHP
Open source enthusiasts
Contributing to various
FLOSS projects
Testen State of the Art 2 / 40
4. About us
Manuel & Toby
Degree in computer sience
More than 10 years of
Qafoo
passion for software quality
professional PHP
Open source enthusiasts
Contributing to various
FLOSS projects
Testen State of the Art 2 / 40
5. About us
Co-founders of
Manuel & Toby
Degree in computer sience
More than 10 years of
Qafoo
passion for software quality
professional PHP
Open source enthusiasts
Contributing to various
FLOSS projects
Testen State of the Art 2 / 40
6. About us
Co-founders of
Manuel & Toby
Degree in computer sience
More than 10 years of
Qafoo
passion for software quality
professional PHP
Helping people to create
Open source enthusiasts high quality web
Contributing to various applications.
FLOSS projects
Testen State of the Art 2 / 40
7. About us
Co-founders of
Manuel & Toby
Degree in computer sience
More than 10 years of
Qafoo
passion for software quality
professional PHP
Helping people to create
Open source enthusiasts high quality web
Contributing to various applications.
FLOSS projects
http://qafoo.com
Testen State of the Art 2 / 40
8. Outline
Unit Testing
Testing File Uploads
Behavior Driven Development
Automated Acceptance Tests
Testing JavaScript
Outlook
Testen State of the Art 3 / 40
9. PHPUnit schematics
TestRunner
MyComponentSuite TestSuite
MyClassTest TestCase
MyClass
Testen State of the Art 4 / 40
10. PHPUnit schematics
TestRunner
MyComponentSuite TestSuite
MyClassTest TestCase
MyClass
Testen State of the Art 4 / 40
11. Test suites
File system based
phpunit.dist.xml / phpunit.xml
@group annotations
Testen State of the Art 5 / 40
12. phpunit.xml
In the dir you execute $ phpunit
phpunit.xml available?
phpunit.dist.xml available?
$ phpunit -c my-phpunit.xml
Testen State of the Art 6 / 40
13. phpunit.xml
1 < p h p u n i t b o o t s t r a p = ” l i b r a r i e s / a u t o l o a d . php ”
2 colors=” true ”
3 convertErrorsToExceptions=” true ”
4 convertNoticesToExceptions= ” true ”
5 convertWarningsToExceptions = ” t r u e ”
6 >
7 < groups>
8 < exclude >< group> i n t e g r a t i o n < / group>< / exclude >
9 < / groups>
10
11 <t e s t s u i t e s>
12 < t e s t s u i t e name= ” A l l Tests ” >
13 < d i r e c t o r y s u f f i x = ” . php ” > t e s t s / < / d i r e c t o r y >
14 </ t e s t s u i t e>
15 </ t e s t s u i t e s>
16
17 <filter>
18 <w h i t e l i s t>
19 < d i r e c t o r y s u f f i x = ” . php ” > . . / s r c / < / d i r e c t o r y >
20 < exclude >
21 < f i l e > . . / s r c / b o o t s t r a p . php< / f i l e >
22 < / exclude >
23 </ w h i t e l i s t>
24 </ f i l t e r >
25 < / p h p u n i t>
Testen State of the Art 7 / 40
14. phpunit.xml
1 < p h p u n i t b o o t s t r a p = ” l i b r a r i e s / a u t o l o a d . php ”
2 colors=” true ”
3 convertErrorsToExceptions=” true ”
4 convertNoticesToExceptions= ” true ”
5 convertWarningsToExceptions = ” t r u e ”
6 >
7 < groups>
8 < exclude >< group> i n t e g r a t i o n < / group>< / exclude >
9 < / groups>
10
11 <t e s t s u i t e s>
12 < t e s t s u i t e name= ” A l l Tests ” >
13 < d i r e c t o r y s u f f i x = ” . php ” > t e s t s / < / d i r e c t o r y >
14 </ t e s t s u i t e>
15 </ t e s t s u i t e s>
16
17 <filter>
18 <w h i t e l i s t>
19 < d i r e c t o r y s u f f i x = ” . php ” > . . / s r c / < / d i r e c t o r y >
20 < exclude >
21 < f i l e > . . / s r c / b o o t s t r a p . php< / f i l e >
22 < / exclude >
23 </ w h i t e l i s t>
24 </ f i l t e r >
25 < / p h p u n i t>
Testen State of the Art 7 / 40
15. phpunit.xml
1 < p h p u n i t b o o t s t r a p = ” l i b r a r i e s / a u t o l o a d . php ”
2 colors=” true ”
3 convertErrorsToExceptions=” true ”
4 convertNoticesToExceptions= ” true ”
5 convertWarningsToExceptions = ” t r u e ”
6 >
7 < groups>
8 < exclude >< group> i n t e g r a t i o n < / group>< / exclude >
9 < / groups>
10
11 <t e s t s u i t e s>
12 < t e s t s u i t e name= ” A l l Tests ” >
13 < d i r e c t o r y s u f f i x = ” . php ” > t e s t s / < / d i r e c t o r y >
14 </ t e s t s u i t e>
15 </ t e s t s u i t e s>
16
17 <filter>
18 <w h i t e l i s t>
19 < d i r e c t o r y s u f f i x = ” . php ” > . . / s r c / < / d i r e c t o r y >
20 < exclude >
21 < f i l e > . . / s r c / b o o t s t r a p . php< / f i l e >
22 < / exclude >
23 </ w h i t e l i s t>
24 </ f i l t e r >
25 < / p h p u n i t>
Testen State of the Art 7 / 40
16. phpunit.xml
1 < p h p u n i t b o o t s t r a p = ” l i b r a r i e s / a u t o l o a d . php ”
2 colors=” true ”
3 convertErrorsToExceptions=” true ”
4 convertNoticesToExceptions= ” true ”
5 convertWarningsToExceptions = ” t r u e ”
6 >
7 < groups>
8 < exclude >< group> i n t e g r a t i o n < / group>< / exclude >
9 < / groups>
10
11 <t e s t s u i t e s>
12 < t e s t s u i t e name= ” A l l Tests ” >
13 < d i r e c t o r y s u f f i x = ” . php ” > t e s t s / < / d i r e c t o r y >
14 </ t e s t s u i t e>
15 </ t e s t s u i t e s>
16
17 <filter>
18 <w h i t e l i s t>
19 < d i r e c t o r y s u f f i x = ” . php ” > . . / s r c / < / d i r e c t o r y >
20 < exclude >
21 < f i l e > . . / s r c / b o o t s t r a p . php< / f i l e >
22 < / exclude >
23 </ w h i t e l i s t>
24 </ f i l t e r >
25 < / p h p u n i t>
Testen State of the Art 7 / 40
17. Useful Annotations
@covers
@group
@ticket
@dataProvider
@expectedException*
@codeCoverageIgnore*
@depends
Testen State of the Art 8 / 40
18. Useful Annotations
@covers
@group
@ticket
@dataProvider
@expectedException*
@codeCoverageIgnore*
@depends
Testen State of the Art 8 / 40
19. Useful Annotations
@covers
@group
@ticket
@dataProvider
@expectedException*
@codeCoverageIgnore*
@depends
Testen State of the Art 8 / 40
20. Useful Annotations
@covers
@group
@ticket
@dataProvider
@expectedException*
@codeCoverageIgnore*
@depends
Testen State of the Art 8 / 40
21. Useful Annotations
@covers
@group
@ticket
@dataProvider
@expectedException*
@codeCoverageIgnore*
@depends
Testen State of the Art 8 / 40
22. Useful Annotations
@covers
@group
@ticket
@dataProvider
@expectedException*
@codeCoverageIgnore*
@depends
Testen State of the Art 8 / 40
23. Outline
Unit Testing
Testing File Uploads
Behavior Driven Development
Automated Acceptance Tests
Testing JavaScript
Outlook
Testen State of the Art 9 / 40
24. A Minimal File Upload
1 <?php
2 class UploadExample
3 {
4 protected $dest ;
5 public function c o n s t r u c t ( $dest )
6 {
7 $ t h i s −> d e s t = r t r i m ( $dest , ’ / ’ ) . ’ / ’ ;
8 }
9 public function handle ( $name )
10 {
11 i f ( i s u p l o a d e d f i l e ( $ FILES [ $name ] [ ’ tmp name ’ ] ) ) {
12 move uploaded file (
13 $ FILES [ $name ] [ ’ tmp name ’ ] ,
14 $ t h i s −> d e s t . $ FILES [ $name ] [ ’ name ’ ]
15 );
16 }
17 }
18 }
Testen State of the Art 10 / 40
25. A Minimal File Upload
1 <?php
2 class UploadExample
3 {
4 protected $dest ;
5 public function c o n s t r u c t ( $dest )
6 {
7 $ t h i s −> d e s t = r t r i m ( $dest , ’ / ’ ) . ’ / ’ ;
8 }
9 public function handle ( $name )
10 {
11 i f ( i s u p l o a d e d f i l e ( $ FILES [ $name ] [ ’ tmp name ’ ] ) ) {
12 move uploaded file (
13 $ FILES [ $name ] [ ’ tmp name ’ ] ,
14 $ t h i s −> d e s t . $ FILES [ $name ] [ ’ name ’ ]
15 );
16 }
17 }
18 }
Testen State of the Art 10 / 40
26. PHPT
PHP Testing Framework (PHPT)
Used to test PHP core
$ pear run-tests example-test-case.phpt
Integrates into PHPUnit
Details: http://qa.php.net/phpt_details.php
Testen State of the Art 11 / 40
27. A Simple PHPT Test
1 −−TEST−−
2 Example t e s t case
3 −−FILE −−
4 <?php
5 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’P ’ ) ) ;
6 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’ Z ’ ) ) ;
7 −−EXPECT−−
8 int (7)
9 bool ( false )
Testen State of the Art 12 / 40
28. A Simple PHPT Test
1 −−TEST−−
2 Example t e s t case
3 −−FILE −−
4 <?php
5 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’P ’ ) ) ;
6 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’ Z ’ ) ) ;
7 −−EXPECT−−
8 int (7)
9 bool ( false )
Testen State of the Art 12 / 40
29. A Simple PHPT Test
1 −−TEST−−
2 Example t e s t case
3 −−FILE −−
4 <?php
5 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’P ’ ) ) ;
6 var dump ( s t r p o s ( ’ Manuel P i c h l e r ’ , ’ Z ’ ) ) ;
7 −−EXPECT−−
8 int (7)
9 bool ( false )
Testen State of the Art 12 / 40
30. File Uploads with PHPT
1 −−TEST−−
2 Example t e s t e m u l a t i n g a f i l e upload
3 −−POST RAW−−
4 Content −Type : m u l t i p a r t / form − data ; boundary=−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
5
6 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
7 Content − D i s p o s i t i o n : form − data ; name= ” f i l e ” ; f i l e n a m e = ” example . t x t ”
8 Content −Type : t e x t / p l a i n
9
10 Qafoo p r o v i d e s q u a l i t y assurance s u p p o r t and c o n s u l t i n g
11
12 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
13 Content − D i s p o s i t i o n : form − data ; name= ” submit ”
14
15 Upload
16 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn −−
17 −−FILE −−
18 <?php
19 require DIR . ’ / UploadExample . php ’ ;
20
21 $upload = new UploadExample ( ’ / tmp ’ ) ;
22 $upload −> handle ( ’ f i l e ’ ) ;
23
24 var dump ( f i l e e x i s t s ( ’ / tmp / example . t x t ’ ) ) ;
25 ?>
26 −−EXPECT−−
27 bool ( true )
Testen State of the Art 13 / 40
31. File Uploads with PHPT
1 −−TEST−−
2 Example t e s t e m u l a t i n g a f i l e upload
3 −−POST RAW−−
4 Content −Type : m u l t i p a r t / form − data ; boundary=−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
5
6 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
7 Content − D i s p o s i t i o n : form − data ; name= ” f i l e ” ; f i l e n a m e = ” example . t x t ”
8 Content −Type : t e x t / p l a i n
9
10 Qafoo p r o v i d e s q u a l i t y assurance s u p p o r t and c o n s u l t i n g
11
12 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
13 Content − D i s p o s i t i o n : form − data ; name= ” submit ”
14
15 Upload
16 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn −−
17 −−FILE −−
18 <?php
19 require DIR . ’ / UploadExample . php ’ ;
20
21 $upload = new UploadExample ( ’ / tmp ’ ) ;
22 $upload −> handle ( ’ f i l e ’ ) ;
23
24 var dump ( f i l e e x i s t s ( ’ / tmp / example . t x t ’ ) ) ;
25 ?>
26 −−EXPECT−−
27 bool ( true )
Testen State of the Art 13 / 40
32. File Uploads with PHPT
1 −−TEST−−
2 Example t e s t e m u l a t i n g a f i l e upload
3 −−POST RAW−−
4 Content −Type : m u l t i p a r t / form − data ; boundary=−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
5
6 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
7 Content − D i s p o s i t i o n : form − data ; name= ” f i l e ” ; f i l e n a m e = ” example . t x t ”
8 Content −Type : t e x t / p l a i n
9
10 Qafoo p r o v i d e s q u a l i t y assurance s u p p o r t and c o n s u l t i n g
11
12 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
13 Content − D i s p o s i t i o n : form − data ; name= ” submit ”
14
15 Upload
16 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn −−
17 −−FILE −−
18 <?php
19 require DIR . ’ / UploadExample . php ’ ;
20
21 $upload = new UploadExample ( ’ / tmp ’ ) ;
22 $upload −> handle ( ’ f i l e ’ ) ;
23
24 var dump ( f i l e e x i s t s ( ’ / tmp / example . t x t ’ ) ) ;
25 ?>
26 −−EXPECT−−
27 bool ( true )
Testen State of the Art 13 / 40
33. File Uploads with PHPT
1 −−TEST−−
2 Example t e s t e m u l a t i n g a f i l e upload
3 −−POST RAW−−
4 Content −Type : m u l t i p a r t / form − data ; boundary=−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
5
6 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
7 Content − D i s p o s i t i o n : form − data ; name= ” f i l e ” ; f i l e n a m e = ” example . t x t ”
8 Content −Type : t e x t / p l a i n
9
10 Qafoo p r o v i d e s q u a l i t y assurance s u p p o r t and c o n s u l t i n g
11
12 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn
13 Content − D i s p o s i t i o n : form − data ; name= ” submit ”
14
15 Upload
16 −−−−−−WebKitFormBoundaryfywL8UCjFtqUBTQn −−
17 −−FILE −−
18 <?php
19 require DIR . ’ / UploadExample . php ’ ;
20
21 $upload = new UploadExample ( ’ / tmp ’ ) ;
22 $upload −> handle ( ’ f i l e ’ ) ;
23
24 var dump ( f i l e e x i s t s ( ’ / tmp / example . t x t ’ ) ) ;
25 ?>
26 −−EXPECT−−
27 bool ( true )
Testen State of the Art 13 / 40
34. PHPT within PHPUnit
1 <ph punit >
2 <testsuites >
3 < t e s t s u i t e name= ”PHPT” >
4 < d i r e c t o r y s u f f i x = ” . phpt ” > . / < / d i r e c t o r y >
5 </ t e s t s u i t e >
6 </ t e s t s u i t e s >
7 </ p h p u n i t >
Testen State of the Art 14 / 40
35. Outline
Unit Testing
Testing File Uploads
Behavior Driven Development
Automated Acceptance Tests
Testing JavaScript
Outlook
Testen State of the Art 15 / 40
36. Behavior Driven Development
Behavior Driven Development (BDD)
Deeper coupling between
Business Analysis
Quality Assurance
Basic ideas:
Use cases as requirements
Make use cases executable
Verify functionality
Advantages:
Communication
Documentation
Short Human readable tests
Testen State of the Art 16 / 40
37. Behavior Driven Development
Behavior Driven Development (BDD)
Deeper coupling between
Business Analysis
Quality Assurance
Basic ideas:
Use cases as requirements
Make use cases executable
Verify functionality
Advantages:
Communication
Documentation
Short Human readable tests
Testen State of the Art 16 / 40
38. Behavior Driven Development
Behavior Driven Development (BDD)
Deeper coupling between
Business Analysis
Quality Assurance
Basic ideas:
Use cases as requirements
Make use cases executable
Verify functionality
Advantages:
Communication
Documentation
Short Human readable tests
Testen State of the Art 16 / 40
39. Behavior Driven Development
Behavior Driven Development (BDD)
Deeper coupling between
Business Analysis
Quality Assurance
Basic ideas:
Use cases as requirements
Make use cases executable
Verify functionality
Advantages:
Communication
Documentation
Short Human readable tests
Testen State of the Art 16 / 40
40. The BDD process
1. Define a Domain Specific Language (DSL)
2. Describe Use Cases
3. Implement against Use Case execution
Testen State of the Art 17 / 40
41. Behat
BDD framework for PHP
http://behat.org/
Inspired by Ruby’s Cucumber
Getting started:
http://docs.behat.org/quick_intro.html
Testen State of the Art 18 / 40
42. Defining Features
1 Feature :
2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API
3
4 Scenario : Admin user can d e l e t e a s e c t i o n
5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
6 And I am logged i n as user ” admin ”
7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s
9
10 Scenario : Anonymous user cannot d e l e t e a s e c t i o n
11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
12 And I am logged i n as user ” anonymous ”
13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”
Testen State of the Art 19 / 40
43. Defining Features
1 Feature :
2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API
3
4 Scenario : Admin user can d e l e t e a s e c t i o n
5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
6 And I am logged i n as user ” admin ”
7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s
9
10 Scenario : Anonymous user cannot d e l e t e a s e c t i o n
11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
12 And I am logged i n as user ” anonymous ”
13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”
Testen State of the Art 19 / 40
44. Defining Features
1 Feature :
2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API
3
4 Scenario : Admin user can d e l e t e a s e c t i o n
5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
6 And I am logged i n as user ” admin ”
7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s
9
10 Scenario : Anonymous user cannot d e l e t e a s e c t i o n
11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
12 And I am logged i n as user ” anonymous ”
13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”
Testen State of the Art 19 / 40
45. Defining Features
1 Feature :
2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API
3
4 Scenario : Admin user can d e l e t e a s e c t i o n
5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
6 And I am logged i n as user ” admin ”
7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s
9
10 Scenario : Anonymous user cannot d e l e t e a s e c t i o n
11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
12 And I am logged i n as user ” anonymous ”
13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”
Testen State of the Art 19 / 40
46. Defining Features
1 Feature :
2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API
3
4 Scenario : Admin user can d e l e t e a s e c t i o n
5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
6 And I am logged i n as user ” admin ”
7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s
9
10 Scenario : Anonymous user cannot d e l e t e a s e c t i o n
11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
12 And I am logged i n as user ” anonymous ”
13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”
Testen State of the Art 19 / 40
47. Defining Features
1 Feature :
2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API
3
4 Scenario : Admin user can d e l e t e a s e c t i o n
5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
6 And I am logged i n as user ” admin ”
7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s
9
10 Scenario : Anonymous user cannot d e l e t e a s e c t i o n
11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
12 And I am logged i n as user ” anonymous ”
13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”
Testen State of the Art 19 / 40
48. Defining Features
1 Feature :
2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API
3
4 Scenario : Admin user can d e l e t e a s e c t i o n
5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
6 And I am logged i n as user ” admin ”
7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s
9
10 Scenario : Anonymous user cannot d e l e t e a s e c t i o n
11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
12 And I am logged i n as user ” anonymous ”
13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”
Testen State of the Art 19 / 40
49. Implementing the DSL
16 class F ea tu re Co nt ex t extends BehatContext
17 {
48 /∗ ∗
49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /
50 ∗/
51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )
52 {
53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;
54
55 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;
56 $ s e c t i o n C r e a t e −>name = $name ;
57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;
58
59 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e
);
60 }
112 }
Testen State of the Art 20 / 40
50. Implementing the DSL
16 class F ea tu re Co nt ex t extends BehatContext
17 {
48 /∗ ∗
49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /
50 ∗/
51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )
52 {
53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;
54
55 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;
56 $ s e c t i o n C r e a t e −>name = $name ;
57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;
58
59 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e
);
60 }
112 }
Testen State of the Art 20 / 40
51. Implementing the DSL
16 class F ea tu re Co nt ex t extends BehatContext
17 {
48 /∗ ∗
49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /
50 ∗/
51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )
52 {
53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;
54
55 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;
56 $ s e c t i o n C r e a t e −>name = $name ;
57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;
58
59 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e
);
60 }
112 }
Testen State of the Art 20 / 40
52. Implementing the DSL
16 class F ea tu re Co nt ex t extends BehatContext
17 {
48 /∗ ∗
49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /
50 ∗/
51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )
52 {
53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;
54
55 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;
56 $ s e c t i o n C r e a t e −>name = $name ;
57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;
58
59 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e
);
60 }
112 }
Testen State of the Art 20 / 40
53. Implementing the DSL
16 class F ea tu re Co nt ex t extends BehatContext
17 {
48 /∗ ∗
49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /
50 ∗/
51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )
52 {
53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;
54
55 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;
56 $ s e c t i o n C r e a t e −>name = $name ;
57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;
58
59 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e
);
60 }
112 }
Testen State of the Art 20 / 40
54. Implementing the DSL
16 class F ea tu re Co nt ex t extends BehatContext
17 {
48 /∗ ∗
49 ∗ @Given / ˆ I have a s e c t i o n w i t h name ” ( [ ˆ ” ] + ) ” and i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /
50 ∗/
51 public function i H a v e A S e c t i o n W i t h N a m e A n d I d e n t i f i e r ( $name , $ i d e n t i f i e r )
52 {
53 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;
54
55 $sectionCreate = $ s e c t i o n S e r v i c e −> n e w S e c t i o n C r e a t e S t r u c t ( ) ;
56 $ s e c t i o n C r e a t e −>name = $name ;
57 $ s e c t i o n C r e a t e −> i d e n t i f i e r = $ i d e n t i f i e r ;
58
59 $ t h i s −> runtimeData [ $ i d e n t i f i e r ] = $ s e c t i o n S e r v i c e −> c r e a t e S e c t i o n ( $ s e c t i o n C r e a t e
);
60 }
112 }
Testen State of the Art 20 / 40
55. When . . .
1 Feature :
2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API
3
4 Scenario : Admin user can d e l e t e a s e c t i o n
5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
6 And I am logged i n as user ” admin ”
7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s
9
10 Scenario : Anonymous user cannot d e l e t e a s e c t i o n
11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
12 And I am logged i n as user ” anonymous ”
13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”
Testen State of the Art 21 / 40
56. Implementing the DSL
16 class F ea tu re Co nt ex t extends BehatContext
17 {
69 /∗ ∗
70 ∗ @When / ˆ I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /
71 ∗/
72 public function i D e l e t e S e c t i o n W i t h I d e n t i f i e r ( $ i d e n t i f i e r )
73 {
74 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;
75
76 try
77 {
78 $ s e c t i o n = $ s e c t i o n S e r v i c e −> l o a d S e c t i o n B y I d e n t i f i e r ( $ i d e n t i f i e r ) ;
79 $ s e c t i o n S e r v i c e −> d e l e t e S e c t i o n ( $ s e c t i o n ) ;
80 }
81 catch ( E x c e p t io n $e )
82 {
83 $ t h i s −> runtimeData [ ’ e x c e p t i o n ’ ] = $e ;
84 }
85 }
112 }
Testen State of the Art 22 / 40
57. Implementing the DSL
16 class F ea tu re Co nt ex t extends BehatContext
17 {
69 /∗ ∗
70 ∗ @When / ˆ I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” ( [ ˆ ” ] + ) ” $ /
71 ∗/
72 public function i D e l e t e S e c t i o n W i t h I d e n t i f i e r ( $ i d e n t i f i e r )
73 {
74 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;
75
76 try
77 {
78 $ s e c t i o n = $ s e c t i o n S e r v i c e −> l o a d S e c t i o n B y I d e n t i f i e r ( $ i d e n t i f i e r ) ;
79 $ s e c t i o n S e r v i c e −> d e l e t e S e c t i o n ( $ s e c t i o n ) ;
80 }
81 catch ( E x c e p t io n $e )
82 {
83 $ t h i s −> runtimeData [ ’ e x c e p t i o n ’ ] = $e ;
84 }
85 }
112 }
Testen State of the Art 22 / 40
58. Then . . .
1 Feature :
2 BDD t e s t s f o r t h e eZ P u b l i s h S e c t i o n API
3
4 Scenario : Admin user can d e l e t e a s e c t i o n
5 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
6 And I am logged i n as user ” admin ”
7 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
8 Then I expect no s e c t i o n f o r i d e n t i f i e r ” MySectionKey ” e x i s t s
9
10 Scenario : Anonymous user cannot d e l e t e a s e c t i o n
11 Given I have a s e c t i o n w i t h name ” MySection ” and i d e n t i f i e r ” MySectionKey ”
12 And I am logged i n as user ” anonymous ”
13 When I d e l e t e s e c t i o n w i t h i d e n t i f i e r ” MySectionKey ”
14 Then I expect d e l e t e f a i l e s w i t h an e x c e p t i o n o f t y p e ” u n a u t h o r i z e d ”
Testen State of the Art 23 / 40
59. Implementing the DSL
16 class F ea tu re Co nt ex t extends BehatContext
17 {
87 /∗ ∗
88 ∗ @Then / ˆ I expect no s e c t i o n f o r i d e n t i f i e r ” ( [ ˆ ” ] + ) ” e x i s t s $ /
89 ∗/
90 public function i E x p e c t N o S e c t i o n F o r I d e n t i f i e r E x i s t s ( $ i d e n t i f i e r )
91 {
92 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;
93 try
94 {
95 $ s e c t i o n S e r v i c e −> l o a d S e c t i o n B y I d e n t i f i e r ( $ i d e n t i f i e r ) ;
96 throw new RuntimeException ( ” I d e n t i f i e r { $ i d e n t i f i e r } s t i l l exists . ” ) ;
97 }
98 catch ( eZ P u b l i s h API R e p o s i t o r y E x c ep t i o ns NotFoundException $e )
99 {
100 }
101 }
112 }
Testen State of the Art 24 / 40
60. Implementing the DSL
16 class F ea tu re Co nt ex t extends BehatContext
17 {
87 /∗ ∗
88 ∗ @Then / ˆ I expect no s e c t i o n f o r i d e n t i f i e r ” ( [ ˆ ” ] + ) ” e x i s t s $ /
89 ∗/
90 public function i E x p e c t N o S e c t i o n F o r I d e n t i f i e r E x i s t s ( $ i d e n t i f i e r )
91 {
92 $ s e c t i o n S e r v i c e = $ t h i s −> r e p o s i t o r y −> g e t S e c t i o n S e r v i c e ( ) ;
93 try
94 {
95 $ s e c t i o n S e r v i c e −> l o a d S e c t i o n B y I d e n t i f i e r ( $ i d e n t i f i e r ) ;
96 throw new RuntimeException ( ” I d e n t i f i e r { $ i d e n t i f i e r } s t i l l exists . ” ) ;
97 }
98 catch ( eZ P u b l i s h API R e p o s i t o r y E x c ep t i o ns NotFoundException $e )
99 {
100 }
101 }
112 }
Testen State of the Art 24 / 40
61. Outline
Unit Testing
Testing File Uploads
Behavior Driven Development
Automated Acceptance Tests
Testing JavaScript
Outlook
Testen State of the Art 25 / 40
62. Automated Acceptance Tests
High level system tests
Assert user expectations
Good start for refactoring
Approaches
HTTP spider (e.g. Goutte)
Browser controller (e.g. Selenium, Sahi)
Browser emulator (e.g. Zombie.js)
Testen State of the Art 26 / 40
63. Mink
Acceptance Test Framework
http://mink.behat.org
Abstraction for underlying tools
Full integration into Behat
Testen State of the Art 27 / 40
64. Testing a Registration Form
1 Feature : R e g i s t r a t i o n
2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r
3 As a normal user
4 I must perform a r e g i s t r a t i o n
5
6 Scenario : Test a d e f a u l t r e g i s t r a t i o n
7 Given I am on ” / r e g i s t e r ”
8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ”
9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”
10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”
11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”
12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”
13 And I f i l l in ” register personal password ” with ” sindelfingen ”
14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”
15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”
16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”
17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”
18 And I f i l l i n ” zipcode ” w i t h ”45886”
19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”
20 And I s e l e c t ” 2 ” from ” c o u n t r y ”
21 And I press ” r e g i s t e r b u t t o n ”
22 Then I should see ” Willkommen , Tobias S c h l i t t ”
Testen State of the Art 28 / 40
65. Testing a Registration Form
1 Feature : R e g i s t r a t i o n
2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r
3 As a normal user
4 I must perform a r e g i s t r a t i o n
5
6 Scenario : Test a d e f a u l t r e g i s t r a t i o n
7 Given I am on ” / r e g i s t e r ”
8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ”
9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”
10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”
11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”
12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”
13 And I f i l l in ” register personal password ” with ” sindelfingen ”
14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”
15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”
16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”
17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”
18 And I f i l l i n ” zipcode ” w i t h ”45886”
19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”
20 And I s e l e c t ” 2 ” from ” c o u n t r y ”
21 And I press ” r e g i s t e r b u t t o n ”
22 Then I should see ” Willkommen , Tobias S c h l i t t ”
Testen State of the Art 28 / 40
66. Testing a Registration Form
1 Feature : R e g i s t r a t i o n
2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r
3 As a normal user
4 I must perform a r e g i s t r a t i o n
5
6 Scenario : Test a d e f a u l t r e g i s t r a t i o n
7 Given I am on ” / r e g i s t e r ”
8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ”
9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”
10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”
11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”
12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”
13 And I f i l l in ” register personal password ” with ” sindelfingen ”
14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”
15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”
16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”
17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”
18 And I f i l l i n ” zipcode ” w i t h ”45886”
19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”
20 And I s e l e c t ” 2 ” from ” c o u n t r y ”
21 And I press ” r e g i s t e r b u t t o n ”
22 Then I should see ” Willkommen , Tobias S c h l i t t ”
Testen State of the Art 28 / 40
67. Testing a Registration Form
1 Feature : R e g i s t r a t i o n
2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r
3 As a normal user
4 I must perform a r e g i s t r a t i o n
5
6 Scenario : Test a d e f a u l t r e g i s t r a t i o n
7 Given I am on ” / r e g i s t e r ”
8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ”
9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”
10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”
11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”
12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”
13 And I f i l l in ” register personal password ” with ” sindelfingen ”
14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”
15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”
16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”
17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”
18 And I f i l l i n ” zipcode ” w i t h ”45886”
19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”
20 And I s e l e c t ” 2 ” from ” c o u n t r y ”
21 And I press ” r e g i s t e r b u t t o n ”
22 Then I should see ” Willkommen , Tobias S c h l i t t ”
Testen State of the Art 28 / 40
68. Testing a Registration Form
1 Feature : R e g i s t r a t i o n
2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r
3 As a normal user
4 I must perform a r e g i s t r a t i o n
5
6 Scenario : Test a d e f a u l t r e g i s t r a t i o n
7 Given I am on ” / r e g i s t e r ”
8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ”
9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”
10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”
11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”
12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”
13 And I f i l l in ” register personal password ” with ” sindelfingen ”
14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”
15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”
16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”
17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”
18 And I f i l l i n ” zipcode ” w i t h ”45886”
19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”
20 And I s e l e c t ” 2 ” from ” c o u n t r y ”
21 And I press ” r e g i s t e r b u t t o n ”
22 Then I should see ” Willkommen , Tobias S c h l i t t ”
Testen State of the Art 28 / 40
69. Testing a Registration Form
1 Feature : R e g i s t r a t i o n
2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r
3 As a normal user
4 I must perform a r e g i s t r a t i o n
5
6 Scenario : Test a d e f a u l t r e g i s t r a t i o n
7 Given I am on ” / r e g i s t e r ”
8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ”
9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”
10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”
11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”
12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”
13 And I f i l l in ” register personal password ” with ” sindelfingen ”
14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”
15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”
16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”
17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”
18 And I f i l l i n ” zipcode ” w i t h ”45886”
19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”
20 And I s e l e c t ” 2 ” from ” c o u n t r y ”
21 And I press ” r e g i s t e r b u t t o n ”
22 Then I should see ” Willkommen , Tobias S c h l i t t ”
Testen State of the Art 28 / 40
70. Testing a Registration Form
1 Feature : R e g i s t r a t i o n
2 I n o r d e r t o perform a c t i o n s l i k e an o r d e r
3 As a normal user
4 I must perform a r e g i s t r a t i o n
5
6 Scenario : Test a d e f a u l t r e g i s t r a t i o n
7 Given I am on ” / r e g i s t e r ”
8 When I s e l e c t ” p r i v a t e ” from ” r e g i s t e r p e r s o n a l c u s t o m e r t y p e ”
9 And I s e l e c t ” mr ” from ” r e g i s t e r p e r s o n a l s a l u t a t i o n m r ”
10 And I f i l l i n ” f i r s t n a m e ” w i t h ” Tobias ”
11 And I f i l l i n ” lastname ” w i t h ” S c h l i t t ”
12 And I f i l l i n ” r e g i s t e r p e r s o n a l e m a i l ” w i t h ” ts01@qafoo . com ”
13 And I f i l l in ” register personal password ” with ” sindelfingen ”
14 And I f i l l in ” register personal passwordConfirmation ” with ” sindelfingen ”
15 And I f i l l i n ” phone ” w i t h ”+49 ( 0 ) 209 40501252”
16 And I f i l l i n ” s t r e e t ” w i t h ” Bochumer S t r a s s e ”
17 And I f i l l i n ” s t re e tn u mb e r ” w i t h ” 2 2 6 ”
18 And I f i l l i n ” zipcode ” w i t h ”45886”
19 And I f i l l i n ” c i t y ” w i t h ” G el s en k ir c he n ”
20 And I s e l e c t ” 2 ” from ” c o u n t r y ”
21 And I press ” r e g i s t e r b u t t o n ”
22 Then I should see ” Willkommen , Tobias S c h l i t t ”
Testen State of the Art 28 / 40
71. Outline
Unit Testing
Testing File Uploads
Behavior Driven Development
Automated Acceptance Tests
Testing JavaScript
Outlook
Testen State of the Art 29 / 40
72. Testing JavaScript
You test PHP . . .
. . . do you test JavaScript?
Testen State of the Art 30 / 40
73. js-test-driver
Test runner for JavaScript
https://code.google.com/p/js-test-driver/
Test JS inside browsers
Run JS tests from command line
No HTML required
Testen State of the Art 31 / 40
74. Simple JS Code
1 var H e l l o = function ( ) { } ;
2 Hello . prototype = {
3 say : function ( who ) {
4 i f ( who === u n d e fi n e d ) {
5 who = ” World ” ;
6 }
7 r e t u r n ” H e l l o ” + who + ” ! ” ;
8 }
9 }
Testen State of the Art 32 / 40
75. Simple JS Code
1 var H e l l o = function ( ) { } ;
2 Hello . prototype = {
3 say : function ( who ) {
4 i f ( who === u n d e fi n e d ) {
5 who = ” World ” ;
6 }
7 r e t u r n ” H e l l o ” + who + ” ! ” ;
8 }
9 }
Testen State of the Art 32 / 40
76. Simple JS Code
1 var H e l l o = function ( ) { } ;
2 Hello . prototype = {
3 say : function ( who ) {
4 i f ( who === u n d e fi n e d ) {
5 who = ” World ” ;
6 }
7 r e t u r n ” H e l l o ” + who + ” ! ” ;
8 }
9 }
Testen State of the Art 32 / 40
77. Simple JS Code
1 var H e l l o = function ( ) { } ;
2 Hello . prototype = {
3 say : function ( who ) {
4 i f ( who === u n d e fi n e d ) {
5 who = ” World ” ;
6 }
7 r e t u r n ” H e l l o ” + who + ” ! ” ;
8 }
9 }
Testen State of the Art 32 / 40
78. A Simple Test Case
1 TestCase ( ” H e l l o o b j e c t t e s t s ” , {
2 ” setUp ” : function ( ) {
3 t h i s . h e l l o = new H e l l o ( ) ;
4 },
5 ” t e s t C a l l a b l e W i t h o u t A r g u m e n t ” : function ( ) {
6 a s s e r t E q u a l s ( ” H e l l o World ! ” , t h i s . h e l l o . say ( ) ) ;
7 },
8 ” t e s t C a l l a b l e W i t h A r g u m e n t ” : function ( ) {
9 a s s e r t E q u a l s ( ” H e l l o Jakob ! ” , t h i s . h e l l o . say ( ” Jakob ” ) ) ;
10 }
11 }) ;
Testen State of the Art 33 / 40
79. A Simple Test Case
1 TestCase ( ” H e l l o o b j e c t t e s t s ” , {
2 ” setUp ” : function ( ) {
3 t h i s . h e l l o = new H e l l o ( ) ;
4 },
5 ” t e s t C a l l a b l e W i t h o u t A r g u m e n t ” : function ( ) {
6 a s s e r t E q u a l s ( ” H e l l o World ! ” , t h i s . h e l l o . say ( ) ) ;
7 },
8 ” t e s t C a l l a b l e W i t h A r g u m e n t ” : function ( ) {
9 a s s e r t E q u a l s ( ” H e l l o Jakob ! ” , t h i s . h e l l o . say ( ” Jakob ” ) ) ;
10 }
11 }) ;
Testen State of the Art 33 / 40
80. A Simple Test Case
1 TestCase ( ” H e l l o o b j e c t t e s t s ” , {
2 ” setUp ” : function ( ) {
3 t h i s . h e l l o = new H e l l o ( ) ;
4 },
5 ” t e s t C a l l a b l e W i t h o u t A r g u m e n t ” : function ( ) {
6 a s s e r t E q u a l s ( ” H e l l o World ! ” , t h i s . h e l l o . say ( ) ) ;
7 },
8 ” t e s t C a l l a b l e W i t h A r g u m e n t ” : function ( ) {
9 a s s e r t E q u a l s ( ” H e l l o Jakob ! ” , t h i s . h e l l o . say ( ” Jakob ” ) ) ;
10 }
11 }) ;
Testen State of the Art 33 / 40
81. A Simple Test Case
1 TestCase ( ” H e l l o o b j e c t t e s t s ” , {
2 ” setUp ” : function ( ) {
3 t h i s . h e l l o = new H e l l o ( ) ;
4 },
5 ” t e s t C a l l a b l e W i t h o u t A r g u m e n t ” : function ( ) {
6 a s s e r t E q u a l s ( ” H e l l o World ! ” , t h i s . h e l l o . say ( ) ) ;
7 },
8 ” t e s t C a l l a b l e W i t h A r g u m e n t ” : function ( ) {
9 a s s e r t E q u a l s ( ” H e l l o Jakob ! ” , t h i s . h e l l o . say ( ” Jakob ” ) ) ;
10 }
11 }) ;
Testen State of the Art 33 / 40
82. Configuring js-test-driver
1 s e r v e r : h t t p : / / l o c a l h o s t :4223
2
3 load :
4 − src / hello . j s
5 − tests / hello . js
Testen State of the Art 34 / 40
83. Configuring js-test-driver
1 s e r v e r : h t t p : / / l o c a l h o s t :4223
2
3 load :
4 − src / hello . j s
5 − tests / hello . js
Testen State of the Art 34 / 40
84. Running the Tests
First shell
1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− p o r t 4223
Capture browser
http://localhost:4223/capture
Second shell
1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− t e s t s a l l
2 s e t t i n g runnermode QUIET
3 ....
4 T o t a l 4 t e s t s ( Passed : 4 ; F a i l s : 0 ; E r r o r s : 0 ) ( 1 . 0 0 ms)
5 F i r e f o x 12.0 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 1 . 0 0 ms)
6 Chrome 18 .0 .1 025 .1 68 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 0 . 0 0 ms)
Testen State of the Art 35 / 40
85. Running the Tests
First shell
1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− p o r t 4223
Capture browser
http://localhost:4223/capture
Second shell
1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− t e s t s a l l
2 s e t t i n g runnermode QUIET
3 ....
4 T o t a l 4 t e s t s ( Passed : 4 ; F a i l s : 0 ; E r r o r s : 0 ) ( 1 . 0 0 ms)
5 F i r e f o x 12.0 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 1 . 0 0 ms)
6 Chrome 1 8 .0 .1 025 .1 68 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 0 . 0 0 ms)
Testen State of the Art 35 / 40
86. Running the Tests
First shell
1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− p o r t 4223
Capture browser
http://localhost:4223/capture
Second shell
1 $ j a v a − j a r J s T e s t D r i v e r − 1 . 3 . 4 . b . j a r −− t e s t s a l l
2 s e t t i n g runnermode QUIET
3 ....
4 T o t a l 4 t e s t s ( Passed : 4 ; F a i l s : 0 ; E r r o r s : 0 ) ( 1 . 0 0 ms)
5 F i r e f o x 12.0 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 1 . 0 0 ms)
6 Chrome 18 .0 .1 025 .1 68 L i n u x : Run 2 t e s t s ( Passed : 2 ; F a i l s : 0 ; E r r o r s 0 ) ( 0 . 0 0 ms)
Testen State of the Art 35 / 40
87. More on JS Testing
JavaScript QA: js-test-driver und Sinon.JS richtig nutzen
Jakob Westhoff (Qafoo)
Wednesday, 14:00, Salon 2
Testen State of the Art 36 / 40
88. Outline
Unit Testing
Testing File Uploads
Behavior Driven Development
Automated Acceptance Tests
Testing JavaScript
Outlook
Testen State of the Art 37 / 40
89. Unified Build Process
https://github.com/manuelpichler/build-commons
Framework for build scripts
Unified targets, like
test
package
verify
deploy
39 supported tool extensions
PHPUnit, CodeSniffer, PDepend, PHPMD, PHPCPD, . . .
DB, DBDeploy, PHAR, Pirum, . . .
js-test-driver, JSLint, JSHint, . . .
...
Testen State of the Art 38 / 40
90. Unified Build Process
https://github.com/manuelpichler/build-commons
Framework for build scripts
Unified targets, like
test
package
verify
deploy
39 supported tool extensions
PHPUnit, CodeSniffer, PDepend, PHPMD, PHPCPD, . . .
DB, DBDeploy, PHAR, Pirum, . . .
js-test-driver, JSLint, JSHint, . . .
...
Testen State of the Art 38 / 40
91. Unified Build Process
https://github.com/manuelpichler/build-commons
Framework for build scripts
Unified targets, like
test
package
verify
deploy
39 supported tool extensions
PHPUnit, CodeSniffer, PDepend, PHPMD, PHPCPD, . . .
DB, DBDeploy, PHAR, Pirum, . . .
js-test-driver, JSLint, JSHint, . . .
...
Testen State of the Art 38 / 40
92. Continuous Performance
Does your app still perform?
Find performance regressions
Ensure optimizations persist
Apache JMeter
https://jmeter.apache.org/
Complex user paths
Concurrend requests
Record on proxy
Clustering
Integrate into Continuous Integration
Immer am Optimum, Continuous Performance
Wednesday, 16:15, Salon 2
Kore Nordmann, Manuel Pichler
Testen State of the Art 39 / 40
93. Continuous Performance
Does your app still perform?
Find performance regressions
Ensure optimizations persist
Apache JMeter
https://jmeter.apache.org/
Complex user paths
Concurrend requests
Record on proxy
Clustering
Integrate into Continuous Integration
Immer am Optimum, Continuous Performance
Wednesday, 16:15, Salon 2
Kore Nordmann, Manuel Pichler
Testen State of the Art 39 / 40
94. Continuous Performance
Does your app still perform?
Find performance regressions
Ensure optimizations persist
Apache JMeter
https://jmeter.apache.org/
Complex user paths
Concurrend requests
Record on proxy
Clustering
Integrate into Continuous Integration
Immer am Optimum, Continuous Performance
Wednesday, 16:15, Salon 2
Kore Nordmann, Manuel Pichler
Testen State of the Art 39 / 40
95. Continuous Performance
Does your app still perform?
Find performance regressions
Ensure optimizations persist
Apache JMeter
https://jmeter.apache.org/
Complex user paths
Concurrend requests
Record on proxy
Clustering
Integrate into Continuous Integration
Immer am Optimum, Continuous Performance
Wednesday, 16:15, Salon 2
Kore Nordmann, Manuel Pichler
Testen State of the Art 39 / 40
96. Thanks for Listening
Please rate this talk at
https://joind.in/event/IPC2012
(Slides: http://talks.qafoo.com)
Stay in touch
Manuel Pichler Tobias Schlitt
manuel@qafoo.com toby@qafoo.com
@manuelp / @qafoo @tobySen / @qafoo
Rent a PHP quality expert:
http://qafoo.com
Testen State of the Art 40 / 40
97. Thanks for Listening
Please rate this talk at
https://joind.in/event/IPC2012
(Slides: http://talks.qafoo.com)
Stay in touch
Manuel Pichler Tobias Schlitt
manuel@qafoo.com toby@qafoo.com
@manuelp / @qafoo @tobySen / @qafoo
Rent a PHP quality expert:
http://qafoo.com
Testen State of the Art 40 / 40
98. Thanks for Listening
Please rate this talk at
https://joind.in/event/IPC2012
(Slides: http://talks.qafoo.com)
Stay in touch
Manuel Pichler Tobias Schlitt
manuel@qafoo.com toby@qafoo.com
@manuelp / @qafoo @tobySen / @qafoo
Rent a PHP quality expert:
http://qafoo.com
Testen State of the Art 40 / 40