This session will build up on the ColdBox RESTful services session by creating a TestBox test suite for the RESTful service. We will review everything from installing TestBox, creating BDD specs, running them in different formats and finally automate it via Jenkins for continuous integration.

1. BDD Testing & Automation
from the Trenches Into the Box 2016
Gavin Pickin

2. ★Who Am I?
★Gavin Pickin – developing Web Apps since late 90s
○Ortus Solutions Software Consultant
○ContentBox Evangelist
★What else do you need to know?
○CFMLRepo.com http://www.cfmlrepo.com
○Blog - http://www.gpickin.com
○Twitter – http://twitter.com/gpickin
○Github - https://github.com/gpickin
★Lets get on with the show.

3. ★State of the Room
★ A few questions for you guys
★ If you have arms, use them.

4. ★State of the Room
Testing? What’s testing?

5. ★State of the Room
Yeah,
I’ve heard of it.
Why do you
think I’m here?

6. ★State of the Room
Yes I know I should be testing,
but I’m not sure how to do it

7. ★State of the Room
My Boss and my Customers wouldn’t
let me

8. ★State of the Room
I’m a tester

9. ★State of the Room
I’m a test writing ninja
Call me Majano,
Luis Majano

10. ★Ways to Test your Code
★Click around in
the browser yourself
★Setup Selenium / Web Driver to click around for
you
★Structured Programmatic Tests

11. ★Types of Testing

12. ★Types of Testing
★Black/White Box
★Unit Testing
★Integration Testing
★Functional Tests
★System Tests
★End to End Tests
★Sanity Testing
★Regression Test
★Acceptance Tests
★Load Testing
★Stress Test
★Performance Tests
★Usability Tests
★+ More

13. ★Levels of Testing

14. ★Cost of a Bug
The bug will cost one way or another

15. ★Integration Testing

16. ★Integration Testing
★Integration Tests several of the pieces
together
★Most of the types of tests are variations of
an Integration Test
★Can include mocks but can full end to end
tests including DB / APIs

17. ★Unit Testing

18. ★Unit Testing
“unit testing is a software verification
and validation method in which a
programmer tests if individual units of
source code are fit for use. A unit is the
smallest testable part of an application”
- wikipedia

19. ★Unit Testing
★Can improve code quality -> quick error
discovery
★Code confidence via immediate verification
★Can expose high coupling
★Will encourage refactoring to produce >
testable code
★Remember: Testing is all about behavior and
expectations

20. ★Styles – TDD vs BDD
★TDD = Test Driven Development
○Write Tests
○Run them and they Fail
○Write Functions to Fulfill the Tests
○Tests should pass
○Refactor in confidence
★Test focus on Functionality

21. ★Styles – TDD vs BDD
★BDD = Behavior Driven Development
Actually similar to TDD except:
○Focuses on Behavior and Specifications
○Specs (tests) are fluent and readable
○Readability makes them great for all levels of
testing in the organization
★Hard to find TDD examples in JS that are not
using BDD describe and it blocks

22. ★TDD Example
Test( ‘Email address must not be blank’,
function(){
notEqual(email, “”, "failed");
});

23. ★BDD Example
Describe( ‘Email Address’, function(){
It(‘should not be blank’, function(){
expect(email).not.toBe(“”);
});
});

24. ★Matchers
expect(true).toBe(true);
expect(true).toBe(true);
expect(true).toBe(true);
expect(true).toBe(true);

25. ★Matchers
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);

26. ★Matcher Samples
expect(true).toBe(true);
expect(a).not.toBe(null);
expect(a).toEqual(12);
expect(message).toMatch(/bar/);
expect(message).toMatch("bar");
expect(message).not.toMatch(/quux/);
expect(a.foo).toBeDefined();
expect(a.bar).not.toBeDefined();

27. ★CF Testing Tools
★MxUnit was the standard
★TestBox is the new standard
★Other options

28. ★TestBox
TestBox is a next generation testing framework for
ColdFusion (CFML) that is based on BDD (Behavior Driven
Development) for providing a clean obvious syntax for
writing tests.
It contains not only a testing framework, runner,
assertions and expectations library but also ships with
MockBox, A Mocking & Stubbing Framework,.
It also supports xUnit style of testing and MXUnit
compatibilities.

29. ★TestBox TDD Example
function testHelloWorld(){
$assert.includes( helloWorld(), ”world" );
}

30. ★TestBox BDD Example
describe("Hello world function", function() {
it(”contains the word world", function() {
expect(helloWorld()).toContain("world");
});
});

31. ★TestBox New BDD Example
feature( "Box Size", function(){
describe( "In order to know what size box I need
As a distribution manager
I want to know the volume of the box", function(){
scenario( "Get box volume", function(){
given( "I have entered a width of 20
And a height of 30
And a depth of 40", function(){
when( "I run the calculation", function(){
then( "the result should be 24000", function(){
// call the method with the arguments and test the outcome
expect( myObject.myFunction(20,30,40) ).toBe( 24000 );
});
});
});

32. ★Using Testing in your
Workflow
★Using HTML Test Runners
○Keep a Browser open
○F5 refresh tests

33. ★Command Line Tests
★Run testbox – manual
○Run tests at the end of each section of work
★Run Grunt-Watch – automatic
○Runs testbox on every file change
○Grunt can run other tasks as well,
minification etc

34. ★Testing in your IDE
★Browser Views
○Eclipse allows you to open files in web view –
uses HTML Runner
★Run testbox/ Grunt / in IDE Console
○Fairly Easy to setup
○See Demo– Sublime Text 3 (if we have time)

35. ★Installing Testbox
★Install Testbox – Thanks to Commandbox
○box install testbox
★Decide how you want to run Testbox

36. ★Create a runner.cfm
<cfsetting showDebugOutput="false">
<!--- Executes all tests in the 'specs' folder with simple reporter by
default --->
<cfparam name="url.reporter" default="simple">
<cfparam name="url.directory" default="tests.specs">
<cfparam name="url.recurse" default="true" type="boolean">
<cfparam name="url.bundles" default="">
<cfparam name="url.labels" default="">
<!--- Include the TestBox HTML Runner --->
<cfinclude template="/testbox/system/runners/HTMLRunner.cfm" >

37. ★Create a Test Suite
// tests/specs/CFCTest.cfc
component extends="testbox.system.BaseSpec" {
function run() {
it( "will error with incorrect login", function(){
var oTest = new cfcs.userServiceRemote();
expect( oTest.login( 'gavin@gavin.com',
'topsecret').result ).toBe('400');
});
}
}

38. ★Create a 2nd
Test Suite
// tests/specs/APITest.cfc
component extends="testbox.system.BaseSpec" {
function run() {
describe("userService API Login", function(){
it( "will error with incorrect login", function(){
var email = "gavin@gavin.com";
var password = "topsecret”;
var result = "";
http url="http://127.0.0.1:8504/cfcs/userServiceRemote.cfc?
method=login&email=#email#&password=#password#" result="result”;
expect( DeserializeJSON(result.filecontent).result ).toBe('400');
});
});
}
}

39. ★ Running Testbox with runner.cfm
*Install Testbox Runner – Thanks Sean Coyne
*npm install testbox-runner
*Install Grunt Shell
*npm install grunt-shell
*Add Grunt Configuration

40. ★Running Testbox
with Grunt Watch
★Install Testbox Runner – Thanks Sean Coyne
○npm install testbox-runner
★Install Grunt Shell
○npm install grunt-shell
★Add Grunt Configuration

41. ★Adding TextBox Config 1
module.exports = function (grunt) {
grunt.loadNpmTasks('grunt-shell');
grunt.initConfig({ … })
}

42. ★Adding TextBox Config 2
Watch: {
…
cfml: {
files: [ "tests/*.cfc"],
tasks: [ "testbox" ]
}
}

43. ★Adding TextBox Config 3
shell: {
testbox: {
command: "./node_modules/testbox-
runner/index.js --colors --runner http://127.
0.0.1:53874/tests/runner.cfm --directory
/tests/specs --recurse true”
}
}

44. ★Adding TextBox Config 4
grunt.registerTask("testbox", [ "shell:testbox" ]);
grunt.loadNpmTasks('grunt-contrib-jasmine');
grunt.loadNpmTasks('grunt-contrib-watch');

45. ★GruntFile.js Gists
Simple Jasmine + Testbox Example
https://gist.github.
com/gpickin/9fc82df3667eeb63c7e7

46. ★Testbox output with Grunt

47. ★Testbox Runner JSON
★Testbox has several runners, you have seen the
HTML one, this Runner uses the JSON runner and
then formats it.
★http://127.0.0.1:53874/tests/runner.cfm?
reporter=json

48. ★Testbox Runner JSON
{"totalSuites":3,"startTime":1465879026042,"bundleStats":[{"TOTALSUITES":1,"STARTTIME":
1465879026042,"TOTALPASS":0,"TOTALDURATION":60,"TOTALSKIPPED":0,"TOTALFAIL":0,"TOTALSPECS":
0,"PATH":"tests.specs.integration.api.BaseAPITest","ENDTIME":1465879026102,"DEBUGBUFFER":[],"
TOTALERROR":0,"NAME":"tests.specs.integration.api.BaseAPITest","ID":"1DDCA037-FF86-4E9B-
B62231A290304E9F","SUITESTATS":[{"STARTTIME":1465879026102,"TOTALPASS":0,"TOTALDURATION":0,"
TOTALSKIPPED":0,"TOTALFAIL":0,"TOTALSPECS":0,"BUNDLEID":"1DDCA037-FF86-4E9B-
B62231A290304E9F","STATUS":"Skipped","PARENTID":"","SPECSTATS":[],"ENDTIME":1465879026102,"
TOTALERROR":0,"NAME":"tests.specs.integration.api.BaseAPITest","ID":"231E5335-408A-41A4-
AFC0A1199989F05E","SUITESTATS":[]}],"GLOBALEXCEPTION":""},{"TOTALSUITES":2,"STARTTIME":
1465879026102,"TOTALPASS":4,"TOTALDURATION":2395,"TOTALSKIPPED":0,"TOTALFAIL":0,"
TOTALSPECS":4,"PATH":"tests.specs.integration.api.UsersAPITest","ENDTIME":1465879028497,"
DEBUGBUFFER":[],"TOTALERROR":0,"NAME":"tests.specs.integration.api.UsersAPITest","ID":"8B3B6F5D-
C7F2-4E77-B701A9F646061CB3","SUITESTATS":[{"STARTTIME":1465879026571,"TOTALPASS":1,"
TOTALDURATION":472,"TOTALSKIPPED":0,"TOTALFAIL":0,"TOTALSPECS":1,"BUNDLEID":"8B3B6F5D-C7F2-
4E77-B701A9F646061CB3","STATUS":"Passed","PARENTID":"","SPECSTATS":[{"ERROR":{},"STARTTIME":
1465879026571,"TOTALDURATION":472,"FAILORIGIN":{},"STATUS":"Passed","SUITEID":"60F5FB8B-3E97-
46C0-9165FDD893DF08B4","ENDTIME":1465879027043,"NAME":"Tests the ability to create a user","ID":"
EBF1E06B-80A9-476E-9AA4889200FD48DC","FAILMESSAGE":""}],"ENDTIME":1465879027043,"TOTALERROR":
0,"NAME":"CRUD API Methods and Retrieval","ID":"60F5FB8B-3E97-46C0-9165FDD893DF08B4","
SUITESTATS":[]},{"STARTTIME":1465879027043,"TOTALPASS":3,"TOTALDURATION":1354,"TOTALSKIPPED":
0,"TOTALFAIL":0,"TOTALSPECS":3,"BUNDLEID":"8B3B6F5D-C7F2-4E77-B701A9F646061CB3","STATUS":"
Passed","PARENTID":"","SPECSTATS":[{"ERROR":{},"STARTTIME":1465879027043,"TOTALDURATION":501,"
FAILORIGIN":{},"STATUS":"Passed","SUITEID":"4CBC4C98-BD18-4C0D-BE187715F5C1BFAB","ENDTIME":

49. ★Running in Sublime Text 2
★Install PackageControl into Sublime Text
★Install Grunt from PackageControl
○https://packagecontrol.io/packages/Grunt
★Update Grunt Sublime Settings for paths
{
"exec_args": { "path": "/bin:/usr/bin:/usr/local/bin” }
}
★Then Ctrl / Command Shift P – grunt

50. ★Running in Sublime Text 2

51. ★Continuous Integration

52. Travis CI

53. Travis CI

55. Jenkins

56. Jenkins

57. ★Q&A
★Any questions?