Developing complex web applications without systematic strategies for its quality assurance will lead to maintenance nightmares. End-to-end testing, from the moment the implementation is being planned up to the stage of deployment, requires careful planning and a suitable set of tools. In this talk, the use of various tools to assist such testing strategies will be uncovered. This involves the use of scriptable headless page automation (PhantomJS, Zombie.js, Selenium) for fast smoke testing as part of the precommit process and for more comprehensive preflight checks in the continuous integration system. In addition, a wide spectrum of testing possibilities will be achieved with extra tools such as static code analyzer, full-stack execution logger, profiler, and code coverage/instrumentation.

1. Strategies for End-to-End
Ariya Hidayat
Web Apps Testing

2. Amazon Kindle
Cloud Reader
Financial Times

3. Edit
Develop
QA

4. Code Editing

5. Do not provoke ambiguities
Avoid silly mistakes
Learn better code pattern
Write readable code
Improve future maintenance

6. JavaScript Engine Building Blocks
Built-in objects,
host objects, ... Runtime
Syntax
Tree
Virtual
Source Parser Machine/
Interpreter
Fast and
conservative

7. Parsing Stage
identifier number
var answer = 42;
keyword equal sign semicolon
Variable Declaration
Identifier
Literal Constant

8. JavaScript Parser (Written in JavaScript)
Esprima UglifyJS
Narcissus
ZeParser
Traceur
Es-Lab

9. Syntax Tree Visualization
answer = 42

10. Style Formatter https://github.com/fawek/codepainter
Sample code
Source CodePainter Formatted source
Indentation
Infer coding Quote for string
styles literal
Whitespace

11. Code Outline Eclipse
Functions
Variables

12. Application Structure
Ext.define('My.sample.Person', { Class manifest
name: 'Mr. Unknown',
age: 42,
{
constructor: function(name) {}, className: 'My.sample.Person',
functions: ['walk', 'run'],
walk: function(steps) {} properties: ['name', 'age']
run: function(steps) {} }
});

13. Content Assist (aka Intellisense aka Autocomplete)
Eclipse
http://contraptionsforprogramming.blogspot.com/2012/02/better-javascript-content-assist-in.html

14. Error Tolerant
Mismatched quote var msg =
"Hello’;
Too many dots person..age = 18;
Incomplete, still typing if (person.
'use strict';
Strict mode violation with (person) {
}

15. Code Linting
var fs = require('fs'),
Not a strict equal esprima = require('./esprima'),
files = process.argv.splice(2);
files.forEach(function (filename) {
if (x == 9) { var content = fs.readFileSync(filename, 'utf-8'),
// do Something syntax = esprima.parse(content, { loc: true });
} JSON.stringify(syntax, function (key, value) {
if (key === 'test' && value.operator === '==')
console.log('Line', value.loc.start.line);
return value;
});
});

16. Copy Paste Mistake
function inside(point, rect) {
return (point.x >= rect.x1) && (point.y >= rect.y1) &&
(point.x <= rect.x2) && (point.y <= rect.y1);
}
Wrong
check

17. “Boolean Trap” Finder http://ariya.ofilabs.com/2011/08/hall-of-api-shame-boolean-trap.html
Obfuscated choice var volumeSlider = new Slider(false);
Double-negative component.setHidden(false);
filter.setCaseInsensitive(false);
Can you make up your mind? treeItem.setState(true, false);
The more the merrier event.initKeyEvent("keypress", true, true,
null, null, false, false, false, false, 9,
0);

18. Refactoring Helper
// Add two numbers // Add two numbers
function add(firt, two) { function add(first, two) {
return firt + two; return first + two;
} }

19. Syntax = Message
var Foo = (function () {
return { module Foo {
bar: function (x, y) { export function bar (x, y) {
// do Something // do something
} }
}; }
};
ES 5.1 Harmony
Today ES 6/7, Future

20. Transpilation
// Object literal property shorthand.
function Point(x, y) {
Harmony return { x, y };
}
// Object literal property shorthand.
ES 5.1 function Point(x, y) {
return { x: x, y: y };
}

21. Development

22. Unit Test Framework
http://www.trollquotes.org/image/5-gandalf-troll-quote.jpg

23. Automatic Reload
Modified source Reload content

24. Remote Interaction
http://www.sencha.com/blog/remote-javascript-debugging-on-android/
Weinre, Adobe Shadow, Remote Web Inspector,...

25. Fast Smoke Test
Feedback
Cycle

26. Precommit Check
Typical Scenario
This is
awesome!
un
t to r
forge tests
the
Git: precommit hook
http://ariya.ofilabs.com/2012/03/git-pre-commit-hook-and-smoke-testing.html

27. Zombie.js
var Browser = require("zombie");
browser = new Browser();
browser.visit("http://mobileconference.nl", function () {
console.log(browser.text("title"));
});

28. PhantomJS (Headless WebKit)
Paint
Normal Browser
Layout Display
Paint
X
Headless Operation
Layout Display

29. Screen Capture = Pixel-Perfect Comparison
phantomjs rasterize.js URL output

30. CasperJS
var casper = require('casper').create();
casper.start('http://www.mobileconference.nl/', function() {
this.echo('Page title is: ' + this.evaluate(function() {
return document.title;
}), 'INFO');
});
casper.run();

31. Postcommit Quality Metrics
Quality Factor 1
Check-in
Quality Factor 2

32. Identifier Length Distribution
mean of the identifier length is 8.27 characters
750
500
250
0
0 5 10 15 20 25 30 35 40 45
prototype-1.7.0.0.js SCRIPT_ELEMENT_REJECTS_TEXTNODE_APPENDING
prototype-1.7.0.0.js MOUSEENTER_MOUSELEAVE_EVENTS_SUPPORTED
jquery-1.7.1.js subtractsBorderForOverflowNotVisible
jquery.mobile-1.0.js getClosestElementWithVirtualBinding
prototype-1.7.0.0.js HAS_EXTENDED_CREATE_ELEMENT_SYNTAX
http://ariya.ofilabs.com/2012/05/javascript-identifier-length-distribution.html

33. Statement Distribution
ExpressionStatement 6728
BlockStatement 6353
IfStatement 3063
ReturnStatement 2878
var fs = require('fs'),
VariableDeclaration 2116 esprima = require('esprima'),
FunctionDeclaration 371 files = process.argv.splice(2);
ForStatement 293
files.forEach(function (filename) {
ForInStatement 143
var content = fs.readFileSync(filename, 'utf-8'),
WhileStatement 131 syntax = esprima.parse(content);
BreakStatement 115
JSON.stringify(syntax, function (key, value) {
TryStatement 84
if (key === 'type') {
EmptyStatement 66
if (value.match(/Declaration$/) ||
ThrowStatement 38
value.match(/Statement$/)) {
SwitchStatement 35 console.log(value);
ContinueStatement 25 }
}
DoWhileStatement 12
return value;
LabeledStatement 6
});
});
http://ariya.ofilabs.com/2012/04/most-popular-javascript-statements.html

34. QA

35. Platform Proliferation

36. BusterJS
Test Suite
Server Interface
Test Server

37. Selenium
Firefox
Opera WebDriver Chrome
WebDriver WebDriver
WebDriver
WebDriver Client

38. Code Coverage https://github.com/itay/node-cover
https://github.com/coveraje/coveraje
https://github.com/pmlopes/coberturajs
x = 42;
if (false)
x = -1;
http://ariya.ofilabs.com/2012/03/javascript-code-coverage-and-esprima.html

39. Instrumentation for Code Coverage
{
var a = 5;
Statement __statement_ZyyqFc(1);
var a = 5;
}
{
__statement_ZyyqFc(2);
foo();
Expression __expression_kC$jur(3), foo();
}
function foo() {
function foo() { __block_n53cJc(1);
Block ... ...
}; }
http://itay.github.com/snug_codecoverage_slides/

40. Performance via Benchmarks.js
var suite = new Benchmark.Suite;
suite.add('String#indexOf', function() {
'Hello World!'.indexOf('o') > -1;
})
.on('complete', function() {
console.log('Fastest is ' +
this.filter('fastest').pluck('name'));
})
.run();
JSPerf.com

41. Performance Baseline
Baseline
Execution time
Application revision

42. Scalability via Run-time Complexity Analysis
Array.prototype.swap = function (i, j) {
var k = this[i]; this[i] = this[j]; this[j] = k;
}
Array.prototype.swap = function (i, j) {
Log({ name: 'Array.prototype.swap', lineNumber: 1,
range: [23, 94] });
var k = this[i]; this[i] = this[j]; this[j] = k;
}
http://ariya.ofilabs.com/2012/01/scalable-web-apps-the-complexity-issue.html

43. Execution Tracing
4640 function calls
https://gist.github.com/1823129
jQuery Mobile startup log
jquery.js 26 jQuery
jquery.js 103 init undefined, undefined,
[object Object]
jquery.js 274 each (Function)
jquery.js 631 each [object Object],
(Function), undefined
jquery.js 495 isFunction [object Object]
jquery.js 512 type [object Object]
jquery.mobile.js 1857 [Anonymous]
jquery.mobile.js 642 [Anonymous]
jquery.mobile.js 624 enableMouseBindings
jquery.mobile.js 620 disableTouchBindings
http://ariya.ofilabs.com/2012/02/tracking-javascript-execution-during-startup.html

44. Wrap-up

45. Coding style/lint, semantic outline, autocomplete,
Edit API usage, refactoring, better syntax
Test framework, remote control, precommit check,
Develop automatic reload, fast smoke testing, preflight
metrics
Comprehensive browser driver, code coverage,
QA performance baseline, execution tracking

46. To boldly analyze what no man has analyzed before...

47. Thank You
ariya.hidayat@gmail.com
ariya.ofilabs.com
@AriyaHidayat