The slides from my presentation @Frontmania on the 16th of November 2018

1. Made each other
!!
How ,
and

2. The app
Automation setup
Happy developers
Script once
Optimisation
Time
User experience
Pro’s and cons
Menu

3. Conversational
" #
User interface
The app

4. " #
User interface
After install
$ %
onBoarding
The app

5. Movie
The app

6. The app
Automation setup
Happy developers
Script once
Optimisation
Time
User experience
Pro’s and cons
Menu

7. Appium + Appium Desktop
webdriver.io
CucumberJS
multiple-cucumber-html-report
Amsterdam 2017
Automation setup
Detox versus Appium
London 2018

8. The app
Automation setup
Happy developers
Script once
Optimisation
Time
User experience
Pro’s and cons
Menu

9. Happy developers
Why?

10. Remove flakiness / inconsistent testdata problem
Test errors / delays / edge cases
Speed up manual and automated tests
Who doesn’t want to be happy? &
Happy developers
Why?

11. Remove flakiness / inconsistent testdata problem
Test errors / delays / edge cases
Speed up manual and automated tests
Who doesn’t want to be happy? &
Happy developers
Why?
With ng-apimock
How

12. Ng-apimock

13. Happy developers
Why?
Remove flakiness / inconsistent testdata problem
Test errors / delays / edge cases
Speed up manual and automated tests
Who doesn’t want to be happy? &
With ng-apimock
How

14. The app
Automation setup
Happy developers
Script once
Optimisation
Time
User experience
Pro’s and cons
Menu

15. Script once
AccessibilityLabels (98%)
XPath (2%)
HyBrid solution
Testproperties
Implementation
Platform specific
Code

16. Testproperties
/**
* If we have a test build, add a unique test id for iOS and Android
*
* @param {string} id
*
* @return {object|*}
*/
function testProperties(id) {
if (IS_AUTOMATION_BUILD) {
if (IS_IOS) {
return {
testID: `test-${id}`,
};
}
return {
accessibilityLabel: `test-${id}`,
};
}
return null;
}

17. Testproperties
/**
* If we have a test build, add a unique test id for iOS and Android
*
* @param {string} id
*
* @return {object|*}
*/
function testProperties(id) {
if (IS_AUTOMATION_BUILD) {
if (IS_IOS) {
return {
testID: `test-${id}`,
};
}
return {
accessibilityLabel: `test-${id}`,
};
}
return null;
}

18. Implementation
import React, { Component } from 'react';
import { View } from ‘react-native';
import { testProperties } from './config/automation/TestProperties';
class MultipleChoice extends Component {
// Some code
render() {
const { options } = this.props;
return (
<View
style={styles.container}
accessibilityLabel="Wijzig uw keuze"
{…testProperties('Select menu')}
>
// The select implementation
</View>
);
}
}

19. Implementation
import React, { Component } from 'react';
import { View } from ‘react-native';
import { testProperties } from './config/automation/TestProperties';
class MultipleChoice extends Component {
// Some code
render() {
const { options } = this.props;
return (
<View
style={styles.container}
accessibilityLabel="Wijzig uw keuze"
{…testProperties('Select menu')}
>
// The select implementation
</View>
);
}
}

20. AccessiBility laBel

21. AccessiBility laBel

22. Script once
AccessibilityLabels (98%)
XPath (2%)
HyBrid solution
Testproperties
Implementation
Platform specific
Code

23. The app
Automation setup
Happy developers
Script once
Optimisation
Time
User experience
Pro’s and cons
Menu

24. Shortcuts
Parallel execution
Animations
Optimisation

25. 95%Behind onBoarding
So….
Shortcuts

26. Behind onBoarding
So….
Or
$ %
Best test feature
Or Do some
Shortcuts

27. $ %
Best test feature
Or Do some
'
magic?
Magic with
Shortcuts

28. '
magic?
Magic with
(
For 5 features
Shortcuts

29. Testlink screen
import TestLinkScreen from './TestLinkScreen';
/**
* Create the test link stack if it's a mocked build
*
* @return {*}
*/
function testLinkStack() {
if (MOCKED_ENVIRONMENTS.includes(ENVIRONMENT)) {
return {
[TEST_LINK_SCREEN]: {
screen: TestLinkScreen,
},
};
}
return {};
}

30. 95%Behind onBoarding
So….
Shortcuts

31. Shortcuts
Parallel execution
Animations
Optimisation

32. Parallel execution
Not
… each scenario on each )
… all )) have the same ⚡
… all )) start with same scenario at same ⏱
… same API response on all ))
Prevent

33. Ng-apimock
Mockserver
Post: api/version/login
"Content-Type": "application/json",
"Accept": “application/json"
response:
{
"status": 200,
"firstName": "Wim",
"lastName": “Selles"
}

34. Ng-apimock
Mockserver
weBdriver.io
PUT:
headers: {
'Content-Type': ‘application/json'
},
method: 'PUT',
body: {
identifier: 'token',
scenario: 'unauthorised'
}

35. Ng-apimock
Mockserver
weBdriver.io

36. The idea
Mockserver
weBdriver.io
IDFV: 51A82F08-CCAC-410F

37. The idea
Mockserver
weBdriver.io
IDFV: 51A82F08-CCAC-410F

38. The idea
Mockserver
weBdriver.io
PUT:
headers: {
'Content-Type': ‘application/json’,
ngapimockid: '51A82F08-CCAC-410F'
},
method: 'PUT',
body: {
identifier: 'token',
scenario: 'unauthorised'
}
IDFV: 51A82F08-CCAC-410F

39. response:
{
"status": 401,
“message": “unauthorised"
}
The idea
Mockserver
weBdriver.io
PUT:
headers: {
'Content-Type': ‘application/json’,
ngapimockid: '51A82F08-CCAC-410F'
},
method: 'PUT',
body: {
identifier: 'token',
scenario: 'unauthorised'
}
IDFV: 51A82F08-CCAC-410F
Post: api/version/login
"Content-Type": "application/json",
"Accept": “application/json”,
ngapimockid: '51A82F08-CCAC-410F'

40. /**
* This is the config-file we use for calling all the API’s.
* Below you will see a part of the config to attach the headers
* to all API calls
*/
import { uniqueAutomationApiHeaderId } from './automation/TestProperties';
const apiConfig = create({
baseURL: BASE_URL,
headers: {
...API_HEADERS,
...uniqueAutomationApiHeaderId(),
},
});
Header

41. /**
* This is a part from our `TestProperties`-file.
*/
import { getUniqueID } from 'react-native-device-info';
/**
* If it is an automation build, then add a new header for the API calls
*
* @return {Object}
*
* For iOS it will return something like: FCDBD8EF-62FC-4ECB-B2F5-92C9E79AC7F9
* For Android it will return something like: dd96dec43fb81c97
*/
function uniqueAutomationApiHeaderId() {
if (IS_AUTOMATION_BUILD) {
return {
ngapimockid: getUniqueID(),
};
}
return {};
}
app method

42. Testlink screen

43. Testlink screen

44. Testlink screen

45. Testlink screen

46. /**
* Executes the api call with the provided information.
*
* @param {string} httpMethod PUT|DELETE|POST|GET
* @param {string} urlSuffix Which path to httpMethod the data to
* @param {Object} options The data that needs to be httpMethod
* @param {string} errorMessage The error that needs to be shown
*
* @return {Promise} The promise.
*
* @private
*/
function execute(httpMethod, urlSuffix, options, errorMessage) {
const data = {
headers: {
'Content-Type': 'application/json',
ngapimockid: device.uniqueID,
},
method: httpMethod,
};
if (options !== undefined) {
data.body = JSON.stringify(options);
}
return handleRequest(urlSuffix, data, errorMessage);
}
Set state
headers: {
'Content-Type': ‘application/json’,
ngapimockid: '51A82F08-CCAC-410F'
},
method: 'PUT',
body: {
identifier: 'token',
scenario: 'unauthorised'
}

47. Parallel execution
Not
… each scenario on each )
… all )) have the same ⚡
… all )) start with same scenario at same ⏱
… same API response on all ))
Prevent

48. Shortcuts
Parallel execution
Animations
Optimisation

49. Animations
onBoarding
Each bubble takes 500 ms to show
Min 15 bubbles => 7,5 sec
Max 25 bubbles => 12,5 sec
Bubble delay is 800 ms
Between 19 (15*800+7,5) and 32 sec (25*800+12,5) animations
What?!

50. Remove animations
import { AppRegistry } from 'react-native';
import App from './app/App';
import { setupAutomation } from './app/config/automation/TestProperties';
// Run the automation setup
setupAutomation();
AppRegistry.registerComponent('Tele2CustomerApp', () => App);

51. Remove animations
const stubs = require('stubs');
/**
* Setup the app for a specific automation build
*/
function setupAutomation() {
if (!IS_AUTOMATION_BUILD) {
return;
}
disableAnimations();
}
/**
* Disable all animations
*/
function disableAnimations() {
const AnimatedTiming = Animated.timing;
stubs(Animated, 'timing', (...props) => {
props[1].duration = 0;
props[1].delay = 0;
return AnimatedTiming(...props);
});
}

52. Result
Animations + real services No animation + mocking

53. Result
Animations + real services No animation + mocking
15 sec!40 sec!

54. Animations
onBoarding
Each bubble takes 500 ms to show
Min 15 bubbles => 7,5 sec
Max 25 bubbles => 12,5 sec
Bubble delay is 800 ms
Between 19 (15*800+7,5) and 32 sec (25*800+12,5) animations
What?!

55. Shortcuts
Parallel execution
Animations
Optimisation

56. The app
Automation setup
Happy developers
Script once
Optimisation
Time
User experience
Pro’s and cons
Menu

57. time
per scenario
Mocking: ~4 API calls (4 x 2 sec.) ~ 8 sec.
Animations: ~ 9 sec.
Testlinkscreen: ~15 sec.
Currently 65 scenario’s
Average savings

58. time
per scenario
Mocking: ~4 API calls (4 x 2 sec.) ~ 8 sec.
Animations: ~ 9 sec.
Testlinkscreen: ~15 sec.
Currently 65 scenario’s
Average savings
Wow!
We just saved more than
30 min
on total execution time!
Please revert! Proceed!

59. time
per scenario
Mocking: ~4 API calls (4 x 2 sec.) ~ 8 sec.
Animations: ~ 9 sec.
Testlinkscreen: ~15 sec.
Currently 65 scenario’s
Average savings

60. The app
Automation setup
Happy developers
Script once
Optimisation
Time
User experience
Pro’s and cons
Menu

61. User experience

62. How?
Automate comparing images

63. How?
Automate comparing images
Core:
• Comparison:
• ResembleJS (https://github.com/HuddleEng/Resemble.js)

64. How?
Automate comparing images
Core:
• Comparison:
• ResembleJS (https://github.com/HuddleEng/Resemble.js)
• Logic:
• WebdriverIO element selection
• Appium understanding (UI-hierarchy)
• Experience from the past with my protractor-image-comparison module

65. HOW-2-USE
// wdio.conf.js
exports.config = {
// ...
plugins: {
'wdio-native-app-compare': {
baselineFolder: '.dist/image-compare/baseline',
screenshotPath: '.dist/image-compare/screenshots',
// See Options for more options
// ..
},
},
// ...
};

66. HOW-2-USE
/**
* Save element
*/
// Default
device.saveElement(device.element('~your-accessibility-id'), 'name-of-your-file');
// Shorthand
device.saveElement($('~your-accessibility-id'), 'name-of-your-file');
/**
* Save screen
*/
device.saveScreen(‘name-of-your-file');

67. HOW-2-USE
/**
* Compare element
*/
expect(
device.compareElement(
$('~your-accessibility-id'),
'name-of-your-file',
{
// .. options here
},
).misMatchPercentage
).toEqual(0);
/**
* Compare screen
*/
expect(
device.compareScreen(
'name-of-your-file',
{
// .. options here
},
).misMatchPercentage
).toEqual(0);

68. React Native
0.54
React Native
0.57
, - .

69. React Native
0.54
React Native
0.57
, - .

70. , - .

71. Report

72. Report

73. User experience

74. The app
Automation setup
Happy developers
Script once
Optimisation
Time
User experience
Pro’s and cons
Menu

75. Pro’s and cons
Pro’s
Increase quality
Speed up development
Test what really needs to be tested
Speed up execution time
Cons
No real services tested (automated)
App is not a production app
Developers often debug app for seeing
no animations.
(use the wrong build /)

76. The app
Automation setup
Happy developers
Script once
Optimisation
Time
User experience
Pro’s and cons
Menu

78. I hope we have some time left, so are there any
questions?

79. Can you show everybody my contact details?
I hope we have some time left, so are there any
questions?
Thank you very much!
Sure!
Contact
0: Wim Selles - Sauce Labs
1: @wswebcreation
2: wswebcreation.nl
2: gitHub.com/wswebcreation
3 Goodbye! 3