** Full webinar recording can be seen here: https://youtu.be/b2D8WQCOCJw ** In this session -- hosted by Sumeet Mandloi, Engineering Director @ Dow Jones (Wall Street Journal), and Eran Barlev (Sr. Customer Success Engineer @ Applitools) -- you’ll learn how you can easily avoid front-end bugs and visual regressions, as well as substantially increase coverage, by adding automated visual testing to your existing automated tests. In addition, Mr. Mandloi shared real-life tips on how to run automated visual testing at scale, with implementation examples from Dow Jones. Watch this session, and learn how to: -- Successfully perform large-scale automated visual testing -- Leverage visual testing to increase coverage, while reducing maintenance efforts -- Run cross-browser tests with visual validation in the cloud -- Add visual validations to your existing automated functional and unit tests

1. 1
Automated Visual Testing at Scale:
A Dow Jones Case Study
Sumeet Mandloi – Dow Jones
Eran Barlev - Applitools

2. Automated
Visual
Testing:
It’s Really
Happening…
1) Why we need it.
2) What it is.
3) How we scale it.
4) Dow Jones case study.

3. Why We Need It
3

4. Your Brand Experience Is Now Digital
4
OS Types Browser Versions Devices & Screen TypesMicroservices

5. Releases Are Expected Faster
5
Watch
TestCode
ReleaseOperate
• Customers today expect new
features to be added routinely
• CI-CD is an emerging practice
necessary to compete in today’s
environment

6. Brand Experiences Suffers as a Result
6
Watch
TestCode
ReleaseOperate
• Customers today expect new
features to be added routinely
• CI-CD is an emerging practice
necessary to compete in today’s
environment
• These two trends intersect at the
UI and break it routinely,
negatively impacting customer
experience

7. It Happens to Microsoft
7

8. It Happens to
UPS
8

9. It happens to
Twitter
9

10. It Happens to Amazon
10

11. It Used to Happen to Dow Jones
11

12. What It Is
12

13. Our Mission
13
To ensure visually
perfect user experiences
on every browser and
device through
automated visual-AI
testing and monitoring.

14. What Is Visual Testing?
14
• Visual Testing is the process
of validating all the visual
aspects of an application’s UI
on all platforms
• Goes beyond functional
testing tools like Selenium to
ensure things like colors, fonts,
buttons appear correctly

15. Visual AI Is The Key
15
RELIABLE
MATURE
SCALABLE
The World’s Largest
Database of Visual UI Tests
Seamless Integration
Virtually No False-Positives

16. It’s Really Easy
16
Drive the AUT and
take screenshots
Compare screenshots
with baseline images
Report differences
Update the baseline
Step 1 Step 2
Step 3Step 4

17. Complete the Test Automation Chain
17
Phase
Tool
Unit Testing Functional Testing

18. Complete the Test Automation Chain
18
Phase
Tool
Unit Testing Functional Testing Visual Testing

19. How We Scale It
19

20. Tip #1
20
USE A ROBUST IMAGE
COMPARISON ENGINE

21. Say Sikuli One More Time!

22. 22

23. 23

24. 24

25. 25

26. 26

27. 27

28. 28

29. - 1 pixel offsets in element positioning
- dynamic content moving elements
- images of different size
- performance
And More....
29

30. Tip #2
30
USE A SINGLE BASELINE ACROSS
DEVICES AND BROWSERS

31. 31
S
T
R
I
C
T
V 1.0
V 2.0
V 3.0
LAYOUT
LAYOUT
LAYOUT

32. Tip #3
32
BASELINE MAINTENANCE COULD BE
CODELESS
(Everybody Can Do It, Immediately)

33. Automated Visual
Testing at Dow Jones
33

34. 34

35. “Coming together is a
beginning, staying together
is progress, and working
together is success.”
- Henry Ford
35

36. Goals of Quality Engineering
36
True partnering with business
Switch from defect tracking to assuring
superior customer experience
Speeding development and test process
Preventing, just not detecting defects
Improve CoQ (Cost of Quality)
1
2
3
4
5

37. Digital is Visual
37
• Web platforms and mobile apps are the face of the
business
• Customers demand the best and the first impression
matters the most
• Respond to business needs quickly

38. Digital Is Visual
38
Mobile apps quality is critical
• Harder to roll back
changes
• Can’t push daily
• Updates take
battery and data
• Higher quality bar

39. Digital Is Visual
39
The test matrix is too big to cover manually
• Web browsers
• Devices
• Operating systems
• Screen resolutions
• Responsive design
• L10n
• 3rd party upgrades

40. Digital Is Fast
40
Business needs today demand shorter release cycles

41. QA Is The Bottleneck
63%
Believe QA is the
bottleneck in
DevOps
70%
Of testing is still
done manually
72%
Agree continuous
testing requires a
major shift in
culture

42. Our Solution: Applitools
42
Validates the content,
data, layout and
appearance of all the
UI elements with a
single command
Validates the visual
aspects of your
application/ website UI
on all platforms
Covers functional
testing of all the data
that is visible through
the UI

43. It Actually Works
43
SOFTWARE
BANKING & FINANCIAL
RETAIL
INSURANCE
HEALTHCARE

44. Features We Value at Dow Jones
44
Single page validation replaces hundreds of lines of validation
code and hours of manual testing
Automatic maintenance – approved changes are automatically
propagated to other tests and execution environments
Cognitive vision – ignore differences that are invisible to the
human eye and avoid false positives

45. Features We Value At Dow Jones
45
Easily test complex application pages including dynamic content, size
and moving elements
Turnkey Solution – no setup costs
Automatically test your app in multiple languages
Seamless integration with existing test automation and ALM solution

46. 46

47. 47

48. 48

49. Key Integration
49
• Applitools easily integrates with any CI-CD framework
• 40+ SDKs for leading functional testing platforms, such as: Selenium, Appium, HP
UFT/QTP, MS Coded UI, etc…
• Supports JAVA, python, JS, Ruby, .Net
• Integration with Perfecto, Sauce Labs, cloud device farms
• Jenkins automation environment plug-in

50. ROI
50
• Reduce time spent on manual visual QA by 90%
• Reduce visual testing coding efforts by 50%
• Improve test automation coverage across browsers and devices up to 3 times
• Capture 20% more regressions and UI bugs pre-release
POC – Reduced total testing efforts from 30-40 hours to 2 hours

51. Who can benefit from this?
51
• Software Engineers building Web Platforms and Mobile Apps
• Design and UX Teams
• Marketing and Advertising
• Customer Experience
• Operations

52. Thank You!
52