Join Danny McKeown, Paychex’s lead test enterprise automation architect, to learn how to climb the testing maturity curve and increase predictability and reuse, all while accelerating repeatable and reliable testing. Learn how Paychex iteratively built a well-defined web and mobile app test automation architecture. By evolving the areas of strategy, environment pre-conditions, continuous integration, and understanding their IT users, Paychex executes a mature program automating test readiness, scheduling, execution, and report distribution. Hear their lessons about strategy, and how the Test Automation Pyramid helps structure their automation architecture. Discover their environment pre-conditions, and how they are able to minimize false negative results (derailment factors) due to non-automation issues. See how Paychex uses continuous integration to bring it all together in an integrated, scalable, and parallel execution. Danny discusses lessons learned about their IT Users and how defining user test automation abilities enables better expectations for the user and project team.

1. T16
Mobile
Testing
5/5/16
13:30
Stay
Ahead
of
the
Mobile
and
Web
Testing
Maturity
Curve
Presented
by:
Danny
McKeown
Paychex
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-­‐-­‐-­‐268-­‐-­‐-­‐8770
·∙·∙
904-­‐-­‐-­‐278-­‐-­‐-­‐0524
-­‐
info@techwell.com
-­‐
http://www.stareast.techwell.com/

2. Danny
McKeown
Paychex
Danny
McKeown
has
more
than
thirty
years
of
technical
and
management
experience
in
information
technology.
With
Paychex
for
twelve
years,
Danny
has
spent
the
last
six
years
as
the
test
automation
architect.
In
this
role,
he
is
instrumental
in
implementing
a
secure
integration
of
Perfecto
Mobile's
solution
with
the
Paychex
automation
framework
that
leverages
HP
UFT
and
Selenium
technologies.
In
addition,
Danny
is
an
adjunct
lecturer
in
the
software
engineering
department
at
the
Rochester
Institute
of
Technology.

3. Unit
API
UI Unit
API
UI
T16
Danny McKeown
Staying Ahead of the Mobile and Web
Testing Maturity Curve

4. Unit
API
UI
About the Speaker
Danny McKeown has more than 30 years of technical
and management experience in information
technology. With Paychex for 12 years , Danny has
spent the last six years as the test automation
architect. In this role, he is instrumental in
implementing a secure integration of Perfecto
Mobile’s solution with the Paychex automation
framework that leverages HP UFT and Selenium
technologies . In addition, Danny is an adjunct
lecturer in the software engineering department at
the Rochester Institute of Technology.

5. Unit
API
UI
Unit
API
UI
About Paychex
 Leader in payroll, human resource, and
benefits outsourcing solutions
 Small- to medium-sized businesses
 Over half million clients
 24 x 7, 365 days per year availability
 Mobile application – Paychex Flex

6. Unit
API
UI
Unit
API
UI
Today’s Focus
 Automation Strategy
 Environment Readiness Check
 Continuous Delivery
 Automation Adoption
 Impact of Agile Process
 Impact of Responsive Technology
 Our Mobile Timeline
 What’s next…

7. Unit
API
UI
Unit
API
UI
Change does not come easy

8. Unit
API
UI
Unit
API
UI
Why Test Automation?
Continuous improvement of:
 Raising the bar on Quality
“Degree of Excellence”
 Driving Efficiency
Time to Market
 Delighting the Customers

9. Unit
API
UI
Unit
API
UI
Automation Strategy
 Does your organization have a
test automation strategy?
 Paychex adopted the
Test Automation Pyramid
 Introduced by Mike Cohn
 Built an architecture around it

10. Unit
API
UI
Unit
API
UI
Test Automation Pyramid
Page #8
Unit Tests
API Tests
Headless, Non-UI, Service,
Database, & Business Logic Testing
UI
Tests
Supports Agile and Continuous Delivery

11. Unit
API
UI
Unit
API
UI
Test Automation Software StackEncapsulation, Scalability, Reuse
Page #9
Operating System
Windows, IE, Chrome, Firefox, Edge
API
Test/Job Execution, Reports
IT Tool
Integration
Resource
Objects, Scenarios, Scenario Drivers, Tests
Repository
Engine
Database, Services, Web (10 Engines)
Core Code
Language/IDE & 3rd Party Plugins
HTML 5, C#/.Net, Visual Studio, Perfecto, Applitools, Selenium, Ranorex
Vendor
Integration
Persistent Storage
SQL, Server, Entity, Framework, NAS
SOX
Application
Suite
User

12. Unit
API
UI
Unit
API
UI
Test Environments
Contribute to false negative test results?
99.9% availability?
99% availability?
90% availability?
What is the availability?
Configured correctly?
Patches up to date?
Monitored?
Change Control?

13. Unit
API
UI
Unit
API
UI
Apply Pre-Condition Testing
Input + Pre-Conditions = Post Conditions + Output
 Execute readiness check at each pyramid tier

14. Unit
API
UI
Unit
API
UI
Executing Automated Scripts
Execute scripts often to get automation value!
On DemandScheduled
Any time, for anyone, and any IT tool?
Paychex implemented an Automation Execution
Center (AEC) that supports Continuous Delivery (CD)

15. Unit
API
UI
Unit
API
UI
Automation Execution Center
Page #13
TRANSPOSE JOB
Exposed API Server
Exposed Web ServerLDAP
Tool
Independence
QUEUE JOB
STORE RESULTS
ARCHIVE
Automation DB
Tool Agnostic
Test Job
Secure
Controlled
Physical
Access
Account
TARGET
ENVIRONMENT
Non-Prod & Prod
Environments
Cloud
Computing
TEST JOB
SUBMISSION
No Test
Automation
Tools
Your IT platform
Continuous Delivery
Integrated, Scalable, Parallel Execution
EXECUTE JOB
Tool Dependent
Parallel Processing
Load Balancing
VM Farm
(also direct log in)
Tools installed

16. Unit
API
UI
Execute mobile “build/deploy/test” 3+ cycles per day
Release
Web Service & other Non-UI Regression
Majority of Business Logic is within the Web Services
Majority of User Experience is within the Mobile UI
Perfecto Cloud Connectivity/Latest App Version Copied
Sanity
Regression
Jenkins – Orchestrates Build, Deploy, & Test

17. Unit
API
UI
Unit
API
UI
Automation Adoption
At Paychex many implement automation:
 Development – Create unit tests
 Test Automation – Develop framework & automation apps
 Build Automation – Develop build, deploy, & test pipeline
 Testers - Utilize automation apps to create & execute test scripts
Automation is a cultural shift…needed some definitions
(and goals) to assist in the transformation

18. Unit
API
UI
Unit
API
UI
Test Automation Belts
Covers from Concepts to Non-UI & UI understanding
to Script Debugging

19. Unit
API
UI
Unit
API
UI
Agile impact on automation
Page #17
CORE TEAM
Developers
Testers
Business Analysts
Scrum Master
Dev Solution Lead Product Owner
Automation
Solution Lead
assigned to
each team
 Definition of “Done” expanded to include
automation, i.e., more automation
 Developers & Testers both
implement functional auto scripts
 Developers & Testers
naturally collaborating more

20. Unit
API
UI
Unit
API
UI
Responsive Technology
 Accelerated the need for mobile
web testing:
 Chrome, Firefox, IE, Edge (future)
 Safari
 Chrome (mobile)
 Accelerated the need for CD:
 Create a sustainable build, deployment
and test pipeline
 Promotion process from early to late
environments (Test to Prod)
 Test Automation went from a
“want” to a “need”!

21. Unit
API
UI
Unit
API
UI
Mobile Timeline
Page #19
Automation Solution
Manual
Testing
2012 2013
Automated
Testing
2014
Continuous
Delivery
2015
Agile
2016
Cross
Browser

22. Unit
API
UI
Unit
API
UI
What’s next…
 Testing desktop browsers in the cloud
 Profiling our test users
 Advance cross browser testing
 Seamless testing from test to prod with same suite of scripts
 Create tools to create test data for spun-up environments (CD)

23. Unit
API
UI
Unit
API
UI
We are all trained but…

24. Unit
API
UI
Thank You