This document discusses extending continuous integration (CI) practices to mobile projects by integrating real devices into the testing process. It summarizes:
1. Plugging real devices into Jenkins allows for automatically deploying builds and performing automated functional testing on each build cycle.
2. A demonstration shows connecting a mobile testing cloud to a Jenkins build to deploy apps and run tests in parallel on real devices.
3. Using real devices for testing provides real feedback that emulators cannot, and helps shift testing left in the development cycle for continuous quality.
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Â
Automate mobile testing in your Jenkins process with real devices
1. Plug Real Devices into Your
Jenkins Process
Carlo Cadet, Director Product Marketing
Uzi Elion, Director Technology
2. Session Abstract
Plug real devices into your Jenkins process and automate testing
Join Perfecto Mobileâs discussion and say Yes to extending your CI process to mobile
projects.
â Yes, you can automatically deploy builds on real devices in every Jenkins build.
â Yes, you can perform automated functional testing every build cycle.
â Yes, you can shift left with early single user performance profiling.
More than theory, watch Perfecto Mobile demonstrate an end-to-end Jenkins build
while pointing out the âgotchasâ of automated app testing on real devices executed in
parallel.
3. Audience Poll
âą Have you extended CI practices to mobile projects?
âą Satisfied
â Not Really
â Somewhat
â Getting it Done!
âą Blockers?
4. Agenda
âą Assembling the ingredients to successfully
extend CI practices to mobile
âą Demonstrating it in action
5. Mobile Moments are Transforming Enterprises
Business Perspective
âą Complex processes are
being reconceived as
discrete single function
tasks
⊠what does the user want
to do right now?
Development Perspective
âą Rapidly deliver Minimum
Marketable Features (MMF)
7. Emulators Suck
Why?
âą False sense of confidence
Real devices = Real Feedback
â Pace of market innovation
â Device and OS fragmentation
â Network conditions
â Interoperability
â Context (sensors)
â User interaction modes
â Client side processing
â Public app reviews
â Immediate download/engage/abandon
decision cycle
8. Continuous Quality Requires the Right Lab
1. Real end user conditions
2. Enterprise grade
â Test ready, Consistent, Stable,
Secure
3. Open and integrated
4. Global
5. Collaboration friendly
6. Hybrid cloud deployment
model
7. Elastic
9. Managing a Device Lab is a B*#!ch
Test
readiness is
the Achilles
heel of
extending
CI practices
to mobile
10. Out of Cycle Testing is the Enemy of Velocity
Source: IBM
12. Mobile CI Practice ⊠Adapt Locally
Job
Test
Duration
Test Scope
Device
Coverage
Unit System Smoke Regression
Performan
ce
Per commit 15-30 Min ±2
Hourly 20-40 Min ±4
3-4 times a day 30-60 Min ±6
Nightly 2-7 Hours ±10
Weekend 10-48 Hours ±20
14. MobileCloud ⊠Open and Integrated
Existing remote web driver
clients
Device Cloud Visual DOM Device
Grid Remote web driver server
REST API
remote web driver perfecto
extensions
Cloud Auto
Remote Interactive access
& Script Recorders
15. Demo Flow
âą Intro to
â Continuous Quality Lab
â Test automation
âą Build process
â Deploy your app on real devices
â Run the tests on real devices
â Get results from Jenkins
15
16. Why CQ Lab
âą Real devices
âą Open API â work with my existing processes
âą Devices Availability
âą Rich reports embedded to Jenkins
16
Yesterday I was delighted by a new feature USAA introduced into there mobile app. While depositing checks into my daughters accounts using the remote check deposit function I discovered a new feature allowing me to share the deposit confirmation via any one of the communication apps on my device with each of them.
Jobs that are executed more frequently have low coverage from one hand but detect issues very early and provide immediate feedback for the developers.
Usually there will be Per Commit or Hourly or 3-4 times a day + Nightly + Weekend
Load tests in CI -> Single user performance + Load in low scale (e.g. 50 VU)
# of devices -> Env Coverage % ->may change based on type of change \ new devices that were released.
In addition to the CI Jobs there are additional testing activities that happens in staging environment â Exploratory testing \ Additional automation cycle on other environments \ Load testing \ Security testing \...