SlideShare ist ein Scribd-Unternehmen logo
1 von 32
Downloaden Sie, um offline zu lesen
Speed to Market in
Mobile Development
Finding the right solution with continuous
integration on real devices
Tina Su, Director of Development, Intuit
About me
Nearly 20 years of industry experience in R&D,
quality and engineering management, with a strong
passion for automation.
 Engineering Director at Intuit, Developer Productivity
 Sr. Engineering Director at Yahoo, Mobile Platforms
 Sr. Engineering Manager at Sun, Java Standard and
Mobile Editions
 Compiler Developer at MIPS and Tandem
Tina Su, Intuit
Agenda
 Mobile trends & developers’ challenges
 Speed & quality through continuous integration
- Intuit Virtual Device Lab
- Intuit mobile test automation
 Takeaways
Aggressive mobile momentum
“2013 Internet Trends,” KPCB
Will surpass desktop
by 2014
It should just work!
Proliferation of platforms,
OS versions & device types
Developer challenges
 Immense configuration matrix
- OS platforms (Android, iOS, Windows, phone vs. tablet, …)
- Inputs (touch/gesture, real buttons, voice, camera, GPS receiver)
- Outputs (portrait/landscape, screen resolution, language)
 Application delivery through the markets
- App signing, provision (app & device), submission
 Offline experience
- Network, different network bandwidth/latency, no-network
- HTML5 offline app
 Media challenges
- Some codecs don’t work on a device
- Streaming, sound
 Security & privacy
Intuit mobile app portfolio today
Apple
Google
Amazon
SMS
0 5 10 15 20 490
22
15
11
3
491 mobile apps for financial institutions
Ramping up Intuit’s innovation engine
Pain: Takes days per build to test
combinations of device/OS configurations
physically & manually, one by one …
QA can’t keep up with the changes
Goal: Reduce development iteration cycle
from days to minutes on an ever-growing
number of devices
Strategy: Speed & quality through mobile
continuous integration
• iPhone 4
• iPhone 4S
• iPhone 5
• iPad 2
• iPad 3
• iPad Mini
Devices
OS versions
• iOS 5.0
• iOS 5.1
• iOS 6.0
• iOS 6.1.x
• iOS 7.0
Builds
X X
Daily
builds
• Apple
• Google
• Amazon
Platforms
X
Continuous integration (CI)
What is CI?
 Automate build, test & deployment process
 Integrate & build early, often, daily, for every check-in
 Best practice of Agile development
 Jenkins is widely adopted, flexible & extensible
Benefits of CI in a mobile world
 Instant feedback on quality, functionality & system impact
 Less complex, no integration hell
 Detect and address device configuration issues early
 Involve stakeholders early
 Ultimately achieve shippable release any time
Speed and quality
through mobile CI
Key enablers:
• Mobile device lab
• Test automation
Test processorReal
devices
Jenkins
SCM:
source
repository
Developers
/IDE
Automatic
test run
Test
results
Submit the app to
app store if
passed
Check in
Check-in
triggers
automated CI
Install the test app to
real devices in
parallel
Run tests on real
devices in parallel
Analyze
test
results
Trigger testing in
mobile device lab
Process starts here
3rd-party vendor solutions didn’t work for us
 Couldn’t handle E2E automation of testing pre-release apps
- Dependency on internal resources & infrastructure
 Vendor lock-in: customized CI & test automation solutions
- No flexibility to use our existing tools & frameworks
 Disadvantages of vendors’ shared devices
- Security concern on testing on shared devices
- No access to corporate network
- Cost-prohibitive to reserve devices
 Private cloud (on-site setup) with local devices not practical
- With sensitive customer data, we need private cloud
- Extensive use of devices for daily testing of hundreds of mobile apps is expensive
Why test on real devices?
Most mobile features can be tested on simulators, but simulators
have limitations:
 Device capabilities (GPS, camera, accelerometer, gyros), form
factors & pixel density
 Real-world experience over the real connection
(3G, 4G, crappy Wi-Fi, etc.)
 True performance, security tests
 Testing on non-default Web browsers (e.g., Opera Mini)
 Not all simulator/emulators have good quality
(don’t reflect real rendering/behavior, bugs)
 Testing SMS & app integration
Intuit Virtual Device Lab
Intuit Virtual Device Lab (VDL)
Gives access to real mobile devices using a browser
Intranet
VDL
User machine
User machine
User machine
3-click access
Always on
Integrated with
dev infrastructure
Globally available
iPhone
iPad
Galaxy
Captivate
Optimus Nexus One
Desire
Incredible
Manufacturers/
devices:
Droid
Milestone
Moto
VDL architecture – Android
Windows PC
Tomcat
Java image
transferring
application
Browser &
Java applet
Internet/intr
anet
Device
interaction
app
Developer’s laptop
VDL
Image transfer
USB connection
VDL architecture – iOS
Mac with Xcode
installed
Tomcat
Java VNC
client
Port
forwarding
Browser &
Java applet
Internet/intr
anet
VNC server
USB connection
Developer’s laptop
VDL
Open source tools & technologies used in VDL
Key capabilities Android iOS
Installing mobile applications into
the real device
adb install <APK file> Fruitstrap
https://github.com/ghughes/fruitstra
p
Transferring device screen images,
controlling device
Screencast tool
http://code.google.com/p/androidsc
reencast
Veency VNC app
Fetching device local files adb pull <filename> sftp command
(jailbroken devices only)
VDL key features & functionality today
Control the device through standard
browser/RESTful APIs
Change orientation to landscape or portrait
Copy local file to the device from your PC
Get file from the device onto your local PC
Execute an ADB command on the device
See current running logs in your browser
Download history logs to your local PC
Continuous integration – how it works
Product
builds
Trigger file launches test
VDL device
inventoryResource
manager
Select a device
Test execution
engine
Results
Monito
r for
new
builds
Mobile continuous integration
Leveraging the Virtual Device Lab for test automation on multiple, real devices simultaneously.
Submit, build, test … in minutes vs. hours
Select devices & tests1 Tests run automatically across selected
devices with each build
2 See the results!3
Next step: over-the-air replaces USB connection
Fast onboarding
Scalable
Any device
Internet/intr
anet
App
installation
Test
execution
Results
retrieval
VDL
Mobile Test Automation
Mobile test automation tools
Instrumentation-based
Non-instrumentation
AndroidiOS
MonkeyRunner
MOET
Sikuli
Android SDK
Robotium
TestDroid (BitBar)
Calabash
(LessPainful)
Soasta
iOS SDK
UIAutomation
UISpec
Frank
MonkeyTalk
DA/PerfectoMob
ile
eggPlant
Cross-platform solution
iOS Driver Appium
• Assessed as of 2012
Mobile test automation trends
 Abstraction with domain-specific language (DSL)
- Common test script over different devices (e.g., Cucumber, MonkeyScript, MOET)
- Support behavior-driven development (BDD)
 Client/server-based approach (remotely driven tests via HTTP)
 Instrumentation & non-instrumentation solutions complement
each other
 Mobile cloud testing environment
- From hardware (cradle) to software approach to drive tests on
real devices
 Automated OTA app installation & tests
Intuit automation framework of choice/recommendation
 Native apps:
- Cucumber + Calabash for BDD on iOS & Android
 Mobile Web:
- Selenium/WebDriver
What is Cucumber?
It lets stakeholders (customers, QE or developers) describe how
software should behave in plain text.
 BDD: Written in business domain-specific language (DSL)
- Good abstraction over different device types
 One format for multiple purposes
- UI feature specification
- Automated tests
 Cucumber itself is NOT a test automation tool
- Add-on to any test automation framework
- Examples: Calabash, Sikuli & Frank mobile test frameworks
What is Calabash?
 Open source cross-platform mobile test framework
- iOS + Android, simulator + real device, native + hybrid
 Supports Cucumber for BDD
 Application needs instrumentation that embeds a Calabash HTTP server
- Tests run from a remote client (in a Jenkins slave)
- Tests can be driven via Wi-Fi (no USB wire)
 Ruby clients in API level (Java clients in experiment)
 Works great in touch simulation & real device testing
Intuit shared test library – rapid test development
Internal open source repository
Teams focus on
creating
Use shared test lib
… and then contribute to
common test library
Tests
Contribute
Tests
Common test library Product tests
Tests
Tests
Globalization test framework
(I18N/L10N)
Performance/load test framework
Security test framework
Native/hybrid test framework
High-level test domain-specific language
(Cucumber steps)
Mobile
back-end
Mobile
front-end
Code
build
Deploy
Test
Results
Connect
device
Notify
email
Get
devices
Consolidate
report
Upload
report
From: manual
50*5
= 250
15 30
5
10
20
15
30 10
= 6 hours
Code
build
Trigger
Execute tests
in parallel
Consolidated
report in archive
Deploy on
devices
0 5 10 0
= 15 mins
To: automated CI
TIME SAVED: 95%Release quality increased
*
Stunning results: reduced iteration cycle from hours to minutes
Tested on 5 devices
VDL cost saving over vendors’ private clouds (today)
Annual
cost ($)
# of devices
Intuit Virtual Device Lab
150
3rd-party vendor 1
local device in cradle
Assumptions: 50 native
apps & daily check-in
Note: As vendor pricing
changes, so do the savings
Savings
25
3rd-party vendor 2
local device w/o cradle
(Software approach)
Savings
Private cloud setup behind corp firewall
Key takeaways
Building custom solution vs. 3rd-party vendor solution
 Understand your technical/platform/global spread
 Many vendor solutions provide a larger set of devices/OS versions
in the global market
 Quite a few offer mobile test services – might be tightly coupled with their
device access solution & automation framework
 Connecting to your corporate network is a challenge – some provide
private cloud (on-site setup within your network) but it’s cost-prohibitive
Invest in automation and CI
 Automate tests with robust automation framework
 Enable rapid test development through shared test library
 Adopt CI in mobile development for speed & quality
More mobile innovations at Intuit
 Mobile crash reporter
 Mobile components & design patterns
 Mobile analytics SDK & gateway
Visit booth #601 – Accelerate Development
 Demos & conversations
 Contest
 Mobile device charging station
 Foosball
 We’re hiring !
Win a weekend with a TESLA!
Thank you!!
Tina_Su@intuit.com

Weitere ähnliche Inhalte

Was ist angesagt?

Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar Experitest
 
Mobile Application Testing
Mobile Application Testing Mobile Application Testing
Mobile Application Testing Shivaraj R
 
Android & iPhone App Testing
 Android & iPhone App Testing Android & iPhone App Testing
Android & iPhone App TestingSWAAM Tech
 
Mobile Application Testing Training Presentation
Mobile Application Testing Training PresentationMobile Application Testing Training Presentation
Mobile Application Testing Training PresentationMobiGnosis
 
Mobile application testing
Mobile application testingMobile application testing
Mobile application testingvodQA
 
Building the Ultimate Device Matrix
Building the Ultimate Device MatrixBuilding the Ultimate Device Matrix
Building the Ultimate Device MatrixCarly Vanderwert
 
Best Practices for DevOps in Mobile App Testing
Best Practices for DevOps in Mobile App TestingBest Practices for DevOps in Mobile App Testing
Best Practices for DevOps in Mobile App TestingBitbar
 
Cloud based Testing Mobile Apps
Cloud based Testing Mobile AppsCloud based Testing Mobile Apps
Cloud based Testing Mobile AppsIndicThreads
 
SeeTestAutomation - Mobile Test Automation Tool by Experitest
SeeTestAutomation - Mobile Test Automation Tool by ExperitestSeeTestAutomation - Mobile Test Automation Tool by Experitest
SeeTestAutomation - Mobile Test Automation Tool by ExperitestExperitest
 
Mobile application testing tutorial
Mobile application testing tutorialMobile application testing tutorial
Mobile application testing tutorialLokesh Agrawal
 
Mobile Application Testing
Mobile Application TestingMobile Application Testing
Mobile Application TestingNoor Orfahly
 
Selenium training
Selenium trainingSelenium training
Selenium trainingShivaraj R
 
Testing Techniques for Mobile Applications
Testing Techniques for Mobile ApplicationsTesting Techniques for Mobile Applications
Testing Techniques for Mobile ApplicationsIndicThreads
 
Mobile Testing Tools 101
Mobile Testing Tools 101Mobile Testing Tools 101
Mobile Testing Tools 101TechWell
 
Experitest & Hexaware Co-Webinar
Experitest & Hexaware Co-WebinarExperitest & Hexaware Co-Webinar
Experitest & Hexaware Co-WebinarExperitest
 
Mobile Testing Types and Basic Process
Mobile Testing Types and Basic ProcessMobile Testing Types and Basic Process
Mobile Testing Types and Basic ProcessOlesia Hirnyk
 
Introduction To Mobile-Automation
Introduction To Mobile-AutomationIntroduction To Mobile-Automation
Introduction To Mobile-AutomationMindfire Solutions
 

Was ist angesagt? (18)

Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar
 
Mobile Application Testing
Mobile Application Testing Mobile Application Testing
Mobile Application Testing
 
Android & iPhone App Testing
 Android & iPhone App Testing Android & iPhone App Testing
Android & iPhone App Testing
 
Mobile Application Testing Training Presentation
Mobile Application Testing Training PresentationMobile Application Testing Training Presentation
Mobile Application Testing Training Presentation
 
Mobile application testing
Mobile application testingMobile application testing
Mobile application testing
 
Building the Ultimate Device Matrix
Building the Ultimate Device MatrixBuilding the Ultimate Device Matrix
Building the Ultimate Device Matrix
 
Best Practices for DevOps in Mobile App Testing
Best Practices for DevOps in Mobile App TestingBest Practices for DevOps in Mobile App Testing
Best Practices for DevOps in Mobile App Testing
 
Cloud based Testing Mobile Apps
Cloud based Testing Mobile AppsCloud based Testing Mobile Apps
Cloud based Testing Mobile Apps
 
SeeTestAutomation - Mobile Test Automation Tool by Experitest
SeeTestAutomation - Mobile Test Automation Tool by ExperitestSeeTestAutomation - Mobile Test Automation Tool by Experitest
SeeTestAutomation - Mobile Test Automation Tool by Experitest
 
Mobile application testing tutorial
Mobile application testing tutorialMobile application testing tutorial
Mobile application testing tutorial
 
Mobile Application Testing
Mobile Application TestingMobile Application Testing
Mobile Application Testing
 
Selenium training
Selenium trainingSelenium training
Selenium training
 
Testing Techniques for Mobile Applications
Testing Techniques for Mobile ApplicationsTesting Techniques for Mobile Applications
Testing Techniques for Mobile Applications
 
Mobile Testing Tools 101
Mobile Testing Tools 101Mobile Testing Tools 101
Mobile Testing Tools 101
 
Experitest & Hexaware Co-Webinar
Experitest & Hexaware Co-WebinarExperitest & Hexaware Co-Webinar
Experitest & Hexaware Co-Webinar
 
Mobile Testing Types and Basic Process
Mobile Testing Types and Basic ProcessMobile Testing Types and Basic Process
Mobile Testing Types and Basic Process
 
Main Challenges of Mobile Testing
Main Challenges of Mobile TestingMain Challenges of Mobile Testing
Main Challenges of Mobile Testing
 
Introduction To Mobile-Automation
Introduction To Mobile-AutomationIntroduction To Mobile-Automation
Introduction To Mobile-Automation
 

Andere mochten auch

Intuit CTOF 2011 - Netflix for Mobile in the Cloud
Intuit CTOF 2011 - Netflix for Mobile in the CloudIntuit CTOF 2011 - Netflix for Mobile in the Cloud
Intuit CTOF 2011 - Netflix for Mobile in the CloudSid Anand
 
My startupxx prototyping v2
My startupxx prototyping v2My startupxx prototyping v2
My startupxx prototyping v2Aliza Carpio
 
探討Web ui自動化測試工具
探討Web ui自動化測試工具探討Web ui自動化測試工具
探討Web ui自動化測試工具政億 林
 
Driving JIRA Adoption Through Simple Configuration
Driving JIRA Adoption Through Simple ConfigurationDriving JIRA Adoption Through Simple Configuration
Driving JIRA Adoption Through Simple ConfigurationAtlassian
 
Combining Operations and Support: Incident Management with Atlassian
Combining Operations and Support: Incident Management with AtlassianCombining Operations and Support: Incident Management with Atlassian
Combining Operations and Support: Incident Management with AtlassianAtlassian
 
A Habit of Innovation
A Habit of InnovationA Habit of Innovation
A Habit of InnovationAtlassian
 
Alex Balazs on Scalable Services at GlueCon 2016
Alex Balazs on Scalable Services at GlueCon 2016Alex Balazs on Scalable Services at GlueCon 2016
Alex Balazs on Scalable Services at GlueCon 2016Intuit Inc.
 
Transforming Organizations – from the Perspective of a Technical Mind #GHC15
Transforming Organizations – from the Perspective of a Technical Mind #GHC15Transforming Organizations – from the Perspective of a Technical Mind #GHC15
Transforming Organizations – from the Perspective of a Technical Mind #GHC15Intuit Inc.
 
The Real Value of Small Business Apps
The Real Value of Small Business AppsThe Real Value of Small Business Apps
The Real Value of Small Business AppsIntuit Inc.
 
Dispatches From the New Economy: The On-Demand Workforce
Dispatches From the New Economy: The On-Demand WorkforceDispatches From the New Economy: The On-Demand Workforce
Dispatches From the New Economy: The On-Demand WorkforceIntuit Inc.
 

Andere mochten auch (11)

Intuit CTOF 2011 - Netflix for Mobile in the Cloud
Intuit CTOF 2011 - Netflix for Mobile in the CloudIntuit CTOF 2011 - Netflix for Mobile in the Cloud
Intuit CTOF 2011 - Netflix for Mobile in the Cloud
 
My startupxx prototyping v2
My startupxx prototyping v2My startupxx prototyping v2
My startupxx prototyping v2
 
探討Web ui自動化測試工具
探討Web ui自動化測試工具探討Web ui自動化測試工具
探討Web ui自動化測試工具
 
Driving JIRA Adoption Through Simple Configuration
Driving JIRA Adoption Through Simple ConfigurationDriving JIRA Adoption Through Simple Configuration
Driving JIRA Adoption Through Simple Configuration
 
Combining Operations and Support: Incident Management with Atlassian
Combining Operations and Support: Incident Management with AtlassianCombining Operations and Support: Incident Management with Atlassian
Combining Operations and Support: Incident Management with Atlassian
 
A Habit of Innovation
A Habit of InnovationA Habit of Innovation
A Habit of Innovation
 
Alex Balazs on Scalable Services at GlueCon 2016
Alex Balazs on Scalable Services at GlueCon 2016Alex Balazs on Scalable Services at GlueCon 2016
Alex Balazs on Scalable Services at GlueCon 2016
 
Entreprenariat feminin 2016
Entreprenariat feminin 2016Entreprenariat feminin 2016
Entreprenariat feminin 2016
 
Transforming Organizations – from the Perspective of a Technical Mind #GHC15
Transforming Organizations – from the Perspective of a Technical Mind #GHC15Transforming Organizations – from the Perspective of a Technical Mind #GHC15
Transforming Organizations – from the Perspective of a Technical Mind #GHC15
 
The Real Value of Small Business Apps
The Real Value of Small Business AppsThe Real Value of Small Business Apps
The Real Value of Small Business Apps
 
Dispatches From the New Economy: The On-Demand Workforce
Dispatches From the New Economy: The On-Demand WorkforceDispatches From the New Economy: The On-Demand Workforce
Dispatches From the New Economy: The On-Demand Workforce
 

Ähnlich wie Velocity Conference: Increasing Speed To Market In Mobile Development Through Continuous Integration

Speed to Market in Mobile Development
Speed to Market in Mobile DevelopmentSpeed to Market in Mobile Development
Speed to Market in Mobile DevelopmentTina Su
 
Velocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuitVelocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuitTina Su
 
ICTestAutomation TrendIC 2016 ENGLISH
ICTestAutomation TrendIC 2016 ENGLISHICTestAutomation TrendIC 2016 ENGLISH
ICTestAutomation TrendIC 2016 ENGLISHMarcel Diepenbroek
 
Mdc2010 Automated Mobile Testing
Mdc2010 Automated Mobile TestingMdc2010 Automated Mobile Testing
Mdc2010 Automated Mobile Testingmomobangalore
 
Webinar: Automate Your Environment Provisioning for Mobile App Development
Webinar: Automate Your Environment Provisioning for Mobile App Development Webinar: Automate Your Environment Provisioning for Mobile App Development
Webinar: Automate Your Environment Provisioning for Mobile App Development Skytap Cloud
 
Learning's from mobile testing
Learning's from mobile testingLearning's from mobile testing
Learning's from mobile testingVikrant Chauhan
 
Mobile QA Metrics Risks and Automation Presentation By Yuval Golan
Mobile QA Metrics Risks and Automation Presentation  By Yuval GolanMobile QA Metrics Risks and Automation Presentation  By Yuval Golan
Mobile QA Metrics Risks and Automation Presentation By Yuval GolanYuval Golan
 
2012 mobile testingsummit-moet
2012 mobile testingsummit-moet2012 mobile testingsummit-moet
2012 mobile testingsummit-moetEing Ong
 
Mobile and Web App Testing for Visual Studio Teams
Mobile and Web App Testing for Visual Studio TeamsMobile and Web App Testing for Visual Studio Teams
Mobile and Web App Testing for Visual Studio TeamsPerfecto by Perforce
 
Mobile applications and automation testing
Mobile applications and automation testingMobile applications and automation testing
Mobile applications and automation testingDipesh Bhatewara
 
Continuous testing on emulators using the espresso test automation framework
Continuous testing on emulators using the espresso test automation frameworkContinuous testing on emulators using the espresso test automation framework
Continuous testing on emulators using the espresso test automation frameworkSauce Labs
 
Meetup kickoff slides - Perfecto Mobile, September 2014
Meetup kickoff slides - Perfecto Mobile, September 2014Meetup kickoff slides - Perfecto Mobile, September 2014
Meetup kickoff slides - Perfecto Mobile, September 2014Perfecto Mobile
 
Mobile testing
Mobile testingMobile testing
Mobile testingAlex Hung
 
Gnana Prasuna B_5.5 years
Gnana Prasuna B_5.5 yearsGnana Prasuna B_5.5 years
Gnana Prasuna B_5.5 yearsGnana Bocha
 
Mobile app testing
Mobile app testingMobile app testing
Mobile app testingsanpalan
 

Ähnlich wie Velocity Conference: Increasing Speed To Market In Mobile Development Through Continuous Integration (20)

Speed to Market in Mobile Development
Speed to Market in Mobile DevelopmentSpeed to Market in Mobile Development
Speed to Market in Mobile Development
 
Velocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuitVelocity2013 mobile ci_intuit
Velocity2013 mobile ci_intuit
 
ICTestAutomation TrendIC 2016 ENGLISH
ICTestAutomation TrendIC 2016 ENGLISHICTestAutomation TrendIC 2016 ENGLISH
ICTestAutomation TrendIC 2016 ENGLISH
 
Mdc2010 Automated Mobile Testing
Mdc2010 Automated Mobile TestingMdc2010 Automated Mobile Testing
Mdc2010 Automated Mobile Testing
 
Webinar: Automate Your Environment Provisioning for Mobile App Development
Webinar: Automate Your Environment Provisioning for Mobile App Development Webinar: Automate Your Environment Provisioning for Mobile App Development
Webinar: Automate Your Environment Provisioning for Mobile App Development
 
Learning's from mobile testing
Learning's from mobile testingLearning's from mobile testing
Learning's from mobile testing
 
Mobile QA Metrics Risks and Automation Presentation By Yuval Golan
Mobile QA Metrics Risks and Automation Presentation  By Yuval GolanMobile QA Metrics Risks and Automation Presentation  By Yuval Golan
Mobile QA Metrics Risks and Automation Presentation By Yuval Golan
 
2012 mobile testingsummit-moet
2012 mobile testingsummit-moet2012 mobile testingsummit-moet
2012 mobile testingsummit-moet
 
Mobile Testing
Mobile TestingMobile Testing
Mobile Testing
 
Mobile and Web App Testing for Visual Studio Teams
Mobile and Web App Testing for Visual Studio TeamsMobile and Web App Testing for Visual Studio Teams
Mobile and Web App Testing for Visual Studio Teams
 
CV@Bipin
CV@BipinCV@Bipin
CV@Bipin
 
SynapseIndia mobile apps
SynapseIndia mobile appsSynapseIndia mobile apps
SynapseIndia mobile apps
 
Mobile applications and automation testing
Mobile applications and automation testingMobile applications and automation testing
Mobile applications and automation testing
 
Continuous testing on emulators using the espresso test automation framework
Continuous testing on emulators using the espresso test automation frameworkContinuous testing on emulators using the espresso test automation framework
Continuous testing on emulators using the espresso test automation framework
 
Meetup kickoff slides - Perfecto Mobile, September 2014
Meetup kickoff slides - Perfecto Mobile, September 2014Meetup kickoff slides - Perfecto Mobile, September 2014
Meetup kickoff slides - Perfecto Mobile, September 2014
 
Appium vs Appium with Perfecto
Appium vs Appium with PerfectoAppium vs Appium with Perfecto
Appium vs Appium with Perfecto
 
Mobile testing
Mobile testingMobile testing
Mobile testing
 
Appium vs. Appium with Perfecto
Appium vs. Appium with PerfectoAppium vs. Appium with Perfecto
Appium vs. Appium with Perfecto
 
Gnana Prasuna B_5.5 years
Gnana Prasuna B_5.5 yearsGnana Prasuna B_5.5 years
Gnana Prasuna B_5.5 years
 
Mobile app testing
Mobile app testingMobile app testing
Mobile app testing
 

Mehr von Intuit Inc.

State of Small Business – Growth and Success Report
State of Small Business – Growth and Success ReportState of Small Business – Growth and Success Report
State of Small Business – Growth and Success ReportIntuit Inc.
 
The State of Small Business Cash Flow
The State of Small Business Cash FlowThe State of Small Business Cash Flow
The State of Small Business Cash FlowIntuit Inc.
 
Small Business in the Age of AI
Small Business in the Age of AI Small Business in the Age of AI
Small Business in the Age of AI Intuit Inc.
 
Get financially Fit: Tips for Using QuickBooks
Get financially Fit: Tips for Using QuickBooksGet financially Fit: Tips for Using QuickBooks
Get financially Fit: Tips for Using QuickBooksIntuit Inc.
 
SEO, Social, and More: Digital Marketing for your Business
SEO, Social, and More: Digital Marketing for your BusinessSEO, Social, and More: Digital Marketing for your Business
SEO, Social, and More: Digital Marketing for your BusinessIntuit Inc.
 
Why Building Your Brand is Key to Getting Customers
Why Building Your Brand is Key to Getting CustomersWhy Building Your Brand is Key to Getting Customers
Why Building Your Brand is Key to Getting CustomersIntuit Inc.
 
Get Found Fast: Google AdWords Strategies for Growth
Get Found Fast: Google AdWords Strategies for GrowthGet Found Fast: Google AdWords Strategies for Growth
Get Found Fast: Google AdWords Strategies for GrowthIntuit Inc.
 
Giving Clients What They Want
Giving Clients What They WantGiving Clients What They Want
Giving Clients What They WantIntuit Inc.
 
What Accounting Will Look Like in 2030
What Accounting Will Look Like in 2030What Accounting Will Look Like in 2030
What Accounting Will Look Like in 2030Intuit Inc.
 
Pricing in the Digital Age
Pricing in the Digital Age Pricing in the Digital Age
Pricing in the Digital Age Intuit Inc.
 
Handbook: Power Panel on Apps you need to give you more time to serve your cl...
Handbook: Power Panel on Apps you need to give you more time to serve your cl...Handbook: Power Panel on Apps you need to give you more time to serve your cl...
Handbook: Power Panel on Apps you need to give you more time to serve your cl...Intuit Inc.
 
Handbook: Advanced QuickBooks Online - Handling Tricky Transactions
Handbook: Advanced QuickBooks Online - Handling Tricky TransactionsHandbook: Advanced QuickBooks Online - Handling Tricky Transactions
Handbook: Advanced QuickBooks Online - Handling Tricky TransactionsIntuit Inc.
 
Advanced QuickBooks Online - Handling Tricky Transactions
Advanced QuickBooks Online - Handling Tricky TransactionsAdvanced QuickBooks Online - Handling Tricky Transactions
Advanced QuickBooks Online - Handling Tricky TransactionsIntuit Inc.
 
Handling tricky transactions in QuickBooks Online
Handling tricky transactions in QuickBooks OnlineHandling tricky transactions in QuickBooks Online
Handling tricky transactions in QuickBooks OnlineIntuit Inc.
 
Social media is social business
Social media is social business  Social media is social business
Social media is social business Intuit Inc.
 
Conversation guide: Forming deep relationships with your clients
Conversation guide: Forming deep relationships with your clientsConversation guide: Forming deep relationships with your clients
Conversation guide: Forming deep relationships with your clientsIntuit Inc.
 
Making tax digital
Making tax digital  Making tax digital
Making tax digital Intuit Inc.
 
Giving clients what they want
Giving clients what they want Giving clients what they want
Giving clients what they want Intuit Inc.
 
100 percent cloud your action plan for success
100 percent cloud your action plan for success 100 percent cloud your action plan for success
100 percent cloud your action plan for success Intuit Inc.
 
Attracting and retaining top talent
Attracting and retaining top talent Attracting and retaining top talent
Attracting and retaining top talent Intuit Inc.
 

Mehr von Intuit Inc. (20)

State of Small Business – Growth and Success Report
State of Small Business – Growth and Success ReportState of Small Business – Growth and Success Report
State of Small Business – Growth and Success Report
 
The State of Small Business Cash Flow
The State of Small Business Cash FlowThe State of Small Business Cash Flow
The State of Small Business Cash Flow
 
Small Business in the Age of AI
Small Business in the Age of AI Small Business in the Age of AI
Small Business in the Age of AI
 
Get financially Fit: Tips for Using QuickBooks
Get financially Fit: Tips for Using QuickBooksGet financially Fit: Tips for Using QuickBooks
Get financially Fit: Tips for Using QuickBooks
 
SEO, Social, and More: Digital Marketing for your Business
SEO, Social, and More: Digital Marketing for your BusinessSEO, Social, and More: Digital Marketing for your Business
SEO, Social, and More: Digital Marketing for your Business
 
Why Building Your Brand is Key to Getting Customers
Why Building Your Brand is Key to Getting CustomersWhy Building Your Brand is Key to Getting Customers
Why Building Your Brand is Key to Getting Customers
 
Get Found Fast: Google AdWords Strategies for Growth
Get Found Fast: Google AdWords Strategies for GrowthGet Found Fast: Google AdWords Strategies for Growth
Get Found Fast: Google AdWords Strategies for Growth
 
Giving Clients What They Want
Giving Clients What They WantGiving Clients What They Want
Giving Clients What They Want
 
What Accounting Will Look Like in 2030
What Accounting Will Look Like in 2030What Accounting Will Look Like in 2030
What Accounting Will Look Like in 2030
 
Pricing in the Digital Age
Pricing in the Digital Age Pricing in the Digital Age
Pricing in the Digital Age
 
Handbook: Power Panel on Apps you need to give you more time to serve your cl...
Handbook: Power Panel on Apps you need to give you more time to serve your cl...Handbook: Power Panel on Apps you need to give you more time to serve your cl...
Handbook: Power Panel on Apps you need to give you more time to serve your cl...
 
Handbook: Advanced QuickBooks Online - Handling Tricky Transactions
Handbook: Advanced QuickBooks Online - Handling Tricky TransactionsHandbook: Advanced QuickBooks Online - Handling Tricky Transactions
Handbook: Advanced QuickBooks Online - Handling Tricky Transactions
 
Advanced QuickBooks Online - Handling Tricky Transactions
Advanced QuickBooks Online - Handling Tricky TransactionsAdvanced QuickBooks Online - Handling Tricky Transactions
Advanced QuickBooks Online - Handling Tricky Transactions
 
Handling tricky transactions in QuickBooks Online
Handling tricky transactions in QuickBooks OnlineHandling tricky transactions in QuickBooks Online
Handling tricky transactions in QuickBooks Online
 
Social media is social business
Social media is social business  Social media is social business
Social media is social business
 
Conversation guide: Forming deep relationships with your clients
Conversation guide: Forming deep relationships with your clientsConversation guide: Forming deep relationships with your clients
Conversation guide: Forming deep relationships with your clients
 
Making tax digital
Making tax digital  Making tax digital
Making tax digital
 
Giving clients what they want
Giving clients what they want Giving clients what they want
Giving clients what they want
 
100 percent cloud your action plan for success
100 percent cloud your action plan for success 100 percent cloud your action plan for success
100 percent cloud your action plan for success
 
Attracting and retaining top talent
Attracting and retaining top talent Attracting and retaining top talent
Attracting and retaining top talent
 

Kürzlich hochgeladen

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 

Kürzlich hochgeladen (20)

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 

Velocity Conference: Increasing Speed To Market In Mobile Development Through Continuous Integration

  • 1. Speed to Market in Mobile Development Finding the right solution with continuous integration on real devices Tina Su, Director of Development, Intuit
  • 2. About me Nearly 20 years of industry experience in R&D, quality and engineering management, with a strong passion for automation.  Engineering Director at Intuit, Developer Productivity  Sr. Engineering Director at Yahoo, Mobile Platforms  Sr. Engineering Manager at Sun, Java Standard and Mobile Editions  Compiler Developer at MIPS and Tandem Tina Su, Intuit
  • 3. Agenda  Mobile trends & developers’ challenges  Speed & quality through continuous integration - Intuit Virtual Device Lab - Intuit mobile test automation  Takeaways
  • 4. Aggressive mobile momentum “2013 Internet Trends,” KPCB Will surpass desktop by 2014 It should just work! Proliferation of platforms, OS versions & device types
  • 5. Developer challenges  Immense configuration matrix - OS platforms (Android, iOS, Windows, phone vs. tablet, …) - Inputs (touch/gesture, real buttons, voice, camera, GPS receiver) - Outputs (portrait/landscape, screen resolution, language)  Application delivery through the markets - App signing, provision (app & device), submission  Offline experience - Network, different network bandwidth/latency, no-network - HTML5 offline app  Media challenges - Some codecs don’t work on a device - Streaming, sound  Security & privacy
  • 6. Intuit mobile app portfolio today Apple Google Amazon SMS 0 5 10 15 20 490 22 15 11 3 491 mobile apps for financial institutions
  • 7. Ramping up Intuit’s innovation engine Pain: Takes days per build to test combinations of device/OS configurations physically & manually, one by one … QA can’t keep up with the changes Goal: Reduce development iteration cycle from days to minutes on an ever-growing number of devices Strategy: Speed & quality through mobile continuous integration • iPhone 4 • iPhone 4S • iPhone 5 • iPad 2 • iPad 3 • iPad Mini Devices OS versions • iOS 5.0 • iOS 5.1 • iOS 6.0 • iOS 6.1.x • iOS 7.0 Builds X X Daily builds • Apple • Google • Amazon Platforms X
  • 8. Continuous integration (CI) What is CI?  Automate build, test & deployment process  Integrate & build early, often, daily, for every check-in  Best practice of Agile development  Jenkins is widely adopted, flexible & extensible Benefits of CI in a mobile world  Instant feedback on quality, functionality & system impact  Less complex, no integration hell  Detect and address device configuration issues early  Involve stakeholders early  Ultimately achieve shippable release any time
  • 9. Speed and quality through mobile CI Key enablers: • Mobile device lab • Test automation Test processorReal devices Jenkins SCM: source repository Developers /IDE Automatic test run Test results Submit the app to app store if passed Check in Check-in triggers automated CI Install the test app to real devices in parallel Run tests on real devices in parallel Analyze test results Trigger testing in mobile device lab Process starts here
  • 10. 3rd-party vendor solutions didn’t work for us  Couldn’t handle E2E automation of testing pre-release apps - Dependency on internal resources & infrastructure  Vendor lock-in: customized CI & test automation solutions - No flexibility to use our existing tools & frameworks  Disadvantages of vendors’ shared devices - Security concern on testing on shared devices - No access to corporate network - Cost-prohibitive to reserve devices  Private cloud (on-site setup) with local devices not practical - With sensitive customer data, we need private cloud - Extensive use of devices for daily testing of hundreds of mobile apps is expensive
  • 11. Why test on real devices? Most mobile features can be tested on simulators, but simulators have limitations:  Device capabilities (GPS, camera, accelerometer, gyros), form factors & pixel density  Real-world experience over the real connection (3G, 4G, crappy Wi-Fi, etc.)  True performance, security tests  Testing on non-default Web browsers (e.g., Opera Mini)  Not all simulator/emulators have good quality (don’t reflect real rendering/behavior, bugs)  Testing SMS & app integration
  • 13. Intuit Virtual Device Lab (VDL) Gives access to real mobile devices using a browser Intranet VDL User machine User machine User machine 3-click access Always on Integrated with dev infrastructure Globally available iPhone iPad Galaxy Captivate Optimus Nexus One Desire Incredible Manufacturers/ devices: Droid Milestone Moto
  • 14. VDL architecture – Android Windows PC Tomcat Java image transferring application Browser & Java applet Internet/intr anet Device interaction app Developer’s laptop VDL Image transfer USB connection
  • 15. VDL architecture – iOS Mac with Xcode installed Tomcat Java VNC client Port forwarding Browser & Java applet Internet/intr anet VNC server USB connection Developer’s laptop VDL
  • 16. Open source tools & technologies used in VDL Key capabilities Android iOS Installing mobile applications into the real device adb install <APK file> Fruitstrap https://github.com/ghughes/fruitstra p Transferring device screen images, controlling device Screencast tool http://code.google.com/p/androidsc reencast Veency VNC app Fetching device local files adb pull <filename> sftp command (jailbroken devices only)
  • 17. VDL key features & functionality today Control the device through standard browser/RESTful APIs Change orientation to landscape or portrait Copy local file to the device from your PC Get file from the device onto your local PC Execute an ADB command on the device See current running logs in your browser Download history logs to your local PC
  • 18. Continuous integration – how it works Product builds Trigger file launches test VDL device inventoryResource manager Select a device Test execution engine Results Monito r for new builds
  • 19. Mobile continuous integration Leveraging the Virtual Device Lab for test automation on multiple, real devices simultaneously. Submit, build, test … in minutes vs. hours Select devices & tests1 Tests run automatically across selected devices with each build 2 See the results!3
  • 20. Next step: over-the-air replaces USB connection Fast onboarding Scalable Any device Internet/intr anet App installation Test execution Results retrieval VDL
  • 22. Mobile test automation tools Instrumentation-based Non-instrumentation AndroidiOS MonkeyRunner MOET Sikuli Android SDK Robotium TestDroid (BitBar) Calabash (LessPainful) Soasta iOS SDK UIAutomation UISpec Frank MonkeyTalk DA/PerfectoMob ile eggPlant Cross-platform solution iOS Driver Appium • Assessed as of 2012
  • 23. Mobile test automation trends  Abstraction with domain-specific language (DSL) - Common test script over different devices (e.g., Cucumber, MonkeyScript, MOET) - Support behavior-driven development (BDD)  Client/server-based approach (remotely driven tests via HTTP)  Instrumentation & non-instrumentation solutions complement each other  Mobile cloud testing environment - From hardware (cradle) to software approach to drive tests on real devices  Automated OTA app installation & tests
  • 24. Intuit automation framework of choice/recommendation  Native apps: - Cucumber + Calabash for BDD on iOS & Android  Mobile Web: - Selenium/WebDriver
  • 25. What is Cucumber? It lets stakeholders (customers, QE or developers) describe how software should behave in plain text.  BDD: Written in business domain-specific language (DSL) - Good abstraction over different device types  One format for multiple purposes - UI feature specification - Automated tests  Cucumber itself is NOT a test automation tool - Add-on to any test automation framework - Examples: Calabash, Sikuli & Frank mobile test frameworks
  • 26. What is Calabash?  Open source cross-platform mobile test framework - iOS + Android, simulator + real device, native + hybrid  Supports Cucumber for BDD  Application needs instrumentation that embeds a Calabash HTTP server - Tests run from a remote client (in a Jenkins slave) - Tests can be driven via Wi-Fi (no USB wire)  Ruby clients in API level (Java clients in experiment)  Works great in touch simulation & real device testing
  • 27. Intuit shared test library – rapid test development Internal open source repository Teams focus on creating Use shared test lib … and then contribute to common test library Tests Contribute Tests Common test library Product tests Tests Tests Globalization test framework (I18N/L10N) Performance/load test framework Security test framework Native/hybrid test framework High-level test domain-specific language (Cucumber steps) Mobile back-end Mobile front-end
  • 28. Code build Deploy Test Results Connect device Notify email Get devices Consolidate report Upload report From: manual 50*5 = 250 15 30 5 10 20 15 30 10 = 6 hours Code build Trigger Execute tests in parallel Consolidated report in archive Deploy on devices 0 5 10 0 = 15 mins To: automated CI TIME SAVED: 95%Release quality increased * Stunning results: reduced iteration cycle from hours to minutes Tested on 5 devices
  • 29. VDL cost saving over vendors’ private clouds (today) Annual cost ($) # of devices Intuit Virtual Device Lab 150 3rd-party vendor 1 local device in cradle Assumptions: 50 native apps & daily check-in Note: As vendor pricing changes, so do the savings Savings 25 3rd-party vendor 2 local device w/o cradle (Software approach) Savings Private cloud setup behind corp firewall
  • 30. Key takeaways Building custom solution vs. 3rd-party vendor solution  Understand your technical/platform/global spread  Many vendor solutions provide a larger set of devices/OS versions in the global market  Quite a few offer mobile test services – might be tightly coupled with their device access solution & automation framework  Connecting to your corporate network is a challenge – some provide private cloud (on-site setup within your network) but it’s cost-prohibitive Invest in automation and CI  Automate tests with robust automation framework  Enable rapid test development through shared test library  Adopt CI in mobile development for speed & quality
  • 31. More mobile innovations at Intuit  Mobile crash reporter  Mobile components & design patterns  Mobile analytics SDK & gateway Visit booth #601 – Accelerate Development  Demos & conversations  Contest  Mobile device charging station  Foosball  We’re hiring ! Win a weekend with a TESLA!