Weitere ähnliche Inhalte Kürzlich hochgeladen (20) Best Practices for Performance Testing Mobile Apps3. © Utopia Solutions3
Key Questions
How is mobile different?
How can I adapt?
Why is performance of mobile
important?
4. © Utopia Solutions4
Why Should I Care?
M-Commerce
66%
OF SHOPPERS
ABANDON
TRANSACTIONS
5 OUT OF 10
BECAUSE OF POOR
PERFORMANCE
1DELAY
SECOND
7%DROP IN
CONVERSIONS
equals a
2013 Mobile Commerce Insights
study conducted by Jumio
6. © Utopia Solutions6
Evolution of Performance Testing
1990 2000 2010
2-Tier
Client Server Web Mobile
Challenges
State of the
Practice
Testing Large
Loads
Complex /
Integrated
Systems
Proprietary
Protocols
Skilled
Resources
In-Lab
Testing
Tool
Maturation
Open Source
Tools
Specialized
Resources
Cloud-based
Testing
9. © Utopia Solutions9
Assess the User and Load Profile
1. Determine the user profile
• Business processes
• Native app, mobile site, full site
• Preferred browser
• Network conditions
• Geographic location
2. Determine the load profile
• Volume of users by business
process and location
• Volume of users by access type
• Frequency / throughput
Incorporate constant feedback from application logs
and other monitoring solutions
10. © Utopia Solutions10
Mobile Challenges - Network
Mobile User Connected User
3G 4G InATunnelG
Network bandwidth and quality affect both end user
experience and system load
11. © Utopia Solutions11
Assess Impact of Network Variability
1. Assess baseline performance
(single user)
Observe UX with real devices
behind network virtualization
solutions
2. Assess performance under
load
Put system under virtual user load
Observe UX with real devices behind
network virtualization solutions
NV
Virtual Users
NV
Real
Devices
12. © Utopia Solutions12
Network Virtualization Tools
Single User (Device and/or
Emulator)
• Apple Network Link Conditioner
• Android Emulator
• Charles Proxy
• Shunra NV
Multiple Virtual Users
• Shunra NV
• Load test tools (built-in)
• Charles Proxy
13. © Utopia Solutions13
Mobile Challenges – User Location and Volume
A high volume of global users is difficult to reproduce in a lab…
14. © Utopia Solutions14
Utilize Cloud-based Testing Solutions
• Achieve production level loads
• Test entire infrastructure (vs.
behind the firewall)
• Distribute load geographically
• Assess user impact on real
devices
15. © Utopia Solutions15
Mobile Challenges – Device Performance
Measuring System / Network Metrics is Not Enough…
Memory
CPU
Battery Drain
Graphics
I / O
16. © Utopia Solutions16
Measuring Device Performance
Critical device metrics:
• CPU
• Memory
• Battery
• Storage
Mobile development platforms
• Android DDMS
• iOS Instruments
Mobile test automation tools
• TouchTest (SOASTA)
• Trust (Mobile Labs)
• MonkeyTalk (Cloud Monkey)
Mobile lab / cloud solutions
• Device Connect (Mobile Labs)
• Keynote DeviceAnywhere
• Perfecto Mobile
Capture metrics while exercising app
Include appropriate devices /
configurations
Incorporate automation for consistency
Device Performance
Measurement Solutions
17. © Utopia Solutions17
Mobile Challenges – Agile Development
Analysis
Design
Develop &
Unit Test
Acceptance
Test
Final
Acceptance
Test
Deploy
Source: Agile and Test Estimation
Sharon Robson
How does performance
testing fit?
20. © Utopia Solutions20
Performance Testing in an Agile Environment
Sprint 1 Sprint 2 Sprint 3 … Deploy
Agile
Performance
Testing
Performance
Testing
Performance
Testing
Performance
Testing
Waterfall Approach
• Rigid
• 4-6 week test cycle
Agile Approach
• Flexible
• 2 week sprints
+ =
21. © Utopia Solutions21
Early Cycle /
Ongoing
Performance
Analysis
Performance Testing in an Agile Environment
Ultimate Goal…
Late Cycle
Performance
Testing
Shift Left
22. © Utopia Solutions22
Performance Testing in an Agile Environment
How?
Map business and end user
requirements onto system components
Benchmark in development / QA
environment
Look for trends in measurements vs.
absolutes
Use service virtualization to stub out 3rd
party or yet to be developed services
Execute unit performance tests as part
of continuous integration
Unit / Component
Performance Testing
23. © Utopia Solutions23
Performance Testing in an Agile Environment
How?
Measure resource / performance
profile of native apps
Use mobile site analysis services / tools
to analyze services and pages
User APM solutions to profile code
Use device profiling solutions to
understand / compare base device
performance
Application / Code
Profiling
24. © Utopia Solutions24
Waterfall Agile
Service Model Perform turnkey
performance test
Timing End of development cycle
Duration Weeks
Tools Traditional load test tools
Environment Dedicated / production like
Role of
development
Throw system over wall
Wait for results
Waterfall Agile
Service Model Perform turnkey
performance test
Provide mentorship to agile
team
Timing End of development cycle Ongoing
Duration Weeks Hours / days
Tools Traditional load test tools Traditional load test tools
APM tools
Service virtualization
Continuous integration
Environment Dedicated / production like Development / desktop
Role of
development
Throw system over wall
Wait for results
Constant focus on performance
Performance Testing in an Agile Environment
What about my Performance
Testing Center of Excellence?
25. © Utopia Solutions25
Summary
Key takeaways…
• Mobile users are not the same as
connected users
• Performance is more than backend /
network
• Push performance analysis to the left
in the development cycle
26. © Utopia Solutions26
Questions…
…. and Answers!
Direct future questions to:
Lee Barnes
Founder and CTO
Utopia Solutions, Inc.
Email: lee.barnes@utopiasolutions.com
Twitter: twitter.com/USI_LeeBarnes
LinkedIn: linkedin.com/in/leebarnes
Blog: www.RaiseYourMQ.com
?