SlideShare ist ein Scribd-Unternehmen logo
1 von 27
Downloaden Sie, um offline zu lesen
Performance Testing in a
Mobile World




            Stuart Moncrieff
            Performance Testing Practice Lead
Presentation Roadmap




    Background   Mobile device     Network        Server
                 performance     performance   performance




2   JDS
Background




    Background   Mobile device     Network        Server
                 performance     performance   performance




3   JDS
Mobile is growing fast
    • There were more smartphones and tablets sold in 2011
      than desktop PCs, laptops and netbooks combined.




    • 20% of web traffic in the US comes from smartphones
      and tablets
4   JDS
Smartphone Sales by Operating System
    • Android is the
      dominant mobile
      phone operating
      system
    • BUT!!
          – The most popular
            handset is iPhone
            (especially in .au)
          – Google Play has
            less than ¼ of the
            revenue of Apple’s
            App Store
          – iPad has more market share than all Android tablets combined
          – App sales are negligible for other stores (Nokia Ovi Store,
            Blackberry App World, Windows Phone Marketplace)
5   JDS
Mobile in the Enterprise
    • Growth of “bring your own device”
      (BYOD) for internal apps – mostly
      web apps and email
    • Devices for field service/delivery workers
      (probably not running iOS or Android)
    • Mobile versions of public-facing websites
      (http://m.yoursite.com.au)
    • Mobile App projects
          – Free apps to reach more customers. E.g. banking app, supermarket app
            (not monetised through app sales)
          – Projects initiated by the Business, instead of the IT department
          – Very small budgets, almost always outsourced (to small, agile companies)
          – Less maturity than typical Enterprise projects (minimal non-functional
            requirements, security testing, performance testing)
    • Large IT projects sometimes have a small mobile component
6   JDS
Mobile performance is important




    • Bounce-rates increase dramatically as response times
      increase
    • 1 star reviews in App Store “this app sucks”
7   JDS
Factors in Mobile Performance




    • Mobile device            • Network                  • Data Centre
          – Varying hardware     ‒ The “last mile”           ‒ Load
                                 ‒ The Internet                balancers, se
          – Mobile app code                                    rvers
                                   (from your telco
                                   to your data centre)      ‒ Server
                                                               code, capacit
                                                               y, configurati
                                                               on
8   JDS
Mobile Device Performance




    Background   Mobile device     Network        Server
                 performance     performance   performance




9   JDS
Apps behaving badly
     • Bandwidth hog
           – “Your app gave me a huge phone bill!”
     • Storage hog
           – “Your app is using up all my storage!”
     • Drains battery
           – Location-based apps, data use patterns
             (polling keeps activating the transmitter)
     • Leaks memory
     • Doesn’t handle network connectivity
       problems gracefully
     • Slow startup or unresponsive user interface

10   JDS
Be careful with WebView
     • Tempting shortcuts:
           – Write parts of your app in JS + HTML, and display in a
             WebView, instead of using native code.
           – HTML5 vs Cross-platform vs Native
           – Make sure you pick the technology with
             the right trade-offs for your application
     • Mobile WebKit has a 300ms delay on
       firing click events after a tap.
           – Work-arounds exist, but impose trade-offs
     • Cross-platform mobile development frameworks
       (PhoneGap, Trigger.io, Appcelerator, etc.)
           – Can be hard to make non-native apps feel “snappy”
           – Overhead imposed by JavaScript to Native bridge
11   JDS
Profiling tools for developers
     • DDMS (Android) or Instruments (iOS)
           – View memory usage and allocation (find leaks), threads, process
             resource usage, slow methods, network usage, radio state (test
             outage conditions).
     • JavaScript
           – Profile it on a desktop
             browser (FireBug for
             Chrome/Firefox, Web
             Inspector for Safari)
           – Use JSLint to ensure
             strict code
             “correctness”




12   JDS
Case Study: SnappyCam+
     • Takes lots of photos very fast
       (great for action sequences)
     • Camera performance was the
       central feature of the app
     • Bottleneck: JPEG compression
           – Existing compression libraries were too
             slow
           – Researched academic papers on JPEG
             compression algorithms
           – Implemented compression algorithm
             highly-optimised for CPU architecture
           – Hand-crafted assembly code for the
             ARM NEON SIMD co-processor

13   JDS
Network Performance




     Background   Mobile device     Network        Server
                  performance     performance   performance




14   JDS
Sloooow network
     • Mobile data speeds are
       highly variable and can
       be very slow (like a
       modem)
     • Websites have become
       bloated with the
       widespread adoption of
       broadband Internet
     • You can’t improve the
       network, but you can
       optimise your mobile
       app/website for slow
       networks

15   JDS
Shunra NetworkCatcher Express Mobile
     • Free iPhone/Android app to check network performance
       from your current location.
     • Benefits:
           – Measure real-world
             mobile network
             conditions
             (bandwidth,
             latency, packet
             loss)
           – Generates an NTX
             file to use with
             Shunra
             PerformanceSuite
             or Shunra vCat


16   JDS
WAN Emulation
     • Most testing done while connected to
           – Your office Wi-Fi
           – Fast, metropolitan 3G internet
     • Use a WAN emulator to measure response times under
       different network conditions




17   JDS
Optimising for slow networks
     • Minimise data transfer
           – Create a mobile version of your website, and serve images scaled for
             small mobile screens
           – Use the HTML5 Application Cache or local storage
     • Minimise “chattiness”.
           – Multiple sequential requests multiply
             the effect of network latency.
     • Pre-fetch data to make it seem fast
     • Use YSlow! for your mobile website
           – Ensure content is cachable (correct
             headers, etc.)
           – Make files smaller (compress with Gzip,
             minify JS/CSS files)
           – Minimise HTTP requests
           – Be aware of slow, third-party sites (e.g. ad networks, analytics)
18   JDS
Case Study: Smart Electricity Meters
     • DPI mandated that all manually read electricity meters
       be replaced with smart meters by the end of 2013.
     • All meters must report usage (in 30 minute intervals) to
       AEMO by 6am the following day.
     • Problems at one electricity distributor:
           – WiMAX (4G) teething problems
               • Patchy data coverage. Meters not
                 “phoning home” reliably
               • Unresponsive meters (requiring site visits)
           – “Thundering herds”. Trying to avoid all the
             meters sending their data at the same time
               • Limited bandwidth on base station
               • Servers cannot handle 700,000 concurrent connections
19   JDS
Server Performance




     Background   Mobile device     Network        Server
                  performance     performance   performance




20   JDS
Performance Testing 101
     • Define Workload Model
           – Identify key business processes
           – Define Peak Hour transaction volumes for each business process
           – Define number of concurrent users, network properties
     • Organise Production-like test environment (with monitoring)
     • Develop Scripts
           – Note: scripts send same traffic as device, so no phones are needed to
             run the load test. But response times will not include client-side time.
           – Record business processes
     • Execute tests
           – Peak load, Stress/Break, Soak
           – Failover, interface outages, etc.
     • Tune, Test, Tune, Test, etc.
           – Problems due to code, capacity,
             or configuration
21   JDS
Difficulties recording mobile apps
     • Record device or simulator/emulator on office network
     • Probably can’t install recording agent on mobile device
     • Most apps are just making web service calls
     Application Type          Recording Method
     Mobile website            • Record from desktop PC with standard
                                 browser. Use a User Agent Switcher add-on.
     Mobile app (HTTP)         • Use a recording proxy
     Mobile app (HTTPS)        • Security features prevent use of a recording
                                 proxy (i.e MITM attack). Some workarounds.
                               • Use packet capture, decode with SSL cert.
     Mobile app (WebSockets    • Beware! Poor tool support.
     or other new protocols)   • Be careful with new technologies
                               • Scoping question: what protocols does the
                                 app use?
22   JDS
Example: Recording a mobile app (using
     HTTPS) with LoadRunner
     1. Server-side packet capture while performing business
        process (e.g. using WireShark)
     2. Export private certificate (from test environment)
     3. New script > Mobile Application - HTTP/HTML
           –   Recording type: Analyze traffic
     4. Select PCAP file,
        set up filters, import
        certificate
     5. Generate script
     6. Add transaction names,
        correlate values, etc.

23   JDS
WAN Emulation under load
     • Users on slow network connections have different load
       characteristics on front-end servers
           – Hold sockets open longer
           – Cause more active TCP sessions on load balancer (licensing
             limits, memory limits for session table)
           – Cause more open connections to web servers (thread/process
             limits, O/S TCP limits, requests queue waiting for a connection)
     • During load testing
           – Emulate network conditions for each virtual user individually
             (bandwidth is not shared by virtual users)
           – Emulate worst-case to find connection limits on servers




24   JDS
Performance Testing Overview




     Test    Device               Network            Server Load/
             Performance          Behaviour          Performance
     Envt    Development          Development        Production-like
     Users   One                  One                Many, emulated/virtual
     Who     Developer            Developer or       Tester
                                  Tester
     Notes   • Manual, using      • Manual, using     • Automated, using load
               profiling tools      network profiling   testing tools
             • Testers can help   • Using WAN         • With/without WAN
               with requirements,   emulation           emulation
               oversight, etc.
25   JDS
Questions?




26   JDS
Thank You / Credits
                      • CoreForm
                           – Android app development & security
                           – http://www.coreform.com.au/
                      • GridStone
                           – Mobile app development studio
                           – http://gridstone.com.au/
                      • SnappyLabs
                           – iOS app development
                           – http://www.snappylabs.com/
                      • MyLoadTest
                           – My blog!
                           – http://www.myloadtest.com/
27   JDS

Weitere ähnliche Inhalte

Was ist angesagt?

Applying DevOps, PaaS and cloud for better citizen service outcomes - IBM Fe...
Applying DevOps, PaaS and cloud for better citizen service  outcomes - IBM Fe...Applying DevOps, PaaS and cloud for better citizen service  outcomes - IBM Fe...
Applying DevOps, PaaS and cloud for better citizen service outcomes - IBM Fe...Sanjeev Sharma
 
UrbanCode Deploy DevOps Best Practices
UrbanCode Deploy  DevOps Best PracticesUrbanCode Deploy  DevOps Best Practices
UrbanCode Deploy DevOps Best PracticesMichael Elder
 
Introduction to IBM UrbanCode Deploy and Release
Introduction to IBM UrbanCode Deploy and ReleaseIntroduction to IBM UrbanCode Deploy and Release
Introduction to IBM UrbanCode Deploy and ReleaseRob Cuddy
 
Improving Software Delivery with DevOps & Software Defined Environments | The...
Improving Software Delivery with DevOps & Software Defined Environments | The...Improving Software Delivery with DevOps & Software Defined Environments | The...
Improving Software Delivery with DevOps & Software Defined Environments | The...IBM UrbanCode Products
 
Creating a DevOps Team that Isn't Evil
Creating a DevOps Team that Isn't EvilCreating a DevOps Team that Isn't Evil
Creating a DevOps Team that Isn't EvilIBM UrbanCode Products
 
Shift Happens - Rapidly Rolling Forward During Production Failure
Shift Happens - Rapidly Rolling Forward During Production FailureShift Happens - Rapidly Rolling Forward During Production Failure
Shift Happens - Rapidly Rolling Forward During Production FailureIBM UrbanCode Products
 
Integrations, UI Enhancements and Cloud – See What’s New with IBM UrbanCode D...
Integrations, UI Enhancements and Cloud – See What’s New with IBM UrbanCode D...Integrations, UI Enhancements and Cloud – See What’s New with IBM UrbanCode D...
Integrations, UI Enhancements and Cloud – See What’s New with IBM UrbanCode D...IBM UrbanCode Products
 
Deployment Automation for Hybrid Cloud and Multi-Platform Environments
Deployment Automation for Hybrid Cloud and Multi-Platform EnvironmentsDeployment Automation for Hybrid Cloud and Multi-Platform Environments
Deployment Automation for Hybrid Cloud and Multi-Platform EnvironmentsIBM UrbanCode Products
 
DevOps in the Hybrid Cloud
DevOps in the Hybrid CloudDevOps in the Hybrid Cloud
DevOps in the Hybrid CloudRichard Irving
 
Citrix Troubleshooting 101: How to Resolve and Prevent Business-Impacting Cit...
Citrix Troubleshooting 101: How to Resolve and Prevent Business-Impacting Cit...Citrix Troubleshooting 101: How to Resolve and Prevent Business-Impacting Cit...
Citrix Troubleshooting 101: How to Resolve and Prevent Business-Impacting Cit...eG Innovations
 
Cloud With DevOps Enabling Rapid Business Development
Cloud With DevOps Enabling Rapid Business DevelopmentCloud With DevOps Enabling Rapid Business Development
Cloud With DevOps Enabling Rapid Business DevelopmentSam Garforth
 
Helping Organizations Realize the Value of DevOps with Continuous Software De...
Helping Organizations Realize the Value of DevOps with Continuous Software De...Helping Organizations Realize the Value of DevOps with Continuous Software De...
Helping Organizations Realize the Value of DevOps with Continuous Software De...IBM UrbanCode Products
 
DevOps for Enterprise Systems Overview
DevOps for Enterprise Systems OverviewDevOps for Enterprise Systems Overview
DevOps for Enterprise Systems OverviewRosalind Radcliffe
 
VDI Performance Assurance With Monitoring and Testing
VDI Performance Assurance With Monitoring and TestingVDI Performance Assurance With Monitoring and Testing
VDI Performance Assurance With Monitoring and TestingeG Innovations
 
A DevOps adoption playbook- achieving business value at scale
A DevOps adoption playbook- achieving business value at scaleA DevOps adoption playbook- achieving business value at scale
A DevOps adoption playbook- achieving business value at scaleSanjeev Sharma
 
Using Lean Thinking to identify and address Delivery Pipeline bottlenecks
Using Lean Thinking to identify and address Delivery Pipeline bottlenecksUsing Lean Thinking to identify and address Delivery Pipeline bottlenecks
Using Lean Thinking to identify and address Delivery Pipeline bottlenecksSanjeev Sharma
 
Bluemix DevOps Meetup
Bluemix DevOps MeetupBluemix DevOps Meetup
Bluemix DevOps MeetupKyle Brown
 

Was ist angesagt? (20)

Applying DevOps, PaaS and cloud for better citizen service outcomes - IBM Fe...
Applying DevOps, PaaS and cloud for better citizen service  outcomes - IBM Fe...Applying DevOps, PaaS and cloud for better citizen service  outcomes - IBM Fe...
Applying DevOps, PaaS and cloud for better citizen service outcomes - IBM Fe...
 
UrbanCode Deploy DevOps Best Practices
UrbanCode Deploy  DevOps Best PracticesUrbanCode Deploy  DevOps Best Practices
UrbanCode Deploy DevOps Best Practices
 
Introduction to IBM UrbanCode Deploy and Release
Introduction to IBM UrbanCode Deploy and ReleaseIntroduction to IBM UrbanCode Deploy and Release
Introduction to IBM UrbanCode Deploy and Release
 
The Future of DevOps and UrbanCode
The Future of DevOps and UrbanCodeThe Future of DevOps and UrbanCode
The Future of DevOps and UrbanCode
 
Improving Software Delivery with DevOps & Software Defined Environments | The...
Improving Software Delivery with DevOps & Software Defined Environments | The...Improving Software Delivery with DevOps & Software Defined Environments | The...
Improving Software Delivery with DevOps & Software Defined Environments | The...
 
Creating a DevOps Team that Isn't Evil
Creating a DevOps Team that Isn't EvilCreating a DevOps Team that Isn't Evil
Creating a DevOps Team that Isn't Evil
 
Shift Happens - Rapidly Rolling Forward During Production Failure
Shift Happens - Rapidly Rolling Forward During Production FailureShift Happens - Rapidly Rolling Forward During Production Failure
Shift Happens - Rapidly Rolling Forward During Production Failure
 
Death to Manual Deployments
Death to Manual DeploymentsDeath to Manual Deployments
Death to Manual Deployments
 
Integrations, UI Enhancements and Cloud – See What’s New with IBM UrbanCode D...
Integrations, UI Enhancements and Cloud – See What’s New with IBM UrbanCode D...Integrations, UI Enhancements and Cloud – See What’s New with IBM UrbanCode D...
Integrations, UI Enhancements and Cloud – See What’s New with IBM UrbanCode D...
 
Deployment Automation for Hybrid Cloud and Multi-Platform Environments
Deployment Automation for Hybrid Cloud and Multi-Platform EnvironmentsDeployment Automation for Hybrid Cloud and Multi-Platform Environments
Deployment Automation for Hybrid Cloud and Multi-Platform Environments
 
DevOps in the Hybrid Cloud
DevOps in the Hybrid CloudDevOps in the Hybrid Cloud
DevOps in the Hybrid Cloud
 
Citrix Troubleshooting 101: How to Resolve and Prevent Business-Impacting Cit...
Citrix Troubleshooting 101: How to Resolve and Prevent Business-Impacting Cit...Citrix Troubleshooting 101: How to Resolve and Prevent Business-Impacting Cit...
Citrix Troubleshooting 101: How to Resolve and Prevent Business-Impacting Cit...
 
Cloud With DevOps Enabling Rapid Business Development
Cloud With DevOps Enabling Rapid Business DevelopmentCloud With DevOps Enabling Rapid Business Development
Cloud With DevOps Enabling Rapid Business Development
 
Helping Organizations Realize the Value of DevOps with Continuous Software De...
Helping Organizations Realize the Value of DevOps with Continuous Software De...Helping Organizations Realize the Value of DevOps with Continuous Software De...
Helping Organizations Realize the Value of DevOps with Continuous Software De...
 
DevOps for Enterprise Systems Overview
DevOps for Enterprise Systems OverviewDevOps for Enterprise Systems Overview
DevOps for Enterprise Systems Overview
 
Adopting DevOps for 2-Speed IT
Adopting DevOps for 2-Speed ITAdopting DevOps for 2-Speed IT
Adopting DevOps for 2-Speed IT
 
VDI Performance Assurance With Monitoring and Testing
VDI Performance Assurance With Monitoring and TestingVDI Performance Assurance With Monitoring and Testing
VDI Performance Assurance With Monitoring and Testing
 
A DevOps adoption playbook- achieving business value at scale
A DevOps adoption playbook- achieving business value at scaleA DevOps adoption playbook- achieving business value at scale
A DevOps adoption playbook- achieving business value at scale
 
Using Lean Thinking to identify and address Delivery Pipeline bottlenecks
Using Lean Thinking to identify and address Delivery Pipeline bottlenecksUsing Lean Thinking to identify and address Delivery Pipeline bottlenecks
Using Lean Thinking to identify and address Delivery Pipeline bottlenecks
 
Bluemix DevOps Meetup
Bluemix DevOps MeetupBluemix DevOps Meetup
Bluemix DevOps Meetup
 

Andere mochten auch

Mobile Performance Testing - Best Practices
Mobile Performance Testing - Best PracticesMobile Performance Testing - Best Practices
Mobile Performance Testing - Best PracticesEran Kinsbrunner
 
Performance testing of mobile apps
Performance testing of mobile appsPerformance testing of mobile apps
Performance testing of mobile appsvodQA
 
Performance Testing for Mobile Apps & Sites using Apache JMeter
Performance Testing for Mobile Apps & Sites using Apache JMeterPerformance Testing for Mobile Apps & Sites using Apache JMeter
Performance Testing for Mobile Apps & Sites using Apache JMeterAlon Girmonsky
 
Performance testing – mobile apps session1
Performance testing – mobile apps   session1Performance testing – mobile apps   session1
Performance testing – mobile apps session1Jyothirmayee Pola
 
Mobile Apps Performance Testing Using Open Source Tool JMeter
Mobile Apps Performance Testing Using Open Source Tool JMeterMobile Apps Performance Testing Using Open Source Tool JMeter
Mobile Apps Performance Testing Using Open Source Tool JMeterDevendra Singh
 
Four Best Practices for Modern Performance Testing
Four Best Practices for Modern Performance TestingFour Best Practices for Modern Performance Testing
Four Best Practices for Modern Performance TestingSOASTA
 
Mobile apps strategydips
Mobile apps strategydipsMobile apps strategydips
Mobile apps strategydipsDeepti Bhutani
 
Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar Experitest
 
CMG imPACt2016 - Mobile performance testing - Vendor training - Federico Tole...
CMG imPACt2016 - Mobile performance testing - Vendor training - Federico Tole...CMG imPACt2016 - Mobile performance testing - Vendor training - Federico Tole...
CMG imPACt2016 - Mobile performance testing - Vendor training - Federico Tole...Federico Toledo
 
Checkout Report Webinar by Elastic Path and Netconcepts
Checkout Report Webinar by Elastic Path and NetconceptsCheckout Report Webinar by Elastic Path and Netconcepts
Checkout Report Webinar by Elastic Path and Netconceptsgetelastic
 
Performance testing – mobile apps session1
Performance testing – mobile apps   session1Performance testing – mobile apps   session1
Performance testing – mobile apps session1Jyothirmayee Pola
 
Mobile Testing Types and Basic Process
Mobile Testing Types and Basic ProcessMobile Testing Types and Basic Process
Mobile Testing Types and Basic ProcessOlesia Hirnyk
 
Mobile Application Testing
Mobile Application TestingMobile Application Testing
Mobile Application TestingNoor Orfahly
 
Training Webinar: Troubleshooting Mobile Apps Performance
Training Webinar: Troubleshooting Mobile Apps Performance Training Webinar: Troubleshooting Mobile Apps Performance
Training Webinar: Troubleshooting Mobile Apps Performance OutSystems
 
SAP performance testing & engineering courseware v01
SAP performance testing & engineering courseware v01SAP performance testing & engineering courseware v01
SAP performance testing & engineering courseware v01Argos
 
Basic Guide For Mobile Application Testing
Basic Guide For Mobile Application TestingBasic Guide For Mobile Application Testing
Basic Guide For Mobile Application TestingSourabh Kasliwal
 

Andere mochten auch (19)

Mobile Performance Testing - Best Practices
Mobile Performance Testing - Best PracticesMobile Performance Testing - Best Practices
Mobile Performance Testing - Best Practices
 
Performance testing of mobile apps
Performance testing of mobile appsPerformance testing of mobile apps
Performance testing of mobile apps
 
Performance Testing for Mobile Apps & Sites using Apache JMeter
Performance Testing for Mobile Apps & Sites using Apache JMeterPerformance Testing for Mobile Apps & Sites using Apache JMeter
Performance Testing for Mobile Apps & Sites using Apache JMeter
 
Performance testing – mobile apps session1
Performance testing – mobile apps   session1Performance testing – mobile apps   session1
Performance testing – mobile apps session1
 
Mobile Apps Performance Testing Using Open Source Tool JMeter
Mobile Apps Performance Testing Using Open Source Tool JMeterMobile Apps Performance Testing Using Open Source Tool JMeter
Mobile Apps Performance Testing Using Open Source Tool JMeter
 
Four Best Practices for Modern Performance Testing
Four Best Practices for Modern Performance TestingFour Best Practices for Modern Performance Testing
Four Best Practices for Modern Performance Testing
 
Mobile apps strategydips
Mobile apps strategydipsMobile apps strategydips
Mobile apps strategydips
 
Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar
 
CMG imPACt2016 - Mobile performance testing - Vendor training - Federico Tole...
CMG imPACt2016 - Mobile performance testing - Vendor training - Federico Tole...CMG imPACt2016 - Mobile performance testing - Vendor training - Federico Tole...
CMG imPACt2016 - Mobile performance testing - Vendor training - Federico Tole...
 
Checkout Report Webinar by Elastic Path and Netconcepts
Checkout Report Webinar by Elastic Path and NetconceptsCheckout Report Webinar by Elastic Path and Netconcepts
Checkout Report Webinar by Elastic Path and Netconcepts
 
Performance testing – mobile apps session1
Performance testing – mobile apps   session1Performance testing – mobile apps   session1
Performance testing – mobile apps session1
 
35602787 mobile-application-testing
35602787 mobile-application-testing35602787 mobile-application-testing
35602787 mobile-application-testing
 
Mobile Testing Types and Basic Process
Mobile Testing Types and Basic ProcessMobile Testing Types and Basic Process
Mobile Testing Types and Basic Process
 
Mobile Application Testing
Mobile Application TestingMobile Application Testing
Mobile Application Testing
 
Training Webinar: Troubleshooting Mobile Apps Performance
Training Webinar: Troubleshooting Mobile Apps Performance Training Webinar: Troubleshooting Mobile Apps Performance
Training Webinar: Troubleshooting Mobile Apps Performance
 
Hp Loadrunner
Hp LoadrunnerHp Loadrunner
Hp Loadrunner
 
Load Runner
Load RunnerLoad Runner
Load Runner
 
SAP performance testing & engineering courseware v01
SAP performance testing & engineering courseware v01SAP performance testing & engineering courseware v01
SAP performance testing & engineering courseware v01
 
Basic Guide For Mobile Application Testing
Basic Guide For Mobile Application TestingBasic Guide For Mobile Application Testing
Basic Guide For Mobile Application Testing
 

Ähnlich wie Performance Testing in a Mobile World

Building enterprise applications using open source
Building enterprise applications using open sourceBuilding enterprise applications using open source
Building enterprise applications using open sourcePeter Batty
 
Keynote | Middleware Everywhere - Ready for Mobile and Cloud | Dr. Mark Little
Keynote | Middleware Everywhere - Ready for Mobile and Cloud | Dr. Mark LittleKeynote | Middleware Everywhere - Ready for Mobile and Cloud | Dr. Mark Little
Keynote | Middleware Everywhere - Ready for Mobile and Cloud | Dr. Mark LittleJAX London
 
redpill Mobile Case Study (Salvation Army)
redpill Mobile Case Study (Salvation Army)redpill Mobile Case Study (Salvation Army)
redpill Mobile Case Study (Salvation Army)Peter Presnell
 
AJAX for Mobile Devices - 04/2009
AJAX for Mobile Devices - 04/2009AJAX for Mobile Devices - 04/2009
AJAX for Mobile Devices - 04/2009Roland Tritsch
 
Managing Complexity in Mobile Application Deployment Using the OSGi Service P...
Managing Complexity in Mobile Application Deployment Using the OSGi Service P...Managing Complexity in Mobile Application Deployment Using the OSGi Service P...
Managing Complexity in Mobile Application Deployment Using the OSGi Service P...mfrancis
 
Mikehall FutureWorld 2010 - enabling connectivity
Mikehall FutureWorld 2010 - enabling connectivityMikehall FutureWorld 2010 - enabling connectivity
Mikehall FutureWorld 2010 - enabling connectivityMicrosoft Windows Embedded
 
AWS Summit Auckland 2014 | Desktops in the Cloud
 AWS Summit Auckland 2014 | Desktops in the Cloud  AWS Summit Auckland 2014 | Desktops in the Cloud
AWS Summit Auckland 2014 | Desktops in the Cloud Amazon Web Services
 
Innovate 2013 session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3Innovate 2013   session 1243 mobile testing.v3
Innovate 2013 session 1243 mobile testing.v3Leigh Williamson
 
IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...
IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...
IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...darwinodb
 
AWS Summit Sydney 2014 | Desktops in the Cloud
AWS Summit Sydney 2014 | Desktops in the CloudAWS Summit Sydney 2014 | Desktops in the Cloud
AWS Summit Sydney 2014 | Desktops in the CloudAmazon Web Services
 
Your Applications Are Distributed, How About Your Network Analysis Solution?
Your Applications Are Distributed, How About Your Network Analysis Solution?Your Applications Are Distributed, How About Your Network Analysis Solution?
Your Applications Are Distributed, How About Your Network Analysis Solution?Savvius, Inc
 
Batty consumerization of geospatial
Batty consumerization of geospatialBatty consumerization of geospatial
Batty consumerization of geospatialGeCo in the Rockies
 
Integrating ADF Mobile with WebCenter
Integrating ADF Mobile with WebCenterIntegrating ADF Mobile with WebCenter
Integrating ADF Mobile with WebCenterBrian Huff
 
Webinar: Enterprise Trends for Database-as-a-Service
Webinar: Enterprise Trends for Database-as-a-ServiceWebinar: Enterprise Trends for Database-as-a-Service
Webinar: Enterprise Trends for Database-as-a-ServiceMongoDB
 
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...NITHIN S.S
 
Mobile CMS - CMSExpo 2010
Mobile CMS - CMSExpo 2010Mobile CMS - CMSExpo 2010
Mobile CMS - CMSExpo 2010Tom Deryckere
 
Roland van leusden mobile performance testing rtc 2014 v0.6
Roland van leusden   mobile performance testing  rtc 2014 v0.6Roland van leusden   mobile performance testing  rtc 2014 v0.6
Roland van leusden mobile performance testing rtc 2014 v0.6Romania Testing
 
ACES QuakeSim 2011
ACES QuakeSim 2011ACES QuakeSim 2011
ACES QuakeSim 2011marpierc
 
IT Technology Trends for Thailand ICT Policy Framework
IT Technology Trends for Thailand ICT Policy FrameworkIT Technology Trends for Thailand ICT Policy Framework
IT Technology Trends for Thailand ICT Policy FrameworkThanachart Numnonda
 
HTML5 Mobile Application Framework
HTML5 Mobile Application FrameworkHTML5 Mobile Application Framework
HTML5 Mobile Application FrameworkThanh Nguyen
 

Ähnlich wie Performance Testing in a Mobile World (20)

Building enterprise applications using open source
Building enterprise applications using open sourceBuilding enterprise applications using open source
Building enterprise applications using open source
 
Keynote | Middleware Everywhere - Ready for Mobile and Cloud | Dr. Mark Little
Keynote | Middleware Everywhere - Ready for Mobile and Cloud | Dr. Mark LittleKeynote | Middleware Everywhere - Ready for Mobile and Cloud | Dr. Mark Little
Keynote | Middleware Everywhere - Ready for Mobile and Cloud | Dr. Mark Little
 
redpill Mobile Case Study (Salvation Army)
redpill Mobile Case Study (Salvation Army)redpill Mobile Case Study (Salvation Army)
redpill Mobile Case Study (Salvation Army)
 
AJAX for Mobile Devices - 04/2009
AJAX for Mobile Devices - 04/2009AJAX for Mobile Devices - 04/2009
AJAX for Mobile Devices - 04/2009
 
Managing Complexity in Mobile Application Deployment Using the OSGi Service P...
Managing Complexity in Mobile Application Deployment Using the OSGi Service P...Managing Complexity in Mobile Application Deployment Using the OSGi Service P...
Managing Complexity in Mobile Application Deployment Using the OSGi Service P...
 
Mikehall FutureWorld 2010 - enabling connectivity
Mikehall FutureWorld 2010 - enabling connectivityMikehall FutureWorld 2010 - enabling connectivity
Mikehall FutureWorld 2010 - enabling connectivity
 
AWS Summit Auckland 2014 | Desktops in the Cloud
 AWS Summit Auckland 2014 | Desktops in the Cloud  AWS Summit Auckland 2014 | Desktops in the Cloud
AWS Summit Auckland 2014 | Desktops in the Cloud
 
Innovate 2013 session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3Innovate 2013   session 1243 mobile testing.v3
Innovate 2013 session 1243 mobile testing.v3
 
IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...
IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...
IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...
 
AWS Summit Sydney 2014 | Desktops in the Cloud
AWS Summit Sydney 2014 | Desktops in the CloudAWS Summit Sydney 2014 | Desktops in the Cloud
AWS Summit Sydney 2014 | Desktops in the Cloud
 
Your Applications Are Distributed, How About Your Network Analysis Solution?
Your Applications Are Distributed, How About Your Network Analysis Solution?Your Applications Are Distributed, How About Your Network Analysis Solution?
Your Applications Are Distributed, How About Your Network Analysis Solution?
 
Batty consumerization of geospatial
Batty consumerization of geospatialBatty consumerization of geospatial
Batty consumerization of geospatial
 
Integrating ADF Mobile with WebCenter
Integrating ADF Mobile with WebCenterIntegrating ADF Mobile with WebCenter
Integrating ADF Mobile with WebCenter
 
Webinar: Enterprise Trends for Database-as-a-Service
Webinar: Enterprise Trends for Database-as-a-ServiceWebinar: Enterprise Trends for Database-as-a-Service
Webinar: Enterprise Trends for Database-as-a-Service
 
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...
 
Mobile CMS - CMSExpo 2010
Mobile CMS - CMSExpo 2010Mobile CMS - CMSExpo 2010
Mobile CMS - CMSExpo 2010
 
Roland van leusden mobile performance testing rtc 2014 v0.6
Roland van leusden   mobile performance testing  rtc 2014 v0.6Roland van leusden   mobile performance testing  rtc 2014 v0.6
Roland van leusden mobile performance testing rtc 2014 v0.6
 
ACES QuakeSim 2011
ACES QuakeSim 2011ACES QuakeSim 2011
ACES QuakeSim 2011
 
IT Technology Trends for Thailand ICT Policy Framework
IT Technology Trends for Thailand ICT Policy FrameworkIT Technology Trends for Thailand ICT Policy Framework
IT Technology Trends for Thailand ICT Policy Framework
 
HTML5 Mobile Application Framework
HTML5 Mobile Application FrameworkHTML5 Mobile Application Framework
HTML5 Mobile Application Framework
 

Kürzlich hochgeladen

Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 

Kürzlich hochgeladen (20)

20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 

Performance Testing in a Mobile World

  • 1. Performance Testing in a Mobile World Stuart Moncrieff Performance Testing Practice Lead
  • 2. Presentation Roadmap Background Mobile device Network Server performance performance performance 2 JDS
  • 3. Background Background Mobile device Network Server performance performance performance 3 JDS
  • 4. Mobile is growing fast • There were more smartphones and tablets sold in 2011 than desktop PCs, laptops and netbooks combined. • 20% of web traffic in the US comes from smartphones and tablets 4 JDS
  • 5. Smartphone Sales by Operating System • Android is the dominant mobile phone operating system • BUT!! – The most popular handset is iPhone (especially in .au) – Google Play has less than ¼ of the revenue of Apple’s App Store – iPad has more market share than all Android tablets combined – App sales are negligible for other stores (Nokia Ovi Store, Blackberry App World, Windows Phone Marketplace) 5 JDS
  • 6. Mobile in the Enterprise • Growth of “bring your own device” (BYOD) for internal apps – mostly web apps and email • Devices for field service/delivery workers (probably not running iOS or Android) • Mobile versions of public-facing websites (http://m.yoursite.com.au) • Mobile App projects – Free apps to reach more customers. E.g. banking app, supermarket app (not monetised through app sales) – Projects initiated by the Business, instead of the IT department – Very small budgets, almost always outsourced (to small, agile companies) – Less maturity than typical Enterprise projects (minimal non-functional requirements, security testing, performance testing) • Large IT projects sometimes have a small mobile component 6 JDS
  • 7. Mobile performance is important • Bounce-rates increase dramatically as response times increase • 1 star reviews in App Store “this app sucks” 7 JDS
  • 8. Factors in Mobile Performance • Mobile device • Network • Data Centre – Varying hardware ‒ The “last mile” ‒ Load ‒ The Internet balancers, se – Mobile app code rvers (from your telco to your data centre) ‒ Server code, capacit y, configurati on 8 JDS
  • 9. Mobile Device Performance Background Mobile device Network Server performance performance performance 9 JDS
  • 10. Apps behaving badly • Bandwidth hog – “Your app gave me a huge phone bill!” • Storage hog – “Your app is using up all my storage!” • Drains battery – Location-based apps, data use patterns (polling keeps activating the transmitter) • Leaks memory • Doesn’t handle network connectivity problems gracefully • Slow startup or unresponsive user interface 10 JDS
  • 11. Be careful with WebView • Tempting shortcuts: – Write parts of your app in JS + HTML, and display in a WebView, instead of using native code. – HTML5 vs Cross-platform vs Native – Make sure you pick the technology with the right trade-offs for your application • Mobile WebKit has a 300ms delay on firing click events after a tap. – Work-arounds exist, but impose trade-offs • Cross-platform mobile development frameworks (PhoneGap, Trigger.io, Appcelerator, etc.) – Can be hard to make non-native apps feel “snappy” – Overhead imposed by JavaScript to Native bridge 11 JDS
  • 12. Profiling tools for developers • DDMS (Android) or Instruments (iOS) – View memory usage and allocation (find leaks), threads, process resource usage, slow methods, network usage, radio state (test outage conditions). • JavaScript – Profile it on a desktop browser (FireBug for Chrome/Firefox, Web Inspector for Safari) – Use JSLint to ensure strict code “correctness” 12 JDS
  • 13. Case Study: SnappyCam+ • Takes lots of photos very fast (great for action sequences) • Camera performance was the central feature of the app • Bottleneck: JPEG compression – Existing compression libraries were too slow – Researched academic papers on JPEG compression algorithms – Implemented compression algorithm highly-optimised for CPU architecture – Hand-crafted assembly code for the ARM NEON SIMD co-processor 13 JDS
  • 14. Network Performance Background Mobile device Network Server performance performance performance 14 JDS
  • 15. Sloooow network • Mobile data speeds are highly variable and can be very slow (like a modem) • Websites have become bloated with the widespread adoption of broadband Internet • You can’t improve the network, but you can optimise your mobile app/website for slow networks 15 JDS
  • 16. Shunra NetworkCatcher Express Mobile • Free iPhone/Android app to check network performance from your current location. • Benefits: – Measure real-world mobile network conditions (bandwidth, latency, packet loss) – Generates an NTX file to use with Shunra PerformanceSuite or Shunra vCat 16 JDS
  • 17. WAN Emulation • Most testing done while connected to – Your office Wi-Fi – Fast, metropolitan 3G internet • Use a WAN emulator to measure response times under different network conditions 17 JDS
  • 18. Optimising for slow networks • Minimise data transfer – Create a mobile version of your website, and serve images scaled for small mobile screens – Use the HTML5 Application Cache or local storage • Minimise “chattiness”. – Multiple sequential requests multiply the effect of network latency. • Pre-fetch data to make it seem fast • Use YSlow! for your mobile website – Ensure content is cachable (correct headers, etc.) – Make files smaller (compress with Gzip, minify JS/CSS files) – Minimise HTTP requests – Be aware of slow, third-party sites (e.g. ad networks, analytics) 18 JDS
  • 19. Case Study: Smart Electricity Meters • DPI mandated that all manually read electricity meters be replaced with smart meters by the end of 2013. • All meters must report usage (in 30 minute intervals) to AEMO by 6am the following day. • Problems at one electricity distributor: – WiMAX (4G) teething problems • Patchy data coverage. Meters not “phoning home” reliably • Unresponsive meters (requiring site visits) – “Thundering herds”. Trying to avoid all the meters sending their data at the same time • Limited bandwidth on base station • Servers cannot handle 700,000 concurrent connections 19 JDS
  • 20. Server Performance Background Mobile device Network Server performance performance performance 20 JDS
  • 21. Performance Testing 101 • Define Workload Model – Identify key business processes – Define Peak Hour transaction volumes for each business process – Define number of concurrent users, network properties • Organise Production-like test environment (with monitoring) • Develop Scripts – Note: scripts send same traffic as device, so no phones are needed to run the load test. But response times will not include client-side time. – Record business processes • Execute tests – Peak load, Stress/Break, Soak – Failover, interface outages, etc. • Tune, Test, Tune, Test, etc. – Problems due to code, capacity, or configuration 21 JDS
  • 22. Difficulties recording mobile apps • Record device or simulator/emulator on office network • Probably can’t install recording agent on mobile device • Most apps are just making web service calls Application Type Recording Method Mobile website • Record from desktop PC with standard browser. Use a User Agent Switcher add-on. Mobile app (HTTP) • Use a recording proxy Mobile app (HTTPS) • Security features prevent use of a recording proxy (i.e MITM attack). Some workarounds. • Use packet capture, decode with SSL cert. Mobile app (WebSockets • Beware! Poor tool support. or other new protocols) • Be careful with new technologies • Scoping question: what protocols does the app use? 22 JDS
  • 23. Example: Recording a mobile app (using HTTPS) with LoadRunner 1. Server-side packet capture while performing business process (e.g. using WireShark) 2. Export private certificate (from test environment) 3. New script > Mobile Application - HTTP/HTML – Recording type: Analyze traffic 4. Select PCAP file, set up filters, import certificate 5. Generate script 6. Add transaction names, correlate values, etc. 23 JDS
  • 24. WAN Emulation under load • Users on slow network connections have different load characteristics on front-end servers – Hold sockets open longer – Cause more active TCP sessions on load balancer (licensing limits, memory limits for session table) – Cause more open connections to web servers (thread/process limits, O/S TCP limits, requests queue waiting for a connection) • During load testing – Emulate network conditions for each virtual user individually (bandwidth is not shared by virtual users) – Emulate worst-case to find connection limits on servers 24 JDS
  • 25. Performance Testing Overview Test Device Network Server Load/ Performance Behaviour Performance Envt Development Development Production-like Users One One Many, emulated/virtual Who Developer Developer or Tester Tester Notes • Manual, using • Manual, using • Automated, using load profiling tools network profiling testing tools • Testers can help • Using WAN • With/without WAN with requirements, emulation emulation oversight, etc. 25 JDS
  • 27. Thank You / Credits • CoreForm – Android app development & security – http://www.coreform.com.au/ • GridStone – Mobile app development studio – http://gridstone.com.au/ • SnappyLabs – iOS app development – http://www.snappylabs.com/ • MyLoadTest – My blog! – http://www.myloadtest.com/ 27 JDS