There has been an increase in the adoption of smartphones, tablets and several mobile devices with the passage of time. And it has led to an enormous growth of mobile applications in recent years. Mobile device is considered to be the primary medium of interaction for the customers and also, businesses worldwide. And mobile applications are, actually, driving the communication. People, generally, do not give much importance to mobile application testing because of its expensive nature. But it is very much essential to ensure that the consumers have a great experience, every time they use the application. The aim of mobile application testing should be to acquire knowledge about the quality of the service that you are offering. Does it work properly or not? Will it provide services as per the expectation of the customer? These questions need to be answered to ensure that the customer comes back to you, for your service again. Mobile testing is becoming more and more complicated and complex with each passing day. Strategies are invented and used to simplify the mobile application testing. This presentation primarily addresses, the strategy to be adopted in mobile app testing, the types of mobile app testing and the stages to be followed, before the application is set live.

1. © RapidValue Solutions
Mobile Application
Testing Strategy

2. © RapidValue Solutions
Mobile App Testing Strategy – Adopted to Deliver Quality Service and Provide Utmost Consumer Satisfaction with the Perfect App !
There has been an increase in the adoption of smartphones, tablets and several mobile devices with the passage of time. And it has led to an enormous growth of mobile applications in recent years. Mobile device is considered to be the primary medium of interaction for the customers and also, businesses worldwide. And mobile applications are, actually, driving the communication.
People, generally, do not give much importance to mobile application testing because of its expensive nature. But it is very much essential to ensure that the consumers have a great experience, every time they use the application.
The aim of mobile application testing should be to acquire knowledge about the quality of the service that you are offering. Does it work properly or not? Will it provide services as per the expectation of the customer? These questions need to be answered to ensure that the customer comes back to you, for your service again.
Mobile testing is becoming more and more complicated and complex with each passing day. Strategies are invented and used to simplify the mobile application testing.
This presentation primarily addresses, the strategy to be adopted in mobile app testing, the types of mobile app testing and the stages to be followed, before the application is set live.

3. © RapidValue Solutions
The main objective of mobile application testing is to help customers to gain assurance that the software applications are performing and functioning as expected with the devices or with the several other applications of the vital players in the mobility ecosystem.
Objectives of Mobile Application Testing
Parameters
Details
Functionality
UI responsiveness, Peak load performance, Operational times
Performance
Network type (Wi-Fi, 2G, 3G, 4G), Impact of connectivity issues
Network
Security of data in motion and data at rest
Security
Mobile platform compatibility, Device model compatibility,
Backward compatibility with previous app version
Compatibility
Marketplace guidelines’ compliance for Apple and Google stores
Conformance
Compare the developed product with user stories
Usability
User Experience
Installation and Provisioning
Installation process, Un-installation process, User provisioning and de- provisioning
Service Testing
Check that the services are up and running and also measure the performance
Resource Testing
Battery performance, Memory usage, Local database growth and Garbage in memory, come under this process
Load Testing on Mobile
With heavy loads of images, videos and also loading the app with more and heavy functions, use third party application like blaze meter to generate load
Localization
With this, the QA team checks the application’s multilingual capability

4. © RapidValue Solutions
Emulators
Cloud Testing Solution
Physical Device
oAll applications can be deployed and tested on emulator without investing in mobile handset for various OS
oEmulators are mostly available free, and we can also perform UI, Stress and performance testing on that
o30-40% test
oThe Mobile devices can be accessed through web interface i.e. the browser
oApplication can be deployed, tested and managed
oAutomation module is available and the solution is secure, if private cloud is used with no maintenance
oUp to 100% test coverage can be achieved
oReal device testing will give most realistic view of test results
oAll possible types of testing activities can be performed including those, that are dependent on hardware
Mobile Testing Strategy is an essential element to ensure the smooth functioning of any application. Emulators are considered to be vital testing platforms while Cloud testing solution is very much secure. Real Device Testing is said to yield the most realistic and practical results.
Mobile Testing Strategy

5. © RapidValue Solutions
•Different Form Factors
•Wide Variety of devices from different manufacturers, incase of Android
•Different versions of Mobile OS
•Different Screen Density
•Infrastructure and Connectivity
•Mobile testing tool availability – still on nascent stage
•Cloud testing platform solves some of these issues, however:
oIt is very expensive
oLock – in with the platform
oScripts cannot be used outside of the platform
oOCR based platform. Because of automation, it is time consuming
Challenges in Mobile Testing
Mobile applications are considered to be “game-changing” technologies, which have further led to the development of various unique and beneficial testing challenges. The growth of the application market has increased the customer’s expectation of obtaining quality service. Companies are also coming up with new apps in the marketplace. Hence, challenges of mobile testing are also increasing each day. Whether it is an enterprise app or a consumer app, companies are finding out ample opportunities to boost their revenues and reaching out to a vast range of consumers. This ambition leads to a greater development of mobile testing strategies which face certain challenges:

6. © RapidValue Solutions
•iOS 7 and above
•iPhone 4, iPhone 4s
•iPhone 5, iPhone 5s, iPhone 5c
•iPhone 6
•iPad Air, iPad2
•iPod touch 4th generation
•iPod 5th generation
•Android 4.x and above
•Samsung galaxy s4, s5
•Samsung Note 3
•Nexus 5
•HTC desire 816,one
•Moto X
•Sony Xperia Z
Popular Mobile Devices for Testing Applications
A great application certainly needs a greater customer service solution. And it is recommended for the application to support all the iOS and Android devices. The following are some of the devices and versions, on which its good to test your application.

7. © RapidValue Solutions
The following statistical chart gives a clear picture of the screen sizes and densities for Android:
Screen Sizes and Densities for Android

8. © RapidValue Solutions
Simulator Testing
Device Testing
(On Wi-Fi Networks + Network Simulation Tools)
Device Testing
(In-Network)
•Supports only device dependent use cases like camera, accelerometer, GPS etc.
Device Testing (In –Network)
•Ideal during the development phase
•Best and economical option for functionality testing
Simulator Testing
Device Testing ( Wi-Fi )
•Use proxy tool Charles proxy to monitor the load and performance of the application
There are certain distinctive features of Simulator and Device Testing. They are summarized below:
Simulator vs. Device Testing

9. © RapidValue Solutions
Advantages of Cloud Testing:
•Variety of plans available, such as hourly, monthly or yearly subscriptions
•Test incoming calls and sms without having any cellphone connection
•Wide variety of devices
•Automated test execution, can be played as videos
•Test can run on several devices, in parallel
•Build integration
The performance testing is conducted on the applications and services provided via cloud computing. Cloud testing ensures optimal performance and scalability under a wide variety of conditions. Cloud testing monitors and reports on the real-world user traffic conditions.
Cloud Testing
User interacts with the device through studio or web front provided by the cloud provider
Server receives the request and delegates that to the actual device
Device sends back the result and that would be displayed on the studio or web front

10. © RapidValue Solutions
The Application testing commences with Unit Testing and then proceeds with Functional Testing, Regression Testing and UI Testing. This is further followed by Automation Testing, Performance & Load Testing and the final stage is the Security Testing.
Application Testing Flow
Unit Testing
Functional Testing
Regression Testing
UI Testing
Automation Testing
Performance & Load Testing
Security Testing
Performed by developers before releasing the build
QA team perform the functional testing based on the test cases that are created using user stories
Regression testing is a suite of testing, that include smoke test and execution of regression test cases
UI testing is a usability testing, focusing on the UI design of the application, carried out by UI & QA teams
Once the development is in shape, all the regression test cases should be automated
QA team should check the UI response, peak load, transaction time as part of the performance testing. This should be conducted after every 3 sprints
Security testing should be considered for data security, authentication and authorization

11. © RapidValue Solutions
App Store
Play Store
Terms & Conditions
Functionality/Reliability
Metadata/Ratings & Location
Intellectual Property Rights
UI conformance & Contents
Spam & Placement
Ads & System Interference
Contents
Personnel & Confidential Information
Content Ratings
Conformance testing is considered to be critical. Conformance is essential and crucial to determine whether the product or the service meets the specified standards. Validation of the App Store and provision of the Google Store Guidelines are the integral part of the effective process, which plays a pivotal role.
Conformance – Validate App Store & Google Play Store Guidelines

12. © RapidValue Solutions
•The network testing should be carried out in physical device by the QA team to ensure the network performance under various technologies like GSM and CDMA
•Cloud testing should also be leveraged to test it in the real environment
•Separate test case suite should be designed to test the various signal strength under tele- signals using 4G,3G,2G etc. and the Wi-Fi signals
•Charles proxy, a third-party tool can be used to control the network bandwidth by creating device and Charles proxy operate under one network
•Applications should be tested under multiple signal strength and quality performance under live environment is ensured
The most important thing is to ensure smooth and seamless network performance which can be evaluated with the help of efficient Network Testing. Performance is evaluated under technologies like CDMA and GSM. The applications are tried and tested under live environment to ensure utmost customer satisfaction.
Network Testing

13. © RapidValue Solutions
•Usability testing should be carried out by the following testing:
oValidation of gray scale wire frames
oTesting the compatibility with all devices and browsers
oUI testing that focuses on user acceptance
oUI comparison testing with benchmarked applications & standards
oReview testing before every UAT by customer
•Usability test should be conducted by UI/UX and QA team
Another important testing procedure is the Usability Testing which should be ideally conducted by UI/UX team and the QA team. It is essential to test the compatibility factor with the various devices and browsers. UI Comparison Testing is also needed to know whether the benchmarked App is in conformity with the existing standards.
Usability

14. © RapidValue Solutions
•SOAP UI is a free testing tool, based on open-source standards as XML, XPATH and GROOVY (Java). It is stable and robust too.
•Using SOAP UI, tester are able to perform
oWeb service regression testing
oWeb service test automation
oWeb service load testing
•SOAP, REST service will be covered as a part of service and all the responses will be evaluated using tools like GET, POST etc.
Service testing will enable the providers to evaluate the performance of the application and work on it, according to the test results. It is essential to deliver a comprehensive testing as a service platform. There has been a rapid growth in the demand for effective application testing tools and services that will leverage mobility services. Service testing is performed to deliver effective and high-quality applications.
Service Testing

15. © RapidValue Solutions
•Resource utilization like memory, CPU, etc. is measured while doing Performance Testing.
•Set criteria for Application/Device performance, Server Performance and Network performance
oQuadrant and Linpack is used for measuring performance
oLint (come with Android SDK), static code analyzer is used for performance and optimization
oOC Lint (IOS), Static code analyzer is used for performance and optimization
oiOS instrumentation tool is used to find the leaks and memory foot print in iOS application
Testing is, indeed, an important aspect of development. It is crucial and critical for both, creating new applications and modifying the existing ones. Utilization of resources is measured effectively while going through the process of Performance Testing. Linpack and Quadrant prove to be effective in measuring performance of the application.
Resource Testing

16. © RapidValue Solutions
There are two methods to perform load testing - Manual & Automation
Automation Testing:
oLoad Testing is trying to simulate the work load of various user scenarios to the applications with third Jmeter
oTesting the load within a specific pagination by monkey testing
Manual Testing:
oThe application under test are loaded with large images (high resolution) and videos to evaluate the peak point where the application can handle the maximum load
oLoad testing are performed under various signal bandwidth to understand the minimum network signal strength to run the application with standard usage
Load Testing

17. © RapidValue Solutions
Automation testing is the key testing strategy to automate Regression testing which include all functionalities:
oAppium is a good tool for Automation testing. And automation should start after the development is in the right stage
oAppium is an open source, cross-platform test automation tool for native, hybrid and mobile web apps. It is tested on simulators (iOS, FirefoxOS), emulators (Android), and real devices (iOS, Android, FirefoxOS)
oAutomation testing should be a part of every sprint to reduce the effort of Regression testing and Smoke testing
oAutomation frame should be designed after couple of sprints
One of the best characteristic of Automation Testing is that it does not require any manual intervention while executing the particular test. The test has certain benefits like greater consistency and the reduced nature of the repetitive work. Automation testing can be relied upon to provide the ease of accessibility to information about the tests.
Automation Testing

18. © RapidValue Solutions
•Automation of regression suite can be incremental starting from 1st sprint
•Automated regression suite to check the impact of latest sprint deliverables on the previous sprint(s) deliverables, for continuous integration
•Re-use the same (with additional manualautomated tests) for hardening sprint before application is launched
Regression Testing is another essential procedure which deals in rerunning test cases just to ensure that the software changes do not have any side-effects. Regression Testing has become more effective and efficient with time. This testing is critical to check or detect whether new issues have cropped up after the change has been applied. Regression Testing is critical for large applications as it is a little difficult to figure out whether any part of the application has given rise to any issue or problem due to the change.
Regression Testing

19. © RapidValue Solutions
•Supporting types: Text, Images, Videos etc.
•Size: Maximum size supported for each category
•Size: Beyond maximum supported size for each category to see the stability / Large volume of data and measure performance
•Mix of different data types with maximum size
•Negative Data (Invalid data): The application is not intented to support
•Data and network (2G,3G,Wifi) combination
•Data and Device type/OS combinations
The following gives a detailed description of the data types, data sizes, blend of different data types and sizes, supporting formats etc. It also provides information regarding the data and device/network combinations. One obtains knowledge about negative data also, which is a particular data type or data size that the application does not support.
Test Data – Type and Size

20. © RapidValue Solutions
•The app diagnostics and crashes on iOS and Android can be monitored using Crittercism
•Google Analytics tool helps to monitor the exit points and identify the reason. It collects user interaction data, manages how the data is processed and provides a report.
The post-production stage is equally important as the pre-production and the production stage. A report has to be provided after collecting user-interaction data and managing the data procedure. Crittercism and Google Analytics provide immense support for monitoring the application.
Post Production

21. © RapidValue Solutions
•Hardening sprint(s) - QA only sprint(s) for Go Live
•On actual environment and configurations
•Regression suite to focus on system level, Integration, Impacted (changes) areas, performance and security
•Real-time data, text, videos and images with maximum supporting size
•Go Live check lists based on the sprint level confidence
oEntry Criteria
oList of devices and configurations
oActual environment & Infrastructure, Test Data
oRegression Suite and Use cases
oInstallation/ Uninstallation
oExit criteria
Testing stage holds utmost importance. It gives the clear picture of the task that has been carried out with such great effort. Testing is done in actual environment and configurations to know whether the application supports it or not.
Go Live – Testing

22. © RapidValue Solutions
The growth of the application market has increased the customer’s expectation of obtaining quality service. Companies are also coming up with new applications in the marketplace. Hence, challenges of mobile testing are also increasing day-by-day. Mobile application testing is, indeed, an integral and essential part of the production process. It is a procedure by which the application software, that is developed for the mobile devices, is tested for its supportive feature, its functionality, usability and for its consistency. The main objective of mobile application testing is to help and assist customers to gain assurance that the software applications are performing and functioning as expected. Mobile testing strategy is an essential element to ensure the smooth functioning of any application.
•The most important thing is to ensure smooth and seamless network performance which can be evaluated with the help of efficient Network Testing.
•Service testing enables the providers to evaluate the performance of the application and work on it, according to the test results.
•The performance testing is conducted on the applications and services provided via cloud computing.
•The Usability testing is equally essential to test the compatibility factor with the various devices and browsers.
•Service testing, Load testing, Regression testing and Automation Testing - all the testing procedures form the significant part of the application testing strategy which is crucial and vital for the production of an application.
Conclusion

23. © RapidValue Solutions
Thank You

24. © RapidValue Solutions
About RapidValue
RapidValue is a leading provider of end-to-end mobility solutions to enterprises worldwide. Armed with a team of 250+ experts in mobility consulting and application development, along with experience delivering over 400 mobility projects, we offer a range of mobility services across industry verticals. RapidValue delivers its services to the world’s top brands and Fortune 1000 companies, and has offices in the United States and India.
www.rapidvaluesolutions.com
www.rapidvaluesolutions.com/blog
+1 877.690.4844
contactus@rapidvaluesolutions.com
This document contains information that is confidential and proprietary to RapidValue Solutions Inc. No part of it may be used, circulated, quoted, or reproduced for distribution outside RapidValue. If you are not the intended recipient of this report, you are hereby notified that the use, circulation, quoting, or reproducing of this report is strictly prohibited and may be unlawful.
Disclaimer:
Presentation by
Rinish KN
CTO, RapidValue Solutions