Cloud Computing and Mobile platforms (e.g. Android, iPhone) have emerged as compelling choices for a large number of software systems and Apps that are built today. While these new paradigms present opportunities for suppliers to provide innovative services, they also present significant challenges to quality engineers. This case study presents practical solutions to overcome those challenges.

1. Challenges and Solutions in Testing Modern
Software Paradigms like
Mobile and Cloud Computing
Satya Kaliki
Chief Technology Officer
satya.kaliki@zanec.com

2. Who am I?
 CTO and Co-founder of ZANEC Technologies
2
 Interim CTO at an Insurance Company
owned by Harvard Medical Institutions
 Passionate about Mobile and Cloud
Computing, Semantic Web, SOA
 13 Years of industry experience

3. Agenda
 Mobile Computing
 Challenges
 Factors to consider in your test strategy
 Case Study – Search App for iPhone, Android, Windows
Phone 7
 Cloud Computing
 Challenges
 Factors to consider in your test strategy
 Case study - Air Cargo Supply Chain
 Q&A
3

4. CHALLENGES

5. Features
5
LocationsDevices
Carriers
Test case
complexity

6. Mobile Apps vs. Traditional Apps
6
vs.

7. Native Apps vs. Mobile Browser App
7
Courtesy: Global Intelligence Alliance

8. Location and Localization
8

9. Field Testing – Signal strength variations
9

10. THINGS TO CONSIDER IN YOUR TEST
STRATEGY

11. 11
Use
Emulators
in the Lab
Use Real
Devices in
the Lab
Use Real
Devices in
the Field
Adopt a
crowdsourcing
strategy
App Maturity and Testing Progress

12. Lowest Common
Denominator
and choose based
on market share
12

13. Usability
13

14. Navigation
14

15. Varied end user expectations
15

16. Field Testing
16
Backgroundnoise

17. Field Testing - Multitasking
17

18. Use Emulators to simulate
network variations
18
Android Emulator Commands
 emulator – netdelay gprs [Emulates
a GPRS, means min of 150 max of
550 ms]
 emulator – netspeed gprs [Transfer
rates of a gprs network]
 emulator – netspeed 14.4 14.4
[Exact transfer rate for up and
down]

19. CASE STUDY
Search App for iPhone, Android,
Windows 7 Mobile

20. App Overview
 The Search App provides secure, real-time access to
documents, structured data and messages - all from a
single click
 Can connect to wide variety of sources including PC’s local hard
drives, Popular social Networking sites, Internet Search engines,
Internet Email Providers, MS SharePoint, MS Exchange Server
 Can distinguish variety of file types including MS Office
 Currently the App is targeting two broad categories of Users
 Personal Use - Social Networking, SMS, Contacts, Media, Photos
 Business Users - Email, Exchange, Contacts etc
20

21. Test Strategy
 iPhone as first Platform and iOS Versions 4.1 (3G,
3GS and 4)
 Extended to Android in Phase 2
 Windows Phone 7 is Phase 3
21
 Test Approach
 Emulators in the Lab
 Real Devices in the Lab
 Real Devices in the Field
 Customer’s priority focus is UK

22. Test Strategy
 Scope of Testing
 Usability
 Navigation
 Intuitiveness
 User Personas
 Varied demographics
 Mobile Literacy not considered for this App
 Performance
 Multi-tasking
 Varying signal and network parameters
22

23. Challenges in
Cloud Computing
Testing

24. Introduction
 Cloud Computing presents significant benefits to
organizations that adopt this paradigm:
 Reduced Cost
 Improved Flexibility
 Significant Mobility
 Increased Service Focus
 Testing in a Cloud Computing environment must
address key risks associated with migrating to
Cloud
 In the absence of a comprehensive Test Strategy
realization of the promise remains a dream
24

25. Cloud
Computing
Risks
Security
Reliability
Manageability
SLAs
Multi-tenancy
Resilience
&
Self-healing
Governance
Location
Independence
25

26. Test teams must understand these models
Consumers
Networks & Firewalls
Data Centers and Power
Physical Servers
Virutalisation
Application Platforms
Operating Systems
Software
Infrastructureasa
Service(IaaS)
PlatformasaService(PaaS)
SoftwareasaService(SaaS)

27. THINGS TO CONSIDER IN YOUR TEST
STRATEGY

28. Choose a Test Environment
On-Premise vs. On-Cloud
28
vs.

29. Must include tests for each of
the Cloud Computing risks
29
Don’t forget to include the
App Functional Testing!

30. Assess Data Security
30
Evaluate App specific
security best practices
Vulnerability
Analysis
Audit,Compliance
andGovernance

31. 31
Performance Testing

32. 32
Multi-tenancy and
data security

33. 33
Test for Reliability

34. 34
Test for Maintainability and
Manageability

35. 35
Estimate optimal Service Levels
and test whether you are able to
meet them

36. 36
Checklist for
assessing Go Live
Readiness

37. CASE STUDY
Air Cargo Supply Chain

38. App Overview
 The App provides secure, real-time access to
shipment documents and role based view of
consignment location
 One-Time Data Entry
 End to end, role based visibility to all stakeholders
(Ground handlers, General Sales Agents, Airlines,
Customs and Freight Forwarders)
38
 Offers great benefits to stakeholders
 Increased reliability on consignment’s current status
 Reduced costs
 Improved Accuracy and Quality
 Enormous Scalability

39. Test Strategy
 Infrastructure as a Service model
 On Cloud Test Environment
 Security model best practices
 Multi-tenant
 Multi-tiered (Same document, different view
based on organisation, role and level within)
 Need to perform well at 1 million consignments
being shipped concurrently
39

40. Test Strategy
 Scope of Testing
 Functional
 Usability
 Various levels of Security
 Performance
 Manageability
 Recovery
 Operational readiness
40

41. Questions?