IDC, the premier global provider of IT market research, and SOASTA, an IDC industry leader in cloud testing know that maintaining leadership means moving quickly to outpace the competition. Both IDC and SOASTA work with clients to realize the benefits that cloud computing brings to delivering high quality, rapidly deployable web and mobile applications. Join them in this webinar where you will hear: IDC speak on: Perspectives on the state of cloud computing for agile web and mobile development Market dynamics and maturity around the cloud and cloud testing Recommendations for getting started with cloud testing SOASTA speak on: The business drivers for cloud and virtualization Customer goals of using and implementing cloud testing The road to implementing cloud testing in a continuous integration model Case studies of customer cloud testing success SOASTA’s services and technology will be highlighted and demonstrated as a solution for continuous web and mobile testing as utilized by the Paychex team. Who Should Attend? Senior IT Management Development and QA Executives and Directors Performance team leads and engineers Test Automation leads and engineers Mobile Development and Testing team leads and engineers

1. 1
© 2013 SOASTA. All rights reserved.
Webinar
Presents
How cloud computing has
impacted the Quality as a
Service marketplace
Joined by:

2. 2
© 2013 SOASTA. All rights reserved.
Market perspectives and applications of cloud testing
TODAY’S SOASTA PRESENTERS
Melinda Ballou: Program Director, Application Life-Cycle Management, IDC
Brad Johnson: VP Product Marketing, SOASTA
Mike Ostenberg: Sr. Solutions Architect, SOASTA
Agenda:
• Poll question
• Market Perspective and Recommendations (IDC)
• Cloud Testing Drivers and Use-Cases (SOASTA)
• Overview and Demonstration of a Leading Cloud Testing Platform
Questions:
Submit in chat box during event

3. 3
© 2013 SOASTA. All rights reserved.
Melinda Ballou

4. Copyright 2013 IDC. Reproduction is forbidden unless authorized. All rights reserved.
Adopting Cloud Testing for Continuous
Delivery: Evolving Quality to Address
Complexity
Melinda Ballou
IDC
Program Director
Application Lifecycle Management
& Executive Strategies

5. Mar-22
© 2013 IDC 5
Industry Highlights: Trends Driving
ASQ & ALM Adoption & Services
 Organizations re-invest, seeking to do more with fewer resources with
financial and staffing constraints; leveraging efficient, agile approaches to
restore and sustain high performing, business-critical software.
 Complex sourcing/off-shoring plus use of open source need strong
teaming, effective code management, testing, change and IT project portfolio
management. Services driven environment (SaaS/cloud,Devops emergence)
 Global economic competition and local compliance across geographies
demand quality, change and portfolio management, adaptability and rigor
 Flexible, agile development paradigm with services creation increasingly
drive technology and business collaboration; agile emergence also disruptive
 Increasing security issues (as driver) and virtualization/cloud (as enabling
technology) for ALM adoption; ad hoc approaches unsustainable
 End-user experience and business impact challenges of rich Internet,
mobile, embedded, with social media collaboration/community opportunities
 SaaS license delivery mechanisms bring new models, de-capitalization
benefits, flexibility, adoption speed, limit complexity, cloud on-ramp

6. Mar-22
© 2013 IDC 6
“Quality Gap”: High Cost of Failure,
Cloud Benefits to Address Challenges
Poor Quality = Increased Business Risk
Lost Revenue
($$$$$)
Lost Customers
Lost Productivity
Increased Costs
Lower Profits
Damaged Brand

7. Mar-22
© 2013 IDC 7
App Development Life-Cycle:
Old View Rapidly Shifting
Assumes: Unidirectional flow, limited # of handoffs
Result:
Silos within project team; gulf with operations
Little understanding how efforts impact overall result
Testing takes place LATE in the cycle!
Define Design Develop Test
OPS
Issues

8. Mar-22
© 2013 IDC 8
Closing the Loop: Leverage Skills &
Tools for Agile, Iterative Approach
Design
Define
Develop
Support
Deploy &
Monitor
Test &
Tune
• Shared goals,
• Shared metrics,
• Shared tests,
tools and skills
• Shared virtual lab
• Quick iterations
can be disruptive
• Demand mgmnt
• Cloud adaptability
• Enabling agile
infrastructure access

9. Mar-22
© 2013 IDC 9
Three Key Challenges for IT
IT must deliver new applications that have greater business
value and higher quality, while managing costs …in the face of
these 3 key challenges:
 Increasing criticality of applications to the business
 Increasing complexity of software systems and applications
(from web to mobile to embedded… encompassing social
systems of engagement to feed systems of record, performance
demand with Big Data Analytics for business optimization)
 Increasingly distributed teams with multi-sourcing of applications
(from onshore to offshore to open source)

10. Mar-22
© 2013 IDC 10
Barriers to Prior Testing Approaches –
Cloud Testing Drivers
• Agile velocity demands immediate, frequent, iterative testing
• Lack of system resources constrains testing usage – expense
limits ability to mirror production configurations (mobile issues)
• User tendency to hoard physical systems for testing creates
bottleneck & expense – need for management & coordination
• Challenges to test system configuration & lengthy set-up
times limit usage and delays testing
• Lack of change management for testing configurations
creates pain points for consistent testing and re-testing
• Need for variable provisioning for complex software
deployment (including mobile) increasingly drives cloud
adoption for testing to cut costs and increase business agility

11. Mar-22
© 2013 IDC 11
Coordinating across the Life-Cycle
• Coordinating requirements, testing, and operational
performance is key across emerging technologies
• Slow response time costs prohibitive for core business areas
• Organizations should leverage quality automation through
requirements, unit test, system integration, pre-deployment &
application performance testing with emerging cloud platforms
• Evaluating test automation for synthetic transactions and real
user measurement and monitoring can help teams react and
manage user application experience
• As business requirements change, a cogent life-cycle
approach enables adaptive testing and monitoring responses
• Look to ALM SaaS alternatives initially as an on-ramp to
cloud where appropriate – strategize through to deployment

12. Mar-22
© 2013 IDC
Q. Your organization's adoption of cloud
services?
Source: IDC CloudTrack Survey, Winter 2012
n=493
% of
Respondents
Adoption Is Building by Cloud Type

13. Mar-22
© 2013 IDC
Source: IDC CloudTrack Survey, Winter 2012
n=493
Mean rating by
respondents
Q. How concerned are you about cloud...?
(1-5 scale; 5 = extremely concerned)
Performance One of Top Three Concerns:
Testing On & Of Cloud Can Help

14. Mar-22
© 2013 IDC
Cloud Computing
Public Cloud Plans: Cloud Testing Emerging
6.7%
9.0%
10.0%
9.7%
9.3%
9.3%
11.3%
12.0%
6.7%
7.3%
8.7%
9.3%
10.0%
11.0%
9.0%
12.0%
0% 20% 40% 60% 80% 100%
Use cloud for developing applications
Use cloud for testing applications
Use cloud for PaaS
Use cloud for application integration
Use cloud compute or storage resources
Use cloud database software
Use cloud for management & monitoring
Use cloud applications
No plans to Use Evaluating Now Beginning to Use
Using Now Will Use in 2012/13 DK
No Cloud Familiarity
14
What plans does your company have regarding public cloud services? (multiple
response)
IDC 2011 NA Cloud Survey, N=300
IT Developer and IT Management Respondents

15. Mar-22
© 2013 IDC
Cloud Computing
Perspective on Cloud Computing Capabilities
0% 20% 40% 60% 80% 100%
Lower overall cost
Fast application development
Lower admin cost
Makes IT far less complex
Applications that always scale up/down
Avoidance of up front costs
Fast application deployment
Standardized AD&D
Pay only for what you use
Immediate access to technology
Access to latest functionality
Completely Agree Mostly Agree Agree
Mostly Disagree Completely Disagree DK
No Cloud Familiarity
15
How much do you agree that cloud computing delivers the following capabilities?
(multiple response)
IDC 2011 NA Cloud Survey, N=300
IT Developer and IT Management Respondents

16. Mar-22
© 2013 IDC
Cloud Computing
Expected Benefits
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
DK or NS
Other
No perceived benefits
Access to best of breed tools
Access to more dev tool functionality
Easier AD&D of SOA applications
Standardized tools
Tool access to test applications
Tool access to deploy applications
Access to integrated tool suite
Access to most up to date tools
Tool access to build applications
More cost effect AD&D
16
What benefits does your organization expect to gain by using public cloud
services? (multiple response)
IDC 2011 NA Cloud Survey, Cloud Knowledgeable, N=277
IT Developer and IT Management Respondents

17. Mar-22
© 2013 IDC
0
20
40
Other
Test data and test bed preparations to
resemble live scenarios under high volume
Reusability of test artifacts to address rapid
change in application functionalities
Complexity of test scenarios to design and
execute
Ability to optimize usage of different platform
resources (e.g. CPU, memory, battery)
Designing to different form factors (e.g.
smartphone versus tablet)
Managing to robust security architecture
and protection of end user application data
Rapid pace of change in mobile OS /
hardware landscape
Expertise / developing skill sets across
multiple mobile OS platforms
Investing in and maintaining varied mobile
infrastructures
1.2%
6.5%
7.1%
13.2%
16.6%
26.5%
26.5%
32.3%
34.8%
35.4%
Total
N=325
Top Challenges in Launching
Mobile Applications
17
Share of Responses (%)
Highlights and Key Messages
Question: What are the biggest challenges you face
in launching mobile applications, whether or not you
are developing applications internally?
 Highlights
 Leading challenges concern:
 Level of investment and maintenance
required to support rapidly evolving mobile
IT infrastructure
 Building up or accessing resource
expertise in the design, development and
testing of mobile apps across multiple
mobile platforms
 Key Messages
 Enterprise business priorities to grow
revenue and innovate while improving
internal operational efficiencies will lead
many organizations to seek help from 3rd
parties in implementing mobile strategies
 It will be a challenge to keep up with the
rapid pace of change in mobile OSs while
addressing mobile security mandates,
designing to different form factors,
providing a smooth, productive customer
experience
Note: Multiple responses allowed

18. Mar-22
© 2013 IDC
-Develop near, mid, and long-term goals for transformation & cloud
• Understand what’s utilized or not, and what’s bought and developed at the line of business
• Base your plan on your organization’s business goals, governance/compliance profile, IT skillsets, and
data center investments
• Develop a “visualization” plan for a hybrid cloud strategy that takes you from your current state to your
end state, based on level of virtualization, focus on infrastructure vs. apps, etc
• Make sure your plan puts your priorities first – whether cost reduction, consolidation, expansion into new
competencies like mobile, analytics, data management, etc.
-Identify easy payback/low downside projects for public cloud services
• Develop/test/deploy and other compute and storage-intensive tasks
• Non-differentiating front-office functions first, the rest can follow -- Land then expand
• Investigate service management and brokerage tools for oversight into performance, service quality, usage
-Go at your own pace
• A majority of companies will always have on-premise gear, and “hybrid” cloud deployments
• Due diligence: compute and storage IaaS clouds are inexpensive, but mistakes are costly
• Be prepared to be more of a services broker, and deal with different types of outside
providers than you’re used to: get ready for change
Guidance for Cloud Testing Adoption
Take a Strategic View of Cloud Sourcing

19. Mar-22
© 2013 IDC 19
Summary
• IDC saw 37.7% revenue growth for cloud testing and ASQ SaaS in 2012 with five year
CAGR through 2017of 31.1 (to reach $1.01B); re-assess options as solutions expand
• ASQ and development teams must bridge the gap to operations to gain access to
appropriate cloud infrastructure for quality; help leverage and coordinate mobile quality
• IDC recommends an encompassing approach to business and IT management that
includes cloud and SaaS testing
 IT organizations should actively lobby the business and operations for access to
cloud testing in the short-term and for other ALM areas over time
 Executive management must plan for incorporation of cloud as part of the overall IT
portfolio to increase business adaptability and cut costs for multi modal software
Companies should view cloud testing and ASQ SaaS as part of a comprehensive
strategy from dev and provisioning through to build and operational planning.
This strategy should include mobile as a bridge from systems of engagement to
systems of record & operations.

20. Mar-22
© 2013 IDC
Melinda Ballou
Program Director,
Application Life-Cycle
& Executive Strategies
1. Business/IT alignment
2. IT-PPM & PPM strategies
3. IT staffing strategies
4. Business strategy
5. IT portfolio mgmt.
Contributing Analysts
18
Melinda Ballou
Program Director, Application Life-Cycle
Mgmt & Executive Strategies
mballou@idc.com

21. 21
© 2013 SOASTA. All rights reserved.
Brad Johnson
@bradjohnsonsv

22. 22
© 2013 SOASTA. All rights reserved.
o Time To Market
6 week delivery plan @ Google
http://www.thechromesource.com/u
nderstanding-chromes-six-week-
release-cycle/
Rapid updates and adoption

23. 23
© 2013 SOASTA. All rights reserved.
o Time to market
o Global access and device options

24. 24
© 2013 SOASTA. All rights reserved.
o Time to market
o Global access and device options
o End user reach and quality expectations

25. 25
© 2013 SOASTA. All rights reserved.
Speed
Global access and device options
• 800,000 servers – 54 locations – 17 countries
Affordability
Time to market
• Test Fast
End user reach and quality expectations
• Frequent, iterative, realistic for high coverage
Scale

26. 26
© 2013 SOASTA. All rights reserved.
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information

27. 27
© 2013 SOASTA. All rights reserved.
http://www.soasta.com/customers/case-studies/

28. 28
© 2013 SOASTA. All rights reserved.
o “a leader in Cloud Testing and ASQ
SaaS”
o “leading innovator in the cloud
testing arena”
o “Overall, SOASTA is well positioned
as a best of breed vendor in this
arena.”
o “Differentiators for SOASTA are its
strength in cloud-based testing and
evolving mobile support and a free
CloudTest Lite version” SOURCE: IDC MarketScape: Worldwide
Cloud Testing and ASQ SaaS 2012–
2013 Vendor Analysis — Enabling
Business Agility and Quality in the Cloud,
by Melinda-Carol Ballou, February 2013,
IDC #237597,
http://www.idc.com/getdoc.jsp?containerI
d=237597
IDC MarketScape vendor analysis model
is designed to provide an overview of the
competitive fitness of ICT suppliers in a
given market. The research
methodology utilizes a rigorous scoring
methodology based on both qualitative
and quantitative criteria that results in a
single graphical illustration of each
vendor’s position within a given market.
The Capabilities score measures vendor
product, go-to-market and business
execution in the short-term. The Strategy
score measures alignment of vendor
strategies with customer requirements in
a 3-5-year timeframe. Vendor market
share is represented by the size of the
circles. Vendor year-over-year growth
rate relative to the given market is
indicated by a plus, neutral or minus next
to the vendor name.

29. 29
© 2013 SOASTA. All rights reserved.
Mike Ostenberg

30. 30
© 2013 SOASTA. All rights reserved.
TouchTest
Continuous
Automated
Testing
of Mobile
Applications
CloudTest
Load &
Performance
mPulse
Real User
Monitoring

31. 31
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
CloudTest
Architecture

32. 32
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
Customers
CloudTest
Architecture

33. 33
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
Customers
CloudTest
Architecture

34. 34
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
CloudTest
Architecture
Analytics
CloudTest
Databas
e
Main
“Conductor”
“Conductor”

35. 35
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
CloudTest
Architecture
Analytics
CloudTest
Databas
e
Main
“Conductor”
“Conductor”
Perf Tester 1

36. 36
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
CloudTest
Architecture
Analytics
CloudTest
Databas
e
Main
“Conductor”
“Conductor”
Perf Tester 1

37. 37
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
CloudTest
Architecture
Analytics
CloudTest
Databas
e
Main
“Conductor”
“Conductor”
Perf Tester 1

38. 38
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
CloudTest
Architecture
Analytics
CloudTest
Databas
e
Main

39. 39
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
CloudTest
Architecture
Analytics
CloudTest
Databas
e
Main
Perf Tester 2
Perf Tester 2

40. 40
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
CloudTest
Architecture
Analytics
CloudTest
Databas
e
Main
Perf Tester 2

41. 41
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
CloudTest
Architecture
Analytics
CloudTest
Databas
e
Main
Perf Tester 2
Analytics
Load
Generator
Load
Generator
Load
Generator
IBM

42. 42
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
CloudTest
Architecture
Analytics
CloudTest
Databas
e
Main
Perf Tester 2
Analytics
Load
Generator
Load
Generator
Load
Generator
IBM
Analytics
Load
Generator
Load
Generator
Load
Generator
RackSpace

43. 43
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
CloudTest
Architecture
Analytics
CloudTest
Databas
e
Main
Perf Tester 2
Analytics
Load
Generator
Load
Generator
Load
Generator
IBM
Analytics
Load
Generator
Load
Generator
Load
Generator
RackSpace
Analytics
Load
Generator
Load
Generator
Load
Generator
AWS East

44. 44
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
Perf Tester 2
Analytics
Load
Generator
Load
Generator
Load
Generator
AWS East
Analytics
Load
Generator
Load
Generator
Load
Generator
IBM
Analytics
Load
Generator
Load
Generator
Load
Generator
RackSpace
Analytics
CloudTest
Databas
e
Main
Massively Scalable
Cost Effective Real
World Testing
CloudTest
Architecture

45. 45
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
Perf Tester 2
Analytics
Load
Generator
Load
Generator
Load
Generator
AWS East
Analytics
Load
Generator
Load
Generator
Load
Generator
IBM
Analytics
Load
Generator
Load
Generator
Load
Generator
RackSpace
Analytics
CloudTest
Databas
e
Main
CloudTest
Architecture
“Conductor”

46. 46
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
Perf Tester 2
Analytics
Load
Generator
Load
Generator
Load
Generator
AWS East
Analytics
Load
Generator
Load
Generator
Load
Generator
IBM
Analytics
Load
Generator
Load
Generator
Load
Generator
RackSpace
Analytics
CloudTest
Databas
e
Main
CloudTest
Architecture
“Conductor”

47. 47
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
Perf Tester 2
Analytics
Load
Generator
Load
Generator
Load
Generator
AWS East
Analytics
Load
Generator
Load
Generator
Load
Generator
IBM
Analytics
Load
Generator
Load
Generator
Load
Generator
RackSpace
CPU, Disk,
Memory
Analytics
CloudTest
Databas
e
Main
CloudTest
Architecture
“Conductor”

48. 48
© 2013 SOASTA. All rights reserved.
SUT/AUT
Web
Server
Web
Server
App Server
Web
Server
Web
Server
App Server
App Server
App Server
Load
Balancer
Cache
Cache
Databas
e
Perf Tester 2
Analytics
Load
Generator
Load
Generator
Load
Generator
AWS East
Analytics
Load
Generator
Load
Generator
Load
Generator
IBM
Analytics
Load
Generator
Load
Generator
Load
Generator
RackSpace
CPU, Disk,
Memory
Analytics
CloudTest
Databas
e
Main
Massively Scalable
Cost Effective Real
World Testing
CloudTest
Architecture
“Conductor”

49. 49
© 2013 SOASTA. All rights reserved.
CloudTest
o Near-immediate near-infinite load capability
o Enables distributed real-time analytics
o Real-world load distribution
o Real-world infrastructure
o Cost efficiency:
• No Cap Ex
• Minimal Op-Ex

50. 50
© 2013 SOASTA. All rights reserved.
Demonstration

51. 51
© 2013 SOASTA. All rights reserved.
Thanks
Contact SOASTA:
www.soasta.com/cloudtest/
info@soasta.com
866.344.8766
Follow us:
twitter.com/cloudtest
facebook.com/cloudtest
Knowledge Center
• White Papers
• Webinar Recordings
• Case Studies
RESOURCES
www.SOASTA.com
CloudLink Community
• Support
• Tutorials
• Video