Series Name: Continuous Delivery Webcast Title: 2013 - Continuous Delivery - Testing for Agile Through Continuous Delivery Date: June 11, 2013 Presenter: Chandra Venkatapathy

1. Continuous Delivery – Agility with Continuous
Testing
Chandra Venkatapathy
Sr Product Manager, DevOps
Chandrav@us.ibm.com
© 2013 IBM Corporation

2. Software innovation is crucial to competitiveness

3. 3
Market shifts are fundamentally changing the way
businesses approach software driven innovation
Macro Business Environment
Increasing regulatory environments drive
the need to reduce risk
Empowered Users
Consumerization of IT drives
the need for apps with high
quality customer experience
Technology Trends
Mobile, social, big data, cloud,
intelligent/connected systems
drive the need for agility
Mobile
Cloud
Intelligent/
Connected Systems
Social
Big Data
Clients are trying to rapidly innovate and become agile with systems of engagement
At the same time need to evolve their systems of records and manage regulatory compliance,
reduce costs by outsourcing and provide world class customer experience
Multi-sourcing
Increasing outsourcing drive
the need for governance
3

4. Enterprises are using Systems of Interactions to drive
business innovation…
Systems of Interaction
Continuous
client experience
Partner value
chain
Cloud-based
Services
Systems of Engagement Systems of Record
CRM HR
DB ERP
Maintain stability and
compliance of core
business systems
Provide differentiating services
and customer experience to
grow market share and
revenue
Rapidly improve and evolve
services based on customer
feedback and success metrics

5. … however, they are struggling to do so
Systems of Interaction
Continuous
client experience
Partner value
chain
Cloud-based
Services
Systems of Engagement Systems of Record
CRM HR
DB ERP
IT Operations:
Pressure for rapid app
release impacts overall
system stability
Line-of-business:
Limited capacity to
innovate on new
products and services
Development/Test:
Speed mismatches between faster moving
front-end and slower moving back-end
systems
applications rolled
back due to quality
issues escaping
into production
51%
experience delays
in integration,
configuration and
testing of applications
41%
Of resources devoted
to maintaining
existing systems and
products
80%
to deliver application
changes to
customers
4-6 Weeks

6. 6
Accelerate Software Delivery
Expanding collaboration to include customers, LOB and others to eliminate organization silos
Balance speed, cost, quality and risk
Automating manual processes across delivery lifecycle to eliminate waste/delays and compliance tracking
Reduce time to customer feedback
Enabling a customer feedback loop for continuous improvement
DevOps
Enterprise capability for continuous software delivery that enables clients
to seize market opportunities and reduce time to customer feedback
DevOps Lifecycle
Operations/ProductionDevelopment/TestCustomers Business Owners
Continuous Innovation, Feedback and Improvements
Monitor and Optimize
Plan and Measure Develop and Test Release and Deploy

7. DevOps Principles and Values
 Develop and test against a production-like
environments
 Iterative and frequent deployments using repeatable
and reliable processes
 Continuously monitor and validate operational quality
characteristics
 Amplify feedback loops
People
Process
Tools
People
Process
Tools

8. Agile in the real world
 Geographically distributed teams
 Independent test teams
– Centers of Excellence
– Economy of scale
 Complex, heterogeneous systems
 Mini-waterfall inside the sprint
– Plan
– Develop
– Test

9. Release Management and Risk
9
B u ildS e r v ic e
A
B
C
D
E
F u n c T e s t S t a g eS y s T e s t P r o d
Large Batch Sizes
• Services: Many
• Dependencies: Many
• Changes: Many
• Complexity: Large
• Impact of Failure: Large
• Failure Analysis: Difficult
Big Bang

10. Patterns of challenges
Differences in dev and
ops environments
cause failures
Backlog of agile
releases that Ops
cannot handle
Manual (tribal)
processes for release
lack
repeatability/speed
Lack of feedback and
quality metric leads to
missed service level
targets
Daily
Build
Monthly
Delivery
Who did
this last
time?
Dave…
Dave’s not
here
man…
Dev
Prod

11. EnvironmentEnvironment
Solution: Virtualize, Standardize and automate the deployment
Output of a process, such,
as Dev Build
Configurations specific to
Deployment definitions, SLAs
ApplicationApplication ConfigurationConfiguration
Physical or Virtual Infrastructure
IaaS
PaaS
ChangeFrequency
Low
High
H/W
OS
OS/Config
Middleware
App
Middeware
Config
App Config

12. Standardize: infrastructure deployment with environment patterns
Communicate and share environment patterns
Issues
– Dev environments different from production
environments
– Difficult to provision test environments
Solution
– Define standard Virtual System Patterns for
environments
– Share patterns between Dev and Ops to improve
communications
– Repeatable and reliable process to provision
base environments (rubber stamps)
Functional
Verification
Test
Environment
Functional
Verification
Test
Environment
User
Acceptance
Test
Environment
User
Acceptance
Test
Environment
Products:
IBM PureSystem
IBM SmartCloud Orchestrator
IBM Workload Deployer

13. Standardize: Content Delivery for the environment
Application Release, and Configuration
Issues
– Difficult to keep track of content, versions and
environments
– Keeping up with speed of development
– Errors in deployment
Solution
– Automate deployments of applications and
configuration across multiple environments
– Repeatable and reliable process to deliver content
linked to dev, test, and production processes
(rubber stamps)
– Release Approvals
Versioned
Artifacts
QA
Environment
Production
Environment
Development
Environment
Artifacts Artifacts Artifacts
Execute Against Execute Against Execute Against
QA
Deployment
Product
Deployment
Development
Deployment
Application Model
Products:
Rational Automation Framework

14. Problem: Dependencies slow down deployment of
production-like environments
14
Week 4Service
A
B
C
D
E
StageSys Test ProdWeek 8
• Service A dependent on Service B: Having different delivery schedules
• Service D dependent on 3rd
party applications; available in system test
ApplicationApplication

15. Virtualize: Test dependencies and test services
 Virtual Services created from
Service specifications or from recording actual
traffic to existing services/applications
 Virtual Services further customized
To simulate simple to complex behaviors (e.g.
negative testing)
To simulate latency, performance profiles, etc.
 Virtual Services published for consumption by
developers, testers
Testing can start earlier: Testers can now create
their tests against virtual services
 Systems can be incrementally tested as sub-systems
become available
15
Virtual
Virtual
Issues
– Test services / System dependencies not
available on-time; or costly to provision and
manage
Solution : Test Virtualization
Product: Rational Test Virtualization

16. Incremental Integration TestingIncremental Integration Testing
Actual Component
Stubbed Component
Units are introduced into the continuous
integration cycle in a prioritised, controlled
fashion.
Units not yet built can be simulated and tested
against.
Incremental Integration Testing
16

17. Integration Testing lags a
step behind the code
Problem: Agile Delivery Backlog
Tester
Setup integration test
environment
Developer
Write & deliver
code
Nightly build(s)
Compile, unit test, publish
Developer
Developer
Developer
Developer
Developer
N days to install & configure
Iteration 1 Design, Code,
Test
Weeks 2 - 6
Integration test
iteration #1
Iteration
planning
Week 1
Iteration 2 Design, Code,
Test
Weeks 8 -12
Integration test
iteration #2
SVT
prep
Iteration
planning
Week 7
N nightly builds are piling up

18. Solution: Continuous Integration testing for Builds (with self-service)
Nightly build is not published
until integration tests run.
Nightly build is not published
until integration tests run.
Compile Unit Test
Install &
Configure
Integration
Test
Environment
Publish
ResultsRun Tests
Rational Quality
Manager
•Smoke Tests
•Sandboxes
•Self-provisioning
•Smoke Tests
•Sandboxes
•Self-provisioning

19. Testing continuously
19
Small Batch Size
• Services: Few
• Dependencies: Few
• Changes: Few
• Complexity: Small
• Impact of Failure: Small
• Failure Analysis: Simple
A
B
C

20. Continuous Delivery Pipeline
Development Build
Verification
Environments
System Integration Test
Environments
-Iterative Testing,
Performance Testing
User Acceptance
Test Environments
Production
Environment
Consistent Delivery of Environments, Content, and Deliverable

21. DevOps Tool Chain
Application Release Management
Cloud Provisioning
Application Deployment Automation
Rational Collaborative Lifecycle
Management Solution
Rational Test Workbench
Rational Test Virtualization Server SmartCloud Control Desk
Jenkins
Rational Build Forge
SmartCloud Orchestrator
IBM Pure Application
System
SmartCloud Application
Performance Management
uBuild
21

22. Continuous testing with virtualized services
Automating deployment and testing to production-like environments
Databases Mainframe
applications
Third-party
Services
Rational Test Virtualization Server
Application
changes
being tested
virtualized services
DevDev QAQA
Deploy
Rational
Team
Concert
Jenkins
Build
Deploy
Rational Test Workbench
Test
Configure
UrbanCode UDeploy

23. Enabling the DevOps lifecycle
23
DevOps Foundation
Open Lifecycle and Service Management Integration Platform
DevOps Lifecycle
Operations/ProductionDevelopment/TestCustomers Business Owners
Continuous Innovation, Feedback and Improvements
Ecosystem
BestPractices
Monitor and Optimize
Plan and Measure Develop and Test Release and Deploy
OSLC

24. Call-to-Action
 Identify the customer pain for Quality
– Patterns of Challenges
– Cost
– Time
 Show how IBM Solves with DevOps / Continuous Delivery for Testing
– Issues, Solution, Capabilities
– IBM UrbanCode, Rational Test Virtualization
 Get Acquainted with IBM UrbanCode, and Rational Test Virtualization

25. Sales Resources
This site is updated weekly with new collateral
BOOKMARK THIS PAGE
https://w3-03.sso.ibm.com/software/xl/portal/content?synKey=O227209M89128T39

26. 26
Contacts for UrbanCode
 Lotus Notes Help ID: AskUrbanCode/Dallas/IBM
 Sellers Resource: http://w3.ibm.com/software/xl/portal/content?synKey=O227209M89128T39
 WW Sales Integration Executive: Ann Marie Somerville: amsomerville@us.ibm.com
 Transition Executive: Peter Spung: paspung@us.ibm.com
 Rational Product Management: Albert (Wing Hong) Ho: winghong@us.ibm.com
 More information on UrbanCode: http://www.youtube.com/user/UrbanCodeSoft
26

27. 27

28. 28
© Copyright IBM Corporation 2013. All rights reserved. The information
contained in these materials is provided for informational purposes only, and is
provided AS IS without warranty of any kind, express or implied. IBM shall not be
responsible for any damages arising out of the use of, or otherwise related to,
these materials. Nothing contained in these materials is intended to, nor shall
have the effect of, creating any warranties or representations from IBM or its
suppliers or licensors, or altering the terms and conditions of the applicable license
agreement governing the use of IBM software. References in these materials to
IBM products, programs, or services do not imply that they will be available in all
countries in which IBM operates. Product release dates and/or capabilities
referenced in these materials may change at any time at IBM’s sole discretion
based on market opportunities or other factors, and are not intended to be a
commitment to future product or feature availability in any way. IBM, the IBM logo,
Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products
and services are trademarks of the International Business Machines Corporation,
in the United States, other countries or both. Other company, product, or service
names may be trademarks or service marks of others.