Weitere ähnliche Inhalte
Ähnlich wie How to Solve Problems of Testing in Agile Environments (20)
Mehr von CA Technologies (20)
Kürzlich hochgeladen (20)
How to Solve Problems of Testing in Agile Environments
- 3. 3 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Abstract
Companies adopting an agile framework for software development know that one area still
contributing to slowdowns is testing. In all its forms—regression, system, performance, etc.—
testing is not optimized in the software development lifecycle. The importance of test data
management—having the right data delivered to the right place, at the right time—for testing
purposes is often overlooked.
Poor test case design and the inefficient provisioning of poor quality data mean that test teams
find themselves unable to keep up with the rate of change while fully testing systems. Quality is
then compromised in favour of delivering software on time and within budget. Organizations
that want to implement continuous delivery must rethink their testing and development
processes rather than simply reshuffle them, and reconsider their approach to test data
management and test case design. By using an end-to-end approach to testing, driven by
requirements, organizations can do shift-left testing, mitigate risk and minimize defect creation,
thereby delivering quality software faster, and for less cost.
Huw Price
CA Technologies
Vice President of
Application Delivery
CA Technologies
Director of Digital
Assurance
Jonathon
Wright
- 8. 8 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
78% of Executives in most B2C industries say their business will be disrupted by digital
technologies in the next 12 months. 20% say their business will be massively disrupted.
- Forrester State of Digital Business - Big Bang Disruptions - Larry Downes - RallyON Keynote
- 23. 23 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
COGNITIVE ADAPTIVE ADOPTION
Manual
Continuous
Autonomics
Lean
Engineering
Connected
Intelligence
BUSINESS VALUE
LEVEL 1 LEVEL 2 LEVEL 3 LEVEL 4 LEVEL 5
?
Optimized
Organization
for DesignOps
Continuous Insight
& Learning
Value Stream
Mapping
Consistency &
Collaboration
Heroes &
Heavy Lifting
DIGITAL MATURITY
§ Insight-Driven (Predictive / Prescriptive)
§ NoOps organized (IoT-Ops/CX-Ops/Sec-Ops)
§ Self-healing end-to-end autonomic orchestration
§ Microcontainerization & Microservices enabled
§ Open Innovation & Connected Intelligence
§ Quantum Teleportation (Shift X)
Where are you?
Minimum Viable Quality
§ Silos, manual handovers, waterfall process
§ One release/year
§ Monolithic apps
§ Long term project/resource planning
§ Error prone dev/test/release processes
Minimum Viable Product
Minimum Viable Delivery
Organization
Change
Methods
and Skills
Insight
Beliefs,
Values,
Culture
Value Behaviors Heuristics
- 26. 26 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Design QA/TESTDEV PRODUCTIONPRE-PROD
Specification
CI/Build Functional
testing UAT Integration
testing
Performance
engineering
Deploy to
pre-prod
Code
commit SCM
Design Spec
Requirements
Business
Users
TEST
Data
TEST
Stub
TEST
Data
User BA
Product
Manager
Customer
Users
Customer
Experience
Customer
Experience
Customer
Experience
Customer
Experience
Design
Business
Value
Business
Value
Business
Value
Improvement
Innovation
Intelligence
Assessment
Insight
Learning
Maintenance
Operations
Delivery
Testing
Support
Customer
Digital Assurance
Value Insight
Learn Learn
Monitor Monitor TEST
Stub
REAL
User
s
TEST
Data
REAL
Data
TEST
Stub
USER
Case
Business
idea
Value-Driven Delivery
- 36. 36 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Modeling Maturity
AMBIVALENT
LEVEL 1
INITIALIZING
LEVEL 2
AWARE
LEVEL 3
MINDFUL
LEVEL 4
EXPERT
LEVEL 5
§ Test data is copied
from production
§ Extensive use of End
to End testing
§ Test data is created
manually by testers
§ PII in test data,
risking breach of
security
Challenges of Level 1 stay
with following
improvements:
§ Some databases are
subsetted for testing
purposed to speed
testing and reduce
storage costs
§ PII is masked in SOME
test databases
§ PII removed from test
data bases
Challenges of Level 2
reduce with following
definitive steps:
§ Test data is created
using
a tool
§ PII removed from test
data bases subsets
§ Test data is explicitly
drawn out in test plans
§ TDM is recognized
as a skill
Level 3 processes get
matured:
§ Test data can be
reserved
§ Test Data coverage is
understood and known
§ The right tests are run,
resulting in complete
testing
§ Test Data on Demand is
widespread.
§ Centralized TDM team
Everything in Level 4 +
§ Enterprise level
Modeling policy and
strategy fully adopted
by the organization
§ Common Enterprise
Model Repository is
leveraged used by all
teams
§ Models are the single
source of truth for
Biz/Dev/Test/Ops
§ Continuous
Development
Framework in place
Most IT
Organizations
- 37. 37 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
> COGNITIVE ADAPTIVE <
RUNDEPLOYTESTBUILDPLAN
INTAKE
Requirements
User Stories
Release Plan
MODEL
Import User Stories to
Automatically Create,
Visualize and Optimize Tests
Initiate Functional, Non-
Functional and Security as
Code
FUNCTIONAL
§ Subset/Mask Test Data
§ Create/Reserve Test Data
§ Test Automation Library
§ Ensure Mobile Experience
INSIGHT-DRIVEN
CONFIG/DEPLOY
§ Provision Entire Stack
§ Confirm Configurations
§ Approve Changes
§ Successfully Deploy
§ Internal or External Cloud
INTEGRATION
Remove Constraints with
Virtual Services
§ Mobile, Web, App Server,
Middleware, Backend, MF
§ 3rd Party Systems / API’s
CODE
Develop and Commit Code,
Scan Code, Version Control,
Continuous Integration.
Complete Build and Initiate
Release
AUTOMATION AUTOMATION AUTOMATION
ü Speed/Time-to-Market
ü Quality/Availability
AUTOMATION
ü Cost/Financial Mix
ü Risk/Compliance
ü Requirements
ü Environments
ü Data
ü Automation
Customer
Experience
NON-FUNCTIONAL
§ Performance Engineering
§ Simulate Backend Load
§ Test Outlier Conditions
§ Ensure Mobile Experience
§ Security / Penetration
Common Goals “Big Picture” Issues
DESIGN SHIFT-LEFT SHIFT-RIGHT OPS
Node.Probe
Node.Test
TDD
BDD
ATDD
BPMN
Nimbus
Visio
Node.Data
Node.Learn
MEASURE/FEEDBACK
§ Customer Experience
§ Business Service View
§ Application View
§ Infrastructure View
§ Dynamic Capacity
§ Feedback loop