As organizations continue to compress development and delivery lifecycles, the risk of regressions, integration errors, and other defects rises. But how can development teams integrate defect prevention strategies into their release cycles to ensure that they're not continuously delivering faulty software? In this presentation, Parasoft & Perforce discuss the key development testing processes to add to your Continuous Delivery system to reduce the risk of automating the release of software defects.

1. Parasoft Proprietary and Confidential 1
2015-03-21
How to Avoid Continuously
Delivering Faulty Software
Mark Lambert, Parasoft
Jonathan Thorpe, Perforce Software

2. Parasoft Proprietary and Confidential 2Parasoft Proprietary and Confidential 2
Today’s speakers
Jonathan Thorpe
Technical Marketing Manager
Perforce Software
Mark Lambert
VP Products and Support
Parasoft

3. Parasoft Proprietary and Confidential 3Parasoft Proprietary and Confidential 3
Drivers of the SDLC
SDLC Speed is the
difference between a
first mover and a
follower
Damages associated
with software failure
are increasing and
very real
Brand equity is critical
and Quality drives
brand loyalty

4. Parasoft Proprietary and Confidential 4Parasoft Proprietary and Confidential 4
SDLC - The Era of Acceleration
 Constant Trade-offs that have business impact
Time
Quality
Scope

5. Parasoft Proprietary and Confidential 5Parasoft Proprietary and Confidential 5
From Automated to Continuous
Continuous testing accelerates the SDLC by
managing quality expectations and actionable tasks
Requirements
Defined
Policy Management
Development
Defect Prevention
Development
Development Testing
Static Analysis
Unit/Component
Peer Review
Automated Tests
Integration Testing
API/Service Tests
Smoke Test
Security Tests
Automated Tests
System Testing
Functional Tests
Scenario Tests
Performance Tests
CI
Build
Defect Remediation Tasks
Go
Release Path
No Go
Business
Decision
Service Virtualization – Test Environment Access

6. Parasoft Proprietary and Confidential 6Parasoft Proprietary and Confidential 6
Best Practices for continuous delivery of
quality software
 Version everything
 Source code, deployment scripts, artifacts, multimedia,
test results etc.)
 Automate as much as possible
 Code reviews as part of workflow
 Including deployment scripts, automated tests, runbooks
 Keep automated build and test execution times short
(minutes not hours)

7. Parasoft Proprietary and Confidential 7Parasoft Proprietary and Confidential 7
Version Everything
 All artifacts should be in version control
 Any size
 Any type
 Version control systems should support how your
organization wants to work
 Trunk based development
 Streams
 Branches

8. Parasoft Proprietary and Confidential 8Parasoft Proprietary and Confidential 8
Architect For Performance
Architect for performance
without adding complexity
Local network performance
even over WAN
Control how much data is
replicated and how often

9. Parasoft Proprietary and Confidential 9Parasoft Proprietary and Confidential 9
Commit High Quality Code
 Limit negative impact on
other developers
 …but still need to commit
frequently
 …without adding complexity
to developer workflows

10. Parasoft Proprietary and Confidential 10Parasoft Proprietary and Confidential 10
Pre-commit Review and Test
 Integrates with builds, code analysis, etc.
 Summary shows up in Swarm; click for detail
 Reviewers are given a more complete picture
Code Check-in or
Pre-commit
Code Review with
Available Results
main
Build
Code
Analysis A
Code
Analysis B
Code
Analysis C
Test A Test B
Detailed Drilldown

11. Parasoft Proprietary and Confidential 11Parasoft Proprietary and Confidential 11
Static Code Analysis
Pattern-Based
Static Analysis
Prevention technique
Analyzes code
structure (parse tree)
to apply best practices
Flow-Based
Static Analysis
Detection technique
Analyzes code flow to
determine “dangerous
paths”
Metric Threshold
Analysis
Advisory technique
Finds complex/hard-to-
test code prone to
errors

12. Parasoft Proprietary and Confidential 12Parasoft Proprietary and Confidential 12
Static Code Analysis
 Well understood often under valued
 Define the goal of the analysis and the Policy for compliance
 Focus on reduction of business risk not pursuit of perfection
 Start small to promote adoption and monitor for areas of
improvement

13. Parasoft Proprietary and Confidential 13Parasoft Proprietary and Confidential 13
Unit vs. Functional Testing
 Unit Testing
 Developer focuses on the code
 Typically not true Unit Test
 Code needs to be built to be testable
 Where is the ROI?
 Did we design it properly
 How much is enough?
 Code Coverage + Peer Review

14. Parasoft Proprietary and Confidential 14Parasoft Proprietary and Confidential 14
Unit vs. Functional Testing
 Functional Testing
 QA focused on the user-story/function
 Where is the ROI?
 Does it function correctly
 Did we break functionality
 How much is enough?
 User-story coverage
 Assoc. code coverage provides additional insight

15. Parasoft Proprietary and Confidential 15Parasoft Proprietary and Confidential 15
Explorative Testing
 Ad-hock/Unstructured
Testing of functional areas
 Important part of
QA/feedback process
 Requires traceability to
user-stories and code
 Should be ‘reinforced’ with
automated tests

16. Parasoft Proprietary and Confidential 16Parasoft Proprietary and Confidential 16
Performance/Security Testing
 Limitations
 Often at the end of the cycle
 Wait until the whole system is ready
 Requires specialized skills and specialized tools
 Often not “real tests”
 Too late for cost effective remediation
 “Shift Left” Performance and Security
 Reuse automated functional tests and tooling
 Eliminated the system constraints … Service Virtualization …

17. Parasoft Proprietary and Confidential 17Parasoft Proprietary and Confidential 17
Service Virtualization
 Complexity is a Barrier to Innovation
 Accessible
 Stable
 Controllable
 Constrained Testing
3rd Party System
Evolving Component
Mainframe
Scheduled Access

18. Parasoft Proprietary and Confidential 18Parasoft Proprietary and Confidential 18
Service Virtualization
 Emulates dependencies for the Test
Environment
 Reduces the complexity for early stage testing
 Increases predictability
 Enables “Test Anytime, Anywhere, Anyway”
 Automated Provisioning for different use-cases
 Automated Test Data Management/Simulation
 Does not eliminate the need for
System/Integration Testing

19. Parasoft Proprietary and Confidential 19Parasoft Proprietary and Confidential 19
Continuous Test Characteristics
 Logically componentized
 Correlated with business
requirements
 Incremental, Repeatable
 Versioned and maintained
 Process is prescriptive
based on results
Continuous
Testing
Policy
Traceability
Analysis Risk
Assessment
Environment
Access
Optimization

20. Parasoft Proprietary and Confidential 20Parasoft Proprietary and Confidential 20
Answer the questions …
• “Can we release?”
• “What is the risk?”
Dev Manager
• “Where is my application weak?”
Architect
• “How do I automate the process?”
DevOps
• “What needs to be fixed?”
• “Did I fix it?”
• “What did I test?”
• “What still needs to be tested?”
Developers / Testers

21. Parasoft Proprietary and Confidential 21Parasoft Proprietary and Confidential 21
Development Testing Platform
 Centralize and Automated
“Quality Hub”
 Provide Controls and visibility
onto variable and ad-hoc usage
of quality tools (incl. open
source)
 Enables centralized policy to
drive consistent results of the
SDLC practices
DTP
Source
Control
Defects
Require-
ments
Code
Review
Static
Analysis
Metrics
Flow
Analysis
Unit
Testing
Coverage
Functional
Testing
Load
Testing

22. Parasoft Proprietary and Confidential 22Parasoft Proprietary and Confidential 22
Provide Clarity on Risk
 Bridge the gap between technical findings and business impact
 Real-time feedback on compliance and certification with industry,
regulatory or standards initiatives during active development.

23. Parasoft Proprietary and Confidential 23Parasoft Proprietary and Confidential 23
Dashboard
and Reporting
Post Analysis
Analysis (PIE)
Prioritization
of Findings
Download to
IDE for
Remediation
Desktop
Execution of
Test/Analysis
Check-in
Source Code
Cont. Test in CI
Infrastructure
Workflow drives improvement
DTP

24. Parasoft Proprietary and Confidential 24Parasoft Proprietary and Confidential 24
Avoid Continuously Delivering Faulty Software
1. Define Business Expectations in a Policy
2. Version everything and be pragmatic
3. Automate Key Software Quality Practices
1. Code Analysis
2. Peer Review
3. Automated Testing with Traceability
4. Apply Continuously and with a Workflow for remediation
5. Translate to Business Impact and Monitor for improvements

25. Parasoft Proprietary and Confidential 25
2015-03-21
Thank you
Questions?