AI and ML can be utilized to improve test management and quality, and the impact of changes from design into production. Learn about the various stages of software development life cycle from planning and design, through coding and testing, and shows how AI and ML can benefit these stages from within a test management system.

2. © 2020 Perforce Software, Inc.
#devopsnext-continuous-testing
LIVE SLACK Q&A

3. © 2020 Perforce Software, Inc.
Leveraging AI and ML in Test
Management Systems.
N I C O K R Ü G E R

4. What is Test
Management?

5. 5 | DevOps Next 2020 perforce.com
Software Development Lifecycle

6. 6 | DevOps Next 2020 perforce.com
Test Management, Planning and Execution…

7. Planning Process

8. 8 | DevOps Next 2020 perforce.com
• What are we building… exactly?
• Identify the risks, early.
• Design quality into your product.
• How do you verify and validate?
• Test planning starts here!
• Cover high risk areas.
Planning Process…

9. 9 | DevOps Next 2020 perforce.com
• Automate requirements / design
reviews.
• Reduce poor requirements that lead
to poor quality!
• Automate test coverage.
• Apply best practices to your design.
• Scan for security vulnerabilities in
your design.
How AI/ML can help…
“IBM Watson AI uses natural language processing and understanding to analyze a
requirement’s text, suggesting improvements that leverage industry best practices for
writing high quality requirements, based on the INCOSE Guidelines for Writing Good
Requirements.”

10. 10 | DevOps Next 2020 perforce.com
• Do I have the right tests?
• What are the best practices that I must apply?
• Are we testing based on what was designed?
• How do I know if something changes?
• Which tests to run first?
Test Planning and Design

11. 11 | DevOps Next 2020 perforce.com
• Auto generate tests based on
Requirements / Epics / Stories.
• Automatically track the impact of
any change on the system.
• Flag tests that are affected.
• Create bug once test fails without
human intervention.
How AI/ML can help…
Traceability matrix created without any manual steps. Auto
generated by the Helix ALM tool.

12. Test Execution

13. 13 | DevOps Next 2020 perforce.com
• Which tests to run first?
• High risk code identification?
• What platforms do I need to
cover?
• Faster feedback is critical.
Test Execution…

14. 14 | DevOps Next 2020 perforce.com
• Identify the tests to run and in
what order.
• Identify high risk areas.
• Identify usage and suggest
platform coverage.
• Analyze code to suggest
improvements.
How AI/ML can help…
Here is an example of ML applied to my code repository that
identifies high risk code based on the number of changes, bug fixes
applied to the individual files over time.

15. Impact of
Change

16. 16 | DevOps Next 2020 perforce.com
• What is the impact of a change?
• How do we know what needs to
be tested?
• Can I see the impact before I
make a change?
• Which tests should run first?
Understanding the Impact of Change…
Test Impact Analysis (TIA) is a technique that helps determine
which subset of tests for a given set of changes.
*Martin Fowler

17. 17 | DevOps Next 2020 perforce.com
• Before a change is made, indicate
the impact.
• Once a change is made, flag tests
and run them.
• Reduce the risk of change.
• Maintain links/relationships
between objects.
How AI/ML can help…
Helix ALM + SurroundSCM, Impact Analysis with change tracking. Show
the impact of a proposed change and flags child items as suspect when a
change is made.

18. 18 | DevOps Next 2020 perforce.com
• End to end Quality Audit.
• Code Commit, Code Review by
AI.
• Understand the code intent.
How AI/ML can help…
“Automated AI Code Review: Our bot reviews your every code commit
and will immediately let you know of critical vulnerabilities and suggest
how to fix them.” – Deepcode.ai

19. 19 | DevOps Next 2020 perforce.com
Increase Quality with DeepCode Learning

20. Wrap Up

21. 21 | DevOps Next 2020 perforce.com
AI/ML in the Software Development Lifecycle
Increase Product Quality Respond Fast
Reduce manual workUncover issues before
they happen

22. 22 | DevOps Next 2020 perforce.com
Classification of Advanced
AI & ML Testing Tools
COMING UP NEXT…
TRACK
Testing Tools
The Rise and Benefits of Robotic
Process Automation (RPA)
TRACK
Continuous Testing
Moving to Modern DevOps
with Fuzzing and ML
TRACK
DevOps & Code

23. © 2020 Perforce Software, Inc.
#devopsnext-continuous-testing
LIVE SLACK Q&A