Manish delivered a session on "Shift Left - Performance Testing" at ATAGTR2020 ATAGTR2020 was the 5th Edition of Global Testing Retreat. Manish is Director of Business and Technology Consulting at Micro Focus, responsible for Enterprise Devops. He brings rich international experience spanning two decades in the IT industry, with major focus on application delivery, automation, DevSecOps and performance engineering The video recording of the session is now available on the following link: https://youtu.be/PAHR3X1TGdc To know more about #ATAGTR2020, please visit: https://gtr.agiletestingalliance.org/

1. Performance Testing:
Shift Left for Agile and DevOps
Manish Gupta

2. Great performance is the foundation of success
High-performing software is
essential to deliver an enhanced
experience and superior quality
Without it, organizations will be
challenged to keep pace with
digital natives and competitors alike
Sources: Domo, Data Doesn’t Sleep 7.0, March 2019 and Lori Lewis, What Happens In An Internet Minute, March 2020
REDUCE COSTS
INCREASE CUSTOMER LOYALTY
GROW BRAND VALUE
GET TO MARKET FASTER

3. Yet performance testing is
often the last mile
Leading to poor application
design, higher costs of
change, release delays, and
production issues

4. Greater Variety
More technologies
and services to test
Putting more pressure on performance teams
Little Alignment
Wider gap between
dev and test efforts
More Complexity
Root cause analysis
is getting harder
Growing Costs
Script upkeep and
increasing loads

5. It’s time to adopt performance engineering
LoadTesting
▪ Identify upper limits and set
SLAs
Performance
Engineering
StressTesting
▪ Identify behavior under
intense loads (data, users)
▪ Understand how the
application recovers from
failure
Performance Tuning
▪ Pinpoint and identify the root
cause of performance issues
in production
▪ Implement code change and
monitor results
Virtualization
▪ Accelerate test efforts with
network and service
virtualization
▪ Eliminate dependencies and
costs associated with
production and external
services

6. Is Performance Testing as Agile as you want it?
Sprint N Sprint N+1 Sprint … Sprint …
Features are stable
Performance issue
found and fixed
Push to prod?
Dev start
Push to prod.
Content is really done
6

7. Performance Engineering is a team sport
DevTestersDevelopers Performance
Engineers
7
“Performance is not just the responsibility of the developer, the tester or the
operations team. It is everyone's responsibility, and "performance first" should be
a mantra for every stakeholder.” – Gartner

8. Performance Testing to Performance Engineering
Add
Performance
criteria to the
Definition of
Done
Have The
Scrum Team to
Create
Performance
Testing Assets
Allow
Pre-Commit
Tests
Integrate With
The CI Pipeline
The
Performance
Engineer Owns
Overall
Performance
Level
8

9. Stay in the IDE
Collaborate
across teams
Execute using
local resources
Integrate to the CI
Scale to the Cloud
Free For
Developers
What is needed from a tool perspective?
9

10. LoadRunner Developer
Designed with developer mindset:
▪ Script and execute load tests as part of a
continuous testing process
▪ Integrate with any IDE (IntelliJ, Visual Studio,
Atom.io, Visual Studio Code) or any CI
(Jenkins, Bamboo, Azure DevOps)
▪ Fully cross-platform (Windows, Linux, macOS)
▪ Ideal for the latest web technologies using an
innovative and simple JavaScript SDK
▪ Based on the DevWeb engine included in all
LoadRunner products
▪ Free up to 50 Vusers

11. Built for collaboration
LoadRunner Developer Benefits:
▪ Create and execute tests using the same
technology
▪ Share assets with between teams
▪ Provide the tools to run and analyze
performance in existing ecosystems
▪ Scale within existing tools
▪ Proficiently test performance to save time
Performance Engineers Developers / DevTesters

12. Develop application and run functional and performance testing within a single IDE
Demo – LR Developer and UFT Developer

13. Quick Scalability with LoadRunner Cloud

14. Maximize testing with extreme cloud scalability
LoadRunner Cloud Benefits:
▪ Augment existing testing needs and minimize
costs associated with physical infrastructure
▪ Scale to over five million virtual users for the
ultimate in test coverage
▪ Distribute virtual users to multiple locations
using AWS or Microsoft Azure cloud locations,
private cloud or on-premises load generators
▪ Flexible licensing supports a consumable,
pay-as-you-go approach for seasonable and
peak testing, and concurrent models for
continuous testing methods

15. Quick testing for fast-moving Agile and DevOps teams
LoadRunner Cloud Benefits:
▪ Design and create tests in under 10 minutes
▪ Leverage LoadRunner scripts, open source
scripting tools or create a test using REST API,
CSV file or HAR file
▪ Easily manage and maintain scripts with Git
integration
▪ Run tests as part of the build process with
Continuous Integration support
▪ Foster collaboration with multi-user, multi-
test execution, asset sharing, and project
management

16. 16
LoadRunner - Rich Integrations for DevOps Tool Chain

17. Analyze smart metrics to understand performance
LoadRunner Cloud Benefits:
▪ Analyze the performance of a selected run
against a range of customizable metrics with a
built-in dashboard
▪ Easily compare runs from same load test or a
benchmark side by side in the same graphs
▪ Quickly generate reports that show statistics
and performance data for test runs
▪ Measure service level agreement (SLA) goals
against performance related data, and
determine whether the SLA passed or failed

18. Choose the right tool for the right job
While leveraging a
connected ecosystem
that delivers smarter
insights, tighter
collaboration and
better cost savings
Project oriented for
the most technical users
Developer-centric
using the IDE of choice
Ultimate scalability
when you need it
CoE ready for
global collaboration

19. Remove Constraints with Service
Virtualization

20. Eliminate Bottlenecks with Service Virtualization
Remove test environment ‘wait time’, Shift testing left, reduce cost
▪ Eliminate constrains in architectures and projects
▪ Simulate data and performance scenarios
▪ Test for real or artificial conditions with insight
▪ Share services; lower infrastructure costs
Existing infrastructure
Third Party
Mainframe
JDBC
SOAP
RFC
MQ
SAP
Existing database
Web service and
Legacy application
REST
Third Party
Pay-per-transaction
Lack of data controls
Virtual Services
SOAP
REST
MQ
JDBC
RFC
CICS
Perf.
Config
Data
Simulation
Application Under Test
Mobile
app
Web
browser
Application
Services
Test Tools
Functional Test
Performance Test
Dev Test