Accelerating Your Digital Agenda with Continuous Testing ft. Forrester

SAUCE LABS PRESENTS: ACCELERATING YOUR
DIGITAL AGENDA WITH CONTINUOUS TESTING
August 13, 2019

Diego Lo Guidice
Vice President, Principal Analyst,
Forrester Research
© Sauce Labs, Inc.
FEATURED SPEAKER

© 2019 FORRESTER. REPRODUCTION PROHIBITED.
Accelerate Your Digital Agenda with
Continuous Testing
Diego Lo Giudice, VP & Principal Analyst serving AD&D
August 2019

4© 2019 FORRESTER. REPRODUCTION PROHIBITED.
We need to test more,
better and faster…
...continuously.

5© 2019 Forrester Research, Inc. Reproduction Prohibited
Why?

http://assets.bwbx.io/images/i0dwq8cFhDmE/v1/-1x-1.jpg
Too much speed?
Poor quality?
Or both?

Poor Quality Software ?

4 Seconds

53%
Base: 232 tech professionals at firms that have implemented or are implementing Agile.
Source: Forrester’s Q3 2017 Global Agile Software Application Development Online Survey
Expect Agile to
improve technical
quality

63%
Expect Agile to
improve functional
quality

76%
Expect Agile to
improve customer
experience

Expectations
are not always
met !

better and faster.
Speed Can’t Come
Without Quality!
Quality@speed

What are the
challenges?

Digital
Transformation
Is The Goal For
Every Industry
Base: 354 business and technology leaders
Source: Forrester’s Q1 2018 Digital Process Automation Survey

But, most digital transformation initiatives are struggling
Source: “The Sorry State Of Digital Transformation In 2018” Forrester report.
Base: 1,559 business and technology decision makers

Many Enterprise
Developers
Aspire To Do
DevOps
Yes, our group
actively
embraces
DevOps
24%
Yes, our group actively
embraces DevOps in
conjunction with Agile
33%
Some of our
processes or
tools resemble
those used in
DevOps, but we
have not
officially
adopted
DevOps
22%
No, but adoption
of DevOps is an
internal priority
5%
No, and we
have no plans to
adopt DevOps
11%
Don't know
5%
Would you characterize your group's current
processes and structure as resembling DevOps?
Base 3,203 Developers who work full time.
Source: Forrester Analytics’ Business Technographics® Developer Survey, 2019.
But only these really
get it!

But Software Delivery Speed Seems Sluggish
Base: 3,294 Global Developers,
Source: Forrester Analytics’ Business Technographics® Global Developer Survey, 2019.
13%
24%
31%
30%
More than once per month
Once per month
Once per quarter
Twice per year or fewer
How frequently do you release applications?

Teams Are Releasing Faster…Barely
Base: 3,228 Developers (2018) and 3,294 Developers (2019)
Source: Forrester Analytics’ Business Technographics Global Developer Survey, 2018 and 2019.
12% 12%
20%
26%
18%
7%
1% 1%
12%
19%
31%
24%
10%
2% 2%
We time new
releases to
coincide with
new versions of
operating
systems or
middleware
We average one
release a year
(or less
frequently)
We average two
releases a year
We average one
release per
quarter
We average one
release a month
We average one
release a week
We average a
release a day
We release
many times per
day
2018 2019
Note! We did take
out a category for
2019. So there’s
12% from 2018
relocated

Testing Is Part Of The Problem
20%
22%
25%
26%
26%
28%
30%
31%
Lack of automation tools and/or infrastructure
Scaling Agile at the enterprise level
Connecting Agile with existing product portfolio management
(PPM)
Lack of dedicated cross functional teams
Lack of Agile testing skills
Lack of Agile skills in downstream or DevOps
Lack of Agile executive leadership
Lack of Agile skills in project management practices
Which of the following are barriers to your organization’s Agile
adoption? (Check all that apply) (not all answers shown)

Base: 3,228 (2018) and 3,294 (2019) Global Developers.
Source: Forrester Analytics Global Business Technographics® Developer Survey, 2018 and 2019
If You Felt This
Was Low….

Base: 2,279 manager and 924 non manager developers.
Source: Forrester Analytics Global Business Technographics® Developer Survey, 2019
Get Ready Since It’s
Probably lower…
The “Managers-
Developers” divide..

Test automation plateaus too, and manual testing
resists to go away

As Continuous Delivery Matures Performance Testing
Is The Next Blocker
Performance done at the end is complex
and costly
Centralized performance testing isn’t scalable
Completely disrupts continuous delivery

How can we address
these challenges?

From Digital Business Excellence To SW Delivery
Execution Excellence….Your new mantra…!
Business Must
Master Digital
To Stay In
Business
Software
Powers The
Digital
Experience
Software Execution
Excellence Equals
Business Execution
Excellence

Modernize Testing With Continuous Testing

Sprint definition of "done" includes
your sprint testing goals (Done
done, test data strategy,
automation, cntrl. Gates…)
Define Test Scenarios using a
DSL (e.g: Gherkin)
Include Performance Testing in
Acceptance Criteria
Feature toggles, AB Multivariate…
Prioritize bugs on backlog
PO, Testers and Developers

End to End Testing
Integration Testing
Functional
Testing
Non-Functional
Testing
Unit Testing
Make Sure Its Clear Who Needs To Test What….
Only Developers
Biz mostly manual, Mostly Technical Testers,
Dev Testers (API testing, automation)
Technical Testers (Specialized),
Dev Testers (slowly catching up...shift left)
Mostly Technical Testers, Dev Testers (SVT
tools)
Biz, Mostly Technical Testers

Design automation of functional
tests (and test data)
Build automation of functional
tests (UI, API)
Build automation for non
functional (SVT, SL PT, Security)
Scriptless, scripting, coding
Exploratory testing (manual)
Technical Testers, Dev-testers,
Business testers

Source: See The Forrester report “Shift Performance Testing Left To Streamline App Delivery”.

Shift Left Performance Testing (SLPT)
Developers
profile code,
smoke test,
leverage
baselines at
unit/
component
level
Product
Owners and
Developers
become
stewards of
Performance
(With Testers)
Overtime E2E
Performance
Testing
Wanes As
SLPT
Becomes The
Norm.
A new set of
lightweight
agile SLPT
tools provide
developers
what they
need to get
the job done

United Airlines experienced
an 80% Performance
Issues drop in production
with SLPT

Version Control test assets
with source code
Integrate in CI/CD to
automate all test execution
(unit, functional, non-
functional)

Automated integration
testing (Service
Virtualization)
Automated governance for
release automation or
feature deployment

Production monitoring
feeds your testing
Machine Learning predicts
what to test next
Integrated AB testing in
SDLC helps improve CX

Finally, Put Intelligence In Your Continuous Testing!
Visual Testing
(ubiquitous )
Test Automation &
Generation
Test Lifecycle
Optimization
Unit test code
generation
API Test
generation
Defect management
(clustering, MTTR
reduction..)
Production
User Experience
Monitoring

We need to test faster,
We need to test better,
We need to test smarter!

Accelerating Your Digital Agenda with Continuous Testing ft. Forrester

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Accelerating Your Digital Agenda with Continuous Testing ft. Forrester

Similar to Accelerating Your Digital Agenda with Continuous Testing ft. Forrester (20)

More from Sauce Labs

More from Sauce Labs (20)

Recently uploaded

Recently uploaded (20)

Accelerating Your Digital Agenda with Continuous Testing ft. Forrester