Why Classic Software Testing Doesn’t Work Anymore

nt Session

Presented by:
Regg
Po re

Brought to you by:

340 Corporate Way, Suite Orange Park, FL 32073
888‐2
W5
Concurre
4/9/2014
12:45 PM

“Why Classic Software Testing
Doesn’t Work Anymore”

Struyk
larion Softwa

300,
68‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Regg Struyk
Polarion Software

With twenty years of commercial software development and testing experience, Regg
Struyk has held many different positions ranging from the head of technical product
management for Agfa HealthCare to, most recently, product evangelist for Polarion
QA. Regg has developed for several software testing tools including test integrity,
iTest, and Polarion QA. Dedicated to the domain of test management, Regg is
continually analyzing testing trends and their potential impact on the discipline of
software testing.

2/11/2014
1
QA Evolves
20 plus years software dev experience
Who am I?
Senior Product Manager @ MKS (PTC)
Group Product Manager @ Agfa Healthcare
Product Manager @ Polarion for QA Software
Polarion Software® Polarion® QA™ - www.polarion.com
g @ Q
2

2/11/2014
2
•Introduction
•Company Profile
•Speed of Light
•Big Data
•Mobile
•Tools•Tools
•Automation
•Resources
“We produce all significant documents for specifications,
QA and test plans easily and rapidly with Polarion QA”
Andreas Deuter , Phoenix Contact Electronics
Waterfall
– Test Specifications
Traditional QA
– Test Cases
– Test Execution
Polarion Software® Polarion® QA™ - www.polarion.com 4

2/11/2014
3
Agile
– Get ready to test
The Speed of Light
– More time constraints
– Originated from development
Hybrids/Spin-Offs
– Scrum, Agile-Fall
– Does not means getting rid of documentation
Market Pressures
– Regulatory
– Quick check-in
5
The Speed of Light
“Testing is not a phase on Agile teams, testing is a way of life. Agile
teams should test continuously. It’s the only way to ensure that the
feautures implemented during a given iteration or sprint are actually
done”

2/11/2014
4
Time Constraints - Deliver Faster
Pressure to test more with Limited
Resources
Focus becomes task completion vs coverage
Quality may suffer (may miss critical bugs)
Examples:
Madden Football 2008 – prematurely released
Xbox – hardware crashes = LAWSUIT
Lack of Planning
– Still need to plan not a free for all
What‘s going on QA – Pitfalls
– More time constraints
Lack of Documentation
– Key component of Agile is removing clunky documentation
– Does not means getting rid
Daily Meeting Hassle
– Waste of time?????
– Quick check-in
“I’m surprised at the name Agile —it should be called couch potato.
There are too many meetings. It’s ironic that there are all these books
explaining how simple it is.”
8

2/11/2014
5
The real goal of testing should be to improve the software
– A continuous search for information
What‘s going on QA
– Instead of pass or fail think in terms of whether or not tests provide
valuable information
Understanding project and goals
– Define coverage requirements
– Understand/establish goals of a sprint
O f %
“Out of 200 participants, 64 % said that switching to Agile
Development was harder than it initially seemed.”
9
•Risk-based testing accommodates change (focus on main areas)
– Reduce the scope of testing based on risk
What‘s going on QA -Solutions
•Careful automated testing can accomodate change*
•Good unit testing is essential (limited bug-finding 30%)
•System Testing will cover more on average
Combine risk-based testing with maintainable automated regression
testing at the system test level
10

2/11/2014
6
The big data construct
What is Big Data?

2/11/2014
7
“Data sets whose size is beyond the ability of commonly used tools to
process it within acceptable time”.
Big Data – What is it?
Transformation in how society processes information.
Explosion of big data is just as transformative as the internet was.
According to IBM, “Every day, we create 2.5 quintillion bytes of data —
so much that 90% of the data in the world today has been created in
so much that 90% of the data in the world today has been created in
the last two years alone”.
Gartner believes Big Data will drive a total IT spend of $34billion in
2013 rising to $232billion by 2016!
13
Big Data – Sources

2/11/2014
8
What is Big Data?
The big data construct theory
Source: Gartner
To gain the right insights, big data is typically broken down by three
characteristics:
Big Data – 3Vs?
•Volume: How much data
•Velocity: How fast data is processed
•Variety: The various types of data
Big data is high-volume, high-velocity and high-variety information
assets that demand cost-effective, innovative forms of information
processing for enhanced insight and decision making.
processing for enhanced insight and decision making.
16

2/11/2014
9
Data Volume
"I have too much data to analyze"
“We live in a world that is getting more digitized and more connected
Big Data Volume - Challenges
every day.”
Data is now created by employees, customers, 3rd party
Extracting information from large data sets
Massive, complex and semi-structured
Regulatory demands increase pressure on testing
Consolidation of data from systems, apps, internet
Mi i d t & d t i t it
Missing data & data integrity
18

2/11/2014
10
Data Velocity
Speed at which new data is being created
Big Data – Velocity
The need for real-time analytics
acquisition of data vs. actionable insight
“Every business is now a digital business”
20

2/11/2014
11
The ability to implement a data driven testing process
Big Data – Velocity
Challenges
Improving response rates for decision making
Expectations – when fast isn’t fast enough
Data Variety

2/11/2014
12
structured/unstructured data
text
Big Data – Variety
text
sensor data
audio
video
click streams
log files
g
23
80% data growth in images & video
Number of data types Unstructured/semi
Big Data – Variety Challenges
Number of data types Unstructured/semi
Volatility of changing data
Sources of data
Mobile devices
Internet
Email

2/11/2014
13
Traditional methods not enough
Big Data – Challenges
Traditional tools are inadequate
Lack of knowledge as to what to test and how much data
to test
Test strategies for structured vs unstructured data
25
Test early – Test fast
Data driven testing
Big Data – Solutions
Data driven testing
Develop a strong testing environment
Multiple testing methodologies
functional/non-functional
Data Sampling
Identify suitable sampling technique
y p g q
Include correct uses cases
Right test data sets
26

2/11/2014
14
Mobile Testing
Mobility application testers, face many critical
challenges compared to conventional software testingchallenges compared to conventional software testing
Number of mobile phones will exceed world population
by 2014 (7.3 billion)
Mobile – Landscape
y ( )

2/11/2014
15
BYOD
Complexity added due to combination of multiple
Mobile– Challenges
platforms each having numerous devices and various OS
versions
Various versions of each platform and device
Mobile user: range from tech savvy to newbie
Testing on various networks and network vendors
CDMA and GSM with their 2G, 3G and 4G versions
29
Rapid Roll-out Schedule
Expectations for apps
Mobile– Challenges
Expectations for apps
Ultra competitive to release new apps
platforms each having numerous devices and various OS
versions

2/11/2014
16
Emulation Software (Zap, Keynote)
Mobile– Solutions
Cloud Computing (Perfecto, Soasta)
Crowd sourcing
Performance Testing (Loadster, Neotys)
Tools

2/11/2014
17
Tools
Unavailability of right tools
– Budget, availability and funding
– Type of Tools:
• DEFFECT TRACKING
• TEST CASE CREATION & Execution
• Test Automation
Integrations: lack there of– Integrations: lack there of
– MS Excel & MS Word equation
• Time consuming to manage and maintain
• Metrics
• Integrations
Tools - Solution
Test Management Software
– Combines Test Cases, Test Executions,
Defects, Test Automation
– Replace or leverage MS Excel/Word
– Customizable workflow
– Automation– Automation
– Integrations

2/11/2014
18
Tools - Solution
Network with other testers to find information about
l k llesser-known test tools.
Online QA/Testing forums
Interoperable tools
Automation

2/11/2014
19
Test Automation
1. Test Automation Will Eliminate Human Testers1. Test Automation Will Eliminate Human Testers
FALSEFALSE
2. What tests should be automated first?2. What tests should be automated first?
Automate your testing procedure when you
have lot of regression work.
Test Automation - Implementation
– Commitment
can take time to set p & maintain• can take time to set-up & maintain
• If not implemented correctly = $$$
Desire to automate tests across apps & platforms
Most automated test tools do not tell you what to test
Initial cost can be high
• purchase and set-up
100% Automation does not exist!

2/11/2014
20
Test Automation - Solutions
The right automation
– Right skill sets (programing scripts)
– Ensure we create reusable tests
• can take time to set-up & maintain
– Unit test will provide highest ROI
There is a large gap between the people who own
automated test tools and the people who actually realize the benefits from test
automation.
5 Lessons Learned

2/11/2014
21
Lessons
To handle the volume and ensure live data
i t ti Bi D t t ti d tintegration, Big Data testing needs to empower
developers, testers and enterprises with freedom to
experiment and innovate.
Lessons
Successful software test automation is possible if
f d t l i dd d d dfundamental issues are addressed and managed.
Automated software testing is truly a different way of
testing and requires adjustments to current test
methods and organizational structures. However,
thepayback from test automation can far outweigh the
costs.

2/11/2014
22
Lessons
With nearly a third of all page views now coming
f bil d t bl t t t d t bfrom mobile and tablets, testers need to embrace
mobile. Mobile is no longer an area that a few UX
people specialize in, and we need to start designing
and testing everything for smartphones and tablets as
well as computers.
Lessons
Development cycles will continue to accelerate
b d i f t ( k tbased on various factors (market, process,
technology), and quality expectations will need to
meet the challenge through increased visibility, new
ways of thinking combined with advances in software.

2/11/2014
23
Lessons
What did you learn????
Resources
“Game Testing Evolves” whitepaper @ Polarion
“Big Data Testing” whitepaper @ PolarionBig Data Testing whitepaper @ Polarion
http://www.polarion.com/products/qa/index.php
regg.struyk@polarion.com
T itt R St k1Twitter: ReggStruyk1
LinkedIn: Software Test Case Management

Why Classic Software Testing Doesn’t Work Anymore

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (9)

Mehr von TechWell

Mehr von TechWell (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Why Classic Software Testing Doesn’t Work Anymore