EuroSTAR Software Testing Conference 2013 presentation on Software Testing Secrets We Dare Not Tell by Graham Thomas.
See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/
Apidays New York 2024 - The value of a flexible API Management solution for O...
Graham Thomas - Software Testing Secrets We Dare Not Tell - EuroSTAR 2013
1. Software Testing Secrets
That We Dare Not Tell !
Graham Thomas
Independent Software Testing Consultant
www.eurostarconferences.com
@esconfs
#esconfs
2. Abstract
I have worked in IT for over 30 years, in software testing for
over 20 years, so you might think that I know a lot about IT
and in particular software testing. Well, I am going to share
some dark secrets about software testing with you, which we
dare not tell, and do that in the form of questions.
These questions are important because they drive at the
very heart of what we are doing in the software testing
industry today.
2
This session has been designed to be a highly interactive
discussion which many people might find challenges their basic
understandings. I will act as facilitator, give an introduction to
each question, then actively moderate the debate and if needed
act as arbiter. Come along, expect to be involved, and if you have
a view then please share it. Help to drive forward the discussion –
and the software testing industry.
3. A bit about you
A feel for the audience today
Current Role
Above Program Test Manager
Program Test Manager or Test
Manager
Test Leader
Test Analyst or Test Execution
New to Testing
Industry Experience
15 years +
Over 10
Over 5
Done a year or more
Still got the L plates on
Industry exposure
Industry body
Workgroup
Contributor
Blog
Speaker
Author
Conferences
First time at EuroSTAR
Other conferences and seminars
Workshops or meetings
3
6. What non-testers think
I was explaining this presentation to my doctor
the other day.
I said “It is questioning the purpose of Software
Testing.”
He said “ That is easy, it is to find all the bugs.”
I said “Not necessarily.”
He gave me a funny look, and I realised that this
was a longer conversation that we had time for!
6
8. 8
2. Just how effective is the way we test –
and how do we know? (Trad, V-Model,
Structured Testing, agile or any other
form of testing for that matter?)
9. Effectiveness
GBP / USD per bug
Overall cost by requirement / use case /story
Organisational cost – If I do this I can’t then do that
Can I afford to test?
How long will it take
Why is it now taking longer
How do I measure it?
Can’t I have it now?
And what will I get for all that
The Time / Cost / Quality Dice
How will I know I have got all of that?
Do I need all that?
What happens if I don’t get all of that?
9
11. 3. If checking isn't software testing, then
why is it that ‘checking’ is what our
stakeholders are paying us to do?
11
12. Drivers for Testing
Checking
Prove it works
Show me it working
Coverage
Understand what I will get
Quantify the risk of delivery / non-delivery
Testing
Michael Bolton quoting Laurent
Bossavit
“A Testers Role is to see through the veil of
conflicting illusions!”
Agile Testing is ‘Exploratory Learning’
Discovery
12
14. 4. If software testing is so
difficult, demanding and
challenging, then why is it
that we keep on assigning
the least skilled or
experienced to perform it?
14
15. Conundrum
And even if we don’t know;
what the purpose of software
testing is,
how effective our software
testing is,
or agree on how we go about
software testing,
surely we should at least be able to
draw together some attributes for
the kind of people we need to
perform software testing?
15
17. 5. Why do software testers
spend so much of their time
running tests that do not find
bugs?
17
Tools Scripting Execution Analysis Resources
18. Inefficiencies
Automation Paradox
To successfully automate tests they all have to pass. However what is
the point of running tests that don’t find any bugs?
Tools to automate testing
If we had tools that could test software we wouldn’t need software
testers!
Though, how would we know whether the tools were working or not?
Automating testing does not save any time, and in most cases
takes longer than manual testing!
The latest trends in test optimisation are to stop running
regression suites that take time effort and resource but no longer
find bugs!
18
20. To Conclude
As an industry, collectively:
“We do not know, or agree on the purpose of software testing”
“We do not know how efficient or effective our test models are, regardless of type”
“We don’t even agree on what it is we should do when we are testing software”
“We don’t understand what skills and experience we need to perform software testing”
“And finally when we do testing, an awful lot of it doesn’t actually make any difference”
I think we should either:
“Keep these secrets very, very quiet”
“Or tell everyone . . . . . .
and set about finding the answers!”
20