Weitere Ă€hnliche Inhalte KĂŒrzlich hochgeladen (20) When Testers Feel Left Out in the Cold2. Hans Buwalda
LogiGear
Â
Hans Buwalda has been working with information technology since his high school
years. In his thirty year career, Hans has gained experience as a developer, manager,
and principal consultant for companies and organizations worldwide. He was a pioneer
of the keyword approach to testing and automation, now widely used throughout the
industry. His approaches to testing, like Action Based Testing and Soap Opera Testing,
have helped a variety of customers achieve scalable and maintainable solutions for
large and complex testing challenges. Hans is a frequent speaker at STAR conferences
and is lead author of Integrated Test Design and Automation: Using the Testframe
Method.
3. 2/5/2014
1
STARCANADA 2014
Toronto, Canada
"Test Management" Track
Session W1
Wednesday, April 9, 10.30 â 11.30
© 2014 LogiGear Corporation. All Rights Reserved
Hans Buwalda
LogiGear
hans @ logigear.com
Who is your speaker
Software testing company, around since 1994
Testing and test automation services:
www.logigear.com
www.testarchitect.com
LogiGear Corporation
Testing and test automation services:
â consultancy, training
â test development and automation services
â "test integrated" development services
â focus on big and complex testing projects
Products:
â TestArchitectâą, TestArchitect for Visual Studioâą
â integrating test development with test management and automation
â based on modularized keyword-driven testing
© 2014 LogiGear Corporation. All Rights Reserved
Dutch guy, in California since 2001
Background in math, computer science, management
Since 1994 focusing on automated testing
â keywords, agile testing, big testing
Hans Buwalda hans @ logigear.com
www.happytester.com
4. 2/5/2014
2
Scope of this talk
Introduction to the management aspects specific for a test
project
Not a project management co rseNot a project management course
A personal view, based on experiences with my own
method
Collection of ideas and experiences, not necessarily a
complete picture
Input and discussions welcome (also talk with each other!)
© 2014 LogiGear Corporation. All Rights Reserved
Input and discussions welcome (also talk with each other!)
The ideas are coming from mixed European, American and
Asian experiences, apply with care
The Challenges for a Test Process
testing should be fun
testing sho ld be effecti etesting should be effective
testing should be efficient
testing should be under control
© 2014 LogiGear Corporation. All Rights Reserved
5. 2/5/2014
3
Keywords, Action Words
number name quantity
new product P-9009 Sledge Hammer 5
number quantity
add quantity P-9009 20
number name quantity
new product P-9009 Sledge Hammer 5
number quantity
add quantity P-9009 20
actions, each with a
keyword and
arguments read from top
to bottom
fragment from a test with actions
Common in automation tools nowadays (but with different styles)
Identify tasks for both test development and automation
q y
add quantity P-9009 3
add quantity P-9009 6
number quantity
check quantity P-9009 34
q y
add quantity P-9009 3
add quantity P-9009 6
number quantity
check quantity P-9009 34
"34" is the
expected value
here
to bottom
© 2014 LogiGear Corporation. All Rights Reserved
y p
The test developer creates tests using actions
Each action consists of a keyword and arguments
The automation task focuses on automating the actions
Each action is automated only once
Test Module Plan
Test Module 1 Test Module 2 Test Module N
Overview
Tests
Objectives
Tests Tests
Actions
. . .
Objectives Objectives
© 2014 LogiGear Corporation. All Rights Reserved
AUTOMATION
user password
log in jdoe StarEast
first last brand model
rent car John Renter Ford Escape
rent car John Renter Chevrolet Volt
last total
check bill Renter 140.42
interaction test business test
window control value
enter log in user name jdoe
enter log in password StarEast
window control property expected
check prop log in ok button enabled true
car guy
6. 2/5/2014
4
Discussing costs and benefits
© 2014 LogiGear Corporation. All Rights Reserved
Examples of costs and benefits
Involvement of users and
business specialists
Preventation of incidents
Shorter time to market
Costs? Benefits?
Need for investments
Build up of a test organisation
License costs
Training, coaching
Maintenance of (much) testware
...
Increased quality and certainty
Lesser dependence on users
and experts
Elimination of dull work
Improved documentation
...
© 2014 LogiGear Corporation. All Rights Reserved
Business Benefits
âą Time-to-Market
âą Quality-to-Market
âą Control
7. 2/5/2014
5
What can make testing miserable . . .
Discussions about costs and benefits
Resistance
Commitment
Politics
Dependence
Unrealistic expectations
© 2014 LogiGear Corporation. All Rights Reserved
Unrealistic expectations
Difficulty
Motivation
Practical issues and problems
Resistance is Normal
âLet's reconsiderâ âI don't understand thisâ
WHEN THEY SAY : THEY COULD THINK :
Let s reconsider
Now is not the time
All the time new objections
âFine, but are we ready for
this?â
Saying nothing
Saying yes, acting no
The method is good but in
I don t understand this
âI didn't expect all thisâ
âThis is going to cost me
my jobâ
I'm the star here, I don't
need the competition
âWe can't achieve thisâ
We will become too
© 2014 LogiGear Corporation. All Rights Reserved
The method is good, but in
this specific case âŠ.
We will become too
dependent on those guys
Now they will find out how
bad we are testing
8. 2/5/2014
6
GETTING COMMITMENT...
© 2014 LogiGear Corporation. All Rights Reserved
Management Commitment
Offer solutions not additional problems
Testing is often not popular . . .
Nobody wants an extra task (extra problem)
Bug are bad
Testing is good
Offer solutions, not additional problems
Tell managers that a good tested system creates a positive
image (not only negative reasons for testing)
Present/show what you're doing (glass box)
Try to get clarity about policies and directions for testing
Keep in mind: managers want things to be under control
© 2014 LogiGear Corporation. All Rights Reserved
Keep in mind: managers want things to be under control
â give clear and timely information about (1) progress and (2) results
Use outsiders and/or books/articles to make your case
Try to find some bugs . . .
9. 2/5/2014
7
Commitment, specifics
âNo time, no money, ...â
â back to the problem
â you should not become the problem owner!you should not become the problem owner!
âIt is so expensive/it is so difficultâ
â testing is expensive and difficult
â test automation is difficult
âThe others should do the testingâ
figure this out
© 2014 LogiGear Corporation. All Rights Reserved
â figure this out
â you can't deal with this yourself (strategic context mismatch)
â "doing things right" versus "doing the right things"
General vagueness
â hidden problems and conflicts
Politics . . .
Who is to blame when things go wrong
Systems are sometimes released while not finished
Testers tend to get the bullet
Notify the responsible managers early about what is going
to happen
Make clear written down procedures, especially for the test
execution phase
Not every bug is equally important (pick your battles)
© 2014 LogiGear Corporation. All Rights Reserved
Make the test process transparent
Maintain an atmosphere of cooperation and
communication with all
Ask for help, sound the alarm,
â donât underestimate the âpolitics of failureâ, it can hurt you
10. 2/5/2014
8
The Life Cycle of an IT Project
An (Old) Joke, Still Valid
Enthusiasm
Disappointment
! ! ! PANIC ! ! !
Looking for guilty ones ("blame-storming")
© 2014 LogiGear Corporation. All Rights Reserved
Punishing the innocent
Rewarding those who had nothing to do with it
The 3 questions for a bug
Ask these question, in the given order
1. Is it a problem?
2. What is the root cause?
© 2014 LogiGear Corporation. All Rights Reserved
3. Why did we not catch it?
11. 2/5/2014
9
Dependencies . . .
Testing and automation is dependent on many factors:
âą Working systems
âą Test environments (like virtual machines)
S ifi ti i t d i k l d
Make clear arrangements with everybody about everything,
as early as possible
â written down
âą Specifications, requirements, domain knowledge
âą People
âą ...
© 2014 LogiGear Corporation. All Rights Reserved
Keep in touch with the rest of the project
Make the high level test products as early as possible
Dependencies, specifics
âThe system under test isn't availableâ
â The automation will be the first in trouble
â Discuss this (early)Discuss this (early)
â Allocate resources when really needed
âThe system under test doesn't workâ
â The automation will be the first in trouble
â Create a âhealth checkâ (smoke test to see if all
functionalities that are relevant to the automation work)
âTh i i â
© 2014 LogiGear Corporation. All Rights Reserved
âThere is no test environmentâ
â Start planning this as early as you can
â Make clear how important this is
â âNo test environment, no testâ
12. 2/5/2014
10
Expectations Around Automation
The impression can arise that automated testing (in
particular with keywords) is just a push of a button . . .
even if nobody said so!
Message: testing is difficult, automated testing is even
more difficult
Make clear what is happening
Communicate manage expectations
. . . even if nobody said so!
© 2014 LogiGear Corporation. All Rights Reserved
Communicate, manage expectations
Don't put too low figures in a test plan
â let somebody else lower them (in writing)
Ask managers and other people involved what they expect
Difficult to Keep People Motivated
Watch the signals
Motivation of team members can erode when time progresses
Happens to both test developers and automation engineers
Watch the signals
Make sure that the right people are assigned
â Automation engineers should have a programming background
â Testers should have a testing and/or business background
Avoid âmonks workâ (Dutch expression for tedious repetitive work)
Differentiate the work
Avoid isolation of the test group
© 2014 LogiGear Corporation. All Rights Reserved
Be prepared to switch roles
Consider agile approaches
Be ready for this problem in advance
Create operational and professional communication structures
â for example special interest groups to regularly discuss professional topics
13. 2/5/2014
11
Complexity: The Work is Difficult
Making tests is supposed to be "difficult":
âą Finding bugs of others
âą Making concrete examples
Automating them is difficult as well
Don't be over-confident, use coaching from experienced
peers or outsiders
Consider reference visits to other projects or sites
Talk about the work, even create special interest groups
Keep in touch with others doing the same work
Automating them is difficult as well
© 2014 LogiGear Corporation. All Rights Reserved
Keep in touch with others doing the same work
Organize interactions (meetings!), for example with:
â users
â developers
â auditors
Delegate tasks (better lazy then crazy...)
Practical Issues and Problems
Does the test tool work here ? ?
Do we have authorizations on the test environment ? ?
Where do we keep our test products ? ? Which tests do we have ? ?
Be prepared for this, automated testing is a âcloud of
detailsâ
Put suitable team members in leading roles, and delegate
responsibilities to them
F l t t j t t t t i f ll ti j b
Questions, chores, details, . . .
© 2014 LogiGear Corporation. All Rights Reserved
For larger test projects test management is a full time job,
make sure you get this time
Try to find easier ways, organize things better, or automate
more
â âIf you have a difficult task, ask a lazy man. He will find an easier wayâ
14. 2/5/2014
12
Some final remarks
Changing a process can mean:
â changing of work processes
â a learning curvea learning curve
â risks and benefits
It is not the same as: buy a tool and all will be well
Changing any process is more than anything an
© 2014 LogiGear Corporation. All Rights Reserved
organisational change and should be managed as
such
Thank you for listening
âŠand...
keep cool, even in the cold
© 2014 LogiGear Corporation. All Rights Reserved