'Lessons Learned in Test Management' by Bob van de Burgt

www.eurostarconferences.com
@esconfs
#esconfs
Lessons Learned in Test
Management
Bob van de Burgt , Professional Testing BV
Bob van de Burgt is test advisor at Professional Testing BV. He contributed
to the development of the testing method TestFrame® and the test
management approach of Logica for which he also was co-author of the
published books. Bob has given many testing courses (including ISTQB) and
is a frequent speaker at (inter)national congresses. He was member of the
board of the Dutch SIGIST TestNet for 10 years of which 6 year as
chairman. Bob was Programme Chair of EuroSTAR 2008 in The Hague.

@esconfs
#esconfs
Join the conversation on Twitter
#esconfs
@esconfs
Continue the conversation with the speaker on Twitter after the show!

@esconfs
#esconfs
3
Agenda
-Introduction
- Test Management Model
- 10 years later
- Agile and Test Management
- Conclusions

@esconfs
#esconfs
Test Management
Model
• Risk & Requirement Based Testing
(RRBT)
• Preparation and Execution
• Eight phases
• Secure in Test Management File
4
Preparation
Test
Organisation
Planning Issue
Management
Reporting
And
Implementation
advice
Transfer
And
Maintenance
Estimation
Risk Analysis
And
Test Strategy
TestManagementFile
Execution
Progress
Management
Risk & Requirement
Based Testing

@esconfs
#esconfs
Requirements
RRBT: Risks versus
RequirementsA risk, no requirement:
• Adding requirements
(finding faults earlier)
• Remove risk (don’t test
more than necessary)
A requirement, no risk:
• Adjusting risk list
(improved coverage
test)
• Removing requirement
(don’t develop more
than necessary, no
“frills”)
Matching risks and requirements
Product
Risks
Matching requirements and risks
5

@esconfs
#esconfs
Priorities product risks: an
example
6
If this
goes
wrong
than
...
… it has non-
financial
consequences
for our
customer.
… it has
financial
consequences
for our
customer.
… it has non-
financial
consequences
for our
department.
… it has
financial
consequences
for our
department.
All
customers
One
customer
All
customers
One
customer
No
workaround
Workaround
No
workaround
Workaround
Must test
Should test
Should test
Could test
Could test
Won t test
Could test
Won t test

@esconfs
#esconfs
Won’t test
conditions:
…
…
The Strategic Test Slicing
Method (STSM)
7
Could test
conditions:
…
…
Should test
conditions:
…
…
Must test
conditions:
…
…
Ouch !!!

@esconfs
#esconfs
8
Agenda
- Introduction
- 10 years later
- Conclusions

@esconfs
#esconfs
Experience RRBT
Challenges:
• Difficulty in defining risks
• When is my risk assessment
complete?
• When do I start the risk
analysis?
• The requirements are not
good or not complete
Achievements:
• Speaking the language of
higher management
• Looking at requirements
with a “testing” eye
• Applicable in traditional
and agile environments
9
Preparation
Test
Organisation
Planning Issue
Management
Reporting
And
Implementation
advice
Transfer
And
Maintenance
Estimation
Risk Analysis
And
Test Strategy
TestManagementFile
Execution
Progress
Management
Risk & Requirement
Based Testing

@esconfs
#esconfs
Challenges:
•Difficulty with risk analysis
•Difficulty with using risks as basis for test
strategy
10
Experience Risk Analysis & Test
Strategy
Achievements:
• Missing requirements became
apparent
• Stakeholder buy-in for testing
• The first difficult questions are asked
Preparation
Test
Organisation
Planning Issue
Management
Reporting
And
Implementation
advice
Transfer
And
Maintenance
Estimation
Risk Analysis
And
Test Strategy
TestManagementFile
Execution
Progress
Management
Risk & Requirement
Based Testing

@esconfs
#esconfs
Challenges
•Still very difficult
•Lack of maturity in keeping metrics
•Project budget has no actual basis
•Moving targets
11
Experience Estimation
Achievements:
• Model will help
• Estimate based on risks is easier to
sell
Preparation
Test
Organisation
Planning Issue
Management
Reporting
And
Implementation
advice
Transfer
And
Maintenance
Estimation
Risk Analysis
And
Test Strategy
TestManagementFile
Execution
Progress
Management
Risk & Requirement
Based Testing

@esconfs
#esconfs
Challenges:
•Planning is always a rough estimate
•Overall project planning often not
realistic
12
Experience Planning
Achievements:
• Planning in Earned Value
method gives advantages
• Planning got better as we now
aim for testing the most
important first
Preparation
Test
Organisation
Planning Issue
Management
Reporting
And
Implementation
advice
Transfer
And
Maintenance
Estimation
Risk Analysis
And
Test Strategy
TestManagementFile
Execution
Progress
Management
Risk & Requirement
Based Testing

@esconfs
#esconfs
Challenges:
•Moving to agile teams
•Limited possibilities to assemble your
dream team
•Test environments still difficult
•A better structured test process leads to
the best test organisation
•Testing does not always gets the most
attention, so not always the best fitting
organization or resources
13
Experience Test Organisation
Achievements:
• Moving to virtual environments
saves time and money
Preparation
Test
Organisation
Planning Issue
Management
Reporting
And
Implementation
advice
Transfer
And
Maintenance
Estimation
Risk Analysis
And
Test Strategy
TestManagementFile
Execution
Progress
Management
Risk & Requirement
Based Testing

@esconfs
#esconfs
Challenges:
•Excel is the most used tool
•Align your reports to your stakeholders
14
Experience Progress Management
Achievements:
• In control with EVM
• TM becomes trusted advisor
with controls over time,
budget and quality
Preparation
Test
Organisation
Planning Issue
Management
Reporting
And
Implementation
advice
Transfer
And
Maintenance
Estimation
Risk Analysis
And
Test Strategy
TestManagementFile
Execution
Progress
Management
Risk & Requirement
Based Testing

@esconfs
#esconfs
Challenges:
•Issue management needs a good process
•Issues should always be input for progress
management
•Adjust existing tooling where necessary
15
Experience Issue Management
Achievements:
• Stakeholder involvement is
crucial for success
• Excellent tools available
(e.g. JIRA)
Preparation
Test
Organisation
Planning Issue
Management
Reporting
And
Implementation
advice
Transfer
And
Maintenance
Estimation
Risk Analysis
And
Test Strategy
TestManagementFile
Execution
Progress
Management
Risk & Requirement
Based Testing

@esconfs
#esconfs
Challenges:
•Adjust frequency to audience
•Report to all stakeholders
•Measuring is key
16
Experience Reporting &
Implementation Advice
Achievements:
• Reporting is easy with the controls of
progress management!
Preparation
Test
Organisation
Planning Issue
Management
Reporting
And
Implementation
advice
Transfer
And
Maintenance
Estimation
Risk Analysis
And
Test Strategy
TestManagementFile
Execution
Progress
Management
Risk & Requirement
Based Testing

@esconfs
#esconfs
Challenges:
•Maintenance is done better for manual
test cases then test automation
•Good release management is key
17
Experience Transfer &
Maintenance
Achievements:
• Plan for transfer
• Plan time to update regression
test ware
• Train the maintenance
department
Preparation
Test
Organisation
Planning Issue
Management
Reporting
And
Implementation
advice
Transfer
And
Maintenance
Estimation
Risk Analysis
And
Test Strategy
TestManagementFile
Execution
Progress
Management
Risk & Requirement
Based Testing

@esconfs
#esconfs
18
Agenda
- Introduction
- 10 years later
- Conclusions

@esconfs
#esconfs
How do we see Agile?
• Agile is
• A method, an approach
• Highly demanding!
• A team effort
• Flexible
19

@esconfs
#esconfs
The customer
• What are the characteristics of an agile method?
• Increments/iterations
• ‘Reduce the waste’
• Continues feedback
• How do we trace these to our customers?
20

@esconfs
#esconfs
Agile is flexible
• The customer expects flexibility
• Scope decisions between iterations
• Good structure and light documentation
• Don’t be too flexible!
21

@esconfs
#esconfs
Challenges of Agile
• Team takes too many decisions
• Signals of testers are not recognized
• Testers are only seen as testers
• Not all team members are involved in
decision making
• Test environment not available fast enough
• New builds are not delivered flexible enough
22

@esconfs
#esconfs
Benefits of Agile
• More added value for the customer
• Highly flexible organization
• Better structured risk management
• High acceptance level of end users
23

@esconfs
#esconfs
The role of the Test Manager
• Plans, monitors and controls testing
• Typical tasks include:
• Define the test approach / strategy
• Contribute to other project activities
• Manage end to end tests
• Introduce suitable metrics for test progress & Quality control
• Write summary reports
24

@esconfs
#esconfs
Conclusions
• The Test Management Model still applies to
all projects
• Product risk assessment still difficult
• Agile will challenge the test manager to
adjust accents within the Test Management
model
• The test manager has to be more flexible:
result focus instead of process focus
25

@esconfs
#esconfs
Questions?

@esconfs
#esconfs
EuroSTAR Webinar Archive
Access almost 50 on-demand software testing webinars by the world's leading
testing experts. Topics include….
Agile, Automation, Test Management, Process Improvements, Risk-based Testing, Performance
Testing, Test Documentation, Cloud Testing, Session-based Testing, Weekend Testers
Visit the archive!
http://www.eurostarconferences.com/community/member/webinar-archive
@esconfs
#esconfs

@esconfs
#esconfs
The EuroSTAR Community.
Are you a member?
Did you know we have an online resource area with almost …
600 software testing presentations, 50 topical
webinars, over 40 videos and podcasts and 20 expert
eBooks
Do you know that membership is completely free and you can access these
premium testing resources anytime?
Join today!
http://www.eurostarconferences.com/community/
member
@esconfs
#esconfs

@esconfs
#esconfs
Call for Submissions to speak at the 2013 EuroSTAR
Conference is now open!
Speak at Europe’s #1 Software Testing Event
Theme: Questioning Testing Deadline for submissions February 13th
For more information http://www.eurostarconferences.com/speakers/conference-2013-call-
for-submissions

'Lessons Learned in Test Management' by Bob van de Burgt

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (7)

Ähnlich wie 'Lessons Learned in Test Management' by Bob van de Burgt

Ähnlich wie 'Lessons Learned in Test Management' by Bob van de Burgt (20)

Mehr von TEST Huddle

Mehr von TEST Huddle (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

'Lessons Learned in Test Management' by Bob van de Burgt

Hinweis der Redaktion