7 bullets to help create a test plan that adds value with little effort

1. Test Plans
Simplicity
1
2013-11-12
PA1
Confidential

2. Introduction
•
In this presentation I will use James Bach’s definition of
a test plan [1]
•
Test Plan: the set of ideas that guide a test project
•
Test Strategy: the set of ideas that guide test design
•
Test Logistics: the set of ideas that guide the application of
resources to fulfill a test strategy
•
Every artifact created must add value
•
The process of creating a test plan adds value, but the
document itself is of limited value
•
2
2013-11-12
PA1
The value of a test plan is the ability to communicate the
set of ideas that guides the test project to different
stakeholders
Confidential

3. Simplicity
•
•
We also want to limit the amount of effort we spend on
creating the test plan, and minimize the amount of
unnecessary artifacts created
•
2013-11-12
There is a value in keeping the test plan simple which
relates to the burden which the test plan puts on
someone trying to explain or understand it
•
3
Simplicity is the state or quality of being simple [2]
On Google and Microsoft there is a concept of a 10
minute test plan, which should in accordance with the
name only take a short period of time to create [3]
PA1
Confidential

4. Simplicity
Cost of Simplicity
Value of Simplicity
Complicated
4
2013-11-12
PA1
Simple
Confidential

5. Dynamic Test Plan
•
•
2013-11-12
Static information can be moved from a test plan to a
Test Strategy or a Test Logistics document
•
5
The key to reducing the size of the test plan is to
remove anything that cannot change over time
Anything that does not add value to any stakeholder
should also be removed
PA1
Confidential

6. Content of a Test Plan
•
•
Definition of Done
•
System Risk Matrix
•
Test Activities
•
Test Schedule
•
2013-11-12
System Configuration Under Test
•
6
Test Ownership
Self-Learning Loop (SLL)
PA1
Confidential

7. Overview
Ownership - Test Configuration - Definition of Done - Activity Description
Identify Risks
7
2013-11-12
PA1
Plan Test
Activities
Self-Learning
Loop
Confidential

8. Test Ownership
•
Which person is responsible for performing what tests?
•
This could be done on a overview scope level, for
different test charters, or for specific test cases
•
A test ownership for a specific person or group could be
defined in the following ways:
•
•
Performance
•
PA1
All or specific charters related to performance
•
2013-11-12
All performance tests
•
8
250 specific performance test cases
Running performance tests during a specific time period
or project phase
Confidential

9. System Configuration Under
Test
•
Most systems can be configured in many different ways
•
It is necessary to describe the configuration under test
in order to
•
•
2013-11-12
PA1
Divide test ownership between different configurations
•
9
Write bug reports
•
•
Reproduce tests
Let all testers know what to test on
This can change over time depending on the
configurability of the system
Confidential

10. Definition of Done
•
It is necessary to have a Definition of Done [4] for
different phases of the test project
•
This is used to define when testing stops as implied by
the name
•
Definition of Done should include:
•
•
2013-11-12
PA1
Expected Risk Level
•
10
Expected Test Coverage
•
•
Expected System Quality
Formal Documentation Needed
Definition of Done can change over time dependant on
stakeholder needs
Confidential

11. System Risk Matrix
•
What risks threaten the system from reaching the
definition of done?
•
These risks change continuously over the life cycle of the
project
•
Risks can include (but are not limited to):
•
•
Hardware delta between configurations
•
Unknown system dependencies
•
Unknown environment dependencies
•
PA1
Software delta between configurations
•
2013-11-12
New hardware builds
•
11
Software changes
Changed stakeholder focus
Confidential

12. Risk Quantification
•
The Risk Matrix needs to be filled quantifiable risks
•
There are many different models available
•
Severity - Occurrence - Detection [5] is one
•
Low risk, Medium risk, High risk, is another
•
•
12
2013-11-12
If a system has many configurations, each
configuration needs a column in the risk matrix
Risks most likely also need to be categorized in a good
way to make the matrix usable
PA1
Confidential

13. Test Activities
•
•
2013-11-12
The scope of these test activities should not be
static, since it is seldom efficient to run static activities
•
13
All different test activities need to be explained in the
test plan
Different activities can be added and removed during
the life cycle of the project
PA1
Confidential

14. Test Schedule
•
•
14
2013-11-12
A schedule or plan of when each test activity should be
executed in time
The schedule should be dynamic [6] and based on input
from what is happening in the project, and what has
happened during previous test activities
PA1
Confidential

15. Self-Learning Loop (SLL)
•
•
2013-11-12
Continuous improvement of gaps in test scope
•
15
It is critical to describe how the test plan will evolve
over the project life cycle, and then continuously
document what we have learned from this feedback
loop
Continuous improvement of understanding of system
dependencies
PA1
Confidential

16. Conclusion
•
•
2013-11-12
Test plans should be dynamic – static parts should be
included in other documents which can be reused
between projects, or not documented at all
•
16
Test plans should add value and not include any waste
These 7 bullets help include critical parts in the test
plan, and not introduce unnecessary waste
PA1
Confidential

17. References
[1] A question about test strategy
http://www.satisfice.com/blog/archives/63
[2] Simplicity
http://en.wikipedia.org/wiki/Simplicity
[3] 10 Minute Test Plan
http://googletesting.blogspot.se/2011/09/10-minute-test-plan.html
[4] Definition of Done
https://www.scrum.org/Resources/Scrum-Glossary/Definition-of-Done
[5] Severity, Occurrence, and Detection Criteria for Process FMEA
http://www.fmeainfocentre.com/guides/ProcessPktNewRatings.pdf
[6] Dynamic Test Plans
http://www.slideshare.net/JohanHoberg/dynamic-test-plans
17
2013-11-12
PA1
Confidential