A test strategy is the set of ideas that guides your test design. It's what explains why you test this instead of that, and why you test this way instead of that way. Strategic thinking matters because testers must make quick decisions about what needs testing right now and what can be left alone. You must be able to work through major threads without being overwhelmed by tiny details. James Bach describes how test strategy is organized around risk but is not defined before testing begins. Rather, it evolves alongside testing as we learn more about the product. We start with a vague idea of our strategy, organize it quickly, and document as needed in a concise way. In the end, the strategy can be as formal and detailed as you want it to be. In the beginning, though, we start small. If you want to focus on testing and not paperwork, this approach is for you.

1. MH
Half-day Tutorials
5/5/2014 8:30:00 AM
Rapid Software Testing:
Strategy
Presented by:
James Bach
Satisfice, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. James Bach
Satisfice, Inc.
James Bach is founder and principal consultant of Satisfice, Inc., a software testing and
quality assurance company. In the eighties, James cut his teeth as a programmer, tester,
and SQA manager in Silicon Valley in the world of market-driven software development. For
nearly ten years, he has traveled the world teaching rapid software testing skills and serving
as an expert witness on court cases involving software testing. James is the author
of Lessons Learned in Software Testing and Secrets of a Buccaneer-Scholar: How Self-
Education and the Pursuit of Passion Can Lead to a Lifetime of Success.

3. Rapid Software Testing:
Strategy
James Bach, Satisfice, Inc.
james@satisfice.com
www.satisfice.com

4. What is Rapid Testing?
Rapid testing is a mind-set
and a skill-set of testing
focused on how to do testing
more quickly,
less expensively,
with excellent results.
This is a general testing methodology.
It adapts to any kind of project or product.

5. The Premises of Rapid Testing
1. Software projects and products are relationships between people, who are
creatures both of emotion and rational thought.
2. Each project occurs under conditions of uncertainty and time pressure.
3. Despite our best hopes and intentions, some degree of inexperience,
carelessness, and incompetence is normal.
4. A test is an activity; it is performance, not artifacts.
5. Testing’s purpose is to discover the status of the product and any threats to
its value, so that our clients can make informed decisions about it.
6. We commit to performing credible, cost-effective testing, and we will inform
our clients of anything that threatens that commitment.
7. We will not knowingly or negligently mislead our clients and colleagues.
8. Testers accept responsibility for the quality of their work, although they
cannot control the quality of the product.

6. What is a test strategy?
Test strategy is the set of ideas that guide your choice of tests.
A set of ideas does not necessarily mean
a document. The test strategy may be
entirely in your head. Or it may be in
several heads, and emerge through
discussion, over time.
It may be documented partially on a
whiteboard or Post-Its or in a mindmap.
Or it could be in a formal document all
dressed like Cinderella at the royal ball.

7. What is a test strategy?
Test strategy is the set of ideas that guide your choice of tests.
To guide is to influence but not
necessarily to determine. Testing is
shaped by many factors in addition to
strategy, including opportunities, skills,
mistakes, time pressures, limitations of
tools, testability, and unconscious biases.

8. What is a test strategy?
Test strategy is the set of ideas that guide your choice of tests.
I mean choice in the most expansive sense of the word,
not simply the selection of existing test cases.
Choice of tests includes choices of what tests to design
and how to design them and all decisions made during
test design. It includes choices made during test
execution, too, including how to perform tests and
what mix of tests to perform in response to which
perceived risks.

9. Why have a test strategy?
If you test, then you already have a test strategy, so that’s
not a meaningful question… Here are some better questions:
Why have an explicit test strategy?
Why worry about your test strategy?
Why explain it? Why document it?
Why not let the tests “speak for themselves?”
1. To get more credibility, control, agility, and accountability, for
less time and effort.
2. Tests don’t talk.

10. (Test Strategy is a Middle Level Tool)

11. Strategic Thinking Begins with the Context
Test Team
Expertise
Loading
Cohesion
Motivation
Leadership
Project Integration
Test Lab
Test Platforms
Test Tools
Test Library
Problem Tracking System
Office Facilities
Product Mission
Stakeholders
Quality Criteria
Reference Material
Requirements
Product
Project Lifecycle
Project Management
Configuration Management
Defect Prevention
Development Team
Development
Mission
Find Important Problems
Assess Quality
Certify to Standard
Fulfill Process Mandates
Satisfy Stakeholders
Assure Accountability
Advise about QA
Advise about Testing
Advise about Quality
Maximize Efficiency
Minimize Cost
Minimize Time
Test
Process
Strategy
Logistics
Work-products

12. Ask yourself
 What testing is easy (even if not very important)?
 What testing is important (to find big bugs)?
 What testing is expected (by people who matter)?
 How can tools help?
 Have we advocated for testability?
 Are we minimizing administrative costs?

13.  Happy Path
 Tour the Product
– Sample Data
– Variables
– Files
– Complexity
– Menus & Windows
– Keyboard & Mouse
A quick test is a cheap test that has some value
but requires little preparation, knowledge,
or time to perform.
 Interruptions
 Undermining
 Adjustments
 Dog Piling
 Continuous Use
 Feature Interactions
 Click on Help
Cost as a Simplifying Factor
Try quick tests as well as careful tests

14.  Input Constraint Attack
 Click Frenzy
 Shoe Test
 Blink Test
 Error Message Hangover
A quick test is a cheap test that has some value
but requires little preparation, knowledge,
or time to perform.
 Resource Starvation
 Multiple Instances
 Crazy Configs
 Cheap Tools
Cost as a Simplifying Factor
Try quick tests as well as careful tests

15. Value (or Risk) as a Simplifying Factor
Find problems that matter
Instead of thinking pass vs. fail,
consider thinking problem vs. no problem.
 In general it can vastly simplify testing if we focus on
whether the product has a problem that matters,
rather than whether the product merely satisfies all
relevant standards.
 Effective testing requires that we understand
standards as they relate to how our clients value the
product.

16. One way to make a strategy…
1. Learn the product.
2. Think of important potential problems.
3. Think of how to search the product for those problems.
4. Think of how to search the product, in general.
Think of ways that:
 will take advantage of the resources you have.
 comprise a mix of different techniques.
 comprise something that you really can actually do.
 serve the specific mission you are expected to fulfill.

17. Test Strategy for DecideRight

18. Test Strategy Text

19. Risk-Based Test Project Cycle:
Testing itself is risk analysis.
Experience
Problems
In the Field
Analyze
Potential
Risks
Analyze
Actual
Risks
Short
loop
Long
loop
New
Project
ship
Exploratory
vs. ScriptedPerform
Appropriate
Testing
Experience
Problems &
Potentialities

20. 18
Tests
Project
Environment
Product
Elements
Quality
Criteria
Perceived
Quality
A Heuristic Test Strategy Model

21. 19
Project
Environment
Tests
Product
Elements
Quality
Criteria
Perceived
Quality
A Heuristic Test Strategy Model

22. Example: Medical Device