Short summary about implementing testing strategies based on the book "continuous delivery" by Jez Humble & David Farley

1. Implementing a
Testing Strategy
Chapter 4

2. Who is in charge of testing and when to
do it.
 Testing is a cross-functional activity that involves the whole team, and should
be done continuously from the beginning of the project. Building quality in
means writing automated tests at multiple levels (unit, component, and
acceptance) and running them as part of the deployment pipeline, which is
triggered every time a change is made to your application, its configuration,
or the environment and software stack that it runs on.
 Testers collaborate with developers and users to write automated tests from
the start of the project. These tests are written before developers start work
on the features that they test. Together, these tests form an executable
specification of the behavior of the system, and when they pass, they
demonstrate that the functionality required by the customer has been
implemented completely and correctly.

3. What is the purpouse of a testing
strategy?
 The design of a testing strategy is primarily a process of identifying and
prioritizing project risks and deciding what actions to take to mitigate them.
A good testing strategy has many positive effects. Testing establishes
confidence that the software is working as it should, which means fewer bugs,
reduced support costs, and improved reputation.
 Testing also provides a constraint on the development process which
encourages good development practices. A comprehensive automated test
suite even provides the most complete and up-to-date form of application
documentation, in the form of an executable specification not just of how the
system should work, but also of how it actually does work.

4. Types of tests

5. Business-Facing Tests That Support the
Development Process
 The tests in this quadrant are more commonly known as functional or
acceptance tests. Acceptance testing ensures that the acceptance criteria for
a story are met. Acceptance tests should be written, and ideally automated,
before development starts on a story.
 Acceptance tests are critical in an agile environment because they answer the
questions, “How do I know when I am done?” for developers and “Did I get
what I wanted?” for users.

6. Technology-Facing Tests That Support
the Development Process
 These automated tests are written and maintained exclusively by developers.
There are three kinds of tests that fall into this category: unit tests,
component tests, and deployment tests.
 Unit tests test a particular piece of the code in isolation. For this reason, they
often rely on simulating other parts of the system using test doubles.
 Unit tests should not involve calling the database, using the filesystem,
talking to external systems, or, in general, interaction between components
of a system.
 Component tests test larger clusters of functionality, so that they can catch
problems like these. They are typically slower, since they can require more
involved setup and perform more I/O, talking to databases, the filesystem, or
other systems. Sometimes, component tests are known as “integration tests”

7. Technology-Facing Tests That Support
the Development Process
 Deployment tests are performed whenever you deploy your application. They
check that the deployment worked—in other words, that your application is
correctly installed, correctly configured, able to contact any services it
requires, and that it is responding.

8. Business-Facing Tests That Critique the
Project
 These manual tests verify that the application will in fact deliver to the users
the value they are expecting. This is not just a matter of verifying that the
application meets its specifications; it is also about checking that the
specifications are correct.
 A particularly important form of business-facing, project-critique tests are
showcases. Agile teams perform showcases to users at the end of every
iteration to demonstrate the new functionality that they have delivered.

9. Technology-Facing Tests That Critique
the Project
 Although users rarely spend a lot of time specifying capacity and security
characteristics up front, they will certainly be very upset if their credit card
details are stolen or if a website is constantly down due to capacity problems.
 These tests often require considerable resources such as special environments
to run on and specialized knowledge to set up and implement, and they often
take a long time to run (whether or not they are automated). Therefore, their
implementation tends to be deferred. Even when they are fully automated,
they tend to be run less frequently and further down the deployment pipeline
than the functional acceptance tests.

10. Real-Life Situations and Strategies:
New Projects.
 At this stage, the cost of change is low and, by establishing some relatively
simple ground rules and creating some relatively simple test infrastructure,
you can give a great start to your process of continuous integration. In this
situation, the important thing is to start writing automated acceptance tests
from the very beginning.

11. Real-Life Situations and Strategies:
Midproject
 Although it’s always pleasant to be starting a project from scratch, the reality
is that we often find ourselves working on a large, resource-starved team
developing a rapidly changing codebase, under pressure to deliver.
 The best way to introduce automated testing is to begin with the most
common, important, and high-value use cases of the application. This will
require conversations with your customer to clearly identify where the real
business value lies, and then defending this functionality against regressions
with tests. Based on these conversations you should automate happy path
tests that cover these high-value scenarios.

12. Real-Life Situations and Strategies:
Integration
 If your application is conversing with a variety of external systems through a
series of different protocols, or if your application itself consists of a series of
loosely coupled modules with complex interactions between them, then
integration tests become very important.
 It is essential to ensure that you don’t hit a real external system unless you
are in production, or you have some way of telling the service that you are
sending it dummy transactions for testing purposes.

13. Real-Life Situations and Strategies:
Process
 The production of acceptance tests can be an expensive and even laborious
task if communication between the team members isn’t effective. Many
projects rely on testers examining upcoming requirements in detail, going
through all possible scenarios, and designing complex test scripts they will
follow later. The results of this process might be sent to the customer for
approval, following which the tests are implemented.
 There are several points at which this process can be very simply optimized.
We find that the best solution is to have a single meeting with all of the
stakeholders at the beginning of each iteration, or about a week before a
story will start development if you’re not using iterations.

14. Summary
 In many projects, testing is treated as a distinct
phase carried out by specialists. However, high-
quality software is only possible if testing
becomes the responsibility of everybody involved
in delivering software and is practiced right from
the beginning of the project and throughout its
life.