1. ATMM
Agile Testing Maturity Model
practical view
Shirly Ronen-Harel
shirly@agilesparks.com
http://il.linkedin.com/pub/shirly-ronen-harel/0/653/249
2010

2. How can we know, what are the best testing
agile practices for a specific team in a
specific point of time ?
When is the best time to use them?
Why does it sometime fail ?
What are the next steps of implementing the
agile testing methodologies?

3. Agile testing maturity model (ATMM)
will help you get some answers.

4. Traditional TMM
There is no doubt that testing matures !

5. Traditional TMM

6. Agile TMM
Testing maturity as a Team related maturity model
As the main guide
And
Traditional process maturity model
As the secondary guide

7. Maturing the testing activities
inside an agile team relates to
the current agile team
maturity level

8. expectation from testing
practices must be aligned with
team ability to execute them

9. This is a Diagnostic model of testing
maturity
Allow recognizing testing maturity level
and plan the next steps accordingly

10. L0 - Waterfall
0
L1 - Forming
1
L2 - Agile Bonding
2
L3 - Performing
3
L4 - Scaling
Agile TMM Building blocks : team maturity levels
4

11. Shu –Ha -Ri
"hold, break, leave" and illustrates the levels of learning of a person. -- DanielSvennberg.

12. • shu (守?) "protect", "obey" — traditional wisdom
— learning fundamentals, techniques, heuristics,
proverbs
• ha (破?) "detach", "digress" — breaking with
tradition — detachment from the illusions of self
• ri (離?) "leave", "separate" — transcendence —
there are no techniques or proverbs, all moves are
natural, becoming one with spirit alone without
clinging to forms; transcending the physical
© http://en.wikipedia.org/wiki/Shuhari

13. L0 - L1 - Forming – SHU L2 - Agile Bonding L3 - Performing - L4 - Scaling
Waterfall Ha RI
Start point : Team team starts to get used to Performing agile Team develop its own
starts to perform as new rules and start to bond team – mind set paste and methods
agile team , ceremonies as a team . agile scrum – level with continuous
and roles are set and Basic agile mind set is 2. improvement mindset.
team starts to run starting to emerge.
iterations.
Technical scrum / agile
Team learn the rules

14. Testing Procedures matures in
deferent testing categories
At a deferent paste

15. Building blocks – Testing related Categories
Testing mind set Testing iterations Automation Testing Planning Provide feedback
Testers speak the Testers speak the
Lead testing activities Defect management
domain language of the technical language of Exploratory testing
in the team
business the development team
Test cases Tasks Environment Repository Reporting/measures
Delivery to testing Regression Tractability Testing manager role Testing team role

16. Surly , there will be
procedures that we can not
achieve if we don’t achieve a
team high level maturity
performance first

17. Testing Procedures matures within the
team maturity levels , but , deferent
testing categories matures
At a deferent paste over the team
maturity levels.

18. Issue L0 - L1 - L2 - Agile L3 - L4 - Scaling
Waterfall Forming Bonding Performing
Testing mind set
Testing iterations
Automation testing - look at
automation maturity level
Planning
Provide feedback.
Testers speak the domain
language of the business.
Testers speak the technical
language of the development
team.
Lead testing activities in the
team
Exploratory testing
Defect management
Test cases
Tasks
Environment
Repository
Reporting/measures
Delivery to testing
Regression
Tractability
Testing manger
Testing team

19. Example
L1 - Forming
L4 - Scaling
L3 - Performing
L0 - Waterfall
L2 - Agile Bonding
0 1 2 3 4
Testing mind set Defects
Repository
Planning
Moving from one level to another require completion of
previous level issues

20. Testing mature as a
value related
concept.

21. The model can be used to scan the
testing maturity model separately
from fixed testing procedure
maturity with a strong relation to
other agile team needs.

22. We can also use testing
maturity model to evaluate
team maturity and promote
team maturity via testing.

23. Correlation with team maturity behavior model
Its also relevant, but for this
presentation we will not deal
with it into the details.

24. We need to identify the former slide maturity
stage in order to know how to approach the
team with promoting practices
1. Forming: - >Directing
2. Storming: - > Selling
3. Norming: - > Supporting
4. Performing: - > Delegating
© Forming Storming Norming Performing Developmental Model

25. Diagnostic model of testing maturity
Allow practically define each testing
category needed maturity level at a
specific point in time.
Allow progress separately on value testing
categories only.

26. Lets really start the session!
Practical view

27. Exercise :
Identify your Team/ Teams, each testing
category maturity phase
Use the blank table as an example .

28. Example (based on few real cases) :
R&D business group holds 4 agile teams
Team 1:
Team 2:
Team 3:
Team 4:
Identify in each team ,each testing category maturity phase

29. Issue L0 - Waterfall L1 - Forming L2 - Agile Bonding L3 - Performing L4 - Scaling
Testing mind set
Testing iterations
Automation testing
Planning
Provide feedback.
Testers speak the domain language of
the business.
Testers speak the technical language of
the development team.
Lead testing activities in the team
Exploratory testing
Defect management
Test cases
Tasks
Environment
Repository
Reporting/measures
Delivery to testing
Regression
Tractability
Testing manger
Testing team

30. Good visibility over team maturity/
Testing maturity.
next steps analysis is easy.

31. Exercise cont.
• What is your testing team maturity level
• What is the team maturity level?
• Who is the most/less mature team?
• Can you conclude over entire R&D maturity
level?
• What are the most common testing issues?
• What are the most problematic/good
issues?
• What can be your next step ?

32. Example (based on few real cases) :
The most mature agile team:
The less matured agile team:
R&D group testing maturity : 2Avr

33. Most Problematic testing area:
We can draw many more conclusion and even use
this model as a progress KPI
R&D can use this model and create improvement
plan:
•What value do we have changing each area?
•Who can assist (stronger teams?!)
•Who needs help?

34. Issue L0 - Waterfall L1 - Forming L2 - Agile Bonding L3 - Performing L4 - Scaling
Testing mind set
Testing iterations
Automation testing
Planning
Provide feedback.
Testers speak the domain language of
the business.
Testers speak the technical language of
the development team.
Lead testing activities in the team
Exploratory testing
Defect management
Test cases
Tasks
Environment
Repository
Reporting/measures
Delivery to testing
Regression
Tractability
Testing manger
Testing team

35. So what should the
R&D manager / testing
manger do?

36. Understand agile team
maturity model

37. Identify agile team testing and maturity stages
L1 - Forming
L4 - Scaling
L3 - Performing
L0 - Waterfall
L2 - Agile Bonding
0 1 2 3 4
Testing mind set Defects
Repository
Planning

38. Give management support

39. Understand the team behavioral maturity
model and make sure the ―how to
implement new practices ― fits the
team current needs

40. 1. Forming: 3. Norming:
* Formation of team happens & the team * Work as a team starts
comes together * Roles and responsibilities are clear and
* Members feel anxious and spend their accepted
time finding out about each other * Team begin to exhibit participative
* Individual roles and responsibilities are behavior & decision making happens by group
unclear agreement
* Highly depending on the manager/leader * Commitment, trust and unity increases
* Equivalent Situational Leadership style: * Equivalent Situational Leadership style:
Directing Supporting
2. Storming: 4. Performing:
* Team members come up with ideas through * This stage is characterized by high levels
debates on how to proceed with the task of:
- about task priorities; - goal orientation,
- clarity on purpose of the task; - interpersonal relations,
- roles & responsibilities and - independence, motivation,
- processes to follow - knowledge and
* Influence of ideas and power struggles - competence in team members
may arise * Team know what,why & how of the task
* Compromises may be required to enable they are executing
progress * High level of respect in the
* Team members may challenge the leader communication between team members
& leader coaches * Team expects delegation of task instead
* Equivalent Situational Leadership style: of instruction/assistance
Selling * Equivalent Situational Leadership style:
Delegating
© Forming Storming Norming Performing Developmental Model

41. Plan to mature the R&D group

42. Detailed model
There is a detailed model describing all
deferent patterns, practices in the various
agile maturity and testing maturity level
Please refer to shirly@agilesparks.com
Agilesparks.com
Agile sparks page On FaceBook

43. High level model
Enjoy

44. Issue L0 - Waterfall L1 - Forming L2 - Agile L3 - Performing L4 - Scaling
Bonding
Testing mind Mind set of separation. Testers starts being Involved in Whole team approach, Team develops a Testers Leads the
set Testers vs. developers. every aspect of the team: testers are working as mind set of team to a quality
meeting , ceremonies, kick part of an integrated understanding and concept :entire
offs, planning ,Cost estimation team. act upon testing team is aware and
closer to the events happening needs. active over quality
– in real-time. issues: bugs, user
stories, delays
In projects and
quality gaps.
Testing Testing is performed at the Hardening sprint at the end of Tests are identified Much small ongoing Performing Short
iterations End of development. series of development sprint. and produced as part iteration with and fast iterations of
And maybe at the end of of a story creation. developers including automated tests.
sprint. Tests are performed tasks testing . Value/Risk driven
for each user story. testing
Automation Recordings of tests Start dividing and Start Building the Ongoing automation Any developer can
testing . Mainly GUI related tests. understanding the automation testing coverage run automation
mainly effort and practice to the automation flow of levels as needed.
regression/progression use deferent levels of agile work according to
cases end to end scenarios. automation testing. Starting deferent automation
Unit tests are not a priority minimal non traditional levels.
and related to a high effort automation.
mind set concept.
Or no automation.
Testing Planning the Entire product Big deference in the planning Testing and Test Planning's are Live planning, high
Planning in advanced. session between the development planning visible to the level and detailed-
Big long term plans. discussion over testing tasks are coordinated. developers and using automation
Formal STP and development tasks which Whole team approach approved by the repository and
Formal STD are the focus. Still mind set of – team cost team. exploratory
Or , no plans at all separation between testing estimations (testing Responding to technique.

45. Issue L0 - Waterfall L1 - Forming L2 - Agile L3 - Performing L4 - Scaling
Bonding
Provide feedback. Quality Feedback is provided Confusion business feedback a Feedback context of user Continues,
Mostly during testing and Business feedback long with PO for a story , release approach automation ,
not during development. comes to PO at the user story rather then defects and business , unit
Most important feedback is end of iteration or /bug/priority tests approach. tests , CI , NB
considered to be the one at end of hardening ongoing feedback in any feedbacks
the end of development and sprint. User story , step of tasks , us,
end of release. business, release architecture
business feedback is development – technical
not completely and business.
delivered. Still
componential
feedback.
Testers speak the Depends on personal tester Testers Exposed as a Tester understands Understands the domain The team is also
domain language of abilities to communicate. company policy to deeply the tested outside team domain. involved and
the business. business issues. First product business. Represents the product takes the lead
company attention to inside the team. over the testing
the need to massively role of
expose testers to the Representing the
business on ongoing PO.
event : meeting,
training, daily
activities, information
sharing.
Testers speak the Depends on personal tester Testers are part of Testers understand Dev technical issues
technical language of abilities to communicate. development design. the developer's tasks reflect in testing tasks
the development Depends of tester exposal to Aware of most HL before and after and tests.
team. technical issues by technical issues. developing them.
management and personal
management skills of testing

46. Issue L0 - Waterfall L1 - Forming L2 - Agile L3 - Performing L4 - Scaling
Bonding
Exploratory testing Ad hoc testing considered, not Performing minor user Performing, planning Team is involved in
planed. stories and bugs exploratory, building performing and
exploratory testing the testing brain storming over
sessions. exploratory exploratory testing
repository.
Defect Usually controlled and managed Bugs are followed and Formal bug's PO and testers are
Management by the testing team or testing managed by the team management flow in the focal points.
manager. tester. the team, including Bugs are not the
Big bug status meetings and Bugs are still major Po involvement. Only quality mirror of the
reports. focus of project. 'important/high project
Bugs are left to the end priority /future
of iteration or release use/cross functions'
or to the next iteration bugs are reported.
or to hardening
iteration.
Test cases Heavy Documented. Confusion regarding Just enough tests Developers write
the testing test cases cases level of tests and adding
Tester and testing managers are writing documentation. tests on a 'need
the sole accounted for testing Starting the first steps only' basis.
scripts planning and execution. of risk based testing
Tasks Testing tasks are visible to the Testers have tasks in Testing tasks are part Testing tasks are
entire project to the details. the sprint. But yet , of all user stories and shared by other
Or testing time is considered separately evaluated. cost affected the team members.
infinite and testing holds sprint outcome.
overload of tasks that are not Testing tasks are

47. Issue L0 - Waterfall L1 - Forming L2 - Agile Bonding L3 - Performing L4 - Scaling
Environment Usually, separate testing confusion– Deliveries to staging and Testing environment is Full NB
environment controlled , disagreements of testing environment when set and has a major automated
managed and build by testers or where to test when needed only for system role of performing testing
IT. Not always up to date to small and what issues. and end 2 end or end uploads tests, system environment
development changes. Testing environment game tests and NB tests.
are still testers Entire team is responsible
problem. for its testing environment
maintenance.
Repository Usually big , detailed and heavy. Tests are organized Start thinking about Tests can be acceded
Managed by the testing team. in each team. general automation cross and run by all teams
product /teams repository
functional and not
componential related.
Reporting/measur Big status reports. Quality reports User story/epic related Integrated tests reports
es include also the user quality. to automated reports
story but still relays On the spot bugs status and working software
on bug's and tests handling instead of big and progress. . Regression
componential vast bus status reports and pass each iteration
coverage status. matrices.
Trend defects and progress
matrix.

48. Issue L0 - Waterfall L1 - Forming L2 - Agile L3 - Performing L4 - Scaling
Bonding
Delivery to testing At the end of development. Hardening at the end End of iteration Ongoing delivery to a On going PSP; CI , NB
of release. delivery. testing environment
Mini hardenings
Regression Performed at the end, Performed and Visible and Available to all teams Team performed
Performed many times. managed, usually documented and reported as part exploratory regression
Usually stopped. waiting to hardening Mini hardenings. of quality status as an ongoing task
session. Performed during
iterations and rest at
the end game.
Tractability Big tractability matrixes of Continue trace testing Tracing tests to a Tests are traced to Automated testing
tests to requirements-design in old faction way: to user story . functionality . repository with
–coding - builds requirements. minimum manual
functionality oriented
tractability
Testing manager Accountable to quality. Sigh Confusion. Inspect the testing Testing manager
role the release from What shell the manger team activities. deals with testing
development to product do/ Coaching testers vision , high level
organization. Defining the first process , procedures
boundaries of the and quality
testing manager role technology
within a development
organization
Testing team Separate from development Testing as an integral
role mind set. Testing is not development activity
considered a development
activity

49. Now, diagnose your own testing maturity
( c ) This model was created and developed by Shirly Ronen-Harel 2010
http://il.linkedin.com/pub/shirly-ronen-harel/0/653/249