This document summarizes Conor Fitzgerald's presentation on Modern Testing principles. The presentation outlines 7 principles of Modern Testing: 1) Business First, 2) Accelerate the Team, 3) Always Learn and Improve, 4) Lead a Quality Culture, 5) The Customer is King, 6) Data Trumps Intuition, and 7) Everyone Can Test. For each principle, Conor provides an explanation, example, and suggestion for applying the principle. He discusses both challenges in implementing the principles and key learnings. The overall message is that Modern Testing principles can help teams accelerate achieving shippable quality.
1. @conorfi
Modern Testing Principles
The Evolution of Quality in Agile teams
Based on the work of Alan Page and Brent Jensen,
who have built on the foundations of many others.
Conor Fitzgerald: Head of Testing – Poppulo
https://conorfi.com/talks/
3. @conorfi
Modern Testing (MT) Podcast
Talking about similar topics of interest
7 Concise and clear Principles
Community - Slack OneOfTheThree
Influenced by similar books
5. @conorfi
What is Modern Testing (MT)?
“It’s not very modern, and it’s not really about testing”
Lean Principles and Quality
“Modern Testing is the Antidote to Traditional Testing Methods”
6. @conorfi
Why Modern Testing (MT)?
Shipping faster (Agile, DevOps, CD)
Technology has increased competition
Lower Customer switching costs(friction free)
Traditional Testing viewed as a Cost and Delay
“Accelerate the Achievement of Shippable Quality”
8. @conorfi
What are the MT Principles?
Principles In Brief - TestBash Home Talk by Alan Page
Everyone can test
Lead a Quality Culture
Business First
Accelerate the Team2
1
Always Learn and Improve3
4
The Customer is King5
Data trumps intuition6
7
9. @conorfi
My Journey with the MT Principles
1. Each principle explained
2. A real world example
3. A suggestion for you to try
10. @conorfi
1. Business First
“Our priority is improving the business.”
“A good test mind applied to a feature
team is a value-add, not a cost.”
17. @conorfi
2. Accelerate the Team
“We accelerate the team, and use
models like Lean Thinking and the
Theory of Constraints to help identify,
prioritize and mitigate bottlenecks from
the system”
21. @conorfi
3. Always Learn and Improve
“We are a force for continuous
improvement, helping the team adapt
and optimize in order to succeed,
rather than providing a safety net to
catch failures.”
24. @conorfi
4. Lead a Quality Culture
“We care deeply about the quality
culture of our team, and we coach,
lead, and nurture the team towards a
more mature quality culture.”
27. @conorfi
Quality Culture Definition
“Shared Mindset that delivering high
quality to our customers is our top
priority, and that all our practices
support this effort”
Alan Page
28. @conorfi
Quality Culture Transition Guide
A framework that teams can use to drive discussion
Not a Capability Maturity Model
Focus is more on areas for improvement than levels
9 Capabilities and 4 levels
http://bit.ly/QCTGuide
29. @conorfi
Chaos Growing Competent Optimizing
Testing Breadth X
Quality and Test Ownership X
Technical Debt and
Maintenance
X
Code Quality and Tools X
Customer Data Analysis X
Development Approach X
Learning & Improvement X
Customer Success X
Leadership Emphasis X
32. @conorfi
5. The Customer is King
“We believe that the customer is the
only one capable to judge and
evaluate the quality of our product”
33. @conorfi
Controversial?
“The Customer is the only one that knows
whether or not the product is solving their
problem.”
“Quality is customer happiness. (not leads
to… is)”
Brent Jensen https://testastic.wordpress.com/
36. @conorfi
6. Data Trumps intuition
“We use data extensively to deeply
understand customer usage and then
close the gaps between product
hypotheses and business impact.”
37. @conorfi
Data can help you answer
Before we update this feature, how many customers are
using it?
If usage is low can we freeze or remove the feature?
When we deliver this feature what does success look like?
Can we create a user journey to represent success?
39. @conorfi
7. Everyone can Test
“We expand testing abilities and
knowhow across the team;
understanding that this may reduce
(or eliminate) the need for a
dedicated testing specialist.“
40. @conorfi
Controversial?
NO – Context Based
Wording: reduce (or eliminate)
If you successfully implement the
principles, you will always be in
demand
42. @conorfi
Exploratory Testing
“Tested = Checked and Explored”
Explore It!
Elisabeth Hendrickson
Checking(Binary) - Known Unknowns (automatable)
Exploring(Open Ended) - Unknown Unknowns
A thoughtful approach which involves “simultaneous
learning, test design, and test execution”
Cem Kaner
43. @conorfi
Suggestion
If your team has a tester, what would
happen if they stopped testing?
What happens when they go on
holidays?
46. @conorfi
Learnings
Principles are a guide, not rules
Testers can be a force multiplier
Shared Ownership of Quality - no safety net
Importance of Collaboration & Communication
48. @conorfi
Conclusion
Testing is not Dead, it’s Evolving
MT is another Evolution
Complements and builds on other Principles
“A good test mind applied to a feature team
is a value-add, not a cost.”
49. @conorfi
Could the Modern Testing
Principles help your team
Accelerate the Achievement of
Shippable Quality?
52. @conorfi
References
Introduction to Modern Testing
https://www.ministryoftesting.com/dojo/courses/introduction-
to-modern-testing-alan-page
Modern Testing Podcast
https://www.angryweasel.com/ABTesting/
Modern Testing Mind Map
https://github.com/mwyrodek/ModernTestingMindMap
53. @conorfi
Recommendations
Lessons Learned in Software Testing: A Context-Driven Approach. Cem Kaner, James
Bach, Bret Pettichord
Explore It! Reduce Risk and Increase Confidence with Exploratory Testing. Elisabeth
Hendrickson
More Agile Testing: Learning Journeys for the Whole Team. Janet Gregory and Lisa
Crispin
Perfect Software: And Other Illusions about Testing. Gerald M. Weinberg
Leading Quality: How Great Leaders Deliver High-Quality Software and Accelerate Growth:
Owais Peer and Ronald Cummings-John
A Practical Guide to Testing in DevOps. Katrina Clokie
54. @conorfi
Recommendations
Kanban in Action. Marcus Hammarberg and Joakim Sundén
The Goal: A Process of Ongoing Improvement. Eliyahu M Goldratt
The Lean Startup. Eric Ries
The Phoenix Project. by Gene Kim, Kevin Behr and George Spafford
Accelerate: The Science of Lean Software and Devops. by Nicole Forsgren , Jez Humble
and Gene Kim
The Devops Handbook: How to Create World-Class Agility, Reliability, and Security in
Technology Organizations by Gene Kim, Patrick Debois, John Willis
Team Topologies: Organizing Business and Technology Teams for Fast Flow by Manuel
Pais and Matthew Skelton
Hinweis der Redaktion
Feature toggles help mitigate risk, customers derive value when the feature toggle is enabled, and we directly/indirectly gain revenue
But when feature toggles are not managing correctly, the can become part of our tech debt and in some case delay delivery to customers
WIP = 2
No customers - 2
All customers = 6
Some customers = 6
Benefits
Deliver value to customer
Reduce tech debt
Working together
Walking the board Right to left
Reduced WIP and in particular new work
Tester not a safety net
More work done
Retros were stale, a few dominant voices
3 Rules for a Retro
1. Blameless
Deal with any blame & move on. Then focus on team successes & areas for improvement
2. Facilitation
Ensure everyone has an opportunity for input and timebox each topic
3. Variety
Regularly change the format, location etc.
Code Correctness != Quality
Bugs != Quality
Code Correctness != Quality
Bugs != Quality
Quality consists of 3 things: value (usefulness), goodness (emotive experience), and correctness()
You are not the customer
We must talk to customers .. building the right product to solve their problems
Qualitative:
Beta Customers Calls
Shadowing Support
Production Issues(Support)
Quanantive: Analytics Events - next principle