Weitere ähnliche Inhalte Ähnlich wie Seven Keys to Navigating Your Agile Testing Transition (20) Kürzlich hochgeladen (20) Seven Keys to Navigating Your Agile Testing Transition1. MF
AM Tutorial
4/29/13 8:30AM
Seven Keys to Navigating Your Agile
Testing Transition
Presented by:
Bob Galen
RGalen Consulting
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
2. Bob Galen
Bob Galen is an agile coach at RGalen Consulting and director of agile solutions at Zenergy
Technologies, a North Carolina-based firm specializing in agile testing and leading agile adoption
initiatives. Bob regularly speaks at international conferences and professional groups on topics related to
software development, project management, software testing, and team leadership. He is a Certified
Scrum Master Practicing (CSC), Certified Scrum Product Owner (CSPO), and an active member of the
Agile Alliance and Scrum Alliance. Bob published Scrum Product Ownership–Balancing Value from the
Inside Out, which addresses the gap in guidance toward effective agile product management. Contact Bob
at bob@rgalen.com or bob.galen@zenergytechnologies.com.
3. Keys for Transitioning to Agile
Testing
Myths & Realities from the Trenches
Bob Galen
President & Principal Consultant
RGCG, LLC
bob@rgalen.com
Fifteen
Seven Keys for Navigating your
Agile Testing Transition
Myths & Realities from the Trenches
Bob Galen – RGCG
bob@rgalen.com
Mary Thorn – Deutsche Bank
mary.thorn@gmail.com
1
4. Introduction
Bob Galen
Somewhere ‘north’ of 30 years experience ☺
Various lifecycles – Waterfall variants, RUP, Agile, Chaos
Various domains – SaaS, Medical, Financial Services, Computer
& Storage Systems, eCommerce, and Telecommunications
Developer first, then Project Management / Leadership, then
Testing
Leveraged ‘pieces’ of Scrum in late 90’s; before ‘agile’ was ‘Agile’
Agility @ Lucent in 2000 – 2001 using Extreme Programming
Formally using Scrum since 2000
Currently an independent Agile Coach (CSC – Certified Scrum
Coach, one of 50 world-wide; 20+ in North America)
at RGCG, LLC and Director of Agile Solutions at Zenergy Technologies
From Cary, North Carolina
Connect w/ me via LinkedIn and Twitter if you wish
Bias Disclaimer:
Agile is THE BEST Methodology for Software Development
However, NOT a Silver Bullet!
Copyright © 2013 RGCG, LLC
3
Introduction
Mary Thorn
A VP of QA and Test Manager at Deutsche Bank
Global Technologies in Cary, North Carolina,
Mary Thorn has a broad testing background that spans
automation, data warehouses, and web-based systems
in a wide variety of technologies and testing
techniques.
During her more than fifteen years of experience in
healthcare, HR, agriculture, and SaaS-based products,
Mary has held manager and contributor level positions
in software development organizations.
She has a strong interest in agile testing methodologies
and direct experience leading agile teams through
Scrum adoption & beyond.
Copyright © 2013 RGCG, LLC
4
2
5. Outline – Myths & Realities
Introduction
1. Transforming your
Team
2. Automation
3. Developers &
Automation
4. Developers Testing
5. Test Planning & Scripts
6. Testing within the Sprint
7. Exploratory Testing
8. Role of Testers
9. Developer to Tester
Workflow
10. Managing Agile Testers
11. Test Metrics
12. Retrospectives – The
Secret Sauce
13. Continuous Improvement
14. The Customer
15. Agile Requirements – The
Product Backlog
Copyright © 2013 RGCG, LLC
5
#1, Transforming your team
Myth: You need all programmers or highly technical
testers when you move to agile
Reality: A mix is best –
Manual, domain-centric and technical skills
Some programming / scripting skills
Soft / collaborative skills
Reality: And throw out all of that Developer-to-Tester
ratio ‘stuff’.
Copyright © 2013 RGCG, LLC
6
3
6. #2, Automation
Myth: You need 100% automation to start
agile testing.
Reality: You simply need to have a strategy AND
doggedly pursue automation where it makes sense
Make it part of the Backlog and work it every sprint
Reality: There are some excellent Open Source tools
that supplement agile automation development
Reality: The Agile Test Automation Pyramid is the right
overall strategy
Copyright © 2013 RGCG, LLC
7
Agile Test Automation Pyramid
Mike Cohn; Lisa Crispin & Janet Gregory
http://behaviordrivendevelopment.wikispaces.com/Testing
Copyright © 2013 RGCG, LLC
8
4
7. #3, Developers & Automation
Myth: QA designs, writes & runs all of the test
automation
Reality: Everyone should be responsible for automation
Developers need to minimally attend to Unit Level
Participate in any framework or re-use development
Writing ‘glue’ code – fixtures, step files, etc.
Reality: It also extends into your Build & Continuous
Integration systems
All automation should be ‘wired’ into CI
Dashboards, trending, lava lamps, etc. for all to see
Copyright © 2013 RGCG, LLC
9
#4, Developers Testing
Myth: Developers can’t test their own code—they’re not
independent enough nor skilled enough to do it properly.
Reality: We need to stop stereotyping team members,
their strengths and their abilities.
Developers can absolutely test their own code.
Some are better at it than others
Pair with them to help test appropriately
Copyright © 2013 RGCG, LLC
10
5
8. #5, Test Planning & Scripts
Myth: You don’t need to plan
(it just happens
)
and you don’t need functional test cases
(automation takes care of everything )
Reality: Plans help the team focus on the risk-based
testing required within an iteration AND across a release
Reality: Scripts (test cases) help formalize and drive
your testing;
Absolutely required in regulatory environments
Reality: You’ll never actually automate every test
Copyright © 2013 RGCG, LLC
11
#6, Testing within the Sprint
Myth: You simply need to run 100%
of the tests within the constraints of the Sprint
“Agile”
that’s
Reality: Rarely possible in most contexts.
You first need a high-degree of automation and business support
(for example: equipment costs)
Very mature test automation and CI / CD environments
Reality: Most agile teams adopt some sort of risk-based
testing approach for within the sprints
Dealing with Technical Test Debt
Then leverage Hardening / Stabilization pre-release sprints
Copyright © 2013 RGCG, LLC
12
6
9. The Agile Release Train
Synchronized
Internal
Release
Team 1
Iterate Iterate Harden
Continuous
Team 2
Iterate Iterate Harden
Continuous
Team 3
Iterate Iterate Harden
Continuous
Team 4
Iterate Iterate Harden
Continuous
External
Release
Iterate Iterate Iterate
Integration
Docs,
Training,
X-team Support,
Integration
Harden UAT,
Comp.
Iterate Iterate Iterate
Iterate Iterate Iterate
Integration
Iterate Iterate Iterate
Integration
Team n
Copyright © 2013 RGCG, LLC
13
The Agile Release Train
Example: eCommerce / SaaS Model
10 days
Team 1
10 days
Iterate
Iterate
5 + 2 days
External
Release
Harden
Continuous Integration
Team 2
Iterate
Iterate
Harden
X-team
Harden
Rinse
Harden
Docs,
Training
Repeat
Continuous Integration
Team 3
Iterate
Iterate
&
Continuous Integration
Team 4
Team 8
Iterate
Iterate
Harden
Continuous Integration
Environment
Evolution
Dev + QA
Copyright © 2013 RGCG, LLC
Dev + QA
QA -> Staging
Production
14
7
10. The Agile Release Train
Example: iContact / SaaS Model
3 weeks / 15 days
SBET, Exploratory –
Regression Testing
4-5 days
Production
Release
Team 1
Iterate
Harden
Continuous Integration
Team 2
Iterate
Harden
X-team
Harden
Continuous Integration
Team 3
Iterate
Harden
Docs,
Training
Rinse
&
Repeat
Continuous Integration
Team 4
Team 10
Iterate
Harden
Continuous Integration
Environment
Evolution
Dev + QA
QA -> Staging
Production
Copyright © 2013 RGCG, LLC
15
Brainstorm…
“Your” Agile Release Train
Either individually or in small groups from the same
company
Take a few minutes and think about your current release
constraints
timing, customers, domain, competition, # of teams, technology,
etc.
Design a release train model for your organization
Overlay it with testing activities, plans, and milestones
Present it to your larger table/group; gain feedback &
adjust
Be prepared to share
Copyright © 2013 RGCG, LLC
16
8
11. #7, Exploratory Testing
Myth: There is no place for Session Based Exploratory
Testing in agile contexts.
Reality: ET and SBET are a beautiful complement to
agile testing.
Helping nurture pairing & collaboration across teams and
functions
Defining new (more valuable) test cases
Quickly gaining quality & usability feedback
Copyright © 2013 RGCG, LLC
17
#8, Role of Testers
Myth: That the testers alone own quality & testing
practices within each team and sprint
Reality: The testers foster a “Whole Team” view
towards quality—focusing less on “Testing” and more on
“Quality Practices & the Customer”
Serving as guides for the team; Testing the “hard bits”
Facilitating exploratory testing sessions—finding more
interesting / valuable tests
Working with the Product Owners—are we solving the
customers problems?
Copyright © 2013 RGCG, LLC
18
9
12. #9, Developer to Tester Workflow
Myth: There is always a hand-off from developers to
testers; usually quite late in the sprint. That’s simply the
“way of things” in software development.
Reality: Scrummer-fall is alive and well but, Wrong!
Teams need to swarm on their work, as flow &
throughput matter the most.
WIP limits and close proximity / collaboration help establish a
healthy tempo of developer & tester pairing
Micro-handoffs – testing as development progresses!
Do you log bugs? Or do you fix bugs?
Copyright © 2013 RGCG, LLC
19
#10, Managing Agile Testers
Myth: The functional test manager is
in charge of deciding how, who, when , etc. for the test
team.
Reality: You still absolutely need functional leadership
within agile teams;
However, it’s focused towards quality practices, strategy &
coaching, and handling impediments / escalations
Encouraging transparency, transforming metrics & reporting
Supporting & protecting the teams
Encouraging risk-taking, innovation & creativity (Slack Time)
Copyright © 2013 RGCG, LLC
20
10
13. Levels of Done-Ness Criteria
Activity
Criteria
Basic Team
Work Products
Done’ness criteria
User Story or
Theme Level
Acceptance Tests
Sprint or
Iteration Level
Done’ness criteria
Release Level
Release criteria
Example
Pairing or pair inspections of code prior to check-in; or
development, execution and passing of unit tests.
Development of FitNesse based acceptance tests with the
customer AND their successful execution and passing.
Developed toward individual stories and/or themes for sets
of stories.
Defining a Sprint Goal that clarifies the feature
development and all external dependencies associcated with
a sprint.
Defining a broad set of conditions (artifacts, testing
activities or coverage levels, results/metrics, collaboration
with other groups, meeting compliance levels, etc.) that IF
MET would mean the release could occur.
Copyright © 2013 RGCG, LLC
21
#11, Test Metrics
Myth: You can and should move
forward reporting everything exactly as
you have before.
Including any ‘dysfunctional’ metrics that your process and/or
PMO dictates.
Reality: The metrics should change immediately.
From QA and Test centric towards Team-Centric metrics (Value,
Throughput, Quality, Team)
Stop reporting out on “Testing”; it’s irrelevant!
This effects planning as well—estimation, progress, risk, etc.
Contribute quality-centric Information radiators to the mix
Copyright © 2013 RGCG, LLC
22
11
14. Brainstorm…
Morphing your Metrics
Get together in small groups of
of 4-6.
What are you measuring today? Why?
How are they driving your success and behaviors?
As you move to agile, what can/should you be
measuring in the 4 key areas:
Value, Quality, Throughput & Predictability, and Team
How will you change your existing metrics? What
behaviors are you trying to inspire?
Copyright © 2013 RGCG, LLC
23
#12, Retrospectives:
The Secret Sauce
Myth: Testers are “Second Class” citizens who don’t
play an active part in the project & team
Reality: There are many places to “make a difference”
Getting the 800 lb. Gorillas out on the table; Showing courage;
telling truth
Fostering continuous improvement within the team
Setting the example; showing vulnerability—admitting you’re
wrong
Team listening; active planning; dependencies; pairing
Risk-taking; Failure!
Copyright © 2013 RGCG, LLC
24
12
15. #13, Continuous Improvement
Myth: We’re generally ‘stuck’ in our
approaches so just accept them and do
the “best you can”.
Reality: Continuous improvement is everyone’s
responsibility—to engage, suggest, take ownership of
current results, explore root causes, etc.
Active participation in your teams Retrospectives is a key way to
guide quality, testing, and customer-centric improvements.
Courage!
Copyright © 2013 RGCG, LLC
25
#14, The Customer
Myth: Business Analysts capture
customer requirements and testers test them for
completeness.
Reality: You need to begin to partner with the Customer
– Stakeholders – Product Owners to produce software
that solves the their problems.
Move to the “front” and help define & refine User Stories with your
Product Owner
Actively participate in Sprint Reviews
Show value for automation; placing test investments in the
Backlog
Copyright © 2013 RGCG, LLC
26
13
16. #15, Agile Requirements –
The Product Backlog
Myth: We can’t start testing until the requirements are
finished or stable; no matter how ‘agile’ we are.
Reality: Hogwash! Get over it
Ambiguity and incompleteness need to become your friend and
ally.
As does working with your Product Owners and Customers to
help define the requirements
Realizing that the requirements (User Stories) are only complete
at the end of each sprint.
Copyright © 2013 RGCG, LLC
27
Agile Test Transformation Strategy:
3 Pillars of Agile Quality
Copyright © 2013 RGCG, LLC
28
14
17. 3 Pillars of Agile Quality
Development &
Test Automation
•
Pyramid-based
Strategy: (Unit +
Cucumber + Selenium)
Software Testing
•
Risk-based testing:
Functional & NonFunctional
•
•
•
Test planning @
Release & Sprint levels
•
Exploratory Testing
•
Standards – checklists,
templates, repositories
•
Balanced across
manual, exploratory &
automation
•
•
•
Agile-centric Metrics
Continuous Integration
Attack legacy technical
debt in the Backlog
Visual Feedback –
Dashboards
Actively practice ATDD
and BDD
Cross-Functional
Team Practices
•
Team-based Pairing
•
Stop-the-Line
•
Code Reviews &
Standards
•
•
Active Done-Ness
•
Aggressive Refactoring
•
User Stories – 3 Amigo
based Conversations
•
Building the ‘Right’
Solutions
29
Copyright © 2013 RGCG, LLC
3 Pillars of Agile Quality
Development &
Test Automation
•
Pyramid-based
Strategy: (Unit +
Cucumber + Selenium)
•
Continuous Integration
•
Attack legacy technical
debt in the Backlog
•
Visual Feedback –
Dashboards
•
Actively practice ATDD
and BDD
A central part of agile adoption is focusing on CI, 3tiered automation development, and Dashboards to
begin incrementally building coverage for faster
feedback on changes.
In the interim, Hardening or Stabilization Sprints and
having a risk-based Release Train concept help
It’s important that Test or QA not ‘own’ the tooling or
all of the automation efforts. The strategy can come
from Test, but the automation development is best
left to the team.
Mature teams invest in automation as part of Doneness and continually on their backlogs
30
Copyright © 2013 RGCG, LLC
15
18. 3 Pillars of Agile Quality
Software Testing
•
Risk-based testing:
Functional & NonFunctional
•
Test planning @
Release & Sprint levels
•
Exploratory Testing
•
Standards – checklists,
templates, repositories
•
Balanced across
manual, exploratory &
automation
•
Agile-centric Metrics
Exploratory Testing (Charter / Session based and
paired) can be an incredibly effective way to
establish a whole-team, collaborative view towards
quality and testing. It also emerges new tests.
Leverage ‘plans’ as a whole-team collaboration
mechanism and do plan.
Do not measure testing or tester progress; instead,
measure throughput, output, sprint outcomes, and
done-ness escapes at a team level.
You need a balanced test team; not everyone needs
to be able to program. But everyone needs to be
skilled testers.
Agile testing is a Risk-Based play in every Sprint and
across a release sequence. Don’t forget your
techniques!
31
Copyright © 2013 RGCG, LLC
3 Pillars of Agile Quality
Cross-Functional
Team Practices
•
Team-based Pairing
•
Stop-the-Line
•
Code Reviews &
Standards
•
•
Active Done-Ness
•
Aggressive Refactoring
•
User Stories – 3 Amigo
based Conversations
•
Building the ‘Right’
Solutions
One of the hardest areas to get ‘right’ culturally. It
needs leadership alignment from Quality/Testing to
Product to Development and a consistent voice of
whole-team approaches.
This is where LEAN lives, where whole-team
collaboration happens, where professionalism and
craftsmanship are held dear.
I like the view of testers becoming the VOC,
champions of quality, and consistent questioners of
what is being build. Are we solving the right
problems as simply as possible.
MMF, MVP, MMP, etc.
And yes Virginia, there ARE standards and
consistency!
32
Copyright © 2013 RGCG, LLC
16
19. Software Testing
Strategies
It ALL starts with empowering testers AND creating a
Whole-Team view towards Quality
Critical Early Steps:
Creating a sense of empowered Functional Team
Applying Testing Standards across all teams
Deploying Exploratory Testing across all teams
Defining a core set of Agile KPI / metrics
ACTIVE participants in Sprint Planning
Copyright © 2013 RGCG, LLC
33
Cross-Functional Team Practices
Strategies
Training
Agile / Lean in general, Story writing, Acceptance, Unit testing,
etc.
Teaming – for example: feedback or 5 Dysfunctions / Trust
Critical Early Steps:
Coaches & Scrum Masters to reinforce: Pairing / Swarming; WIP
Limits across teams
Define prescriptive and aggressive Done-Ness for ALL teams
Implement coding standards & Crucible / code reviews across the
center (appropriate for technology stacks)
Release Planning BEFORE allowing a team to start Sprint #1
Backlogs have Bug + Refactoring + Automation targets (20%)?
Copyright © 2013 RGCG, LLC
34
17
20. Organizational Quality
Strategies
Continuously communicate your unified Vision
Your strategy must be aligned/shared across:
Development, Quality/Testing, and Product
Keep working your strategy across the pillars
Don’t get stuck with too narrow a focus (easy road)
Make your strategy visible (Information Radiators)
Show progress (Ex: burn up of test automation coverage
across tiers)
Visualize organizational impediments to your Agile Quality
strategies
Attack them!
Quarterly read-outs on progress, plans and adjustments
Listen to your teams
Celebrate successes!
Copyright © 2013 RGCG, LLC
35
What will be (your) agile strategy when you get
back home?
Either in groups or individually
Consider the 3-Pillars discussion
Consider your current team / organization agile ‘state’
Define a broad, 3-pillar view towards some immediate
focus points when you get back into the office
What will you focus on? Why?
How will you communicate the need for change?
How will you measure results?
What will come immediately afterwards?
Copyright © 2013 RGCG, LLC
36
18
21. Wrapping up…
Agile is the best thing that’s
happened to testers since
The Great Depression
Whole Team view
Testing, Metrics, Automation
Planning, Reporting, Quality
Facilitate feedback
Multi-tiered automation
Just-in-Time, risk-based testing
Continuous improvement
Trust the Team
Retrospective
Copyright © 2013 RGCG, LLC
37
Contact Info
Bob Galen
Principal Consultant,
RGalen Consulting Group, L.L.C.
Director of Agile Solutions,
Zenergy Technologies,
Blogs
Project Times http://www.projecttimes.com/robert-galen/
Business Analyst – BA Times http://www.batimes.com/robert-galen/
My Podcast on all things ‘agile’ http://www.meta-cast.com/
Experience-driven agile focused
training, coaching & consulting
Contact: (919) 272-0719
bob@rgalen.com
bob.galen@zenergytechnologies.com
www.rgalen.com
Copyright © 2013 RGCG, LLC
38
19
22. Additional Topics
Copyright © 2013 RGCG, LLC
39
Two Pillars of Lean ‘Thinking’
Respect for
People
Customer, Employees,
Vendors
Develop your teams
Trust & coach
No wasteful work
Continuous
Improvement
Embrace change,
challenge everything
Kaizen – small, incremental
change
Kaikaku – larger scale,
fundamental
From http://www.leanprimer.com
Copyright © 2013 RGCG, LLC
40
20
23. Agile Testing Quadrants
Brian Marick; Lisa Crispin & Janet Gregory
Business Facing
Functional tests
Story tests
Examples
Prototypes
Simulations
Exploratory testing
Scenarios
Usability testing
UAT
Alpha / Beta
Q2
Q3
Q1
Q4
Unit tests
Component tests
API tests
Automated &
Manual
Manual
Performance testing
Load testing
Security testing
Non-functional requirements
Technology Facing
Critique the Product
Supporting the Team
Automated &
Manual
Automation,
Tools, and
Manual
Copyright © 2013 RGCG, LLC
10 Tenets of Agile Testing
41
Jean Tabaka, Rally Software
1.
The system always runs
Continuous Integration
2.
Stop the line, vs. logging
defects
Lean – fix it now!
3.
If it’s not tested, it’s not
“Done”
Early feedback; Earned
Value
4.
Testing comes first, not last
Collaborative testing, focus
on building in quality
5.
Finding defects after
Development is “Done” is too
late
Early feedback; fix it now!
Copyright © 2013 RGCG, LLC
42
21
24. 10 Tenets of Agile Testing
Jean Tabaka, Rally Software
6.
“Development Complete” is
meaningless
Whole Team complete view
– no “partial credit”
7.
Use testing, not analysis, to
explore requirements
Executable requirements
8.
Automation is “how” not a
“whether” or “when”
Automate all testing;
feedback
9.
Tests are your second most
detailed specification
Code is first; later is
traditional specifications
10. Testers are CustomerDeveloper liaisons
VOC; guide effective team
collaboration; ask the right
questions
Copyright © 2013 RGCG, LLC
10 Commitments of Agile Testing
43
Jean Tabaka,
Rally Software
1. We commit to not moving forward if a hole is found
through root cause analysis without first writing a test
2. We commit to not relying solely on just automated
testing or just manual testing
3. We commit to not sitting behind a QA wall (no
boundaries!)
4. We commit to not allowing a code complete without
test code harness complete
5. We commit to not waiting for a test phase but rather
working in smaller and smaller pieces, sooner and
sooner
Copyright © 2013 RGCG, LLC
44
22
25. 10 Commitments of Agile Testing Jean Tabaka,
Rally Software
6. We commit to not testing one iteration after
development is “Done”
7. We commit to not allowing surprises to accumulate for
large end-to-end testing (“mock it now”)
8. We commit to not leaving the riskiest tests to the end
9. We commit to being an equal participant with the
customer and the developer in defining “Doneness”
10. We commit to not taking this oath lightly
Copyright © 2013 RGCG, LLC
45
23