More Related Content Similar to Testing in an Agile Context 2011 (20) More from Chris Sterling (18) Testing in an Agile Context 20111. Testing in an Agile Context
“What he needs is some way to pay back. Not some way to borrow more.”
- Will Rogers
Chris Sterling
VP of Engineering
AgileEVM Inc.
Web: www.AgileEVM.com
Email: chris@agileevm.com
Blog: www.GettingAgile.com
Follow Me on Twitter: @csterwa
Hash Tag for Presentation: #swdebt
Wednesday, June 8, 2011
2. Chris Sterling – Sterling Barton, LLC
Partner, Sterling Barton, LLC
Author of Book “Managing Software
Debt: Building for Inevitable
Change”
Consults on software technology,
Agile technical practices, Scrum,
and effective management
techniques
Email: chris@sterlingbarton.com
Web: http://www.agileevm.com
Certified Scrum Trainer Blog: http://www.gettingagile.com
Follow me on Twitter: @csterwa
Innovation Games® Trained
Facilitator
Open Source Developer
© 2009-2010, 2
Wednesday, June 8, 2011
3. Presentation Agenda
Agile - Find Issues Earlier
Effects of Quality Debt
Definition of Done
Quality Dashboard
Agile Test and Integration Strategies
Acceptance Test-Driven Development
Test-Driven Design
Agile Regression Test Triangle
Configuration Management Debt Strategy
Questions and Answers
© 2009-2010, 3
Wednesday, June 8, 2011
5. “Stuff” Rolls Down Hill...
Requirements
Specification
Design
Construction
Testing
(Validation)
Integration
Maintenance
© 2009-2010, 5
Wednesday, June 8, 2011
6. “Stuff” Rolls Down Hill...
Requirements
Specification
Design
Construction
Testing
(Validation)
This is too
late to find issues Integration
and respond
effectively
Maintenance
© 2009-2010, 5
Wednesday, June 8, 2011
7. The Agile Manifesto*
We are uncovering better ways of developing software by doing it and
helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right,
we value the items on the left more.”
Kent Beck James Grenning Robert C. Martin
Mike Beedle Jim Highsmith Steve Mellor
Arie van Bennekum Andrew Hunt Ken Schwaber
Alistair Cockburn Ron Jeffries Jeff Sutherland
Ward Cunningham Jon Kern Dave Thomas
Martin Fowler Brian Marick
55
6
© 2009-2010, * www.agilemanifesto.org
Wednesday, June 8, 2011
8. Principles Behind the Agile Manifesto*
1. Our highest priority is to satisfy the customer through early and continuous delivery of
valuable software.
2. Welcome changing requirements, even late in development. Agile processes harness
change for
the customer's competitive advantage.
3. Deliver working software frequently, from a couple of weeks to a couple of months, with
a preference to the shorter timescale.
4. Business people and developers must work together daily throughout the project.
5. Build projects around motivated individuals. Give them the environment and support
they need, and trust them to get the job done.
6. The most efficient and effective method of conveying information to and within a
development team is face-to-face conversation.
7. Working software is the primary measure of progress.
8. Agile processes promote sustainable development. The sponsors, developers, and
users should be able to maintain a constant pace indefinitely.
9. Continuous attention to technical excellence and good design enhances agility.
10. Simplicity--the art of maximizing the amount of work not done--is essential.
11. The best architectures, requirements, and designs emerge from self-organizing teams.
12. At regular intervals, the team reflects on how to become more effective, then tunes and
© 2009-2010,
adjusts its behavior accordingly. 56
7
* www.agilemanifesto.org/principles
Wednesday, June 8, 2011
10. Effects of Quality Debt
“Promises make debt, and debt makes promises.” - Dutch Proverb
9
Wednesday, June 8, 2011
13. “For every [dollar] of competitive advantage gained by
cutting quality, it costs $4 to restore it; and software is
an organizational asset and decisions to cut quality
must be made by executive management and
reflected in the financial statements.” - Ken Schwaber
© 2009-2010, 11
Wednesday, June 8, 2011
14. Lack of emphasis on software quality attributes
contributes to decay
© 2009-2010, 12
Wednesday, June 8, 2011
18. Agile Test and Integration Strategies
© 2009-2010, 16
Wednesday, June 8, 2011
19. Definition of Done
Defines work products that will be
delivered with each feature as it is
ready for acceptance.
Typical entries...
Code includes unit tests, reviewed,
checked in
Tests described and executed
Build, release notes
Compliance documentation
updated to include current
functionality
Example
of
“average”
DoD
What else?
© 2009-2010, 17
Wednesday, June 8, 2011
20. Definition of Done as a Compliance Checklist
Acceptance defined criteria for each Code checked in with reference to
user story US#/Task#
Unit tests written and passed Tested on FE
Code compiles with no errors and no Integration test written & passes
warnings
Test code reviewed
New code doesn’t break existing code
Environment requirements documented
Test case review (Dev to review test
case written) Interface document updated/added and
checked in to SVN
Architectural impact assessed and
artifacts updated if necessary Acceptance criteria verified complete
Comments in code All P1-P3 bugs for the story are closed
Error codes added Test approves user story
Code reviewed by peer Story demonstrated to product owner
© 2009-2010, 18
Wednesday, June 8, 2011
21. Another Definition of Done Example
Story and
Unit Tests Functional Acceptance Build System
Task Status
Passed Tests Passed Tests Passed Compiles
Updated
Story
Code Code
Code is Code Meets Published to
Implements Comments
Reviewed Standards Dev Server
Logging Updated
Product Past Product No Compile
Demo
Owner Acceptance Owner Warnings in
Prepared
Sprint
Demo Tests Passed Acceptance Code
Bugs Deployment Release Code
Committed in Docs Notes Repository
Sprint Resolved Updated Updated is Tagged
Release to Deployment Deployment Release
Stage Testing Docs Notes
Release
Server Passed Delivered Delivered
Infrastructure Integrated Build
Change Notes Stress Testing Requirements
Delivered Passed Met
© 2009-2010, 19
Wednesday, June 8, 2011
22. No matter what, the cost of addressing software
debt increases with time.
© 2009-2010, 20
Wednesday, June 8, 2011
27. Test-Driven Design (TDD) - Basic “Flow”
Write
Failing
Test
Refactor
to
Make
Test
Acceptable
Design Pass
© 2009-2010, 22
Wednesday, June 8, 2011
28. Test-Driven Design (TDD) - Basic “Flow”
Write
Failing
Test
Refactor
to
Make
Test
Acceptable
Design Pass
© 2009-2010, 22
Wednesday, June 8, 2011
29. Test-Driven Design (TDD)
Lets go through an example session using TDD to drive the implementation of a
user story to meeting its acceptance criteria.
23
Wednesday, June 8, 2011
30. Jitter – Example TDD Session
Fake micro-blogging tool named “Jitter” is made by Seattle-
based fictitious company that focuses on enabling coffee
injected folks to write short messages and have common online
messaging shorthand expanded for easy reading. The user story
we are working on is:
So it is easier to read their kid’s messages, Mothers want to
automatically expand common shorthand notation
The acceptance criteria for this user story are:
LOL, AFAIK, and TTYL are expandable
Expand lower and upper case versions of shorthand
© 2009-2010, 24
Wednesday, June 8, 2011
31. Expand LOL to “laughing out loud”
public class WhenMotherWantsToExpandMessagesThatContainShorthandTest {
@Test
public void shouldExpandLOLToLaughingOutLoud() {
JitterSession session = mock(JitterSession.class);
when(session.getNextMessage()).thenReturn("Expand LOL please");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand laughing out loud please"));
}
}
© 2009-2010, 25
Wednesday, June 8, 2011
32. But wait…what if…?
What if LOL is written in lower case?
What if it is written as “Lol”? Should it be expanded?
What if some variation of LOL is inside a word?
What if characters surrounding LOL are symbols, not letters?
Write these down as upcoming programmer tests as comments so I
don’t forget them.
// shouldExpandLOLIfLowerCase
// shouldNotExpandLOLIfMixedCase
// shouldNotExpandLOLIfInsideWord
// shouldExpandIfSurroundingCharactersAreNotLetters
© 2009-2010, 26
Wednesday, June 8, 2011
33. Expand LOL If Lower Case
@Test
public void shouldExpandLOLIfLowerCase() {
when(session.getNextMessage()).thenReturn("Expand lol please");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand laughing out loud please"));
}
This forced use of java.util.regex.Pattern to handle case
insensitivity.
public String getNextMessage() {
String msg = session.getNextMessage();
return Pattern.compile("LOL”, Pattern.CASE_INSENSITIVE)
.matcher(msg).replaceAll("laughing out loud");
}
© 2009-2010, 27
Wednesday, June 8, 2011
34. Don’t Expand “Lol” – Mixed-Case
@Test
public void shouldNotExpandLOLIfMixedCase() {
String msg = "Do not expand Lol please";
when(session.getNextMessage()).thenReturn(msg);
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(), equalTo(msg));
}
This forced me to stop using Pattern.CASE_INSENSITIVE flag
in pattern compilation. Only use “LOL” or “lol” for now.
public String getNextMessage() {
String msg = session.getNextMessage();
return Pattern.compile("LOL|lol").matcher(msg)
.replaceAll("laughing out loud");
}
© 2009-2010, 28
Wednesday, June 8, 2011
35. Don’t Expand “LOL” If Inside Word
@Test
public void shouldNotExpandLOLIfInsideWord() {
String msg = "Do not expand PLOL or LOLP or PLOLP please";
when(session.getNextMessage()).thenReturn(msg);
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(), equalTo(msg));
}
The pattern matching is now modified to use spaces around
each variation of valid LOL shorthand.
return Pattern.compile("sLOLs|slols").matcher(msg)
.replaceAll("laughing out loud");
© 2009-2010, 29
Wednesday, June 8, 2011
36. Expand “LOL” If Not Inside Word
@Test
public void shouldExpandIfSurroundingCharactersAreNotLetters() {
when(session.getNextMessage()).thenReturn("Expand .lol! please");
MessageExpander expander = new MessageExpander(session);
assertThat(expander.getNextMessage(),
equalTo("Expand .laughing out loud! please"));
}
Final implementation of pattern matching code:
return Pattern.compile("bLOLb|blolb").matcher(msg)
.replaceAll("laughing out loud");
© 2009-2010, 30
Wednesday, June 8, 2011
37. The Agile Regression Testing Triangle*
© 2009-2010, * The Agile Triangle has been modified from Mike Cohn’s original version 31
Wednesday, June 8, 2011
38. The Agile Regression Testing Triangle*
Automated Unit Tests
Make up largest portion of
regression tests and are
developed by programmers
© 2009-2010, * The Agile Triangle has been modified from Mike Cohn’s original version 31
Wednesday, June 8, 2011
39. The Agile Regression Testing Triangle*
Integration Tests
Automated &
Exploratory
Automated Unit Tests
Make up largest portion of
regression tests and are
developed by programmers
© 2009-2010, * The Agile Triangle has been modified from Mike Cohn’s original version 31
Wednesday, June 8, 2011
40. The Agile Regression Testing Triangle*
Smoke++ Tests
Risk-based UI &
Integration Tests
API Automated
Automated &
Tests
Exploratory
Automated Unit Tests
Make up largest portion of
regression tests and are
developed by programmers
© 2009-2010, * The Agile Triangle has been modified from Mike Cohn’s original version 31
Wednesday, June 8, 2011
41. Configuration Management Debt
“If releases are like giving birth, then you must be doing something wrong.”
- Robert Benefield
32
Wednesday, June 8, 2011
42. The Power of 2 Scripts: Deploy and Rollback
© 2009-2010, 33
Wednesday, June 8, 2011
43. Thank you
Questions and Answers
34
Wednesday, June 8, 2011
44. Chris Sterling – Sterling Barton, LLC
Partner, Sterling Barton, LLC
Author of Book “Managing Software
Debt: Building for Inevitable
Change”
Consults on software technology,
Agile technical practices, Scrum,
and effective management
techniques
Email: chris@sterlingbarton.com
Web: http://www.agileevm.com
Certified Scrum Trainer Blog: http://www.gettingagile.com
Follow me on Twitter: @csterwa
Innovation Games® Trained
Facilitator
Open Source Developer
© 2009-2010, 35
Wednesday, June 8, 2011