Introduction to Agile software testing - The 5th seminar in public seminar series from KMS Technology which have been delivering from 2011 in every two months

1. An Introduction to
Agile Software Testing
1

2. Course Objectives
After this seminar, you will:
 Understand what an Agile project
 Know the differences between Kanban and Scrum
 Understand what and how a tester performs the
testing in Scrum process
 Understand what testing techniques required for a
tester in Agile
 Know what challenges of Agile Software Testing
2

3. Course Content
 What is an Agile Project?
 What are Scrum and Kanban?
 What are Scrum & Kanban Processes?
 What are testing Activities in Scrum?
 What are Agile Testing Techniques?
3

4. What is an Agile Project?
All projects are “agile”, but some agile projects
are more agile than others
In practice, “Agile” labeled
How to “agile” a project to projects that its delivery of
An Agile purist would point to agile manifesto and each increment taking no
its principles
longer 4 weeks
Individuals and interactions over process and tools
Is it met level of agility?
 Working SW over comprehensive documents
 Customer collaboration over contract negotiation
 Responding to change over following a plan 4

5. Agile Project
An
introduction
of Scrum and
Kanban
5

6. What is Scrum?
Scrum in 100 words
• Scrum is an agile process that allows us to focus on
delivering the highest business value in the shortest
time.
• It allows us to rapidly and repeatedly inspect actual
working software (every two weeks to one month).
• The business sets the priorities. Teams self-organize to
determine the best way to deliver the highest priority
features.
• Every two weeks to a month anyone can see real
working software and decide to release it as is or
continue to enhance it for another sprint.
6

7. What is Scrum? - Principles
9 Scrum principles
Scrum contains sets of principles (some of them
required while others are optional) and
predefined roles
(1)Demo
(2)Sit together
(3)Daily Scrum
(4)Retrospective
(5)Sprint backlog
(6)Product Backlog
(7)Sprint planning meetings
(8)Time-boxed iterations
(9)Definition of Done 7

8. What is Kanban?
Kanban in 100 words
Visualize the workflow
Split the work into pieces, write each item on a card and put on
the wall.
Use named columns to illustrate where each item is in the
workflow..
Limit Work In Progress (WIP) – assign explicit limits to how many
items may be in progress at each workflow state
Measure the lead time (average time to complete one item,
sometimes called “cycle time”), optimize the process to make
lead time as small and predictable as possible
8

9. Kanban vs Scrum Board
Kanban and Scrum Board
V.S.
Scrum and Kanban over time
V.S.
Scrum resists change within V.S. Kanban limits number of items
an Iteration in workflow state
9

10. An
introduction
of Scrum
Process
10

11. What is Scrum process?
Scrum Roles Key Artifacts Ceremonies Process
Sprint planning
Product Backlog • Hosted by Scrum Master Product
• Requirements – user stories • Pick highest priority items in Backlog
• Desired work Product backlog and the team turns
• Prioritized by Product Owner the items into Sprint Backlog
• Anybody can add to it
Product Owner • Estimate sprint backlog in hours Sprint – 2-4 weeks
• Work breakdown
Sprint Goal Sprint Planning Sprint Goal
• Summary of focused work in • Declare Sprint Goal
sprint Daily Scrum Sprint
• Declared by Product Owner
• Accepted by team • Hosted by Scrum Master Daily Scrum Backlog
• 15 minutes - same time every day
Scrum Master Blocks
Sprint Backlog • Not for problem solving
• Team signs up for work of • 3 questions in meeting: (1) What Daily Work
their own – work never Product
did you do? (2) What will you do?
assigned (3) What’s in your way?
• Owned/managed by the team
• Estimated work remaining is • Team updates sprint backlog
updated daily Sprint Review/
Sprint Review
Blocks List • Hosted by Scrum Master – 2-4 hours Retrospective
Scrum Team • List of blocks & unmade • Accomplishments
decisions • Whole team participants
(max – 9) • Owned by Scrum Master
• Updated daily • Take form of demo for features
Product
Sprint Retrospective
Burndown chart Backlog
• Hosted by SM – 15-30 minutes
• Effort spent over period
• Stories/ features completed • Discuss on “Start doing”, “Continue 11
doing” and “Stop doing”

12. What are roles in Scrum?
Product Owner (PO) is a Scrum Master represents The Team
client’s representation Management 5-9 members
Define features of product Enacting Scrum values Including: developers,
Decide Release Date and testers, designers,…
Ensure team’s productivity
content Full time
Prioritize features according
Prioritize features to market value Work Self-organizing
according to market value  Membership should be
Corporate across all roles
Be responsible for the and functions changed each sprint
profitability of product
 Shield team from external
 Accept or reject work interferences
item result
A practice for prioritizing features (or User Story)
Priority = Max (5 (or 6) x priority that assigned by PO, sum
(priority that assigned by team members) 12

13. What are roles of tester in Scrum?
There is NO tester in formal Scrum Process
Testing is carried out by developer with Unit Test
•Test Driven Development means testing
executed before development and refactoring
performed to optimize development
•Testing coverage
 Testing is carried out by Product Owner/ Client
•Frequently testing by Product Owner – each
sprint
•Testing Acceptance Criteria
Isn’t it really necessary tester position in Scrum?
What happens without tester?
No future for tester? 13

14. What are roles of tester in Scrum ? (cont.)
Are testers obsolete?
Questions:
• Will developers do acceptance
testing well?
•Will developers test their work
product well?
•Is that enough with only one
kind of testing – Unit Test?
• How to test non-functional
requirements?
Benefits when have
dedicated test team:
•Focus on customer usage over
technical implementation
•Focus on uncovering flaws over
confirming completeness
14

15. What are roles of tester in Scrum? (cont.)
• Participant in Release/ Sprint
Planning
• Support developers in Unit Testing
– say them about testing view instead
of development view
• Test User Story when completed,
last gate to confirm testing
completeness
• Collaborate with customer and
Product Owner to define acceptance
criteria
• Provide positive feedback to
customer
• Develop automation testing
15

16. Testing Activity
in Scrum &
Kaban Process
16

17. What are Testing Activities in Scrum Process?
Process Tester must know what model of cadence used
Product
cadence
Backlog
single
Sprint – 2-4 weeks
cadence
Three
Sprint Planning Sprint Goal
Sprint
Daily Scrum Backlog
Blocks
Driven
Daily Work
Product event
In Sprint Planning:
Sprint Review/
Retrospective • As a tester, he/she says what should be picked from
Product Backlog for this Sprint to bring value to customer
& to create high performance
Product •As a tester, he/she says how many hours should have
Backlog to complete testing for each of selected User Stories
•As a tester, he/she must understand what sprint goals
• As a tester, involves into the prioritizing process17

18. What are Testing Activities …? – estimate effort
Effort Estimate techniques:
Heuristic (expert judgment based) Parametric (calculation based)
Comparison to similar systems  Function Points
 Expert Judgment  Use Case Points
 Activity Based (top down)  User Story Points
 Task Based (bottom up)
 Three-point estimate
 50/90 estimate
In Agile, testers are empowered to make decision. It means they
are given authority to estimate effort for each work item that
stated in Story Card for Product Owner reviews
As a result, testers are responsible for all testing activities that
they estimated.
18

19. What are Testing Activities …? – 50/90 estimate
50/90 Estimation
Worst case: number of hours (cost) in the worse-case scenario
 Most -likely: number of hours (cost) in the Most -likely scenario
Total Hours = Most Likely
Task variance = (Worst Case (W) – Most Likely (A))
Deviation = sqrt (sum((Wi-Ai)^2))
19

20. What are Testing Activities …? – 3-point estimate
3 Points Estimate (PERT)
Opt: indicates number of hours (cost) in the best-case scenario
Pess: indicates number of hours (cost) in the worse-case scenario
 Most -likely: indicates number of hours (cost) in the Most -likely scenario
PERT = (Opt + 4 x Most-Likely + Pess) / 6
Variance = (Pess – Opt) / 6
20

21. What are Testing Activities …? – User Story Points
User Story points (USP)
 Story points are a unit of measure for expressing the overall size of a
user story, feature, or other piece of work
 The raw values assigned are unimportant
 Image a story with 2 USP(s) will 2 times bigger than 1-USP one
 Two estimation scales or or
Velocity is a measure of a team’s rate of progress – Ideal Time
The beauty of this is that estimating in story points completely separates
the estimation of effort from the estimation of duration
If we find that a particular work unit is twice the size we thought.
The estimate remains the same, but rate of progress through it
will be slower
21

22. What are Testing Activities in Scrum Process?
Process
Product
Backlog
Sprint – 2-4 weeks
Sprint Planning Sprint Goal
Sprint
Daily Scrum Backlog In Daily Scrum:
Daily Work
Blocks
• As a tester, he/she joins all daily meeting (standup)
Product to speak up
1. what was tested and did yesterday/ progress;
Sprint Review/ 2. What will test and do today?
Retrospective 3. Any in testing progress
•As a tester, he/she can bring any backlog item that
cannot be completed in current sprint and put to next
Product
sprint – EX: I don’t have enough time to execute
Backlog
testing for item #4. it will be tested in next sprint.
22

23. What are Testing Activities in Scrum Process?
Process
Product
Backlog
Sprint – 2-4 weeks In Daily Work:
Sprint Planning Sprint Goal • As a tester, he/she is responsible for develop
Sprint automation scripts (for automation testing
Daily Scrum Backlog •As a tester, he/she schedules automation testing
Blocks with Continuous Integration (CI) System.
Daily Work
Product •As a tester, he/she reviews & reports test results to
stakeholders via CI System or by himself/herself.
•As a tester, he/she executes non-functional testing
Sprint Review/ with approved user stories
Retrospective
Some organizations, Product Owner does not have
enough energy to do acceptance testing each sprint.
Tester is hired for this case
Product
Backlog •As a tester, in end of Sprint, he/she executes UAT
and confirms testing completeness for current sprint.
23

24. What are Testing Activities in Scrum Process?
In Scrum, Software is integrated daily, build and test
each day. Therefore, CI System is deployed to most
of Agile Project. A CI System & test components can
be identified as below
• Hudson / CruiseControl – CI Framework
• Junit (for Java)/ Nunit (.Net) – Unit Test
• Cobertura (Java)/ NCoverage – Testing
Coverage
• Ant (for Java)/ Nant (.Net) – Automation Build
• SVN – Version Control
•Selenium integrated with Hudson - Functional
testing
In Daily Work, tester will responsible for develop
automation scripts, integrate testing automation
testing tools/scripts with CI for regression test and
UAT
As a part of duty, tester is playing role supporter in
helping developers create Unit Tests effectively and
efficient 24

25. What are Testing Activities in Scrum Process?
In Daily Work:
As a practice, “Done” of sprint can be considered when
Code checked in.
 Builds ready
 Unit Test complete successfully Tester’s
supports
 80% code branch coverage on UT
 Non-functional testing completed
(confirmed by testers)
 Finish deploying to Production
25

26. What are Testing Activities in Scrum Process?
Process
Product
Backlog
Sprint – 2-4 weeks In Review & Retrospective:
Sprint Planning Sprint Goal • As a tester, he/she identifies what went right and
Sprint what went wrong in current sprint
Daily Scrum Backlog •As a tester, he/she identifies lesson learned and
Blocks best practices.
Daily Work
Product •As a tester, he/she is encouraged to write user
stories that support testing.
EX: As a tester, I need interface with inputs – A, B, C
Sprint Review/ to execute testing of feature X.
Retrospective
•As a tester, he/she is encouraged to write user
stories that bring value to customer.
Product EX: As a user, I need to be able to sort items.
Backlog All these user stories are put to Product Backlog
for review and Approved
26

27. What are Testing Activities in Scrum Process?
In Review & Retrospective:
• Any items remaining will be
•Review why tester could NOT complete in
current sprint
Problem may be identified by lacking of
experience in estimate or any roadblock
prevented it completed. Whatever reason
identified will be noted as lesson learned
•Put to next sprint for continuing testing
• Any roadblock comes from lacking of support of
Product Owner, should be pointed out for Scrum
Master’s follow-up
To bring effectiveness and efficiency to review sessions,
Scrum Master should request participants to prepare list of
items/ issues at least 1 day prior to meeting date
27

28. What are Testing Activities in Kanban Process?
Kanban is less prescriptive than Scrum
All testing activities are similar with Scrum
No required time-boxed iteration
•Testing is not limited by time duration
•Accept changes at anytime – Scrum protects changing backlog items
in sprint while Kanban still accepts removing and adding new items
• Testing item in workflow state can be added or removed to get the
most effectiveness and efficiency.
Kanban limits number of items for each workflow state
•Testing items should be picked logically to put to workflow stated
28

29. What are Testing Activities in Kanban Process?
An Example of Managing Tasks/ Work Item
1. Backlog: state all work items/ user stories translated into
tasks - As a characteristic of Backlog may be limited
number of items
2. Ready: List of work items that ready for executing
3. Specifics: Identify what criteria to confirm work item
completeness
4. Execute: Work Item in executing
Tuan completed work item - Next, he moves another work He also moves another work
testing for “Sorting Items”. He item – testing for “Filter Items” item from Backlog as a
reviews and confirms that it from ready to “execute” to pipeline item.
satisfied criteria state. He moves start new work
29
this item to “Done”

30. What are Testing Activities in Product Development?
Testing against Testing in live
user story environment - UAT
Non-functional
Unit Testing
testing/ confirms
Integration Test
story is met
In Agile, testing is focused on
Tester is also responsible for
1. Unit Test that developers need support of testers to testing user story that cannot be
create effective set of tests and
verified by UT to confirm testing
2. Acceptance Test that the responsibility is belonged
to tester completed before demo and
release to production.
Integration & regression test are executed daily by
automation. 30

31. What are Testing Activities in Product Development?
Testing against Testing in live
user story environment - UAT
Non-functional
Unit Testing
testing/ confirms
Integration Test
story is met
 Story Exploration Communication within an agile team to explore story
understanding
 Estimation
 Consider flows of between Product Owner and team on story
 Story Planning
 Testers also speak up with questions to clarify any vagueness
 Story Progression in the story
 Story Acceptance

32. What are Testing Activities in Product Development?
EX: For User story: “As a user, I can
reset my account password if I forgot
my password”
Test may have questions relate to this
story as follows
1. What does it mean for an email
address to be ‘unknown’?
2. What does it mean to ‘require
confirmation’ of the password?
And then s/he temporarily states their
understanding as next picture
 Story Exploration Estimation is a part of iteration where tester is using appropriate
techniques
 Estimation
Story Planning occurs after a story has been estimated and be
 Story Planning placed in iteration. Details of test tasks defined as
 Story Progression 1. Prepare test data 2. Extend acceptance test
 Story Acceptance 3. Execute manual 4. Automate CI tests

33. What are Testing Activities in Product Development?
 If acceptance criteria are not in
place, agile teams may collaborate
on writing acceptance tests during
story planning
 Tester can select acceptance
tests for automation. Not all
acceptance tests will be automated
Ex: for Forgot Password story, team may
have scenario as next picture for acceptance
test
 Story Exploration Story Progression: uncover additional tests that reply on
 Estimation exploratory testing and communication. Conducting testing while
monitor results from CI system
 Story Planning
 Story Progression Story acceptance: “done” in test may be understood as all tests
pass or level of test automation is met
 Story Acceptance

34. What are documents of testing in Agile Project?
Agile in common, or Scrum/ Kanban in
particular, is focused on how to deliver
As a part of process, testing in Agile has
value to customer by involving customer lightweight documentation with:
(that presented by Product Owner) in
whole process; transferring product to
live (production) environment to bring Use reusable checklists to
benefits to business soon as possible suggest tests
and expect changes to maximize
Focus on essential of test
customer business value. While Agile
reduces documentation overhead by
simplifying process and number of
artifacts created rather than the incidental details
 Capture test ideas for
Exploratory Testing
In case test-cases required, they should be
created at high level
If any, leverage one artifact for Manual and
Automation Tests
34

35. How to track bugs in Agile Process?
 Do or Do Not need Bug Tracking.
No bug Tracking is still working well when testers perform their tests
in Dev environment
However, in Test environment, testers need to track bugs because they
don’t see fixes for a while
With Bug Tracking, any bugs which are identified (opened) by
developers also will be regress tested by testers. Then test scripts will
be updated to cover the case
Ultimately, bugs can be prioritized with stories
Keep in mind, in Agile, testing is not concrete phase. It is a part of
development
Development Development &
Testing
Testing 35

36. Agile Testing
Techniques
36

37. What are Agile Testing Techniques? Quadrants
Automation &
Business Facing Manual
Manual Test Test
Exploratory Testing
Functional Testing
Scenario Test
Story Test
Usability Test
Prototypes Test
Support the team
Critique Product
User Acceptance Test
Unit Testing Performance & Load Testing
Unit tests combined with Performance Test
Test coverage Load Test
Security Test
Special Tool &
Automation
Automation
Test
Technology Facing Test
37

38. What are Agile Testing Techniques?
 As typical testing, black box testing is used.
 Agile testing is focus on using
Automation testing techniques
Test driven development, auto builds and continuous regression test
daily all reply on automation and will not work without it. And
acceptance test is carried out each Sprint that automation also has
benefits
Exploratory testing technique
This technique simultaneous learning, test design and test execution.
While the software is being tested, the tester learns things that
together with experience and creativity generates new good tests to
run.
Scenario testing technique
Use real-world scenarios that exercise multiple stories
Don’t think of Exploratory is Agile Testing – it is only one of Agile
testing approaches 38

39. What are Agile Testing Techniques? Exploratory
 An example of Exploratory testing
technique.
User Story
As an online shopper, I want to be able to delete
items from my shopping cart
Exploratory is kind of creativity techniques
We start with thinking of some personas or
characters and devise exploratory scenarios which
role may get into. Ex:
1. Senior citizen who never shopped on web
2. Hacker looking for troubles
3. Internet-savvy teen
For each role, behaviors will be different to have
cases of testing generated and tested
39

40. References
Scrum and Kaban at InfoQ
http://www.infoq.com/
Testing Experience
http://www.testingexperience.com/
Mountain Goat Software
http://www.mountaingoatsoftware.com/
 Agile Estimating and Planning – Mike Cohn
 User Stories Applied: For Agile Software Development
40

41. Recommended useful tool for Agile Team
- the ultimate screenshot capture software designed specifically for
testers that:
- Snap a single screenshot or record a sequence of screens
- Automatically generate intelligent step-by-step narration (defect report)
- Auto submit tickets to defect tracking and ALM systems
- calling for BETA user - Full featured Test Management System to
give testers a simple yet productive working environment. Some of the qTest
features and modules include:
- Test Planning
- Requirement Management
- Test Case Management
- Test Execution
- Defect Tracking
- Dashboards and Reports
- Built in workflows and collaboration
Get the tools as free download at http://www.qasymphony.com/
41

42. Thank You
Welcome your questions
Agile
Project 1
Scrum & 2
Kanban
Agile 3
Testing
Others 4
42