2. Introduction
About speaker
• 10+ years in IT
• 9 years in quality
management
• PM, TM, CEO
• She is cool!
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
3. Introduction
About speech
1. Why to measure
2. What to measure
3. How to measure
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
11. What to measure?
Product quality
Project status
Kiev
11/10/2013
• Control
• React in time
Team efficacy
XP days Ukraine
• Make clear
decisions
• Visualize your
goals
• Better planning
• Improvement
strategy
Natalya Rukol
Definition of Qua
12. What to measure
Product quality
How it
looks
XP days Ukraine
What’s
inside
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
13. Product quality
External quality
• Customers’
satisfaction
• Business results
• Refunds (N & %)
• Support & Hotline
statistics
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
15. Product external quality
Examples
Metric
6.5 Release
7.0 Release
Customers’ satisfaction
4.4
4.2
Refunds
115
109
Refunds %
0,06%
0,11%
Support time per client, hours
6
8
Support tickets amount per
month
4112
3640
Support tickets in %
(bugs/questions/praises)
27/61/12
19/79/2
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Status
?
Definition of Qua
16. Product external quality
Examples
Metric
6.5 Release
7.0 Release
Customers’ satisfaction
4.4
4.2
Refunds
115
109
Refunds %
0,06%
0,11%
Support time per client, hours
6
8
Support tickets amount per
month
4112
3640
Support tickets in %
(bugs/questions/praises)
27/61/12
19/79/2
Status
?
OK, and what???
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
17. Product external quality
How to use
• Track changes
while it’s not too
late
• Investigate WHY
problems?
• Ask boss for
increase
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
18. Product external quality
Too late…
Your
software is
shit!
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
19. Product quality
Let’s go inside
• Functionality
(what?)
• Non-functional
requirements
(how?)
• Usability
• Performance
• Etc.
• Stability (when?)
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
20. Product quality
Measure inside
What
How to measure
Functionality
(What?)
• Requirements
accepted
• Users scenarios
execution
Non-functional
requirements
(How?)
• Load results
• Performance results
• Usability scenarios pass
ratio
Stability (When?)
• Tests pass rate
• Builds stability
statistics
• Bugs
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
23. Product internal quality
Not relevant
100% pass rate,
no criticals, blah
blah
Your
software is
still shit!
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
24. Product quality
Consequences
What’s
inside
How it
looks
Customers satisfaction
•
•
•
•
Refunds
• Why refunds?
• Collect critics
• Critical bugs
• Features accepted
Support tickets
• What bothers most?
• Usability scenarios
1.
2.
What do they like?
What they use mostly?
What they don’t like?
Pain features?
•
•
•
•
Critical bugs
Testing pass rate
Usability scenarios
Features accepted
You have to find your own consequences!
First find, then prove!
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
26. Product quality
Why external first?
• CLOKs & bugs are
useless w/o
customers &
sales
• No way to forget
priorities
• Think big!
• Tech depends on
business, not vice
versa!
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
28. How to measure
Do it right!
• Pre-analysis
• Introduction
• Measurement
• Control
• Continuous
improvements
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
29. How to measure
Pre-analysis
• Collaborate with
the whole team
• Remember The
Great Goal (TGG)
• Find your strategy
together
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
30. How to measure
Soft introduction
• Don’t force
• Share
responsibility
• Find enough time
• Provide with
enough resources
• Inside out
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
31. How to measure
Discipline
• Automate all the
measurements
• Show your
respect
• Grow a habit
• Stand it for 2
months
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
32. How to measure
PDCA
• Plan, Do, Check, Act
• Post-Mortems
• Accept mistakes
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
33. Quality Definition:
Results
• Clear view
• Achievements
• Team
collaboration
• Karma increase
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua
34. Quality Definition:
What’s next
Contact with any
questions:
Additional
materials:
http://natalyarukol.ru
http://natalyarukol.ru/definitionof-quality
http://nrukol.moikrug.ru
http://software-testing.ru
Skype: natalya.rukol
mailto: natalya@quality-lab.ru
XP days Ukraine
Kiev
11/10/2013
Natalya Rukol
Definition of Qua