Implementing a successful metrics system in an IT company involves defining goals and key performance indicators from both a top-down and bottom-up approach. Quality metrics should track factors like cost of quality, defects, and rework trends over time. Regular collection and analysis of metrics on projects, processes, and people can provide insights into trends and areas for improvement. Both business objectives and individual project success can be better managed through an iterative process of planning, collecting, analyzing, and evolving organizational metrics.
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Implementing Metrics in IT
1. Implementing a successful metrics system in an IT company
Muntean Ramona
Measurements & Best Practices @ISDC
2. IT’S IMPORTANT TO YOU
IT’S REDUCING UNCERTAINTY
TELLS YOU MORE THAN YOU KNEW BEFORE
Better informed decisions
Tracking the progress of the objectives
Managing the achievements of goals
WE CAN MANAGE WHAT WE CAN EASILY MEASURE
3. GOALS
Goal
Business growth rate
Profit margin
Bids success rate
Personnel turnover
Success
Factors
SUCCES FACTORS
Key Performance
Indicators
Productivity
Predictability
Quality
Competitiveness
Client Satisfaction
Employee Satisfaction
Metrics / Data
Projects, Processes, People
TOP DOWN APPROACH – COMPANY GOALS
4. QUALITY
Cost of Quality
(CoQ)
Cost of Appraisal
(CoA)
Reviews (1st time)
Code
Requirements
Design
Test plans, test scripts
Testing (1st time)
Audits
Cost of Prevention
(CoP)
Training
Procedures, methods
Tools, systems
Planning
Quality improvement
Measurements
Cost of Non-Conformance
(CNC)
Fixing defects
Re-work code review
Re-work doc review
Re-reviews
Re-testing
8. TRENDS
Defect Detection Rate (DDR)
90.00%
We can learn how
review is
influencing the
product quality in
correlation with
defects found by
testers.
80.00%
70.00%
60.00%
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%
S4
S5
S6
DDR_Review
S7
S8
DDR_Testing
S9
S10
S11
DDR_Customer
S12
S13
9. DEVELOPING MEASUREMENT PROCESS
DO
PLAN
Define procedures:
Collect metrics:
- Metrics to be collected
- Collection practices & tooling
- Analysis & reporting
- Feedback integration
- Tailor procedures (if the case)
- Record and store data
- Use proper tools & templates
- Check data accuracy
Identify scope:
- Goals, objectives
- “Problems” that need answer
- Measurements needs
Evolve process:
- Evaluate progress
- Evaluate process
- Recommend improvements
ACT
Analyze metrics:
- Aggregate, correlate data
- Prepare reports
- Discuss trends, identify thresholds
- Revise procedures
CHECK
10. MEASUREMENT PARADOX?
I want a car that uses as little petrol as possible.
I have chosen 2 cars and will do 3 test drives with both of them.
Test drive 1
Test drive 2
Test drive 3
Average
Car A
1 km/l
1 km/l
4 km/l
2 km/l
Car B
1.5 km/l
1.5 km/l
1.5 km/l
1.5 km/l
Test drive 1
Test drive 2
Test drive 3
Average
Car A
1 l/km
1 l/km
0.25 l/km
0.75 l/km
Car B
0.667 l/km
0.667 l/km
0.667 l/km
0.667 l/km
The car that uses less
petrol is car A which does
more kilometres per litre.
But, if I had measured
litres per kilometre, car B
uses less litres per
kilometre and is the better
car.