IT Confidence 2013 - Spago4Q presents a 3D model for Productivity Intelligence

http://itconfidence2013.wordpress.com
Application Lifecycle
Management and process
monitoring through an
integrated and low-cost
solution, mainly based on Open
Source Software Products
1°International Conference on
IT Data collection, Analysis and Benchmarking
Rio de Janeiro (Brazil) - October 3, 2013
N. Bertazzo, D. Gagliardi,N. Bertazzo, D. Gagliardi,
S. Oltolina,S. Oltolina, G. RuffattiG. Ruffatti
Engineering GroupEngineering GroupInsert here a picture

2IT Confidence 2013 – October 3, 2013 http://itconfidence2013.wordpress.com
ALM and…
G1. Measuring company performances through an effective
SPI program
G2. Adopting a multi-dimensional performance model
deployed mostly with 3D instances
G3. Implementing an integrated and low cost – OSS based
– solution for the measurement and governance of software
product & process quality
Presentation Goals

Which is users’ & customers’
level of satisfaction?
How productive is my
organization?
Is there REALLY
a way to measure performance?
Which is the
quality level of my product?
How can I improve the
development process?
How can I compare
different labs?
TopTop
ManagerManager
Is my project
on track?
How can I improve
performance?
ProjectProject
ManagerManager
QualityQuality
ManagerManager
Which are corporate
audits results?
Introduction The problem: Managers’ needs

Engineering At a glance
www.eng.it

Technical, Innovation & Research Division
Engineering’s Software Labs (ESL)
PRODUCTION
ESL3: Application
Management
ESL1-2: Project
development
RFPs technical
support
MANAGED OPERATIONS
Infrastructures & System Services
Architectural design Research
&
Development
Competence
Centers
Resource
management
Business Units (BUs) for different market sectors
Account
Managers
Sales
Managers
Service
Desk
Business Analysis
Project
ManagersBusiness
Competence
Center
Worldwide
Customers
Background Engineering production organization Managers’ needs

• Continuous Quality Improvement in Engineering's projects
• Unified Infrastructure supporting quality processes granting flexibility and
adaptability
• CMMI-DEV and ISO certifications, as independent method to validate the
compliance of processes and infrastructures with quality standards
• Set-up of Engineering Software Labs (ESLs) to enhance and measure
productivity and improve quality practices
Background Compliance of SPI to quality systems
…

Interaction & Information sources (quant. & qual. data)

• Three dimensions of analysis:
1. Economical (EE)
2. Social (SS)
3. Technical (TT)
QEST model: 3 dimensions
•Performance values for each dimension allow to
identify process areas that need improvements

• Spago4Q, the open source platform to measure, analyze and monitor
quality of products, processes and services
http://www.spago4q.org
The model
• QEST nD model, a conceptual framework for measuring process
performance based on multiple analysis dimensions
http://www.semq.eu/leng/modtechqlm.htm
The tool
Productivity Intelligence

The model (1): QEST
Method: Performance is expressed as the combination of the specific ratios
selected for each of the 3 dimensions of the quantitative assessment
(Productivity - PR) and the perceived product quality level of the qualitative
assessment (Quality - Q)
Performance = PR + Q
Model: QEST (Quality factor + Economic,
Social & Technical dimensions) is a
“structured shell” to be filled according
to management objectives in relation to
a specific project
Such a model has the ability to handle
independent sets of dimensions without
predefined ratios and weights - referred
to as an open model

The model (2): geometrical indicators
It is possible to measure performance considering at least 3 distinct geometrical
concepts:
• Distance between the tetrahedron base
center of gravity and the center of the
plane section along the tetrahedron
height – the greater the distance from 0,
the higher the performance level;
• Area of the sloped plane section – the
smaller the area, the higher the
performance level;
• Volume of the lowest part of the
truncated tetrahedron – the greater the
volume, the higher the performance level.
• Target: measuring project performance (p) using 3 distinct viewpoints
• Input Data: list of weighted ratios for each dimension and quality
questionnaires
• Output Data: an integrated normalized value of performance

• Integrated quantitative and qualitative evaluation from 3 concurrent
organisational viewpoints
• A 3D geometrical representation at a single project phase (usually after
the project is completed)
•Use of de facto and de jure standards (e.g. ISO/IEC 9126 for the Quality
Factor  now ISO/IEC 25010:2011)
• Performance Measurement Model to use for consolidating Balanced
Scorecard (BSC) measurement outcomes
• Extension of the original 3D model to n possible dimensions-
perspectives  QEST nD through the simplexsimplex as the mechanism to solve
the problem from the 4th
dimension on
The model (3): key features

The tool: Spago4Q

• QEST model is fully supported by Spago4Q
• The procedure is coherent with the PMAI (Plan-Measure-Assess-
Improve) cycle:
 PLAN, defining a set of metrics, based on the GQM approach, and
possible dimensions of analysis (perspectives) characterizing the
analysis
 MEASURE, including the collection of data, and the computation
of metric values and global performance value
 ASSESS, presenting results through dashboards and reports
 IMPROVE, analyzing in detail each value that is less than the
expected thresholds, in order to find possible problems or
bottlenecks from a process-based viewpoint
The integrated environment: QEST & Spago4Q

The ESL model selected goals for each analysis dimension:
1. Economical (EE)
E.G1 Reduce the effort of corrective maintenance (corrective + preventive,  ISO/IEC14764:2006)
E.G2 Improve ESL resource/assets allocation
E.G3 Reduce effort due to hardware system unavailability (‘downtime’)
E.G4 Reduce rework (Analysis/Design SLC phases)
E.G5 Improve productivity (note: different ‘sizing’ units)
2. Social (S)
S.G1 Reduce the number of non-conformity issues (QA inspection)
S.G2 Improve artifacts reuse (functional reuse)
S.G3 Evaluate training skills for organizational resources
S.G4 Improve customer satisfaction (e.g. Customers/Prospects, Business Units, Developers)
S.G5 Improve knowledge sharing (“social 2.0”, communities)
3.Technical (TT)
T.G1 Reduce the resolution time for defects and technical issues
T.G2 Reduce the number of pre-delivery defects (as in ODC analysis)
T.G3 Improve delivery time for deliverables
T.G4 Improve code quality
T.G5 Improve the testing process (e.g. coverages, # req’s, # tests, ...)
3D analysis: main goals

Metric ID Metric Desc Formula Source
E.M1.1 Incidence of corrective maintenance effort Corrective Maintenance Effort/Development Effort ALM & prj registry
E.M2.1 Allocation of ESL resources Nr. of Res (hours) allocated on prj/Tot of Res (hours) ALM & Corp. Systems
E.M3.1 Hardware System Availability Percentage System Availability System Monitoring
E.M4.1 Incidence of rework Rework Effort / Development Effort ALM & prj registry
E.M5.1 Development capability FP/Effort ALM & prj registry
S.M1.1 n. Of Non Conformity issue % of NC for project ALM & QA Registry
S.M2.1 Incidence of artifact reuse Nr downloads/tot nr of artifacts stored Component repo
S.M3.1 Skill improvement % new (or modify) skills for resource Skill management tool
S.M4.1 Customer Satisfaction Results of survey Survey tool
S.M5.1 Knowledge sharing improvement % of interaction with collab. tools Collaboration tools
T.M1.1 Incidence of defects % nr. of defects (errors + defects) for project ALM
T.M2.1 Defects Mean Resolution Time Tot. resolution time/Tot. defects ALM
T.M3.1 Incidence of delayed deliverables % nr. delayed deliv. / Tot. deliverables ALM
T.M4.1 Code Quality Results of automatic static test Code analysis tool
T.M5.1 Testing process improvement Test coverage ALM
3D analysis: metrics

ESL Chief Manager
ESL Lab Manager
Level 1
Level 2
Level 3
Project Manager (PM)
3D view: drill-down through the organizational levels
PRJ n
ESL
ESL 1 ESL 2 ESL 3
PRJ 1
PRJ n
PRJ 1
PRJ n
PRJ 1
Project developmentProject development Application maintenanceApplication maintenanceProject developmentProject development
Engineering’s Software LabsEngineering’s Software Labs
Top Manager (TM)

3D view: Level 1 - TM dashboard – sample #1
 Unified view on Engineering Software Labs
 Global performance indicator
 Performance comparison (time, labs)

3D view: Level 1 - TM dashboard – sample #2

Metrics
Reqs & Bugs
Risks
Tasks & Issues
Docs
 Detailed view
 Tracking and trends
A single dimension view: Level 3 - PM dashboard

A single dimension view: Level 3 - PM ALM tool

A single dimension view: Levels 1/2 - QA audits

Users & customers
feedbacks are
now integrated
with corporate data
Finally I know how productive my
organization is!
Finally we can REALLY
measure performance!
I can monitor the
quality level of my product
The development process
is under control and
I can improve it!
Now I can compare
Labs performance!
TopTop
ManagerManager
I know if my project
is on track & I can
identify issues
Conclusions
Productivity Intelligence
enables performance
improvement
ProjectProject
ManagerManager
QualityQuality
ManagerManager
Through audit dashboards,
corporate QA
is under control

• Improve reports and KPIs
 Introduce/improve reports for new/modified information needs
 Dynamical update for thresholds
• Integrate tools to collect soft factors measures and indicators
from two categories of ESL customers using on-line surveys:
 “external customer”: feedback on the perceived product quality
 “internal customer”: feedback on the actual process quality from
project managers of the various Business Units
• Build new KPIs on data coming from the Infrastructure
Enhancing Project that collects issues and suggestions by the
ALM users
 e.g. in the BSC ‘Learning & Growth’ perspective
Next steps

25
 Buglione L., Misurare il Software. Quantità, qualità, standards e
miglioramento di processo nell’Information & Communication Technology,
FrancoAngeli, 3/ed, 2008, ISBN 978-88-464-9271-5
 Buglione L. & Abran A., QEST nD: n-dimensional extension and
generalisation of a Software Performance Measurement Model,
International Journal of Advances in Engineering Software, Elsevier Science
Publisher, Vol. 33, No. 1, January 2002, pp.1-7
 Spago4Q website and resources: www.spago4q.org
 Contacts & Info: spago4q@eng.it
References

Q && A
Muito obrigado pela vossa atençaoMuito obrigado pela vossa atençao!!
Thanks for your attentionThanks for your attention!!
ALM and...

Our Contact Data
Nicola
Bertazzo
nicola.bertazzo@eng.it
Daniele
Gagliardi
daniele.gagliardi@eng.it
Gabriele
Ruffatti
gabriele.ruffatti@eng.it
Sergio
Oltolina
sergio.oltolina@eng.it
ALM and...

IT Confidence 2013 - Spago4Q presents a 3D model for Productivity Intelligence

Recommended

Recommended

More Related Content

What's hot

What's hot (14)

Viewers also liked

Viewers also liked (19)

Similar to IT Confidence 2013 - Spago4Q presents a 3D model for Productivity Intelligence

Similar to IT Confidence 2013 - Spago4Q presents a 3D model for Productivity Intelligence (20)

More from SpagoWorld

More from SpagoWorld (20)

Recently uploaded

Recently uploaded (20)

IT Confidence 2013 - Spago4Q presents a 3D model for Productivity Intelligence