5 Quality

SoberIT
Software Business and Engineering Institute

Errata!
  “Estimation of Software” has a bug!
  “The multiplier for high complexity and external output type
is higher than the multiplier for high complexity and external
input type.”
  Instead, you should look up the values from the table for
external inquiry type, as with other user types

  The reason:
  Some descriptions of Albrecht function point method have
instructed to do as described in the material, but this
suggestion has been changed later.

HELSINKI UNIVERSITY OF TECHNOLOGY

SoberIT

Lecture on February 16th
  Student presentations!
  Prepare a presentation of 15-20 minutes on topic of your interest
(related to software project management), and get 5 lecture
summary points
  And/or
  Attend the session, participate in discussions and share your own
experiences, and earn max 5 lecture summary points by writing
a summary of the presentations & discussion

  If you want to give a presentation, please send email to
tuomas.niinimaki@tkk.fi


SoberIT

T-76.5612 Software Project Management

5: Quality in Software Project Management

Tuomas Niinimäki
(many slides by Juha Itkonen & Mika Mäntylä)

Helsinki University of Technology


SoberIT

A Plan

HELSINKI UNIVERSITY OF TECHNOLOGY 4

SoberIT

The Reality


SoberIT

What quality practices can do for my
project?
  Provide headlights (Information on quality)
  Where are we?
  … and what lies ahead?

  Enable changing our plan
  We can get there, but later than planned
  We can get almost there and stick to our
schedule
  But if we run full steam ahead according
to our plan we will definitely crash


SoberIT

Contents
  Setting quality goals
  From goals to quality practices
  Quality information in project planning and tracking


SoberIT

Quality Goals


SoberIT

Strive for good quality
Bad quality leads to rework,
higher costs, delays, and
problems in operation


SoberIT

What is quality (in software)?


SoberIT

ISO 9126 Quality attributes
  Functionality   Efficiency
  Suitability   Time behavior
  Accurateness   Resource behavior
  Interoperability   Maintainability
  Security   Analyzability
  Reliability   Changeability
  Maturity   Stability
  Fault tolerance   Testability
  Recoverability   Portability
  Usability   Adaptability
  Understandability   Installability
  Learnability   Conformance
  Operability   Replaceability
  Attractiveness
+ suggested metrics for each quality attribute

SoberIT

QUALITY

PRODUCT PEOPLE PROCESS


SoberIT

QUALITY

PRODUCT PEOPLE PROCESS
  Product is error-free   The expectations of   Control and
the customer or user management
  Product meets the have been met
specifications   Expectations,
  Someone gets value estimates, forecasts
  Product has certain from the result
measurable attribute   What is value?   Reproducibility
Differs depending
  e.g. durability, on who you ask:
weight, SMS
support
developer, user,
tech. support
  Improvement
  Learning from
mistakes and
  Sustainability successes

  Conformity /
commitment


SoberIT

Quality goals


SoberIT

Setting the target - Quality goals
  Generic quality characteristics are not enough
  Customer’s requirements and needs
  Product characteristics
  Application domain
  Development technologies and environment
  Project characteristics
  Specific quality goals are needed
  Specific goals for this project
  Unambiguous
  Connected to a concrete project, product, and problems

we need to define precisely what qualities we require of a system


SoberIT

From a general to a specific quality goal: An Example

  Adaptability
  Attributes of software that bear on the opportunity for its
adaptation to different specified environments without
applying other actions or means than those provided for this
purpose for the software considered.

  Adaptability
  The MobWebClient software must work with all advertised
compatible browser and e-mail environments, with or without
java, without specific configuration.


SoberIT

Quality goals
  Quality goals point out the important quality characteristics for
  the project,
  the product, and
  the whole organization

  Quality goals
  show where to focus effort and resources
  help selecting suitable practices and methods
  steer every activity of the project into the right direction

  Quality goals can reflect different qualities for different parts of
the system

SoberIT

Focusing the QA efforts – Quality Risks
  Risk – a potential threat to projects objectives
  With uncertain probability
  Quality goals tell us what are the important qualities to achieve
in this project
  Quality risk analysis focuses our QA efforts to the most
important things
  E.g. if functional correctness is one of our quality goals:

What are the most important functions that we should test
thoroughly vs. all the functions we could test?


SoberIT

What we should test vs. what we could test?

Reduced value due to defects
Cost of finding defects


SoberIT

Example – Risk analysis matrix
Characteristic Impact Probability Exposure

Usability 3 4 12

Functional 2 2 4
correctness

Conformance 1 3 3
to standard X


SoberIT

Example – Statistical Risk Analysis Matrix
Impact * Probability = Re
New Design Com- Weigh. Risk
Dev Cust. Avg Func. Qual. Size plexity Sum Exposure
Weight 5 5 1 3

Interest
Calculation 3 3 3 2 3 3 3 37 111

Close
Account 1 3 2 2 2 2 3 31 62

Create 2 1 1,5 3 3 2 3 41 61,5
Account

The numbers and calculations are not important – Idea
is to get the features into priority order Modified slide, originally from Ståle Amland

SoberIT

Prioritising risks (I x P = E)
  I = Impact, P = Probability, E = Exposure

  What does the exposure number mean?
  Scale of a risk
  Used to rank risks

  high I x low P = low I x high P
  Risks with very severe impacts can have average exposure
  Usually impact is easier to estimate than probability
  Can lead to low exposures for high impact risks


SoberIT

Prioritising risks
  Goal is to prioritize
  Which risks deserve most attention
  The discussion about the severities and probabilities is probably
more valuable than the exact ranking
  In QA, prioritization means distribution of efforts, not order of
actions
  Less risky application areas cannot be ignored
  Requirement priority is not test priority


SoberIT

Quality Practices


SoberIT

Quality practices
  Many practices strive for better quality
  Testing
  Reviews, inspections
  Conventions, guidelines
  Documenting
  Design and development methods & techniques
  Project management practices


SoberIT

Quality practices
  Plan which practices are used to achieve and measure each of
the quality goals
  Applying existing practices already in use
  Learning best practices from other projects and organizations
  Improving and combining existing practices
  Inventing new practices

  Quality goals give guidance for performing the practices


SoberIT

Mapping the goals and practices
Quality goals Quality threat
Practices
Functional Conformance Usability Performan DB changes One developer
correctness to standard X ce break existing has no
client sw experience in
J2EE

Functional
testing 3 1
GUI
prototyping 2
Code reviews
3

3 = good practice
2 = helps
1 = might help


SoberIT

Mapping the goals and practices
Quality goals Quality threat
Practices
Functional Conformance Usability Performan DB changes One developer
correctness to standard X ce break existing has no
client sw experience in
J2EE

Functional
testing 3 1
GUI
prototyping 2
Code reviews
3 1 2
Regression
testing client 1 3
sw

  Documents how quality goals are to be achieved
  Focuses activities into right issues
  Reveals and communicates
  explicitly identified threats
  quality goals with weak practices
28

SoberIT

How to measure the quality goals
  Some quality characteristics can be measured
quantitatively
= Hard metrics
  e.g. features passing tests, performance
  Some can be assessed subjectively
= Subjective evaluation
  e.g. usability, conformance to a standard
  Some cannot be measured at all (in a practical
situation)
= Indirect metrics
  e.g. maintainability, reusability

SoberIT

Evaluating and tracking quality - Hard metrics
  Hard metrics are concrete and clear
  If the goal can be quantified, do so
  What measures are used?
  Who, how and when?
  How the measures are used?
  Numbers are easy to present and understand
  Easy to track and observe changes and trends
  Easy to misinterpret
  E.g. what can we infer based on the number of found
defects alone?
  Historical data helps in detecting trends
  Characteristics of defects (severity, technical type)


SoberIT

Evaluating and tracking quality - Subjective evaluation

  Subjective evaluation as a measure
  Define how the goal is assessed
  What are the metrics
  Who, how and when
  How the results are used
  Reviews and assessments
  Subjective opinion
  Qualitative data
  E.g., subjective assessment of quality of a video stream
  Tracking not as straightforward -> must quantify
  E.g., compare to reference video streams of different
quality

SoberIT

Evaluating and tracking quality - Indirect metrics
  Define how to achieve, evaluate and track these goals too!
  Indirect assumptions
  Maintainability
  E.g., following a coding standard leads to better code
quality, understandability and source code documentation
  Reusability
  E.g., writing and executing unit tests for all code leads to
better design, robust code, and better maintainability
  E.g., product line architecture leads to re-usable software
  Hard metrics for following these practices
  Maintainability
  E.g., coding standards violations found in code reviews
  Reusability
  E.g., number / amount of unit tests, unit test peer
reviews
  E.g., amount of project specific glue code

SoberIT

Goal – Question – Metric (GQM)
  Goals define the purpose and objective for measurement
  Purpose
  Quality issue
  Object
  Viewpoint
  Questions characterize and refine the measurement goals
  The goal can be achieved when answers to the questions are
available
  Metrics are identified for each question
  Metrics provide the answers to the questions


SoberIT

Project Planning


SoberIT

Planning the project with quality goals and practices

  Describe quality goals and major threats
  Make sure that each quality goal is meaningful and described in
the context of this project
  Make sure that all required quality practices are planned
  Effort estimates
  Schedule
  Resourcing
  Plan how to track quality practices and measure achieved quality
  What measures are used?
  Who, how and when?
  How the measures are collected and used?
  Remember: Quality assurance takes effort and calendar time


SoberIT

Testing and project planning
  Test releases
  Make clear what and when is released for testing
  Plan what is the required quality level of test releases – and how to
ensure / measure it
  Building and delivering the test release takes time
  Do not forget that fixing takes time and effort
  You don’t know how many bugs will be found
  You must plan how many bugs will be found
  Allocate time and resources to test, fix and re-test (verify the fixes)
  More bugs means more fixing and slower testing
  Test schedules are relative
  Development will be late
  Code will be buggy
  Plan how to handle slipping dates and underestimated workload


SoberIT

Iterative Testing and Deployment
Iteration Heartbeat Testing and deployment
iterations
Release

t
Release to Release to
testing customer

  Testing, stabilizing, and deployment takes time
  Testing requires many iterations
  Found defects has to be fixed
  Fixes has to be re-tested
  Fixing causes more defects…
  If you do more during development the risk is smaller during
testing and deployment
  If all testing is left at the end, the testing phase is hard to
estimate and plan
  Waterfall process

SoberIT

Project Tracking and Reporting


SoberIT

Making quality visible in project tracking
  Keep quality goals visible in project meetings
  Track the planned metrics
  Publish the metrics
  React to deviations
  Connect quality metrics in progress tracking
  What does it mean to get a task done?
  Reviews, unit tests, functional tests, regression tests, demo, …


SoberIT

Making quality visible in project tracking
  Find out the reasons behind the metrics
  small number of found bugs due to ignored testing tasks
  large number of found bugs due to changed specifications -> broken
test cases

Number of
defects
A B

C D

Low QA quality High QA quality


SoberIT

Test reporting in project management
  As a project manager you need test reports
  Not as bug descriptions, not as plain numbers of test cases
and bugs
  Test reports describe what has been tested and assess the
current level of achieved quality based on that testing
  Test reports should provide information to assess how well
the stated goals have been achieved
  Important content in test reports
  Assessment of the achieved quality
  What was tested, how thoroughly, what was coverage
  What was found, bugs by severity, issues, metrics
  Progress according to the plan

SoberIT
Reporting with weak metrics:
Testing Dashboard

Modified from: Kaner et al. 2002. Lessons Learned in Software testing.

  If you cannot have direct quality measures, use indirect
  Use of certain quality assurance practices gives indirectly hints of
the likely quality of the final system
  Qualitative assessment and the reasoning behind is much better
than plain numbers


SoberIT

Test reporting for steering group
  It depends! (Usually higher level of abstraction, focus on specific
issues) 35
30
25 Designed
20 Implemented
15 Tested
10 Needs rework
5
Finished
0
1 3 5 7 9 11 13 15 17 19 21 23

35
30
25
20
Remaining
15
10 Planned
5
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23


SoberIT

Quality information Managing project
• Review results • To re-scope an
iteration
• Test results, bug
counts • To focus
resources into
• Development right areas
progress
• To make sure
• Metrics of achieving good
different qualities enough quality
(performance,
load, …) • To find out we are
making the right
product

we need to forecast the likely quality of the
final system when it is under development

SoberIT

Quality Assurance Through Time Horizons
Iteration Heartbeat

  Heartbeat time horizon Release
  Quality practices are part of each
development task
t

  Iteration time horizon
  Tasks to assure the quality of an
increment

  Release time horizon
  Tasks to assure the quality of the
whole release
Blue = QA practices

SoberIT

Quality information and time horizons
Release time horizon

i
Release project i i i i i i

Time
  Not connected to any iterations or milestones – hard to track and
manage
  Typical approach to manage
  Informal reviews
  System testing
  Acceptance testing
  Testing non-functional quality characteristics
  Performance
  Usability
  Load and stress testing
  …


SoberIT

Iteration time horizon

i
Release project

Iteration i i i i i i i i i i iii

Time
  Quality assurance goals for each iteration
  Activities that provide information by the end of (each) iteration
  Not necessary connected to individual development tasks
  Information available at the end of each iteration, at latest
  Review results
  Unit test and integration test results
  Function test and system test results
  Quality metrics (e.g. performance, bug counts, test reports)


SoberIT

Heartbeat time horizon

i
Release project

Iteration

i i
Heartbeat i i i i i i i i i i i i i i i i

Time
  Quality assurance activities as part of each development task
  Information created early and continuously
  Development progress, bug counts, performed tests, …
  Code review results
  Unit tests and results
  Automatic builds and smoke tests
  Function test and integration test results


SoberIT

i
Release project i i i i i i

Iteration i i i i i i i i i i iii
i i
Heartbeat i i i i i i i i i i i i i i i i

Time
Heartbeat quality practices Release quality practices
•  Information for steering the •  Independent activities
iteration •  Tracking and planning iterations
•  Tracking progress

Iteration quality practices
•  Information for steering the current
and forthcoming iterations
•  Evaluating achieved quality

SoberIT

Gather information on short time
horizons
  Progress against iteration (quality) goals must be tracked in
heartbeat rhythm
  Plan how you get the quality information before it’s too late
  Find actively ways of digging quality information out as early and often
as possible
  Talk to testers and developers
  The best way is to get something completed and tested in short
iterations
  The risk of poor quality is not delayed to the end of the project
  Shorter feedback loop -> easier and faster to fix the problems
  Pay attention to heartbeat practices
  Assure good enough quality in each iteration

  Do not push risks like a growing snowball in front of you


SoberIT

Who is responsible for quality?
  Make sure that quality is everybody’s responsibility
  and it shows in the project plan
  It is good to have a project “QA manager” role
  Responsibility to actively track quality in the heartbeat rhythm and
help project manager to make QA activities to happen
  Reports project’s progress from the quality point of view in the
project meetings
  Tracking quality goals in iteration time horizon
  Tries to be less eager to sacrifice quality for new fancy features
  Keep separate testers tightly in the project
  In the heartbeat rhythm
  Project meetings
  Reviews
  Collaborating with developers


SoberIT

Summary
  Define concrete and meaningful quality goals
  Identify practices and activities that are applied to achieve the
goals and track progress
  Plan activities and tracking
  quality assurance activities in the schedule and effort estimations
  how to react to deviations
  Remember the different time horizons
  When is the information needed in order to be useful?
  Plan activities to provide the information promptly
  Identify also the major quality threats
  and plan how the threats are handled in the project and when
  Do not leave big threats to be revealed at the end of the project


SoberIT

Thank you.

Questions?
Tuomas Niinimäki

tuomas.niinimaki@soberit.hut.fi


5 Quality

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (9)

Ähnlich wie 5 Quality

Ähnlich wie 5 Quality (20)

5 Quality