1. Software Quality
1
Overview
§ Everything we have been talking about all semester has
had something to do with software quality.
§ This is perhaps the most central and important topic of the
course.
§ We will discuss the meaning of quality, and how to
achieve and assure it.
§ We will then turn our attention to defects: where they
come from, what to do with them, and how to prevent
them.
§ We then discuss the Quality Triangle: People, Processes,
and Tools.
§ We conclude with a discussion of software metrics.
2
1
2. Objectives
§ Here is what you should be able to do upon completion
of this module:
§ Define Software Quality.
§ Achieve and assure software quality.
§ Discuss the Return on Investment (ROI) of quality processes.
§ Explain how to prevent defects.
§ Describe the interrelationship of People, Process and Tools.
§ Describe the Quality organization: CM, Testing, and SQA.
§ Discuss the use of metrics in a software engineering process.
3
Outline
§ Software Quality.
§ Definition.
§ Assuring and Achieving quality.
§ ROI (return on investment).
§ Defects:
§ Definition.
§ Preventing defects.
§ Finding and fixing defects.
§ The Quality Triangle: People, Processes, and
Tools.
§ Software Metrics.
4
2
3. Software Quality
§ Definition of quality: The degree to which a
system, component, or process meets (1)
specified requirements, and (2) customer or user
needs or expectations. 1
5
Software Quality Criteria
§ Correctness § Portability
§ Efficiency § Reliability
§ Flexibility § Reusability
§ Robustness § Testability
§ Interoperability § Usability
§ Maintainability § Availability
§ Performance § Understandability
6
3
4. Achieving Quality
§ How do we build quality into a product?
7
Assuring Quality
§ How do we know we have quality?
§ How do we demonstrate this to others?
8
4
5. Return On Investment
§ What is the cost of quality?
§ What is the cost of lack of quality?
9
Defects
§ Definitions.
§ Preventing defects.
§ Finding and fixing defects.
10
5
6. Some Definitions
§ Error: A conceptual, syntactic or clerical
discrepancy which results in one or more faults
in the software. 1
§ Fault: A specific manifestation of an error. A
discrepancy in the software which can impair its
ability to function as intended. An error may be
the cause for several faults. 1
11
More Definitions
§ Failure: A software failure occurs when a fault in
the computer program is evoked by some input
data, resulting in the computer program not
correctly computing the required function in an
exact manner. 2
§ Defect: Either a fault or discrepancy between
code and documentation that results in mischief
in the testing, installation, maintenance, or use
of software. 3
12
6
7. References
1 Glossary
§ of Software Engineering Terminology,
IEEE Std 610.12-1990
§ 2 Lloyd, D.K., and M. Lipow, Reliability,
Management, Methods and Mathematics, 2 nd
Edition, published by the authors, 1977.
§ 3 Dunn, Robert, and Ullman, Richard, Quality
Assurance for Computer Software, McGraw-Hill,
New York, 1982.
13
Categorization of Defects
§ Classified by phase in § Generic Faults:
which defect is § Computational.
introduced: § Logic.
§ Inadequate or incorrect
§ I/O.
requirements definition.
§ Inadequate or non- § Data Handling.
compliant top-level § Interface.
design. § Database & Data
§ Errors in detailed Definition.
design.
§ Other.
§ Coding errors.
14
7
8. Preventing Defects
§ Standards.
§ Software Engineering Methods.
§ Languages (let the compiler do some of the
checking).
§ Formal proofs of correctness.
§ Prototyping.
§ Continuous Process Improvement.
15
Finding Defects
§ Reviews.
§ Walkthroughs.
§ Formal Inspections.
§ Testing.
§ Defect Reporting.
16
8
10. Quality Assurance
§ Organization.
§ Configuration Management.
§ Testing (IV&V).
§ SQA.
§ Standards.
§ Tools / Support Software.
19
Configuration Management
§ Bill of Materials hierarchy.
§ Software Library.
§ Each version of each component.
§ Configuration Matrix.
§ Version control.
§ Change Control Board (CCB).
20
10
11. Testing
§ Purpose of testing:
§ Defect detection.
§ Verification.
§ Validation.
§ Acceptance.
21
Phases of Testing
System Specification Qualification Test
Software Integration Test
Requirements
Software Design String Test
Code Unit Test
22
11
12. Software Quality Assurance
§ Independent organization.
§ Concerned with the software process.
§ Guided by Standards.
23
Standards
§ Software Quality § Library Control.
Assurance Program. § Reviews and Audits.
§ Tools, Techniques § Configuration
and Methodologies. Management.
§ Computer Program § Testing.
Design. § Corrective Action.
§ Work Certification. § Subcontractor
§ Documentation. Control.
Source: Mil-S-52779A, ‘‘Software Quality
Assurance Program Requirements’’
24
12
13. Tools
§ Requirements.
§ Analysis and Design.
§ Coding.
§ Configuration Management.
§ Testing.
§ Software Quality Assurance.
§ Program Management.
25
Metrics
§ Definitions.
§ Product Metrics.
§ Process Metrics.
§ Use of Metrics.
§ Establishing a metrics program.
26
13
14. Metrics
§ Process Metrics – measure aspects of the
process.
§ Product Metrics – measure aspects of the
product.
27
Software Product Metrics
§ Examples of software metrics:
§ Complexity.
§ Reliability.
§ Usability.
§ Types of metrics:
§ Static – can be counted without running the software.
§ Dynamic – must run the software with
instrumentation.
28
14
15. Reliability -- MTBF
§ MTBF: Mean time between failures.
§ As testing proceeds, the rate of defects found
(hopefully) decreases.
§ As the defect rate decreases, the MTBF
increases.
Defect rate
Goal
MTBF
Test time
29
Software Process Metrics
§ Examples:
§ Productivity.
§ Manageability.
§ Scrap & Rework.
§ Turnover in Personnel.
§ Types of metrics:
§ Static – counting things.
§ Dynamic – change over time.
30
15
16. Use of Metrics
§ Proper uses of metrics:
§ Statistical Process Control.
§ Continuous Process Improvement.
§ Improper uses of metrics:
§ Using product or process metrics to measure people.
§ Setting arbitrary measurement targets.
31
Establishing a Metrics Program
§ How do you decide what to measure?
§ Measure everything you can think of, and then see
what is useful.
or
§ Determine what is useful (i.e., what your goals are) in
advance, then see what needs to be measured to
support those goals.
§ GQM – Goal Question Metric
32
16
