Quality Control for Medical Device Software - It Arena Lviv Presentation

1. Quality Control for
Medical Device Software
by Roman Lavriv,
Senior Project Manager
©2014 GlobalLogic Inc. CONFIDENTIAL

2. Introduction
Regulations
Requirements & V-model
Verification & Validation
Going Agile
01
02
03
04
05
©2014 GlobalLogic Inc. CONFIDENTIAL

3. 01 Introduction
3 CONFIDENTIAL

4. Introduction
About Me
− Over 11 years in IT outsourcing as Project
Manager
− Managing software quality for a medical
device for last 4 years
− Team of 48 software testers
− Class C medical device – severe injury or
death possible
4 CONFIDENTIAL

5. 02 Regulations
5 CONFIDENTIAL

6. Regulations
IEC 60601-1-4 – Medical electrical equipment
IEC 60601
• Harmonized standard for medical electrical equipment recognized by
public health authorities in most countries
IEC 60601-1
• Part 1 - General requirements for basic safety and essential
performance
IEC 60601-1-4
• Part 1-4 – Programmable electrical medical systems
6 CONFIDENTIAL

7. Regulations
Software Development Process
IEC 62304
• Medical
Device
Software -
software
lifecycle
process
7 CONFIDENTIAL
AAMI TIR (SW1)
• Guidance on
agile practices
in the Medical
Devise
Software
• (based on IEC
62304)

8. Regulations
Title 21 of the Code of Federal Regulations
21 CFR
Title 21 is the portion of the Code of Federal
Regulations that governs food and drugs within
the US for the Food and Drug Administration
(FDA), the Drug Enforcement Administration
(DEA), and the Office of National Drug Control
Policy (ONDCP)
21 CFR Part 11
Electronic Records;
Electronic Signatures
Validation
8 CONFIDENTIAL
21 CFR Part 820
Quality System
Regulation

9. Regulations
GPSV
General Principles of Software Validation – Guidance for Industry and FDA
staff
− This guidance outlines general validation principles that the Food and
Drug Administration (FDA) considers to be applicable to the validation of
medical device software or the validation of software used to design,
develop, or manufacture medical devices.
9 CONFIDENTIAL

10. Regulations
FDA Premarket Approval (PMA)
PMA (21 CFR Part 814)
Premarket approval (PMA) is the U.S. Food and Drug Administration (FDA)
process of scientific and regulatory review to evaluate the safety and
effectiveness of Class III medical devices
10 CONFIDENTIAL

11. 03 Requirements & V-model
11 CONFIDENTIAL

12. Requirements & V-model
Requirements & Design Hierarchy
User Needs & Use
Cases
System
Requirements
• Can be broken down into
individual subsystems
• Software is one of
subsystems
Software
Requirements
• Identify risk mitigations
• Update software and
system requirements, as
appropriate
12 CONFIDENTIAL
Software Architectural
Design
• Identify SW items
• Identify internal / external
interfaces
• Identify SOUP (including
requirements, HW/SW, &
acceptance criteria)
Software Detailed
Design
• Identify & design SW
units
• Design interfaces

13. Requirements & V-model
V-model
System
Requirements
Software
Requirements
Software
Architecture
Design
Software
Units
Code
Unit Testing
13 CONFIDENTIAL
Integration
Testing
Functional
Testing
System
Testing
Is Tested By
Is Tested By
Is Tested By
Is Tested By

14. Requirements & V-model
Software Risk Identification
− Assign software system safety classification
− Analysis of software contributing to the hazardous situation
− Risk Control
 Traceability
o Hazardous situation to software item
o Software item to software cause
o Software cause to risk control
o Risk control to verification
− Software changes
 Additional causes to hazardous situations?
 Additional risk mitigations?
 Interference with existing mitigations?
14 CONFIDENTIAL

15. Requirements & V-model
Software Safety Classification
Risk Management Severity Software System Safety Classification
Nuisance Class A (No injury or damage to health is
possible)
Minor, Moderate Class B (Non serious injury is possible)
Major, Severe Class C (Death or serious injury is possible)
15 CONFIDENTIAL

16. Requirements & V-model
Change Control
− Record every change request (defect-fix or
enhancement)
− Software Change Review Group (SCRG)
 Approve change requests, features, design changes
 Approve work products (new and changed) to be
delivered
 Notify affected parties of approved changes
 Evaluate residual risks
 Determine acceptability of software for its intended
use
 Review defect trend metrics
 Disposition problem reports
o Problem resolved
o Adverse trends reversed
o Changes implemented appropriately
o Additional problems introduced are addressed
− Configuration items may only be changed in
16 CONFIDENTIAL
response to a change request approved by
SCRG
− Execute applicable design control activities
− Audit trail of change
 Problem reports
 List of changes
 History of change requests

17. 04 Verification & Validation
17 CONFIDENTIAL

18. Verification & Validation
Production Software Release
System testing
Intended Use
testing
Software Development Report (SDR)
Software
Functional Testing
Software Unit Verification
System
Requirements
System Specifications
Product
Requirements
Software Requirements
Specifications
Software
Architecture Design
Traceability for
Test Coverage
Traceability for
Test Coverage
Traceability for
Test Coverage
18 CONFIDENTIAL
(Unit Testing, Code
review and Static code
analysis)
Software Architecture Specification
Software Detail Design
Software Unit
Integration
Product Specifications
SOFTWARE MAINTENANCE PROCESS – Post Production Software Release
SOFTWARE
DESIGN VERIFICATION
SOFTWARE CHANGE MANAGEMENT
(CHANGE CONTROL, PROBLEM RESOLUTION AND CONFIGURATION MANAGEMENT)
SOFTWARE RISK MANAGEMENT
SOFTWARE DESIGN
VALIDATION
Software Detailed
Design
Software
Requirements
Verification Complete Software Release
Software Development Planning
Static Validation
Requirements
Reviews
Static Validation
Requirements
Reviews
Static Verification
Requirements
Reviews
Static Verification
SW Architecture
Reviews
Static Verification
Detailed Design
Reviews
Dynamic Validation
Test Cases / Results
Dynamic Validation
Test Cases / Results
Dynamic Verification
Test Cases / Results
Dynamic Verification
Test Cases / Results
Dynamic Verification
Test Cases / Results Software Unit
Implementation
Controlled Code
Requirements Traceability
Software Development Plan (SDP)
Software Development Report (SDR)
Requirements Traceability
Verification Start Software Release

19. Verification & Validation
Verification vs Validation
Verification
…provides objective evidence that the design outputs of
a particular phase of the software development life cycle
meet all of the specified requirements for that phase.
Software verification looks for consistency,
completeness, and correctness of the software and its
supporting documentation, as it is being developed, and
provides support for a subsequent conclusion that
software is validated. Software testing is one of many
verification activities intended to confirm that software
development output meets its input requirements. Other
verification activities include various static and dynamic
analyses, code and document inspections, walkthroughs,
and other techniques. (GPSV, 3.1.2)
Validation
…confirmation by examination and provision of objective
evidence that software specifications conform to user
needs and intended uses, and that the particular
requirements implemented through software can be
consistently fulfilled. (GPSV)
19 CONFIDENTIAL

20. Verification & Validation
Static vs. Dynamic
• Walkthroughs / Technical
Reviews / Inspections
Static
(IEC 62304:2006, 5.2.6, 5.3.6,
5.4.4, 5.6.5, 5.7.4)
• Cases / Scripts / Procedures /
Protocols / Specs
• Black Box / White Box / Grey Box
• Functional / Structural
• Regression
• Performance / Load / Stress
• Security
• Integration / Unit
• Smoke
• Alpha / Beta
Dynamic
(IEC 62304:2006, 5.5, 5.6, 5.7)
20 CONFIDENTIAL

21. Verification & Validation
Static Verification Elements
Software Requirements
• Implement system requirements
• Traceable to system requirements
• No contradictions
• Not ambiguous
• Uniquely identified
• Testable
Software Architectural Design
• Implements software requirements
• Supports interfaces
• Supports SOUP
Software Detailed Design
• Implements software architecture
• No contradictions
Static code analysis
• Detects defects prior to code
execution
Code Review
• Meets Design inputs
• Implements coding standards
Unit Tests
• Correct
21 CONFIDENTIAL
Integration Tests
• Correct
Functional Tests
• Appropriate
• Traces to software requirements
• Tests or ensures verification of all
requirements
• Pass / fail criteria

22. Verification & Validation
Integration Testing
Two phases:
1. Integration of software units into
software items and the software
system
2. Integration of hardware /
software items; support for
manual operations
Components
 Verifies architecture
 Static verification of test cases
 Controlled code
 Formal problem resolution
 Mandated test case components
 Impact analysis and regression
22 CONFIDENTIAL
testing required

23. Verification & Validation
Functional Testing
 Static verification of test cases
 Trace to SRS
 Prototype software
 Formal problem resolution
 Mandated test case components
 Impact analysis and regression testing required
23 CONFIDENTIAL

24. Verification & Validation
Software Design Validation
 Intended use testing and software system testing
 Static verification of test cases
 Trace to system specification and product specification
 Production or production-equivalent software
 Formal problem resolution
 Mandated test case components
 Impact analysis and regression testing required
24 CONFIDENTIAL

25. Verification & Validation
Impact Analysis
Questions to ask yourself:
1. Did the change work?
2. Did I break something that
worked before?
3. Did I compromise any aspect of
risk management?
a) Introduce new hazards
b) Break existing mitigations
Ways to accomplish:
 Traceability Analysis
 Expert Review
 Architectural Review
 Code Inspection
25 CONFIDENTIAL

26. Verification & Validation
Zero Defects?
IEC 62304:2006
• 3.2 ANOMALY
any condition that deviates from the expected based on requirements specifications, design documents, standards, etc. or from someone’s
perceptions or experiences. ANOMALIES may be found during, but not limited to, the review, test, analysis, compilation, or use of SOFTWARE
PRODUCTS or applicable documentation
• 5.8.2 Document known residual ANOMALIES
The MANUFACTURER shall document all known residual ANOMALIES. [Class B, C]
• 5.8.3 EVALUATE known residual ANOMALIES
The MANUFACTURER shall ensure that all known residual ANOMALIES have been EVALUATED to ensure that they do not contribute to an
unacceptable RISK. [Class B, C]
FDA’s “Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices”
• Unresolved Anomalies (Bugs or Defects)
For Moderate and Major Level of Concern Software Devices, the submission should include a list of all unresolved software anomalies. For
each anomaly, we recommend that you indicate the:
• problem
• impact on device performance
• any plans or timeframes for correcting the problem (where appropriate).
•We recommend that you annotate each item with an explanation of the impact of the anomaly on device safety or effectiveness, including
operator usage and human factors issues. Typically, this list can be generated as an output of a change control board or similar mechanism for
evaluation and disposition of unresolved software anomalies. We recommend that you communicate this list to the end user as appropriate to
assist in the proper operation of the device. In all instances where it is practical to do so, you should include any mitigations or possible
workarounds for unresolved anomalies; this recommendation applies to Blood Establishment Computer Software in particular.
26 CONFIDENTIAL

27. 05 Going Agile
27 CONFIDENTIAL

28. Going Agile
Regulatory Requirements
“But government agencies requires a waterfall method, doesn’t it?”
From IEC 62304
 This standard does not prescribe a specific life cycle model.
The users of this standard are responsible for selecting a life cycle model for the software project
and for mapping the processes, activities, and tasks in this standard onto that model.
 Annex B (informative)
Guidance on the provisions of this standard:
The purpose of this standard is to provide a development process that will consistently produce
high quality, safe medical device software. To accomplish this, the standard identifies the
minimum activities and tasks that need to be accomplished to provide confidence that the
software has been developed in a manner that is likely to produce highly reliable and safe
software products.
28 CONFIDENTIAL

29. Going Agile
V-model Evolution
System
Requirements
Software
Requirements
Software
Architecture
Design
Software
Units
Code
Unit Testing
29 CONFIDENTIAL
Integration
Testing
Functional
Testing
System
Testing
Is Tested By
Is Tested By
Is Tested By
Is Tested By

30. Going Agile
V-model Evolution
30 CONFIDENTIAL

31. Going Agile
Software Development Planning – Increment/Sprint Level
Software Development Planning –
Story Level
Software Requirements
Static Verification: Requirements
Reviews
Software Architecture Design
Static Verification: Architecture Reviews
Software Detailed Design
Static Verification: Requirements
Reviews
Unit Implementation and Verification
(Unit Testing, Code review and Static
code analysis)
Software Integration testing
Dynamic Verification – Test Cases /
Results
Software Functional Testing
Dynamic Verification – Test Cases /
Results
Software
Functional Testing
Integration testing
Dynamic
Verification Test
Cases / Results
Dynamic
Verification Test
Cases / Results
Story
Story
Story
Story
SOFTWARE CHANGE MANAGEMENT
(CHANGE CONTROL, PROBLEM RESOLUTION AND CONFIGURATION MANAGEMENT)
SOFTWARE RISK MANAGEMENT
31 CONFIDENTIAL
System Level
Testing
Intended Use
Testing
Increment
Increment
Increment
Increment
Dynamic
Validation Test
Cases / Results
Dynamic
Validation Test
Cases / Results
Software
Functional Testing
Integration testing
Dynamic
Verification Test
Cases / Results
Dynamic
Verification Test
Cases / Results
Software Development Planning – Release Level
Release
Release
Release
Software Development Planning – Project Level
(Software Development Plan and Software Development Report)
Static Verification: Requirements Reviews
Static Verification: Architecture Reviews
Software Requirements
SW Architecture Design
Static Validation: Requirements Reviews
Static Validation: Requirements Reviews
System Requirements
Product Requirements
SOFTWARE MAINTENANCE PROCESS – Post Production Software Release
SOFTWARE TRACEABILITY
System Requirements to Product Requirements and System testing,
Product Requirements to Software Requirements and Intended Use testing
Software Requirements to Software Functional Testing

32. Questions?
©2014 GlobalLogic Inc. CONFIDENTIAL

33. Thank you!
©2014 GlobalLogic Inc. CONFIDENTIAL