Agile in Medical Software Development

O R T H O G O N A L
orthogonal.io
A G I L E I N A Q M S
What Medical Device Companies Need to Know

PRESENTATION AGENDA
orthogonal.ioAgile in a QMS2
Background
Agile Introduction
Agile for Medical Devices
How to be Agile in a QMS
Automation
How to be Agile in a QMS (for Non Developers)
Agile Implementation Mistakes
Resources

I N T R O D U C T I O N
Background on Bernhard and Orthogonal

ABOUT ME
4 Agile in a QMS orthogonal.io
Bernhard Kappe
CHIEF EXECUTIVE OFFICER, ORTHOGONAL
Bernhard is an expert in lean innovation and the
application of agile methods in regulated
industries. He is the author of the eBook Agile in
an FDA Regulated Environment and a frequent
conference speaker on wireless medical devices
and mHealth. He works with Orthogonal clients
on connected care strategy and at the application
of lean and agile methods.

CONNECTED CARE
TECHNOLOGIES

MEDICAL INDUSTRY CLIENTS

A G I L E I N F I V E
M I N U T E S
A Quick recap of the why and what of Agile

CLASSIC WATERFALL
8
Works great when
problems and
solutions are well
known!

THE COST OF CHANGE IN
WATERFALL
9

ITERATIVE METHODS
Fast Feedback Loops to Reduce Uncertainty and
Efficiently Manage Change:
• OODA Loops
• Shewhart/Deming Cycles (PDCA)
• Six Sigma (DMAIC)
• Toyota Production System
• Lean Manufacturing
• Lean Startup
• Lean UX

AGILE
APPROACH
• Break functionality into the smallest units of customer value: the user
story.
• Organize the stories into a prioritized list called a backlog
• Organize development efforts into short (one to two week0 efforts called
iterations or sprints.
• The goal of each iteration: Demonstrate working software
• Take feedback and improve the process through retrospectives and
the software through refactoring.
• Self Organizing Teams

AGILE ITERATIONS
12
• Story Baking – Detailed specification of user stories, including acceptance criteria and ties to FURPS+.
• Story Points – Unit of estimation measuring complexity and size.
• Iteration Planning Meeting – a negotiation between the team and the product owner about what the team will do during the next
Iteration. The product owner and all team members agree on a set of Iteration goals, which are used to determine which product
backlog items to commit from the uncommitted backlog to the Iteration. Often new backlog items are defined during the meeting.
• Daily Standup Meeting - a 10-minute daily meeting for the team
• Pair Programming – two programmers work together on one user story.
• Test Driven Development (TDD) – repetition of a very short development cycle: first the developer writes a failing automated test case
that defines a desired improvement or new function, then produces code to pass that test and finally refactors the new code to
acceptable standards.
• Continuous Integration – code is checked into a continuous integration environment/build server, which runs automated tests (unit,
integration, static and dynamic analysis, etc.)
• Demo/Design Review - On a per iteration basis, functional software is reviewed by stakeholders, including source code review
• Retrospective - what went well and what to improve in the next Iteration.

THE COST OF CHANGE CURVE
REVISITED
13

THE COST OF CHANGE CURVE
REVISITED
14

A G I L E F O R
M E D I C A L
D E V I C E S
Why we need Agile and why it’s taken so long to adopt it

CHANGE AND UNCERTAINTY
• New Product Development
• Complex Systems
• System Interdependencies
• More Users
• Technology Changes
• Market Changes
• Regulatory Changes

MEDICAL DEVICE
CHALLENGES
The Target
Business Value/
Real-World
Outcomes
Patient Safety
Process
Efficiency
Regulatory
Compliance

BARRIERS TO AGILE
• Agile evolved in software, and Medical Devices used to not have much
software.
• Regulations, standards and guidance documents (CFR 21 Part 820, ISO
13485, IEC 62304) were mostly defined before agile, and are written in a
waterfall way.
• Medical Device regulations and standards have additional requirements
(risk management, design controls, verification and validation) that add
complexity to Agile.

THE FDA DESCRIBES RATHER THAN
PRESCRIBES
“The quality system requirements do not dictate the types of design process that a manufacturer must use.
Manufacturers should use processes best suited to their needs. However, whatever the processes may be,
it is important that the design controls are applied in an appropriate manner.”
FDA Design Control Guidance for Medical Device Manufacturers, p. 8
“Based on the intended use and the safety risk associated with the software to be developed, the software
developer should determine the specific approach, the combination of techniques to be used, and the level
of effort to be applied. While this guidance does not recommend any specific life cycle model or any
specific technique or method, it does recommend that software validation and verification activities be
conducted throughout the entire software life cycle.”
General Principles of Software Validation (GPSV), p. 1

RESOURCES: AAMI TIR45:2012
• Provides recommendations for complying
with international standards and FDA
guidance documents when using agile
practices to develop medical device
software.
• Agile brings value to medical device
software: continuous focus on safety, risk
management and control, impact analyses,
. . .
• Designs coherent (locally constant) and
highly cohesive (vs. tightly coupled)
• Recognized as a standard by the FDA in
2013
20

H O W T O B E
Mapping Agile to IEC 62304 and CFR 21 Part 820

APPLYING AGILE UNDER THE
QSR
• Mapping agile (TIR 45) to IEC 62304
• Layering in: Risk Analysis, Design review, etc
• Verification Driven Development
Testing Automation
22

IEC 62304: MEDICAL DEVICE
SOFTWARE LIFECYCLE
PROCESSES
23
8. Software Configuration Management
9. Software Problem Resolution
7. Software Risk Management
5. Software Development Process
5.2 SW
Requirements
Analysis
5.3 SW
Architectural
Design
5.4 SW
Detailed
Design
5.7 SW
System
Testing
5.8 SW
Release
5.1 SW
Development
Planning
5.5 SW Unit
Implement.
& Verification
5.6 SW
Integration &
Integration
Testing

FDA DESIGN CONTROLS GUIDANCE
Footer Text25

V-MODEL: BENT WATERFALL OR
AGILE?
User Requirements
System
Requirements
Specification
Software Detailed
Design
User Acceptance
Testing
System &
Integration
Testing
Unit Testing
Quality Assurance / Risk Control • Relationship between
artifacts are time-
independent
• Finish to finish parallelism
(vs. start to finish)
• Design Inputs ↔ Design
Outputs
• Synchronize Approvals

BEHAVIOR DRIVEN DEVELOPMENT
(BDD)
“Behavior-driven development is an
“outside-in” methodology. It starts at
the outside by identifying business
outcomes, and then drills down into
the feature set that will achieve those
outcomes. Each feature is captured as
a “story”, which defines the scope of
the feature along with its acceptance
criteria.”
• —Dan North
Title (one line describing the story)
Narrative:
As a [role]
I want [feature]
So that [benefit]
Acceptance Criteria: (presented as Scenarios)
Scenario 1: Title
Given [context]
And [some more context]...
When [event]
Then [outcome]
And [another outcome]...
Scenario 2: ...

VERIFICATION DRIVEN AGILE
Verification Test
• Avoids different interpretations of
requirements and specification that are
typical in waterfall development
• This avoids “verification hell” that can double
the length and cost of a project.
• Allows for continued feedback from end
users on working software – further reducing
risk

R I S K
M A N A G E M E N T
I N A G I L E
It’s Iterative

THE ITERATIVE NATURE OF
RISK
“Risk can be introduced throughout the product lifecycle, and risks that
become apparent at one point in the life-cycle can be managed by
action taken at a completely different point in the life cycle.”
ISO 14971: 2007 Medical devices -- Application of
risk management to medical devices

ITERATIVE RISK MANAGEMENT
• Iterative Risk Analysis Takes inputs from and
provides outputs to Human Factors and
Development
• Risk Mitigations are reflected in new stories.
• Risk Mitigations are verified and validated
• The Iterative Risk Management cadence can
be different than the development cadence

HUMAN FACTORS
ENGINEERING
• Lean and agile cadence ships insights and/or UI each iteration
• User experience architecture maps and assesses human
interaction models and requirements
• Usability studies conducted with real users to identify and assess
the effectiveness of vital system interactions
• Interactive prototypes to simulate interactive functions of the
software
• User interface design of software interface using color, animation,
text and graphics
• Result:
– User Engagement
– Real-World Outcomes

A U T O M A T I O N
The key to maintaining quality at speed

TDD+ BDD + TESTING AUTOMATION
34
From this:
… to this

TESTING AUTOMATION
• Full Code Coverage: Unit, Integration, Acceptance Testing built into continuous
integration environment.
• White-box and black-box testing for all elements of the stack,
from hardware to firmware to web services
• Result:
– Maintain Velocity on Larger, More Complex Projects
– Faster Parallel Development of Hardware and Software
– Faster, More Efficient Maintenance

AGILE DESIGN CONTROLS

AGILE ITERATION AND DOCUMENTATION

H O W T O B E
( F O R N O N -
D E V E L O P E R S )
A selection of connected care solutions Orthogonal has
developed for the medical industry.

Agile development is
wasted If none of the
related processes can
iteratively provide and
receive input.
Risk
Managment
Human
Factors
Verification &
Validation
Post Market
Surveillance
Product
Managment
Document
Managment

PRODUCTION PROCESS

PUTTING IT ALL TOGETHER
Business Value/
Real-World
Outcomes
Patient Safety
Process
Efficiency
Regulatory
Compliance
Better usability,
adherence/compliance = Improved
Engagement and Outcomes
Iterative Risk Management = Safer
Products and Reduced
Regulatory Risk
Faster Development
Streamlined Maintenance and
Manufacturing
Platform Development/Reusability

A G I L E
I M P L E M E N TAT I O N
M I S T A K E S

orthogonal.ioCompany Overview45
Area Finding (Gap) Best Practice Consequences of Gap Recommended Approach Dependencies
Quality Management Development Methodology
and Quality Management
System (QMS) are Tightly
Coupled
Loose Coupling of Quality
Management and Development
Approach.
Changes in development methodology as
needed by individual projects may result in
changes and re-approvals of parts of the
QMS which takes time and consumes
significant organizational resources
Allow for different development methodologies by
stating in QMS that any methodology based on a
standard and that meets relevant regulations is
allowed as long as it is approved by Quality.
Consider templated approach at SDP level.
Approvers of QMS (Executive
management, Quality,
Regulatory)
Risk Management Risk Management is not
iterative and integrated into
agile process
QMS states that Risk Management
Plan determines how risk control is
performed throughout the product
lifecycle.
Risk Management that is not iterative and
integrated into agile process may result in an
"Agile Falls" lifecycle where risk "in-stream"
risk analysis and mitigation suffers and risk
management activities become a bottleneck
for other project activities to continue
QMS does not specify risk management
methodology, only that risk control activities will be
performed throughout the project lifecycle as
required by relevant regulations. Risk
Management Plan outlines risk control activities
and their rationales on a per project basis.
QMS Risk Management
Process.
Verification and Validation Verification and Validation
(VnV) is not iterative and
integrated into agile process
QMS states that Validation Plan
determines how VnV is performed
throughout the product lifecycle
VnV that is not iterative and integrated into
agile process may result in an "Agile Falls"
lifecycle where verification testing fails to
indicate when an issue appears and VnV
activities become a bottleneck for other
project activities to continue.
QMS includes a VnV process definition that is
iterative and can be integrated into agile
processes.
QMS Verification And
Validation Process.
Requirements Management Requirements PRDs do not
incorporate Agile User
Stories
Include "story points" in User Stories
as user requirements
User requirements may miss important user
requirements
Include "story points" in User Stories as user
requirements. Also consider using Human Factors
Engineering and Stakeholder (e.g. clinician and
patient) flow analyses as a way of logically
grouping User Stories.
QMS Requirements
Management Process
Requirements Management Inadequate Software Tools
for Requirements
Management and
Traceability
Analyze and determine which tools
for Requirements Management and
Traceability would best fit the
organization.
Poorly specified requirements, inadequate
traceability; inability to analyze logically
categorize requirements
Follow recommendation for which tools to use for
Requirements Management and Traceability that
would best fit the organization
QMS
Testing Automation No unit or integration tests Implement a "test first" approach to
Unit Tests when specifying low level
requirements (i.e. how would this
requirement be tested?).
Poorly specified and tested requirements. Automate unit and integration testing in a
Continuous Integration Environment
QMS Test Plan, Tools
Validation
Testing Automation No Continuous Integration
Environment
Specify interface requirements and
sub-system integration tests during
system development.
Poorly specified and tested requirements. Automate Continuous Integration Environment;
justify when able to do so.
Validation
Testing Automation No Acceptance tests Specify user acceptance tests when
constructing User Requirements.
Poorly specified and tested requirements. Automate acceptance testing (as reasonably
possible)
Validation, Continuous
Integration Environment
Acceptance test automation

#1: TIGHT COUPLING
Finding: Development Methodology and Quality Management System (QMS) are Tightly Coupled
Consequences: Constraints on Implementing Agile. Changes in development methodology as needed by
individual projects may result in changes and re-approvals of parts of the QMS which takes time and
consumes significant organizational resources.
Best Practice: Loose Coupling of Quality Management and Development Approach.
Recommended Approach: Allow for different development methodologies by stating in QMS that any
methodology based on a standard and that meets relevant regulations is allowed as long as it is approved
by Quality. Consider templated approach at SDP level.
Dependencies: Approvers of QMS (Executive management, Quality, Regulatory)

#2: WATERFALL RISK MANAGEMENT
Finding: Risk Management is not iterative and integrated into agile process
Consequences: Risk Management that is not iterative and integrated into agile process may result in an "Agile
Falls" lifecycle where risk "in-stream" risk analysis and mitigation suffers and risk management activities become a
bottleneck for other project activities to continue
Best Practice: QMS states that Risk Management Plan determines how risk control is performed throughout the
product lifecycle.
Recommended Approach: QMS does not specify risk management methodology, only that risk control activities
will be performed throughout the project lifecycle as required by relevant regulations. Risk Management Plan
outlines risk control activities and their rationales on a per project basis.
Dependencies: QMS Risk Management Process.

#3: V&V NOT INTEGRATED
Finding: Verification and Validation (V&V) is not iterative and integrated into agile process
Consequences: V&V that is not iterative and integrated into agile process may result in an "Agile Falls"
lifecycle where verification testing fails to indicate when an issue appears and V&V activities become a
bottleneck for other project activities to continue.
Best Practice: QMS states that Validation Plan determines how V&V is performed throughout the product
lifecycle.
Recommended Approach: QMS includes a V&V process definition that is iterative and can be integrated
into agile processes.
Dependencies: QMS Verification And Validation Process

#4: NO USER STORIES IN THE
PRD
Finding: Product Requirement Documents do not incorporate Agile User Stories
Consequences: Requirements may miss important user requirements, and require additional translation to
agile-usable requirements
Best Practice: Include story definitions in User Stories as user requirements
Recommended Approach: Include story definitions in User Stories as user requirements. Also consider
using Human Factors Engineering and Stakeholder (e.g. clinician and patient) flow analyses as a way of
logically grouping User Stories.
Dependencies: QMS Requirements Management Process

#5: NO AGILE RM TOOLS
Finding: Inadequate Software Tools for Requirements Management and Traceability
Consequences: Poorly specified requirements, inadequate traceability; inability to analyze logically
categorize requirements, lots of extra work.
Best Practice: Analyze and determine which tools for Requirements Management and Traceability would
best fit the organization.
Recommended Approach: Follow recommendation for which tools to use for Requirements Management
and Traceability that would best fit the organization
Dependencies: QMS, Tools Validation

#6: NO TDD OR BDD
Finding: No or insufficient unit, integration and acceptance test coverage.
Consequences: Poorly specified and tested requirements. Bugs, more laborious maintenance
Best Practice: Implement a "test first" approach to Unit Tests, Integration Tests and Acceptance tests
when specifying requirements (i.e. how would this requirement be tested?).
Recommended Approach: TDD and BDD, Automate unit, integration and acceptance testing in a
Continuous Integration Environment
Dependencies: QMS Test Plan, Tools Validation

#7: NO PAIRING
Finding: No Developer Pairing
Consequences: Slower development, greater dependency on individual developers (lower “bus number”),
no built-in design review, less ability to scale projects up and down, higher training costs
Best Practice: Implement Promiscuous Pairing.
Recommended Approach: Training and Mentoring by developers experienced with pairing.
Dependencies: Developers who can pair

R E S O U R C E S

RESOURCES: AAMI TIR45:2012
• Provides recommendations for complying
with international standards and FDA
guidance documents when using agile
practices to develop medical device
software.
• Agile brings value to medical device
software: continuous focus on safety, risk
management and control, impact analyses,
. . .
• Designs coherent (locally constant) and
highly cohesive (vs. tightly coupled)
• Recognized as a standard by the FDA in
2013
54

RESOURCES: ORTHOGONAL
EBOOK
• Addresses the shortcomings of waterfall
development specifically in regulatory
environments.
• How agile development meets the
safety, reliability and regulatory needs of
the medical device and diagnostics
industry.
• How agile development can help ensure
delivery of successful software.
55

Q U E S T I O N S ?
Bernhard Kappe
bkappe@orthogonal.io

Agile in Medical Software Development

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Agile in Medical Software Development

Similar to Agile in Medical Software Development (20)

Recently uploaded

Recently uploaded (20)

Agile in Medical Software Development

Editor's Notes