This presentation provides an overview of risk-based monitoring and how clinical trial management systems (CTMS) and electronic data capture (EDC) analytics can help identify and manage risks during clinical studies. It discusses how guidance is moving away from traditional on-site monitoring towards more flexible, risk-based approaches. Key performance indicators and aggregate data analysis can be used to generate risk profiles for sites and identify changing risks over time. This allows sponsors to monitor studies more efficiently while still ensuring subject protection and data quality. The role of monitors is changing from on-site verification to activities like data monitoring, root cause analysis, and proactive risk management.

1. Tammy Finnigan
Nov 06, 2012

2. Objectives
This presentation will aim to:
• Provide a brief overview of the guidance that is driving the change in
how the industry approaches monitoring
• Define risk based monitoring
• Examine the process and data required to generate risk profiles for
investigational sites
• Demonstrate how CTMS and EDC analytics can be used to identify
and manage changing risks during study conduct
• Consider the changing role of the monitor
2

3. Background
Legislation The role of the monitor
CFR and EU CTDir ICH-
ICH-GCP
“protect the safety and well being of
“Sponsors of clinical
human subjects and the quality and
investigations are required to integrity of data”
provide oversight to ensure
Guidance from regulators:
adequate protection of the
“most effective way “ to monitor a clinical
rights, welfare and safety of
trial was to “maintain personal contact
human subjects and the between the monitor and the investigator
quality and integrity of the throughout the clinical investigation”
data” interpretation:
Industry interpretation
Site qualification
Regular monitoring visits (4-8wk intervals)
100% data verification
3

4. Change in industry guidance
Recent Industry Guidance Growing consensus
Re-
Re-visited ICH-E6
ICH- Risk-based approaches to monitoring, focusing
Flexibility in how trials are monitored on critical data elements, more likely to:
Advice to sponsors: • ensure subject protection
• overall study quality
Consider “the objective, purpose,
design, complexity, blinding, size and Will permit sponsors to monitor the conduct of
endpoints of a trial” in determining the clinical trials more efficiently and effectively
extent and nature of monitoring than the current approach of routine visits to all
clinical sites and 100% data verification
Guidance specifically provides for the
possibility of reduced, or even no* Interesting point:
onsite monitoring and address data Recent survey by an EDC vendor on >2000
quality and site performance through protocols demonstrated that < 3% of the clinical
centralized methods study data was changed due to onsite
*Guidance is clear only appropriate in monitoring
exceptional circumstances
4

5. Risk Based Monitoring
• Identify likely • Critical data
sources of • Investigator • On-going
• Processes
error experience data analysis
identified in the
Protocol Study Site
• Impact of error Monitoring risk • Historical Proactive • Identify non-
Risk Risk
• Likelihood of Plan assessment performance Risk Mgmt. compliance
Assessment Assessment
error • On site and • Standard of • Identify
• Ability to centralized care in country outliers
detect error activities
Considerations Considerations Considerations Considerations
• Type of data being • Complexity of study • Outcome of protocol risk • Aggregate data analysis
collected design assessment and and review outlying sites
• Specific activities • Types of end points monitoring plan • Analyse site
required for data • Stage of study (taper • Site quality assessment characteristics and
collection monitoring) • Increased areas of risk correlate with poor
• Data critical to reliability • Standard of medical for the site performance and non-
of study findings care • Additional area of risk compliance trends
• Safety concerns • Type of monitoring for the site • Adjust monitoring
• Subject protection activities (frequency, • Site specific monitoring activities based on the
concerns intensity, targeted, plan analysis
random, onsite/central)
5

6. Site Risk Profiles
• Not within the context of a specific study
• Based on historical performance data across studies
Site risk • Set risk threshold for key performance indicators
• On-going quality management and development of investigator and site relationship
profile
• Based on protocol risk assessment and protocol monitoring plan
• Cross reference protocol risk assessment against Site risk profile
Study site • Include outcome of site qualification visit as applicable
risk profile • Identify areas of additional risk for the site
• Modify the protocol monitoring plan to accommodate additional site specific risks
• Study risks should always be addressed in the study site monitoring plan
Study site • Additional focus or intensity of monitoring activities may be planned for individual sites to
mitigate specific risks
monitoring plan
6

7. Key Performance Indicators
The site risk profile should provide a continuous picture of the site
F
Subject Safety Data Quality Performance
- Protocol violations - Data query rate - Data reporting time
- Screen failure rates - % of missing data - Query response time
- Drop out rates - Protocol violations and - Critical site issues
- Compliance with safety deviations - Issue resolution time
and ethics reporting - Source data verification - Recruitment rates
timelines issues - GCP training
7

8. Proactive Risk Management
Improved
site and
study
performance
Non-study
Study related
related
8

9. Centralized vs. On Site Monitoring
When is On Site
Centralized Monitoring
On Site Monitoring Monitoring Effective?
Recent survey by an Inexperienced sites or site Solely data monitoring
EDC vendor on >2000 staff i.e. new to clinical
• Statistical analysis
protocols demonstrated trials, new to therapy area
that < 3% of the clinical • Exception reporting
Sponsor has no/little
study data was changed • Data violations
previous experience with
due to onsite monitoring • Missing data
the site
With advances in • Data aggregation
Training
remote data capture
• Root cause analysis
systems, safety/PVG Maintaining and improving
systems and internet investigator relationships • Trend analysis
access across the When physical verification
globe, is onsite is required i.e. drug Feedback loop into
monitoring effective? accountability protocol and site risk
assessments
9

10. Study Related Risk Management
During the study, off site, centralized monitoring activities and data analysis
can be used to identify risk:
Aggregate Data Analyse Site
Statistical Methods Monitor Data Quality
Analysis Characteristics
Protocol specific rules Missing data Aggregate data across study sites Data anomalies
Parametric algorithms Complete trend analysis and
Inconsistent data Higher frequency of errors
(probability rules) determine ‘expectedness’
Review sites in relation to other
Fraud and bias detection Potential protocol deviations Higher protocol violations
sites i.e. protocol distribution
Exception reporting Data outliers Identify outliers Delays in data reporting
10

11. Aggregate Data Analysis
Review site data in relation to other study sites:
Protocol Violations
PV bubble chart from study site data analysis dashboard to show protocol site distribution of
protocol violations, highlight outlier sites (with hover over displaying # by category i.e. eligibility,
informed consent, IMP)
- In this example there are sites that display disproportionate number of violations
compared with other study sites
- Root cause analysis
- Drill down to the violation type, are there specific risk factors for subject
safety and/or data integrity?
- Mitigation – additional site training visit
11

12. Aggregate Data Analysis
Screen Failure Rates
SF bubble chart from study site data analysis dashboard shows protocol site distribution of
screen failure rates, highlight outlier sites (hover over displaying # by reason i.e. withdrew
consent, abnormal labs)
- Are there sites that display disproportionate number of screen failures compared
with other study sites?
- Root cause analysis
- Drill down to the screen failure reasons, are there specific risk factors for
subject safety?
- What practices are the site following?
- What mitigation needs to be put in place?
- Are there high rates of specific screen failure reasons across sites and countires?
- Root cause analysis
- Feed back into the protocol risk assessment
- Have we detected something unexpected?
- Do we need to change our risk assessment and monitoring plans?
12

13. Aggregate Data Analysis
Review study data to identify trends:
Protocol Dropouts
Early withdrawal scatter diagram from study subject data dashboard show sprotocol distribution
of early withdrawal reasons, and rates by country
- Trend analysis
- Is this the identification of additional risks to subjects and/or study
procedures
- Assess against protocol risk assessment
- Do we need to adjust our monitoring and quality plans
- Do we need to conduct additional training
- Is there a need for country specific amendments
13

14. Site Characteristics
Review site data to identify sites with characteristics correlated
with poor performance and non-compliance:
Study Site Risk Profile
Radar diagram from study site risk dashboard, to show site risk characteristics in relation to
study ‘normal’ rate
- The site is compared with the protocol average for critical risk factors to determine
if there are:
• A higher number of data errors
• A lower number of subject dropouts
• Longer lag time entering data
14

15. Proactive Risk Management
Determine appropriate mitigation,
update to risk assessments
Follow escalation process
identified in monitoring plan
Identify new risks (study and/or
study site)
Review aggregate data
dashboards
15

16. Risk Mitigation
Study level
Unexpected trends identified in the protocol data may result in the need to
change monitoring activities across the study e.g.
• Data previously deemed to be non-critical may need to be monitored more closely
• Trends in subjective end point analysis may result in more onsite monitoring
activities
• Trends in protocol violations may point to the need for additional training of site
staff
16

17. Risk Mitigation
Site level
Unexpected site characteristics identified in the study site data may result in
the need to modify the monitoring activities for the individual site e.g.
• More intense data monitoring if there is an inordinate number of data
errors in the sub-set that is monitored for all sites
• Targeted onsite monitoring visits
• Tapered monitoring visits i.e. additional site visits in the early stages
of a study if high number of eligibility violations or screen failures,
address training
The continuous picture of the site should not be forgotten during the study
• Site profile should be monitored for change
• As this is based on cross study performance, the site may move
above/below the identified thresholds
• Study teams should be informed
17

18. Changing Roles
The role of the monitor as currently defined, is going to change.
Consider the following model:
Site Quality Manager Site Manager Data (Quality) Manager
Create site risk profile Site relationship management Data monitoring and analysis
Assess site against study risk
Site training Root cause and trend analysis
profile
Create study site monitoring
Onsite data verification Ongoing study risk management
plan
18

19. Benefits of risk based monitoring
Real time identification of risk will allow sponsors and CROs to:
• Make informed decisions
• Be proactive in managing potential impact to subject safety
and data integrity
On-going management of site risk profiles will ultimately
improve site and study performance
19

20. Questions
20

21. Presenter’s Biography and Contact details
Tammy Finnigan, COO, Triumph Consultancy Services
tammy.finnigan@triumphconsultancy.co.uk
Tammy’s entire career has been focused on clinical research,
having worked in project management and clinical operations for
10 years, with both large Pharma and CRO businesses prior to
joining Triumph. Her experience both in monitoring, and
managing clinical trials made her a significant hire for Triumph in
2007. Tammy’s experience, passion and eye for quality saw her
promoted to Head of EU Operations within her first year, and in
2011 she was appointed COO to take over global operations
responsibility.
21