Ever since the European Union (EU) introduced new legislation that requires life sciences companies to proactively detect, prioritize, and evaluate safety signals, there has been an increased interest, not only from sponsors and CROs in the EU, but globally, in pharmacovigilance systems that can assist with the signal management process. Perficient's Chris Wocosky, an expert in signal detection and management, shows how your organization can use Empirica Signal, Oracle's state-of-the-art signal detection system to data mine the existing FDA Adverse Event Reporting System (FAERS) to determine safety signals. This presentation and demonstration willhelp you bettter understand how this solution can be used in daily pharmacovigilance activities.

1. Identifying Safety Signals By Data Mining the
FDA Adverse Event Reporting System with
Empirica Signal
Chris Wocosky, Lead Business Consultant
Life Sciences Business Unit
facebook.com/perficient twitter.com/Perficientlinkedin.com/company/perficient

2. Perficient is a leading information technology consulting firm serving clients throughout
North America and Europe.
We help clients implement business-driven technology solutions that integrate business
processes, improve worker productivity, increase customer loyalty and create a more agile
enterprise to better respond to new business opportunities.
About Perficient

3. • Founded in 1997
• Public, NASDAQ: PRFT
• 2013 revenue ~$373 million
• Major market locations throughout North America
Atlanta, Boston, Charlotte, Chicago, Cincinnati,
Columbus, Dallas, Denver, Detroit, Fairfax,
Houston, Indianapolis, Los Angeles, Minneapolis,
New Orleans, New York City, Northern California,
Philadelphia, Southern California, St. Louis, Toronto
and Washington, D.C.
• Global delivery centers in China, Europe and India
• >2,100 colleagues
• Dedicated solution practices
• ~90% repeat business rate
• Alliance partnerships with major technology
vendors
• Multiple vendor/industry technology and growth
awards
Perficient Profile

4. Oracle Products
• Argus Safety Suite
• Argus Safety Japan
• Affiliate
• Interchange
• Reconciliation
• Dossier
• Insight
• AERS
• TMS
• Empirica Signal
• Empirica Topics
Our Solutions Expertise
Life Sciences Business Unit
Safety & Pharmacovigilance
Provides Consulting, Implementation,
Integration, Upgrade, Migration and
Hosting Services for the most widely
used and highly sought after drug
safety, pharmacovigilance, risk, and
signal management systems in the
world.

5. Introduction
Industry Experience: 17 years
• Medical Device Manufacturing
– Edwards Lifesciences (formerly Edwards
Laboratories) Abbott Vascular (formerly Guidant),
J&J Biosense Webster
• Cardiovascular and Electrophysiology
– Tissue & Mechanical Heart Valves, Coronary
Stents and Ablation Catheters
• Quality/Reliability Engineering, Quality Assurance,
Regulatory Compliance and Complaint
Management
Software Experience: 8 years
• Software Application Sales
– ArisGlobal, Phase Forward, Oracle, BioPharm
Systems (now Perficient)
• Safety & Pharmacovigilance Products
– Register, inforMed, ARISg;
– ClinTrace, Empirica Signal, Study and Topics
– Siebel AECM, Argus Safety Suite
• Pre-Sales Consulting, Implementation Consulting,
Training
Christina Wocosky
Lead Business Consultant
Safety and Pharmacovigilance
Life Sciences Business Unit
o: 951-676-9501
m: 951-505-7011
chris.wocosky@perficient.com

6. Agenda
• Background of Empirica Signal
• Why Empirica Signal?
• Product Overview
• Q&A

7. Background
• Developed 2003-2005 as a (CRADA) Cooperative Research and
Development Agreement between FDA and Lincoln Technologies, Inc.
lead by Bill DuMouchel and Ana Szarfman.
• Objective: User friendly / web based tool for FDA
• WebVDME – Web Visual Data Mining Environment
• Conclusion: Tool assists in identifying unusual patterns – BUT – the
patterns need interpretation!
• 2005 - Phase Forward acquired LT – rebranded as Empirica Signal
• 2010 - Oracle acquired Phase Forward
• In use at FDA today

8. Regulatory Landscape
• Increased focus on technology to save time and money
• Companies are expected to do more with tighter budgets
• Global requirements growing – still not harmonized
• An increased focus on safety related issues

9. Drugs Withdrawn Last 5 years*
Withdrawn Drug name Remarks Country(s)
2013 Tetrazepam Serious cutaneous reactions. EU
2011 Drotrecogin alfa
(Xigris)
Lack of efficacy as shown by PROWESS-SHOCK
study
WorldWide
2010 Propoxyphene
(Darvocet/Darvon)
Increased risk of heart attacks and stroke. WorldWide
2010 Gemtuzumab
ozogamicin
(Mylotargz)
No improvement in clinical benefit; risk for
death.
US
2010 Rosiglitazone
(Avandia)
Risk of heart attacks and death. This drug
continues to be available in the U.S.
EU
2010 Sibutramine
(Reductil/Meridia)
Increased risk of heart attack and stroke. Australia, Canada, China, the EU, Hong
Kong, India, Mexico, New Zealand,the
Philippines, Thailand,the UK, and the US
2010 Sitaxentan Hepatotoxicity Germany
2009 Efalizumab (Raptiva) Increased risk of progressive multifocal
leukoencephalopathy
Germany
*See References

10. Why These Drugs Were Withdrawn
• Risks to the patients, serious side effects
• Unexpected AE’s that were not detected during Phase III Clinical
Trials
• Only became apparent from post-market surveillance data from the
wider patient community
• Not limited to drugs that were ever approved by the FDA
• Some were approved in markets other than the US

11. Empirica Signal Solution Area
• Growing government and public concern about drug safety has
increased the importance of pharmacovigilance
– Traditional methods such as individual case review no longer suffice
due to increased data volume and complexity
– New strategies to proactively identify and expeditiously manage
emerging safety risks are needed
• In this highly competitive and risk-averse environment, it is
essential for regulatory authorities as well as pharmaceutical
companies to establish pharmacovigilance programs
– Need to capitalize on multiple data sources using advanced tools
for developing a thorough and well understood safety profile

12. Process Proposed by CIOMS VIII
Signal
Prioritization
Signal
Detection
Signal
Evaluation

13. GVP Module IX
The signal management process shall cover all steps
from detecting signals (signal detection), through their
validation and confirmation, analysis, prioritization and
assessment to recommending action, as well as the
tracking of the steps taken and of any
recommendations made.

14. The GVP Module IX Process
Signal Detection
Signal Validation
Exchange of Information
Empirica Signal
Empirica Topics
Signal Analysis and
Prioritization
Signal Assessment
Recommendation for Action

15. Why Empirica Signal/Topics?
• The same tool developed and in use by FDA, as well as
the MHRA
• Early detection of new potential signals, or changes in signal
profiles over time
• Track and document every step you take from the first moment a
potential signal is identified, monitoring, verification, decisions and
all actions taken
• Provide regulators with a .pdf with all the necessary information to
back up decisions
If you didn’t document it, it didn’t happen!

16. Signal Detection and Management
Signal Detection
• Monitoring of “Drug Profiles”
with visualization tools
• Quick and easy drill down to the
data under the charts and
graphs
• Create your own data mining
runs to further refine your
analysis
• Compare “like” products side by
side with the same charts and
graphs
Signal Management
• Integrate quantitative signal
detection techniques into routine
safety reviews
• Assess the latest information on
all marketed products and how
“signals” evolve as data
accumulates over time
• Store public and private
comments and blanket
comments for multiple drug-event
combinations
• Store findings with all other data
sources

17. Empirica Signal Benefits
• Early detection of potential safety signals for marketed drugs
• Multiple disproportionality scores available
• Powerful graphical visualizations to facilitate medical interpretation
• Intuitive user interface including drill-down capabilities without
requiring any programming
• Built-in functionality to address confounding and assist in signal
assessment such as stratification, logistic regression and extended
logistic regression
• Extensive built-in signal management, tracking and documentation
functionality

18. Empirica Signal Drug Profiles
• Drug profiles support easy and consistent drug reviews
– Presentations of statistical and other information about one or
more drugs (generic, trade or ingredient)
– Users don’t need to navigate Empirica Signal; instead they can
review their drug using a pre-defined set of charts/graphs
• Display can be in slide show mode or desktop review
• Full power of data mining results, including drilldown, without having to
navigate Empirica Signal
• This graphical, easy to use review tool is directly accessible upon login
without the need to issue any commands
– Layouts define the charts (graphs or report outputs), options and
positioning to be included
– All graphs and tables can be printed directly according to the
user-defined layouts
– Access to publicly available labeling information (FDA, NIH)

19. Comprehensive Drug Profile Layout

20. Empirica Signal
• Proactively identify and
expeditiously manage safety
risks
• Detect and quantify safety
signals by using advanced
data mining techniques that
can be applied to any
spontaneous reporting
database
• Use the most advanced set of
tools to develop a thorough
and well understood safety
profile

21. Sector Map (Heatmap, Treemap)
• Overall Safety Profile of a Drug
• Large tiles are the MedDRA System Organ
Class (SOC)
• Smaller tiles are the Preferred Terms (PT)
• Colors represent the strength of the signal –
bright red indicates the AE’s occurring
significantly higher than expected with the
drug – bright green indicates significantly
less occurrences than expected, or
potential indications.
• The chart below, is the ranking in
descending order by the user selected
disproportionality score – the algorythm in
this example is the (EBGM – Empirical
Bayes Geometric Mean) – also available
are EB05,Chi or PRR.
• The focus is on a particular drug, with the
background being the 5M+ cases in the
FDA AERS database

22. Visual Presentation of Safety Signals
In this map, you
can clearly see
the large
number of AE’s
in the Cardiac
SOC as
represented by
the bright red
color.
This led to the
1998 market
withdrawal of
Seldane

23. Sector Map
• The size of PT tiles is based on
the public health impact of PTs,
which is computed as:
(number of times the PT occurs
in serious-or-fatal cases) *
(proportion of cases with that PT
that are serious-or-fatal).
• PTs with values over a minimum
threshold are included in the
sector map.
• A higher Public Health Impact
score corresponds to a larger
tile in the sector map.
• Since size is based on the
number of cases of the PT alone
(not the number of cases that
have the PT and a particular
drug), the tile size is stable over
sector maps for different drugs.

24. Side by Side Comparison

25. Zoom In on A Sector Map

26. Drug Profile: Desktop View

27. Drug Profile: Slide Show View

28. Drug Profiles: Slide Show View

29. Drug Profile: Slide Show View

30. Drug Profile: Slide Show View

31. Access to Safety Report Data
All of the blue underlined
items are hyperlinks to the
underlying data
The sub-menus are the same
on the charts and graphs

32. Access to Safety Report Data

33. Download Safety Report Data

34. Analytical Graphics for Safety Review

35. Data Mining, Statistics and Data Visualization

36. Query & Reporting
Creating a Case Series with the Query Wizard
Use selection options and/or
saved lists
After selecting variable values,
edit the query logic to combine
query elements

37. Flexible Reporting and Tabulation

38. Empirica Signal Management “Dashboard”
• Supports “strategic pharmacovigilance,” helping drug safety
groups prioritize work and focus resources on areas of
greatest impact
• Combines signal scores with metrics based on case counts
and changes over time
• Easy-to-use table and graphs interface with workflow and
annotation tools

39. Integrated single-table overview
 Drug & event details
 Signal statistics for multiple time points
 Number of new cases in current period
 Trending
 50% increase in EBGM from last period
 Current EB05 > EB95 measured previously
 Indication of labeled event (“expectedness”)
 Indication of FDA “sentinel event”
 Percent of database cases with event that were fatal
 Percent of database cases with event that were serious
 Quadrant – A, B, C, D quadrant (Patrick Waller - MHRA)
 Additional columns are easy to add

40. Annotation
• Annotations at the drug, drug-event combination and case levels
• Permits both free-text and controlled vocabulary comments
• Convenient way to build and share knowledge base

41. Empirica Signal Capabilities
• Data mining and statistics
– Disproportionality statistics including empirical Bayes estimates of
relative reporting rate (EBGM) and non-Bayesian statistics such as
PRR and ROR
– Logistic regression using adverse event as a response variable
– Graphical and tabular presentation of results
• Access to safety report data
– Convenient drilldown to case data from graphs and tables
– Download of signal scores and case data
– Available for both public and private safety databases

42. Empirica Signal Capabilities
• Query & Reporting
– Aggregate as well as line listing reports
– Library of standard reports
– Drilldown, download and publishing capabilities
• Fully web-based application environment
– All processing on the server – can be provided as hosted service
– Simple web browser client – operates through firewalls

43. Empirica Signal Management Capabilities
(Optional)
• Extends Empirica Signal functionality for routine ongoing review by
safety evaluators
• Combines disproportionality measures with other relevant information
such as listedness, fatal outcomes, etc. to enhance the review
process
• Purpose: Timely detection of signals for marketed products while
minimizing the overall monitoring effort
• Capabilities include:
– Easy detection of changes in signal scores
– Structured on-line assessments
– Prioritize, track and document signals
– ‘Drill down’ to AE reports in underlying data

44. Empirica Signal Topics Capabilities (Optional)
• Provides workflow, auditing and tracking capabilities for potential
signals
• Supports topic attachments from multiple sources, either from inside
the Empirica Signal application or from files or web site references
• Provides topics workflow assignment and deadline capabilities
• Topics can encompass multiple individual Topic Actions, each with
their own description, attachments, assignees and deadlines
• Topic visibility can be controlled in great detail
• At any point in time a Topic can be represented in the form of a PDF
or ZIP file for communication and archiving purposes

45. Key Features
• Customizable, easy-to-use Drug Profile Portal
• Enhanced, configurable workflow
• Support for a full range of statistical techniques
• Enhanced, higher performance logistic regression
• Bayesian Multi-item Gamma Poisson Shrinker interaction
calculations for runs with three or more dimensions
• Direct access to safety data through drilldown and case series
features
• Streamlined interface
• Interactive reporting with parameter driven case selection
• Options to download data and case details in native Microsoft
Excel (.xls), .pdf, or rich text (.rtf) format
• Robust, comprehensive online help

46. Key Benefits
• Identify and monitor safety trends using the same tools used by
regulators
• Prioritize and execute critical pharmacovigilance activities
• Employ a variety of classical and Bayesian data mining techniques
• Analyze both drug/event combinations and drug/drug interactions
• Filter out false positives to focus on the most important risks
• Detect problems that may have been overlooked during clinical
development
• Gather intelligence on competing drugs’ safety profiles
• Maintain a comprehensive history of safety related topics
• Track signals and manage actions in a regulatory complaint
environment

47. • Hosting
• Training
• Integration: Argus Argus Mart Empirica Signal Topics
Perficient Offerings

48. Empirica Signal Data Provisioning Services
Subscriptions to periodic updates of prepared data for Empirica Signal
• FDA FOI AERS data (1968 – present)
Substantial data cleaning is conducted to prepare a proprietary version of FOI AERS
safety data; these same cleaning techniques are also used to prepare the FDA-internal
version of the AERS data for the FDA’s use with Empirica Signal:
• Integrate incremental information at database and case level
• Harmonize MedDRA coding of adverse events throughout the cumulative
database across dictionary updates
• Map verbatim drug descriptions to a controlled hierarchical vocabulary of trade
and generic names (WHO-DD and WHO-ATC)
• Identify and flag duplicate cases
• FDA VAERS data (1990 – present)
• Offered as a monthly or quarterly subscription
• WHO-UMC Vigibase database of reports (1968 – present)
• In-house corporate adverse event databases
• Occasional spiking of in-house data into public sources

49. Questions?

50. References
1. Tetrazepam:http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/human/referrals/Tetrazepam_containing_medicinal_products/human_re
ferral_prac_000015.jsp&mid=WC0b01ac05805c516f.htm
2. Xigris: https://investor.lilly.com/releasedetail.cfm?ReleaseID=617602.htm
3. PropoxypheneDarvocet/Darvon):http://www.fda.gov/Safety/MedWatch/SafetyInformation/SafetyAlertsforHumanMedicalProducts/ucm234389.htm
4. Gemtuzumab ozogamicin (Mylotargz):http://www.fda.gov/safety/medwatch/safetyinformation/safetyalertsforhumanmedicalproducts/ucm216458.htm
5. Rosiglitazone (Avandia):
http://www.ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/news/2010/09/news_detail_001122.jsp&mid=WC0b01ac058004d5c1
6. Sibutramine (Reductil/Meridia): http://www.fda.gov/safety/medwatch/safetyinformation/safetyalertsforhumanmedicalproducts/ucm228830.htm
7. Sitaxentan: http://www.mhra.gov.uk/Safetyinformation/DrugSafetyUpdate/CON105752
8. Efalizumab (Raptiva): http://www.fda.gov/Drugs/DrugSafety/PostmarketDrugSafetyInformationforPatientsandProviders/ucm133337.htm
9. CIOMS VIII: http://www.cioms.ch/index.php/publications/available-publications?task=view&id=27&catid=54

51. Thank You!
For more information, please contact:
chris.wocosky@perficient.com
LifeSciencesInfo@perficient.com (Sales)
+44 (0) 1865 910200 (U.K. Sales)
+1 877 654 0033 (U.S. Sales)
www.facebook.com/perficient
www.perficient.com
www.twitter.com/perficient
http://www.oracle.com/us/products/applications/health-sciences/safety/empirica-signal/index.html