1. PREVIOUS NEXT
Evaluating and Investigating Drug Safety
Signals with Public Databases
Rodney L. Lemery, MPH, PhD
Vice President
Safety and Pharmacovigilance
BioPharm Systems, Inc.

2. Evaluating and Investigating Drug Safety Signals with Public Databases 2
Contents
• Brief Overview of Common Language and
Pharmacoepidemiology
• Online Free and Fee-based Databases
– Overview of Online Databases Available for Pay
– Overview of Online Health Databases Available for
Free

3. PREVIOUS NEXT
Part I: Common Language and PV
Overview

4. Evaluating and Investigating Drug Safety Signals with Public Databases 4
Common Language
ADR Adverse Drug Reaction
APR Adverse Product Reaction
CIOMS Council for International
Organizations of Medical Sciences
EMA European Medicines Agency

5. Evaluating and Investigating Drug Safety Signals with Public Databases 5
Common Language…
Much debate on the definition
(we will use the following):
Information that arises from one
or multiple sources, which
suggests a new potentially
causal association, or a new
aspect of a known association,
between an intervention and an
event or set of related events,
either adverse or beneficial, that
is judged to be of sufficient
likelihood to justify verificatory
actions.
(CIOMS, 2010 p.14)
Signal

6. Evaluating and Investigating Drug Safety Signals with Public Databases 6
Common Language…
Much debate on the definition
(we will use the following):
The act of looking for and/or
identifying signals using event
data from any source.
(CIOMS, 2010 p.116)
Signal Detection

7. Evaluating and Investigating Drug Safety Signals with Public Databases 7
Common Language…
Waller (2010, p.50) defines this as
an important and controversial
method of ensuring only those
signals worthy of internal resources
are passed into the formal
evaluation process
—The WHO uses a method similar to
Emergency Room triage processes in
hospital settings to quickly evaluate the
aspects of a case that make it critical
for research while placing other cases
on hold until a later investigation period
—The MHRA uses an analytic
methodology comprised of two
mathematical scores contributing to a
final score that will prioritize the case
—Other articles exist in the literature
suggesting valid decision support
methods
Signal
Prioritization

8. Evaluating and Investigating Drug Safety Signals with Public Databases 8
Common Language
The formal process of reviewing
scientific data sources to refute or
confirm the existence of a signal in
a company product safety profile;
this confirmation will elevate the
signal to a potential or identified
risk
CIOMS VIII (2010, p. 90) indicates that
this process should be multi-faceted:
1.Collect evidence to evaluate causal link
between the product and the event
2.Determine if the signal represents an
identified or potential risk
3.Communicate the identified risk and to
propose its further evaluation and mitigation
Signal
Evaluation

9. Evaluating and Investigating Drug Safety Signals with Public Databases 9
Detailed Signal Management Lifecycle
Signal
Prioritization
Signal
Detection
Signal
Evaluation

10. Evaluating and Investigating Drug Safety Signals with Public Databases 10
Simplified Safety Signal Management Lifecycle
Signal
Prioritization
Signal
Evaluation
Signal
Detection
CIOMS (2010, p. 9)

11. Evaluating and Investigating Drug Safety Signals with Public Databases 11
Pharmacoepidemiologic Studies
Study Design Advantages Disadvantages
Randomized
Control Trial
Most convincing design
Only design which can control for
unknown confounders
Only experimental design
Most expensive
Artificial (nothing like the "real-world“)
Logistically difficult
Ethical objections can lead to non-
investigation (children, very sick
patients)
Cohort Studies Can study multiple outcomes
Can study uncommon exposures
Selection bias less likely (than
case/control)
Unbiased exposure data
Incidence data available
Possibly biased outcome data
More expensive
If done prospectively, may take years to
complete
Case-control
Studies
Can study multiple exposures
Can study uncommon diseases
Logistically easier and faster
Less expensive
Control selection problematic (selection
bias)
Possibly biased exposure data
Analyses of
secular trends
Can provide rapid answers Confounding is not controlled
Case Series Easy quantitation of Incidence No control group, so cannot be used for
hypothesis testing
Case Reports Cheap and easy method for
generating hypotheses
Cannot be used for hypothesis testing
OrderofDifficultyandCausalEvidence

12. Evaluating and Investigating Drug Safety Signals with Public Databases 12
Pharmacoepidemiologic Studies
Case/ControlExposure of Interest
(Unknown)
Disease of Interest
(Known)
Prospective
Cohort
Exposure of
Interest
(Known)
Disease of Interest
(Unknown)
Retrospective
Cohort
Exposure of Interest
(Known)
Disease of Interest
(Unknown)

13. Evaluating and Investigating Drug Safety Signals with Public Databases 13
Reasons for Pharmacoepidemiologic Studies…
• Regulatory
– Required for approval
– Response to audit
• Marketing
– Assist in market penetration by further documenting safety
• Comparator studies
– Increase Name recognition
– Repositioning of drug
• New patient populations (age or gender focus)
• Different outcomes (QOL)
• Explore unintended benefits of the product
• Legal
– In anticipation or response to legal action

14. Evaluating and Investigating Drug Safety Signals with Public Databases 14
Reasons for Pharmacoepidemiologic Studies
• Clinical
– Hypothesis generation
• Increasing our knowledge on the safety profile of new entities
in the market
– Hypothesis testing
• Look at beneficial product effects as well as harmful ones
– Case/Control and cohort studies of estrogen compounds and their use in
preventing osteoporotic fractures
(Strom & Kimmel, 2006, p.59)

15. PREVIOUS NEXT
Part II: Overview of Free Online
Databases

16. Evaluating and Investigating Drug Safety Signals with Public Databases 16
General Principles
Since pharmacoepidemiology studies can be large,
long term and ultimately expensive; the use of existing
databases could aid in the conduct of these types of
observational studies.
– There are two broad kinds of databases
available for use:
• For FREE
• For FEE

17. Evaluating and Investigating Drug Safety Signals with Public Databases 17
For Free Databases
A number of entities have developed simple and
complex online databases available via the web for
querying and display of epidemiologic information.
– CDC-WONDER
– EU-ADR

18. Evaluating and Investigating Drug Safety Signals with Public Databases 18
CDC WONDER
• Wide-ranging Online Data for Epidemiologic Research (WONDER)
– An easy-to-use internet based tool that makes the information
resources of the Centers for Disease Control and Prevention
(CDC) available to public health professionals and the public at
large
– It allows us to search for and read published documents on
public health concerns, including reports, recommendations and
guidelines, articles as well as statistical research data
published by CDC
– Query numeric data sets on CDC's mainframe and other
computers, via "fill-in-the blank" web pages.
• Public-use data sets about mortality (deaths), cancer
incidence, HIV and AIDS, TB, natality (births), census data
and many other topics are available for query, and the
requested data are readily summarized and analyzed.

19. Evaluating and Investigating Drug Safety Signals with Public Databases 19
CDC WONDER
The WONDER
homepage provides
a number of
queryable databases

20. Evaluating and Investigating Drug Safety Signals with Public Databases 20
CDC WONDER (Mortality Rates)
• Assuming that not 100% of the sub-population
afflicted with a disease state dies from the disease
state, we may be able to use mortality rates as a
confirmation or refutation of a suspect ADR
– NOTE: Cause of death records may have
information (classification) bias involved that may
not provide a realistic measure

21. Evaluating and Investigating Drug Safety Signals with Public Databases 21
CDC WONDER (Mortality Rates)
Using [Open] allows us to drive into ICD-10 codes used in
the death classifications

22. Evaluating and Investigating Drug Safety Signals with Public Databases 22
CDC WONDER (Mortality Rates)
Select the
region in
which you
are
attempting
to get rates

23. Evaluating and Investigating Drug Safety Signals with Public Databases 23
CDC WONDER (Mortality Rates)
Various demographic breakdowns are also available in this
database

24. Evaluating and Investigating Drug Safety Signals with Public Databases 24
CDC WONDER (Mortality Rates)
Year and Month
categories are also
available to segregate
the data

25. Evaluating and Investigating Drug Safety Signals with Public Databases 25
CDC WONDER (Mortality Rates)
Autopsy sub-grouping choices

26. Evaluating and Investigating Drug Safety Signals with Public Databases 26
CDC WONDER (Mortality Rates)
Using [Open] allows
us to dive into ICD-
10 codes used in the
death classifications

27. Evaluating and Investigating Drug Safety Signals with Public Databases 27
CDC WONDER (Mortality Rates)
• Options for rate display and data export are also
available in WONDER
• Once all options have been entered, clicking [Send] will
execute the report generation

28. Evaluating and Investigating Drug Safety Signals with Public Databases 28
CDC WONDER (Mortality Rates)
WONDER mortality data for “Stomach Cancers”

29. Evaluating and Investigating Drug Safety Signals with Public Databases 29
CDC WONDER Limitations
• The limitations of this database is that the
information provided does not have any
drug information associated to the diseases
of interest
• This makes the use of the system limited to
finding incidence or prevalence rates of
underlying diseases only
– This would limit the use to only confirmation or
refutation of the potential signal with the
appropriate assumptions made earlier

30. Evaluating and Investigating Drug Safety Signals with Public Databases 30
EU-ADR Web-Based System
• Also available now is an amazing online database that brings
together multiple sources into a single queryable system
• The EU-ADR project is the development of an innovative
computerized system to detect adverse drug reactions
(ADRs), supplementing spontaneous reporting systems.
– EU-ADR will exploit clinical data from electronic healthcare
records (EHRs) of over 30 million patients from several
European countries (The Netherlands, Denmark, United
Kingdom, and Italy). In this project a variety of text mining,
epidemiological and other computational techniques will be
used to analyze the EHRs in order to detect ‘signals’
(combinations of drugs and suspected adverse events that
warrant further investigation).

31. Evaluating and Investigating Drug Safety Signals with Public Databases 31
EU-ADR Web-Based System
Once registered and logged in, the home page has 2 main tabs
– Datasets
– Workflow

32. Evaluating and Investigating Drug Safety Signals with Public Databases 32
EU-ADR Web-Based System
Datasets allow you to create Drug and Event pairs
– The Drugs are coded to the WHO Drug ATC Level 5
– The Events are coded to event term abbreviations specific
to this system

33. Evaluating and Investigating Drug Safety Signals with Public Databases 33
EU-ADR Web-Based System
The Workflow tab allows you to select a particular
method of substantiation and a Drug/Event pair
– The Drugs are coded to the WHO Drug ATC Level 5
– The Events are coded to event term abbreviations specific to
this system

34. Evaluating and Investigating Drug Safety Signals with Public Databases 34
EU-ADR Web-Based System
MEDLINE ADR
This search engine
workflow looks at the
available literature in the
MEDLINE literature
database and looks for
situations where 3 or more
articles exist with the event
and product listed with
subheadings of <<Chemical
induced>> and <<Adverse
effects>>
This is one way to
substantiate your ADR

35. Evaluating and Investigating Drug Safety Signals with Public Databases 35
EU-ADR Web-Based System
MEDLINE Co-occurrence
This search engine workflow
looks at the available literature
in the PubMed literature
database

36. Evaluating and Investigating Drug Safety Signals with Public Databases 36
EU-ADR Web-Based System
DailyMed
This search engine workflow
looks at the available
drug/event pair in the
DailyMed database provided
by the Dutch Universitair
Medisch Centrum
Rotterdam, Netherlands

37. Evaluating and Investigating Drug Safety Signals with Public Databases 37
EU-ADR Web-Based System
Drugbank
This search engine workflow
looks at the available
drug/event pair in the
Drugbank database
maintained by the Dutch
Universitair Medisch
Centrum Rotterdam,
Netherlands

38. Evaluating and Investigating Drug Safety Signals with Public Databases 38
EU-ADR Web-Based System
Substantiation
This search engine
workflow looks at the
clinical connection
between the drug and
the event by considering
drug metabolism and
looking up the
phenotypes of this
interaction against a
database of gene-
disease associations
maintained by the IMIM
(Research Unit on
Biomedical Informatics
(GRIB) IMIM/UPF),
Spain

39. Evaluating and Investigating Drug Safety Signals with Public Databases 39
EU-ADR Web-Based System
If there are not results in the selected engine,
then the search will return no results

40. Evaluating and Investigating Drug Safety Signals with Public Databases 40
EU-ADR Web-Based System Limitations
• The limitations of this database is that the
EHR information used originated from EU
countries only and the results found here
may be limited to only the EU and not
generalizable to the US population
• This system does provide a wonderful
method of substantiating the biologic
plausibility of a Drug/AE pair and does allow
the advanced review of the scientific peer-
reviewed literature for Drug/AE pairs

41. PREVIOUS NEXT
Part III: Overview of Fee-based Online
Databases

42. Evaluating and Investigating Drug Safety Signals with Public Databases 42
For Fee Databases
A number of private entities have developed
databases available that can aid in the display
of information that may help conduct
observational studies or aid in the confirmation
or refutation of identified signals
– Group Health Cooperative
– Kaiser Permanente Medical Care Program
– UK Clinical Practice Research Datalink (CPRD
formally known as the GPRD)

43. Evaluating and Investigating Drug Safety Signals with Public Databases 43
Group Health Cooperative

44. Evaluating and Investigating Drug Safety Signals with Public Databases 44
Group Health Cooperative
• Group Health Cooperative (GHC) is a large non-
profit consumer-directed HMO established in 1947
– Provides health care on a prepaid basis to
~600K people in Washington and Idaho
• GHC has a number of automated and manual
databases whose data serves multiple
epidemiologic studies
– The linking of comprehensive EHR to other
datasets of interests using the consumer
(enrollee number)
– Stable population over time

45. Evaluating and Investigating Drug Safety Signals with Public Databases 45
Group Health Cooperative Summary of Use
A retrospective cohort study of the GHC data looked
at perinatal outcomes, congenital malformations and
early growth and development of infants with and
without prenatal exposure to antidepressants
– Discharge data was used to find all live births from 1986-
1998
– Pharmacy data was used to identify all tricyclic and SSRI
antidepressant prescriptions 360 days prior to delivery
– Infants exposed to antidepressants were matched to those
not exposed
– Blinded (to exposure details) medical reviewers looked at
the various outcomes being studied

46. Evaluating and Investigating Drug Safety Signals with Public Databases 46
Group Health Cooperative Summary of Use
• Infants exposed to tricyclic or SSRIs during
pregnancy were not at an increased risk for
congenital malformations or developmental delay
• Exposure to SSRIs in the third trimester was
associated to lower Apgar scores
• Exposure to SSRIs anytime during pregnancy was
associated to premature birth and lower delivery
weight
– Tricyclic exposure did not have associations to
these outcomes
• (Strom & Kimmel, 2006, p.178)

47. Evaluating and Investigating Drug Safety Signals with Public Databases 47
Group Health Cooperative Limitations
• GHC information has been used primarily to study
drug utilization and AE risk/benefit evaluation of
medicinal products and procedures
• The size of the database does indicate that rare
drug/AE combinations are not likely to be found
• GHC does dictate the drugs available to their
members via their formulary so this may limit the
studies on newer medications on the market

48. Evaluating and Investigating Drug Safety Signals with Public Databases 48
Kaiser Permanente Medical Care Program
• Created in the 1930’s KP was a fee-for-service
medical care initiative originally available only to
construction, shipyard and steel mill workers
employed by Kaiser industries
• Today it is one of the US’ largest non-profit HMOs
– It services over 8.2 million individuals
– Covers eight states

49. Evaluating and Investigating Drug Safety Signals with Public Databases 49
Kaiser Permanente Medical Care Program
Since Kaiser is its own full coverage HMO system, the
data collected on the participants ranges from
pharmacy records, hospitalization records, outpatient
lab results and claims received by non-KP providers
plus other sources
– This empowers researchers to perform large
scale observational trials in the databases using
multiple sources

50. Evaluating and Investigating Drug Safety Signals with Public Databases 50
Kaiser Permanente Medical Care
Program Summary of Use
A retrospective cohort study of patients exposed to
troglitazone (Rezulin) in an attempt to understand the
relative risks associated to hepatic failure and
troglitazone exposure
– Cohort included 9600 diabetic patients with over
three years of Rezulin exposure
– Hospital discharge summaries and procedure
documentation indicative of acute hepatic injury
were identified and ~1200 individual medical
records were reviewed
– 109 of these records were sent to a blinded
panel of hepatologist for outcome adjudication

51. Evaluating and Investigating Drug Safety Signals with Public Databases 51
Kaiser Permanente Medical Care
Program Summary of Use
• The blinded panel identified only 35 cases where the hepatic injury
was attributed only to the use of diabetic medications
• Risk of hepatic failure in patients using Rezulin was not any higher
compared to other diabetic patients
– However, the entire diabetic population did have an increased
risk of hepatic injury compared to the general population
• Currently the spontaneous FDA AERS database places the risk of
hepatic failure in those using Rezulin at 20-25 fold higher than any
other reported drug use
• This observational study disputes this finding and suggests the rate of
hepatic failure in Rezulin users is 1 per 10,000 person-years. To put
it in perspective, according to CDC WONDER data, the mortality rate
for hepatic failures NEC is 0.1347 per 10,000 person-years
• (Strom & Kimmel, 2006, p.182)
• (CDC WONDER Mortality Rates, 2013)

52. Evaluating and Investigating Drug Safety Signals with Public Databases 52
Kaiser Permanente Medical Care
Program Limitations
• Drop-out rates are higher than in similar volunteer
studies
• Some of the datasets (like the cancer and
HIV/AIDS registries) collect race, SES and other
demographics useful in multivariate analysis while
other datasets within Kaiser are missing this data
• KP like other HMOs does restrict their prescription
formularies which may bias the drug use data
• KP records the prescriptions filled data but this is
not an accurate measure of drug
consumption/exposure (it is only a proxy measure)

53. Evaluating and Investigating Drug Safety Signals with Public Databases 53
UK Clinical Practice Research Datalink (CPRD) AKA General
Practice Research Database (GPRD)
The databases built from EHR discussed so far can be
generalized into 2 broad categories
– Administrative
• Administrative records are often captured for billing
purposes and may not have accurate diagnosis data
for use in observational studies (depending on the
research questions being asked)
• These datasets may also be missing needed additional
data like family history, lifestyle practice etc.
– Patient Care
• Given that these records are used in the allopathic
care of the individual patient, their collection and
storage may not be appropriate for the research
questions asked in an observational, epidemiologic
study.

54. Evaluating and Investigating Drug Safety Signals with Public Databases 54
• In the UK, the CPRD is considered to be the world’s largest
medical records database in use by epidemiologists for
investigation
• Originally called the Value Added Medical Products (VAMP)
Research Databank, this system originated in 1987 and has
been adding ~3 million patients per year into the database
ever since
– ~1 million of these patients have more than 11 years worth
of data
– The participants represent ~5% of the general UK
population and are generally represented across SES and
demographic attributes
(Strom and Kimmel, 2006, p. 205)
UK Clinical Practice Research Datalink (CPRD) AKA General
Practice Research Database (GPRD)

55. Evaluating and Investigating Drug Safety Signals with Public Databases 55
• A retrospective cohort study looked at acne patients
from 1987-2002 who had been exposed to
antibiotics and those who had not
• Outcome measures were the occurrence of any
upper respiratory infections over a 12 month period
• Results were adjusted for age, sex, year of
diagnosis, number of prescriptions (for acne),
number of office visits, history of diabetes and
history of asthma
– All potential confounders for the outcome
measure
CPRD Summary of Use

56. Evaluating and Investigating Drug Safety Signals with Public Databases 56
• Acne patients exposed to chronic antibiotic
treatments had and increased risk of URI (OR of
2.15, 95% CI 2.05-2.23)
• This finding did not change when adjusting for the
confounders or the measure of health care seeking
behavior
– The etiology of the URI was not evaluated
(bacterial or viral)
– It is also not known if acne patients are more
prone to URIs independent of antibiotic use
CPRD Summary of Use

57. Evaluating and Investigating Drug Safety Signals with Public Databases 57
• CPRD may contain incomplete information on some
data from specialists and the information contained
may be more biased to more serious medical
diagnoses as minor issues are not always captured
• There is incomplete data present pre-2002 as
consistent EHR weren’t used until that year
• The general size and complexity of this database
requires researchers to have IS staff available for
assistance in the query and analysis of the data
– The CPRD can be accessed via a web interface
to mitigate this complexity
• (Strom and Kimmel, 2006, p. 209)
CPRD Limitations

58. Evaluating and Investigating Drug Safety Signals with Public Databases 58
References
• Abenhaim L, Moore N, Begaud B. (1999). The role of pharmacoepidemiology in
pharmacovigilance: a conference at the 6th ESOP Meeting, Budapest, 28 September
1998. Pharmacoepidemiol Drug Saf. (8 Suppl 1) S1-7
• CDC WONDER Mortality Rates. (2013). Retrieved from http://wonder.cdc.gov/ucd-
icd10.html on September 18th, 2013
• Coloma PM, Trifiro` G, Schuemie MJ et al. On behalf of the EUADR Consortium.
Electronic healthcare databases for active drug safety surveillance: is there enough
leverage? Pharmacoepidemiol Drug Saf. Epub 2012 Feb 8.
• Council for International Organizations of Medical Sciences (CIOMS). (2010). Practical
Aspects of Signal Detection in Pharmacovigilance. Report of CIOMS Working Group VIII,
Geneva .
• EMA. (2012). Guideline on good pharmacovigilance practices. Retrieved from
http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2012/06/W
C500129138.pdf on September 10, 2012
• FDA. (2005). Good Pharmacovigilance Practices and Pharmacoepidemiologic
Assessment. Retrieved from
http://www.fda.gov/downloads/regulatoryinformation/guidances/ucm126834.pdf on
September 10, 2012
• Glass T. A., Goodman, S. N., Hernán, M. A., and Samet, J. M. (2013). Causal inference
in public health. Annual Rev Public Health. 34, pp. 61-67
• Hauben M, Reich L. Drug-induced pancreatitis: lessons in data mining. Br J Clin
Pharmacol. 2004;58(5):560–2.
• Strom, B., Kimmel, S. (2006). Textbook of Pharmacoepidemiology. John Wiley and
Sons, England.

59. Evaluating and Investigating Drug Safety Signals with Public Databases 59
References
• Tan, J. (2010). Adaptive Health Management Information Systems. Jones
and Bartlett Publishers, Sudbury MA, USA
• Waller, P. (2010). An Introduction to Pharmacovigilance. Wiley-Blackwell.
Oxford, UK

60. Evaluating and Investigating Drug Safety Signals with Public Databases 60
Contact
Rodney has over 15 years experience in clinical research
including in-hospital epidemiology, laboratory
experimentation, clinical data management, clinical trial
design, dictionary coding and safety
management/pharmacovigilance.
Rodney has worked for BioPharm Systems for eleven years
now serving in a variety of roles all related to the technical
and/or clinical implementations of software systems used in
the clinical trial process.
Prior to coming to BioPharm Systems Rodney worked at
pharmaceutical and technology companies in the Dictionary
Coding, Statistical Programming and Data Management
areas.
In addition to his current work at BioPharm Systems,
Rodney holds an Contributing faculty position at Walden
University teaching Public Health Informatics and disease
surveillance courses.
Rodney holds a Bachelor of Science in Genetic Engineering,
a Masters of Public Health in International Epidemiology and
a Ph.D. in Epidemiology focusing on Social Epidemiology