Medication safety as a use case for argumentation mining
We present a use case for argumentation mining, from biomedical informatics, specifically from medication safety. Tens of thousands of preventable medical errors occur in the U.S. each year, due to limitations in the information available to clinicians. Current knowledge sources about potential drug-drug interactions (PDDIs) often fail to provide essential management recommendations and differ significantly in their coverage, accuracy, and agreement. The Drug Interaction Knowledge Base Project (Boyce, 2006-present; dikb.org) is addressing this problem.
Our current work is using knowledge representations and human annotation in order to represent clinically-relevant claims and evidence. Our data model incorporates an existing argumentation-focused ontology, the Micropublications Ontology. Further, to describe more specific information, such as the types of studies that allow inference of a particular type of claim, we are developing an evidence-focused ontology called DIDEO--Drug-drug Interaction and Drug-drug Interaction Evidence Ontology. On the curation side, we will describe how our research team is hand-extracting knowledge claims and evidence from the primary research literature, case reports, and FDA-approved drug labels for 65 drugs.
We think that medication safety could be an important domain for applying automatic argumentation mining in the future. In discussions at Dagstuhl, we would like to investigate how current argumentation mining techniques might be used to scale up this work. We can also discuss possible implications for representing evidence from other biomedical domains.
Talk for Dagstuhl Seminar 16161: Natural Language Argumentation: Mining, Processing, and Reasoning over Textual Arguments
http://www.dagstuhl.de/en/program/calendar/semhp/?semnr=16161
Medication safety as a use case for argumentation mining, Dagstuhl seminar 16161, 2016 04-19
1. Medication safety as a use case
for argumentation mining
Jodi Schneider and Richard D. Boyce
Dagstuhl Seminar 16161
Natural Language Argumentation: Mining, Processing, and Reasoning
over Textual Arguments
2016-04-19 1
3. Informatics
The management and processing of data, information
and knowledge.
Examples:
o Biomedical informatics
o Dental informatics
o Legal informatics
o Business informatics
o Chemical informatics
o Neurinformatics
o ...
3[Fourman 2002]
5. Evidence Informatics
The management and processing of data, information
and knowledge ABOUT evidence.
Develop end-user applications.
e.g. Information retrieval using arguments & evidence.
o Kevin’s “legal argument roles”
o Benno’s PageRank for arguments
o Retrieve scientific articles by rhetorical or
argumentative features.
5
6. Evidence Informatics
The management and processing of data, information
and knowledge ABOUT evidence.
Develop end-user applications.
e.g. Information retrieval using arguments & evidence.
o Kevin’s “legal argument roles”
o Benno’s PageRank for arguments
o Retrieve scientific articles by rhetorical or
argumentative features.
Seek reusable underlying principles,
shared between several fields.
6
7. My approach to evidence informatics
o Understand user tasks and reasoning.
o Identify domain-specific argumentation schemes.
o Fill a knowledge base with arguments.
• Use domain-specific argumentation schemes as
templates.
• Fill “slots” in the scheme.
• Hand-annotate to bootstrap information extraction.
o Search engine for arguments and evidence
• Use rhetorical structures.
• Use argumentative structures.
7
10. Prescribers consult drug compendia which are
maintained by expert pharmacists.
Medscape EpocratesMicromedex 2.0
10
11. Prescribers consult drug compendia which are
maintained by expert pharmacists.
Medscape EpocratesMicromedex 2.0
11
12. Problem
o Thousands of preventable medication errors occur
each year.
o Clinicians rely on information in drug compendia
(Physician’s Desk Reference, Medscape,
Micromedex, Epocrates, …).
o Compendia have information quality problems:
• differ significantly in their coverage, accuracy, and
agreement
• often fail to provide essential management
recommendations about prescription drugs
12
13. Problem
o Drug compendia synthesize drug interaction
evidence into knowledge claims but:
• Disagree on whether specific evidence items can support
or refute particular knowledge claims
13
14. Problem
o Drug compendia synthesize drug interaction
evidence into knowledge claims but:
• Disagree on whether specific evidence items can support
or refute particular knowledge claims
• May fail to include important evidence
14
15. “Addressing gaps in clinically useful
evidence on drug-drug interactions”
4-year project, U.S. National Library of Medicine R01
grant
(PI, Richard Boyce; R01LM011838)
o Evidence panel of domain experts: Carol Collins,
Amy Grizzle, Lisa Hines, John R Horn, Phil Empey,
Dan Malone
o Informaticists: Jodi Schneider, Harry Hochheiser,
Katrina Romagnoli, Samuel Rosko
o Ontologists: Mathias Brochhausen, Bill Hogan
o Programmers: Yifan Ning, Wen Zhang, Louisa
Zhang
15
16. Goals
o Long-term, provide drug compendia editors with
better information and better tools, to create the
information clinicians use.
o This talk focuses on how we might efficiently
acquire and represent
• knowledge claims about medication safety
• and their supporting evidence
o In a standard computable format.
16
18. Drug Interaction Probability Score
1. Are there previous credible reports in humans?
• If there are case reports or prospective studies that clearly provide evidence
supporting the interaction, answer YES. For case reports, at least one case
should have a “possible” DIPS rating (score of 2 or higher).
• If a study appropriately designed to test for the interaction shows no
evidence of an interaction, answer NO.
…
5. Did the interaction remit upon de-challenge of the
precipitant drug with no change in the object drug? (if no
de-challenge, use Unknown or NA and skip Question 6)
• Stopping the precipitant drug should bring about resolution of the
interaction, even if the object drug is continued without change. …
• If dechallenge of the precipitant drug without a change in object drug did not
result in remission of the interaction, answer NO.
• If no dechallenge occurred, the doses of both drugs were altered, or no
information on dechallenge is provided, answer NA.
[Horn et al. 2007] 18
27. Multiple layers of evidence
Medication Safety
Studies Layer
Clinical Studies and
Experiments
Scientific Evidence Layer
27
28. [Brochhausen, Schneider, Malone, Empey, Hogan and Boyce “Towards a foundational representation of
potential drug-drug interaction knowledge.” First International Workshop on Drug Interaction
Knowledge Representation (DIKR-2014) at ICBO.]
28
30. Scientific Evidence Layer: Micropublications
[Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and
annotations in biomedical communications.]
30
31. Scientific Evidence Layer: Micropublications
[Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and
annotations in biomedical communications] 31
47. Hand-extracting claims and evidence
o Sources
• Primary research literature
• Case reports
• FDA-approved drug labels
o Process
• Spreadsheets
• PDF annotation
47
50. Work to date
o 410 assertions and 519 evidence items transformed
from prior work.
o 609 evidence items (pharmacokinetic potential
drug-drug interactions) annotated by hand from 27
FDA-approved drug labels.
o 230 assertions of drug-drug interactions annotated
by hand from 158 non-regulatory documents,
including full text research articles.
50
52. We are developing a search/retrieval portal
It will:
o Integrate across multiple types of source materials
(FDA drug labels, scientific literature, …)
o Systematize search: Enable ALL drug compendium
editors to access the same info
o Provide direct access to source materials
• E.g. quotes in context
52
54. Evaluation plan for the search/retrieval portal
o 20-person user study
o Measures of
• Completeness of information
• Level of agreement
• Time required
• Perceived ease of use
54
56. My approach to evidence informatics
o Understand user tasks and reasoning
o Identify domain-specific argumentation schemes.
o Create arguments
• Use domain-specific argumentation schemes as
templates.
• Fill “slots” in the scheme.
• Hand-annotate to bootstrap information extraction.
• Automate.
o Provide argument and evidence-based information
retrieval
• Rhetorical functions
• Argumentative structures
56
57. Evidence modeling & curation
o Analogous processes could be used in other fields:
evidence modeling & curation is a general process.
o Biomedical curation is most mature: structured
nature of the evidence interpretation, existing
ontologies, trained curators, information extraction
and natural language processing pipelines
o Curation pipelines need to be designed with
stakeholders in mind.
57
58. Evidence Informatics
The management and processing of data, information
and knowledge ABOUT evidence.
Develop end-user applications such as using
arguments & evidence for information retrieval.
o Kevin’s “legal argument roles”
o Benno’s PageRank for arguments
o Retrieve scientific articles by rhetorical or
argumentative features
Seek reusable underlying principles,
shared between several fields.
58
59. Thanks to collaborators & funders
o Training grant T15LM007059 from the National
Library of Medicine and the National Institute of
Dental and Craniofacial Research
o The entire “Addressing gaps in clinically useful
evidence on drug-drug interactions” team from U.S.
National Library of Medicine R01 grant
(PI, Richard Boyce; R01LM011838) and other
collaborators
59
60. Jodi Schneider, Mathias Brochhausen, Samuel Rosko, Paolo Ciccarese,
William R. Hogan, Daniel Malone, Yifan Ning, Tim Clark and Richard D. Boyce.
“Formalizing knowledge and evidence about potential drug-drug
interactions.” International Workshop on Biomedical Data Mining, Modeling,
and Semantic Integration: A Promising Approach to Solving Unmet Medical
Needs (BDM2I 2015) at ISWC 2015 Bethlehem, Pennsylvania, USA.
Jodi Schneider, Paolo Ciccarese, Tim Clark and Richard D. Boyce. “Using the
Micropublications ontology and the Open Annotation Data Model to
represent evidence within a drug-drug interaction knowledge base.” 4th
Workshop on Linked Science 2014—Making Sense Out of Data (LISC2014) at
ISWC 2014 Riva de Garda, Italy.
Mathias Brochhausen, Jodi Schneider, Daniel Malone, Philip E. Empey, William
R. Hogan and Richard D. Boyce “Towards a foundational representation of
potential drug-drug interaction knowledge.” First International Workshop on
Drug Interaction Knowledge Representation (DIKR-2014) at the International
Conference on Biomedical Ontologies (ICBO 2014) Houston, Texas, USA.
Richard D. Boyce, John Horn, Oktie Hassanzadeh, Anita de Waard, Jodi
Schneider, Joanne S. Luciano, Majid Rastegar-Mojarad, Maria Liakata,
“Dynamic Enhancement of Drug Product Labels to Support Drug Safety,
Efficacy, and Effectiveness.” Journal of Biomedical Semantics. 4(5), 2013.
doi:10.1186/2041-1480-4-5 60
61.
62. Medication Safety Studies Layer:
DIDEO
Brochhausen et al, work in progress, example of Clinical Trial
62
64. Definitions
o Drug-drug interaction
• A biological process that results in a clinically
meaningful change to the response of at least one co-
administrated drug.
o Potential drug-drug interaction
• POSSIBILITY of a drug-drug interaction
• Data from a clinical/physiological study OR reasonable
extrapolation about drug-drug interaction mechanisms
64
67. Existing approaches: Representation
Bradford-Hill criteria (1965)
1. Strength
2. Consistency
3. Specificity
4. Temporality
5. Biological gradient
6. Plausibility
7. Coherence
Bradford-Hill A. The Environment and Disease: Association or Causation?.
Proc R Soc Med. 1965;58:295-300.
67
68. Existing approaches: Representation
Horn, J. R., Hansten, P. D., & Chan, L. N. (2007). Proposal for a new tool to evaluate
drug interaction cases. Annals of Pharmacotherapy, 41(4), 674-680.
68
69. Existing approaches: Representation
Royal Dutch Association for the Advancement of
Pharmacy (2005)
1. Existence & quality of evidence on the interaction
2. Clinical relevance of the potential adverse reaction
resulting from the interaction
3. Risk factors identifying patient, medication or disease
characteristics for which the interaction is of special
importance
4. The incidence of the adverse reaction
Van Roon, E.N. et al: Clinical relevance of drug-drug interactions:
a structured assessment procedure. Drug Saf. 2005;28(12):1131-9.
69
72. Silos: Multiple sources of information
Post-market studies
Reported in
Scientific literature
Pre-market studies Clinical experience
Drug product labels
(US Food and Drug
Administration)
72
Reported in
73. o What arguments are used in medication safety?
o How can these arguments be mined/identified?
o What work needs to be done?
74. Why is a new data model needed?
o Need computer integration
o Want a COMPUTABLE model that can make
inferences
74
Hinweis der Redaktion
http://www.dagstuhl.de/en/program/calendar/semhp/?semnr=16161
We present a use case for argumentation mining, from biomedical informatics, specifically from medication safety. Tens of thousands of preventable medical errors occur in the U.S. each year, due to limitations in the information available to clinicians. Current knowledge sources about potential drug-drug interactions (PDDIs) often fail to provide essential management recommendations and differ significantly in their coverage, accuracy, and agreement. The Drug Interaction Knowledge Base Project (Boyce, 2006-present; dikb.org) is addressing this problem.
Our current work is using knowledge representations and human annotation in order to represent clinically-relevant claims and evidence. Our data model incorporates an existing argumentation-focused ontology, the Micropublications Ontology. Further, to describe more specific information, such as the types of studies that allow inference of a particular type of claim, we are developing an evidence-focused ontology called DIDEO--Drug-drug Interaction and Drug-drug Interaction Evidence Ontology. On the curation side, we will describe how our research team is hand-extracting knowledge claims and evidence from the primary research literature, case reports, and FDA-approved drug labels for 65 drugs.
We think that medication safety could be an important domain for applying automatic argumentation mining in the future. In discussions at Dagstuhl, we would like to investigate how current argumentation mining techniques might be used to scale up this work. We can also discuss possible implications for representing evidence from other biomedical domains.
entry for ‘informatics’ to appear in International Encyclopedia of Information and Library Science (second edition) (0415259010) John Feather and Paul Sturges eds. Routledge 2002
entry for ‘informatics’ to appear in International Encyclopedia of Information and Library Science (second edition) (0415259010) John Feather and Paul Sturges eds. Routledge 2002
entry for ‘informatics’ to appear in International Encyclopedia of Information and Library Science (second edition) (0415259010) John Feather and Paul Sturges eds. Routledge 2002
entry for ‘informatics’ to appear in International Encyclopedia of Information and Library Science (second edition) (0415259010) John Feather and Paul Sturges eds. Routledge 2002
, Joost G. "Rhetorical structure of scientific articles: the case for argumentational analysis in information retrieval." Journal of documentation 47.4 (1991): 354-372.)
entry for ‘informatics’ to appear in International Encyclopedia of Information and Library Science (second edition) (0415259010) John Feather and Paul Sturges eds. Routledge 2002
, Joost G. "Rhetorical structure of scientific articles: the case for argumentational analysis in information retrieval." Journal of documentation 47.4 (1991): 354-372.)
Adverse drug events are a leading cause of death
Image from https://www.njpharmacy.com/wp-content/uploads/2013/02/drug-interactions-checker.png
Image from http://www.clipartbest.com/clipart-McLLpbGKi
Adverse drug events are a leading cause of death
Images from
http://www.knowabouthealth.com/android-version-of-medscape-app-ready-to-download/7568/
Android Play store
http://amazingsgs.blogspot.com/2011/10/top-5-free-android-medical-apps-for.html
Drug Compendia synthesize PDDI evidence into knowledge claims but
May fail to include important evidence
Disagree if specific evidence items can support or refute PDDI knowledge claims
Most sources of clinically-oriented PDDI knowledge disagree substantially in their content,
including about which drug combinations should never be never co-administered. For
example, only one quarter of 59 contraindicated drug pairs were listed in three PDDI
information sources[4], only 18 (28%) of 64 pharmacy information and clinical decisions
support systems correctly identified 13 PDDIs considered clinically significant
by a team of drug interaction experts[5], and four clinically oriented drug information
compendia agreed on only 2.2% of 406 PDDIs considered to be “major” by at least
one source[6].
From our paper: http://ceur-ws.org/Vol-1309/paper2.pdf
4. Wang, L.M., Wong, M., Lightwood, J.M., Cheng, C.M.: Black box
warning contraindicated comedications: concordance among three
major drug interaction screening programs. Ann. Pharmacother. 44,
28–34 (2010).
5. Saverno, K.R., Hines, L.E., Warholak, T.L., Grizzle, A.J., Babits, L.,
Clark, C., Taylor, A.M., Malone, D.C.: Ability of pharmacy clinical
decision-support software to alert users about clinically important
drug-drug interactions. J. Am. Med. Inform. Assoc. JAMIA. 18, 32–
37 (2011).
6. Abarca, J., Malone, D.C., Armstrong, E.P., Grizzle, A.J., Hansten,
P.D., Van Bergen, R.C., Lipton, R.B.: Concordance of severity ratings
provided in four drug interaction compendia. J. Am. Pharm. Assoc.
JAPhA. 44, 136–141 (2004).
Adverse drug events are a leading cause of death
Images from
http://www.knowabouthealth.com/android-version-of-medscape-app-ready-to-download/7568/
Android Play store
http://amazingsgs.blogspot.com/2011/10/top-5-free-android-medical-apps-for.html
I’ve been working with this group since 2012. I’m working on modeling and argumentation.
Horn, J. R., Hansten, P. D., & Chan, L. N. (2007). Proposal for a new tool to evaluate
drug interaction cases. Annals of Pharmacotherapy, 41(4), 674-680.
Hu, M., Mak, V. W. L., & Tomlinson, B. (2011). Simvastatin‐induced myopathy, the role of interaction with diltiazem and genetic predisposition. Journal of clinical pharmacy and therapeutics, 36(3), 419-425.
Hu, M., Mak, V. W. L., & Tomlinson, B. (2011). Simvastatin‐induced myopathy, the role of interaction with diltiazem and genetic predisposition. Journal of clinical pharmacy and therapeutics, 36(3), 419-425.
Hu, M., Mak, V. W. L., & Tomlinson, B. (2011). Simvastatin‐induced myopathy, the role of interaction with diltiazem and genetic predisposition. Journal of clinical pharmacy and therapeutics, 36(3), 419-425.
Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications. Journal of Biomedical Semantics, 5(1), 1. http://dx.doi.org/10.1186/2041-1480-5-28
Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications. Journal of Biomedical Semantics, 5(1), 1. http://dx.doi.org/10.1186/2041-1480-5-28
From http://dailymed.nlm.nih.gov/dailymed/fda/fdaDrugXsl.cfm?setid=13bb8267-1cab-43e5-acae-55a4d957630a&type=display
entry for ‘informatics’ to appear in International Encyclopedia of Information and Library Science (second edition) (0415259010) John Feather and Paul Sturges eds. Routledge 2002
, Joost G. "Rhetorical structure of scientific articles: the case for argumentational analysis in information retrieval." Journal of documentation 47.4 (1991): 354-372.)
DIDEO:
A potential drug-drug interaction (PDDI) is an information content entity that specifies the possibility of a drug-drug interaction based on either reasonable extrapolation about drug-drug interaction mechanisms or a data item created by clinical studies, clinical observation or physiological experiment.
Implementation/specification of Bradford-Hill to DDIs/PDDIs
1. Are there previous credible reports of this interaction in humans?2. Is the observed interaction consistent with the known interactive properties of precipitant drug?3. Is the observed interaction consistent with the known interactive properties of object drug?4. Is the event consistent with the known or reasonable time course of the interaction (onset and/or offset)?
5. Did the interaction remit upon dechallenge of the precipitant drug with no change in the object drug? (if no dechallenge, use Unknown or NA and skip Question 6)
6. Did the interaction reappear when the precipitant drug was readministered in the presence of continued use of object drug?
7. Are there reasonable alternative causes for the event?a8. Was the object drug detected in the blood or other fluids in concentrations consistent with the proposed interaction?9. Was the drug interaction confirmed by any objective evidence consistent with the effects on the object drug (other than drug concentrations from question 8)?10. Was the interaction greater when the precipitant drug dose was increased or less when the precipitant drug dose was decreased?
Animation here
Product labeling is incomplete
Search strategy
No standard way of searching/assessing the evidence
By reducing the variability in searching (more standardize)
(others working on standardizing assessing evidence)
No standard way to synthesize