This document discusses a project to construct a knowledge base linking drug interaction assertions to evidence from source documents. It will use the Micropublications Ontology to represent each assertion's support graph of claims and evidence, and the Open Annotation model to dynamically link support graph elements to quoted text excerpts from sources. The knowledge base will help answer competency questions about assertions, evidence, and their provenance. Challenges include representing both structured and unstructured text claims and efficiently querying the evidence base at scale.
All These Sophisticated Attacks, Can We Really Detect Them - PDF
Linked Science at ISWC 2014 using Micropublications and Open Annotation
1. Linked Science at ISWC 2014
Riva del Garda, Trentino, Italy
19 October 2014
Jodi Schneider, Paolo Ciccarese,
Tim Clark and Richard D. Boyce
2. Goal of this project
Construct & maintain
a knowledge base linking to evidence
i.e. data, methods, materials
where:
• Each ASSERTION in the knowledge base
has a SUPPORT GRAPH of claims and evidence
• Each SUPPORT GRAPH element (claims, data, methods, materials)
is dynamically linked to specific QUOTED
ELEMENTS in source documents on the Web
3. Why? It's time-consuming to find
the state of the art in a field!
• What do we know about field F? assertion X?
• What evidence supports assertion X?
• What assumptions are used in research
supporting assertion X?
4. Application domain: medication safety
• Potential drug-drug interactions
– 2+ drugs, where interaction is known to be possible
• Adverse drug event
– Harm caused by medication
– Huge public health issue
> 1.5 million preventable adverse drug events/year
(USA)
• Post-market safety issues
5. Drug information sources
• Evidence is selected & assessed by editorial boards
– MICROMEDEX, First DataBank, Q-DIPS
• E.g. MICROMEDEX:
– "In-house team of 90+ clinically-trained editorial staff"
(physicians, clinical pharmacists, nurses, medical librarians)
– "Content is reviewed for clinical accuracy and relevance."
– "Critical content areas may undergo an additional review by
members of our Editorial Board."
• Potential problems
– a time-consuming (i.e. expensive), collaborative, process
– maintaining internal and external inconsistency is non-trivial
6. Part of a larger effort
• “Addressing gaps in clinically useful evidence on
drug-drug interactions”
• 4-year project, U.S. National Library of Medicine
R01 grant (PI, Richard Boyce)
• Evidence panel of domain experts
(Carol Collins, Lisa Hines, John R Horn, Phil Empey)
& informaticists
(Tim Clark, Paolo Ciccarese, Jodi Schneider)
• Programmer: Yifan Ning
7. Build on 3 things
• Drug Interaction Knowledge Base [Boyce2007,
Boyce2009]
• Open Annotation Data Model [W3C2013]
• Micropublications Ontology [Clark2014]
8. Drug Interaction Knowledge Base
(DIKB)
– Hand-constructed knowledge base
– Safety issues when 2 drugs are taken together
– Focus is on EVIDENCE
[Boyce2007, Boyce2009]
9. Drug Interaction Knowledge Base
(DIKB) - Boyce 2007-2009
– Hand-constructed knowledge base
– Safety issues when 2 drugs are taken together
– Focus is on EVIDENCE
All assumptions are linked to evidence
Enables the [Boyce2007, system Boyce2009]
to identify when
assumptions are no longer valid
10. DIKB supports queries about
assertions & evidence:
• Get all assertions that are supported by a
U.S. FDA regulatory guidance statement
• Are the evidence use assumptions are
concordant, unique, and non-ambiguous?
• Which assertions are supported/refuted by
just one type of evidence?
[Boyce2007, Boyce2009]
16. Micropublications Ontology (MP)
http://purl.org/mp
Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications
17. Goal of this project
Construct & maintain
a knowledge base linking to evidence
i.e. data, methods, materials
where:
• Each ASSERTION in the knowledge base
has a SUPPORT GRAPH of claims and evidence
• Each SUPPORT GRAPH element (claims, data, methods, materials)
is dynamically linked to specific QUOTED
ELEMENTS in source documents on the Web
18. Modeling strategy
Construct & maintain
a knowledge base linking to evidence
i.e. data, methods, materials
where:
• Each ASSERTION in the knowledge base
has a SUPPORT GRAPH of claims and evidence: MP
• Each SUPPORT GRAPH element (claims, data, methods, materials)
is dynamically linked to specific QUOTED
ELEMENTS in source documents on the Web
19. Modeling strategy
Construct & maintain
a knowledge base linking to evidence
i.e. data, methods, materials
where:
• Each ASSERTION in the knowledge base
has a SUPPORT GRAPH of claims and evidence: MP
• Each SUPPORT GRAPH element (claims, data, methods, materials)
is dynamically linked to specific QUOTED
ELEMENTS in source documents on the Web: OA
20. Quotes integrated (MP using OA)
http://purl.org/mp
Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications
21. Enhancing the DIKB with MP and OA
1. Represent the overall argument of the paper
– Support & challenge relationships
– Data, methods, materials
2. Semantic tagging, so drugs & proteins can be
queried using knowledge from other sources
3. Make quotes actionable (highlight in orig doc)
4. Handle new competency questions
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33. Quote stored in OA, with link to source
ex:annotation-1
ex:body-1 ex:target-1
Predicate Object
rdf:type mp:Method
rdf:value (exact text)
about
Predicate Object
rdf:type oa:SpecificResource
oa:hasSource <http://dailymed…>
oa:hasSelector ex:selector-1
34. Quote stored in OA, with link to source
ex:annotation-1
ex:body-1 ex:target-1
Predicate Object
rdf:type mp:Method
rdf:value (exact text)
about
Predicate Object
rdf:type oa:SpecificResource
oa:hasSource <http://dailymed…>
oa:hasSelector ex:selector-1
ex:selector-1
Predicate Object
oa:prefix (preceding text)
oa:exact (exact text)
oa:postfix (following text)
35. New competency questions to answer
1. Finding assertions and evidence
• List all assertions that are not supported by evidence
– By data, by methods, by materials
• What is the in vitro evidence for assertion X? the in vivo
evidence?
– With provenance: Give me back the original data tables
2. Enabling updates
• List all evidence that has been flagged as rejected from
entry into the knowledge base
– By data, by methods, by materials
36. New competency questions to answer
3. Assessing the evidence
• Which research group conducted the study used for
evidence item X?
• What are the assumptions required for use of this
evidence item to support/refute assertion X?
– Without directly entering them
4. Statistics for analytics/KB maintenance
• Number of evidence items for and against each
assertion type
– By data, by methods, by materials
37. Modeling challenges
• To date, MP has not been used to represent
both unstructured text claims
("escitalopram does not inhibit CYP2D6")
and logical representation of text as
normalized subject-predicate-object
(nanopublication of statement)
• Efficient querying will be needed, even when
the evidence base scales. We are using an
iterative design-and-test approach.
38. Future work
• NLP support: Create a pipeline for extracting
potential drug-drug interaction (PDDI) mentions
from scientific & clinical literature
• Usability tests: Tools usable by domain experts
• NLP + "crowdsourcing" (distributed annotation)
• Resolving links to paywalled PDFs
39. Acknowledgements
• Funding
– ERCIM Alain Bensoussan fellowship Program
under FP7/2007-2013, grant agreement 246016
– National Library of Medicine (1R01LM011838-01)
• Thanks to the Evidence Panel of Addressing
PDDI Evidence Gaps: Carol Collins, Lisa Hines,
and John R Horn, Phil Empey
• Thanks to programmer Yifan Ning
Hinweis der Redaktion
20 min including questions
http://linkedscience.org/events/lisc2014/
Paper: http://jodischneider.com/pubs/lisc2014.pdf
Semantic web technologies can support the rapid and transparent validation of scientific claims by interconnecting the assumptions and evidence used to support or challenge assertions. One important application domain is medication safety, where more efficient acquisition, representation, and synthesis of evidence about potential drug-drug interactions is needed. Exposure to potential drug-drug interactions (PDDIs), defined as two or more drugs for which an interaction is known to be possible, is a significant source of preventable drug-related harm. The combination of poor quality evidence on PDDIs, and a general lack of PDDI knowledge by prescribers, results in many thousands of preventable medication errors each year. While many sources of PDDI evidence exist to help improve prescriber knowledge, they are not concordant in their coverage, accuracy, and agreement. The goal of this project is to research and develop core components of a new model that supports more efficient acquisition, representation, and synthesis of evidence about potential drug-drug interactions. Two Semantic Web models—the Micropublications Ontology and the Open Annotation Data Model—have great potential to provide linkages from PDDI assertions to their supporting evidence: statements in source documents that mention data, materials, and methods. In this paper, we describe the context and goals of our work, propose competency questions for a dynamic PDDI evidence base, outline our new knowledge representation model for PDDIs, and discuss the challenges and potential of our approach.
Create an audit trail between assertions, evidence, and source documents, so that assertions and evidence can be flagged for update in flexible and intelligent ways.
Statistics from Institute of Medicine Report Brief, July 2006 on Preventing Medication Errors
http://www.iom.edu/~/media/Files/Report%20Files/2006/Preventing-Medication-Errors-Quality-Chasm-Series/medicationerrorsnew.pdf
# How many pharmacokinetic studies in the DIKB could be used to support or refute
# an increases AUC assertion?
# How many AUC studies are in the DIKB that are based on data from the
# product label?
# Are the evidence use assumptions are concordant, unique, and non-ambiguous
# Get all assertions that are supported by an FDA guidance statement
# SHOW THE DISTRIBUTION OF THE LEVELS OF EVIDENCE FOR MECHANISTIC ASSERTIONS
# WHAT ASSERTIONS ARE SUPPORTED/REFUTED BY JUST ONE TYPE OF EVIDENCE?
## which items in the DIKB have evidence for and against that is both
## from product labeling?
# number of assertions in the system
# number of evidence items for and against
## find assertions that are not supported by evidence
# what single evidence items act as as support or rebuttal for multiple substrate_of assertions?
#remove a older version of a redundant evidence item
## change evidence for to evidence against
# has this evidence item been rejected
# what other assertions are being supported/challeged by this evidnece item?
Annotations in the data model are a set of RDF resources that connect some target to a set of resources that are in some way about it.
Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications. J. Biomedical Semantics 5: 28 (2014)
Create an audit trail between assertions, evidence, and source documents, so that assertions and evidence can be flagged for update in flexible and intelligent ways.
Create an audit trail between assertions, evidence, and source documents, so that assertions and evidence can be flagged for update in flexible and intelligent ways.
Create an audit trail between assertions, evidence, and source documents, so that assertions and evidence can be flagged for update in flexible and intelligent ways.
Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications. J. Biomedical Semantics 5: 28 (2014)
New competency questions handled:
Assertions that are not supported by evidence already in the evidence base, the evidence that should be checked most thoroughly (e.g. evidence that by itself supports multiple assertions), and the data, methods, and materials associated with a given evidence item as described in source documents.
1. Finding assertions:
(a) List all assertions that are not supported by evidence
(b) Which assertions are supported (or refuted) by just one type of evidence?
(c) Which assertions have evidence from source X (e.g., product labeling)
(d) Which assertions have both evidence for and evidence against from a
single source X?
2. Finding evidence: (a) List all evidence for or against assertion X (by evidence type, drug, drug
4.2
pair, transporter, metabolic enzyme, etc.)
(b) What is the in vitro evidence for assertion X? the in vivo evidence?
(c) List all evidence that has been flagged as rejected from entry into the the knowledge base
(d) Which single evidence items act as support or rebuttal for multiple as- sertions of type X (e.g., substrate of assertions)?
Assessing the evidence:
3. Understanding evidence coming from a given study:
(a) What data, methods, materials, are reported in evidence item X?
(b) Which evidence items are related to and follow-up on evidence item X?
(c) Which research group conducted the study used for evidence item X?
(d) AretheevidenceuseassumptionsforevidenceitemXconcordant?unique?
non-ambiguous?
4.3
Verifying plausibility of an evidence item:
(a) Has evidence item X been rejected for assertion Y? If so, why and by whom?
(b) Which other assertions are being supported/challenged by this evidence item?
(c) What are the assumptions required for use of this evidence item to sup- port/refute assertion X?
Checking assertions about pharmacokinetic parameters (i.e., area under the concentration time curve (AUC))
(a) How many pharmacokinetic studies used for evidence items in the DIKB
could be used to support or refute an assertion about pharmacokinetic
paramater X (e.g., ‘X increases AUC’)?
(b) How many pharmacokinetic studies in the DIKB used for evidence items
for assertion X are based on data from the product label?
(c) What is the result of averaging (or applying some other statistical oper- ation) to the values for pharmacokinetic parameter X across all relevant
studies used for evidence items?
Checking for differences in the product labeling:
(a) Are there differences in the evidence items that were identified across different versions of product labeling for the same drug?
(b) What version of product labeling was used for evidence item X? Original manufacturer or repackager? Most current label or outdated? Is the drug on market in country X or not? American or country X?
Supporting updates to evidence and assertions
1. Changing status of redundant and refuted evidence:
(a) Remove a older version of a redundant evidence item
(b) Change the modality of a supporting evidence item to be a refuting
evidence item
2. Updating when key sources change: (a) Get all assertions that are supported by evidence items identified from
an FDA guidance or other source document just released as an updated version.
4.4 Understanding the evidence base
1. Statistical information about the evidence base:
(a) Number of assertions in the system
(b) Number of evidence items for and against each assertion type
(c) Show the distribution of the levels of evidence for various assertion types
(e.g., pharmacokinetic assertions)
strength
Software responsive
For adding annotations: Existing MP plugin for Domeo
For viewing annotations: Want them highlighted in a web-based interface BUT Resolving annotations requires a method for pointing to paywalled/subscription PDF & HTML
An existing Micropublication plugin for Domeo [Ciccarese2014] is being mod- ified as part of the project. Our plan is to use the revised plugin to support the evidence board with the collection of the evidence and associated annotation data. It will also enable the broader community to access and view annotations of PDDIs highlighted in a web-based interface. We anticipate that this approach will enable a broader community of experts to review each PDDI recorded in the DIKB and examine the underlying research study to confirm its appropriateness and relevance to the evidence base.
The usability of the annotation plug-in is critically important so that the panel of domain experts will not face barriers to annotating and entering ev- idence. This will require usability studies of the new PDDI Micropublication plugin. Another issue is that many PDDI evidence items can be found only in PDF documents. Currently, the tool chain for PDF annotation is relatively weak: compared to text and HTML, PDF annotation tools are not as widely available and not as familiar to end-users. Suitable tools will have to be integrated into the revised plugin.
PDF documents may be in proprietary portals or academic library systems
