Understanding which events are mentioned in unstructured natural language texts, and which relations connect them is a fundamental task for many applications in natural language processing (NLP), such as personalized news systems, question answering and summarization. A notably challenging problem related to event processing is recognizing the relations that hold between events, in particular temporal and causal relations. Having knowledge about such relations is necessary to build event timelines from text and could be useful for future event prediction, risk analysis and decision making support. While there has been some research on temporal relations, the aspect of causality between events from an NLP perspective has hardly been touched, even though it has a long-standing tradition in psychology and formal linguistic fields. We propose an annotation scheme to cover different types of causality between events, techniques for extracting such relations and an investigation into the connection between temporal and causal relations. The latter will be the focus of this thesis work because causality clearly has a temporal constraint. We claim that injecting this precondition may be beneficial for the recognition of both temporal and causal relations.
1. Extracting Temporal and Causal
Relations between Events
Paramita
Under the supervision of Sara Tonelli
10 December 2013
2. Overview
• Introduction to Event Extraction
• Event Relation Extraction
– Problem Statements
– State-of-the-Art
• Research Goals and Plan
– Preliminary Result
3. Information Extraction
Typhoon Haiyan, one of the most powerful typhoons
ever recorded slammed into the Philippines on Friday,
setting off landslides, knocking out power in one entire
province and cutting communications in the country's
central region of island provinces.
Natural Language Text: unstructured
What?
Where?
When?
Typhoon Haiyan
The Philippines
Friday
Knowledge Base: structured
4. Event Extraction
What is an event?
“A thing that happens or takes place, especially one of importance”
─ Oxford dictionary
A Philippine volcano, dormant for six centuries, exploded
last Monday. During the eruption, lava, rocks and red-hot
ash are spewed onto surrounding villages. The explosion
claimed at least 30 lives.
Events and frameworks for events:
Annotation temporal expressions:
event: for six centuries, exploded last Monday
• TimeML dormant“something that happens/occurs or a state
TimeML
temporal link
temporal link
that holds true”
• ACE
temporal link
• ACE
exploded
dormant
• arg-time: six centuries • arg-time: last Monday
5. TempEval-3 (2013)
• Shared task on temporal and event processing
• Automatic identification of temporal expressions, events, and
temporal relations within a text annotated with TimeML
Task
F1
Precision Recall
Task A –Temporal Expression
90.30%
93.09%
87.68%
Task B – Event Extraction
81.05%
81.44%
80.67%
Task ABC – Temporal Awareness
30.98%
34.08%
28.40%
Task C1 – Temporal Relations
(identification + classification)
36.26%
37.32%
35.25%
Task C2 – Temporal Relations (only
classification)
56.45%
55.58%
57.35%
Low performances on temporal relation extraction!
6. Overview
• Introduction to Automatic Event Extraction
• Event Relation Extraction
– Problem Statements
– State-of-the-Art
• Research Goals and Plan
– Preliminary Result
7. The Relationship between Events
Typhoon Haiyan struck the eastern Philippines on Friday,
killing thousands of people.
IS_INCLUDED
BEFORE
CAUSE
Temporal Relations
Causal Relations
Temporal Constraint of Causality
cause BEFORE effect
creating event timelines,
multi-document summarization
predicting future events,
risk analysis,
decision making support
8. Research Questions
“Given a text annotated with events and time
expressions, how to automatically extract temporal
relations and causal relations between them?”
“Given the temporal constraint of causality, how to
utilize the interaction between temporal relations and
causal relations for building an integrated extraction
system for both types of relations?”
9. Temporal Relation Types: TimeML
• Based on Allen’s interval algebra (James F. Allen, 1983): a
calculus for temporal reasoning, capturing 13 relations
between two intervals
Allen’s Relation
X<Y,Y>X
X m Y , Y mi X
Illustration
X
X
X s Y , Y si X
X
Y
X d Y , Y di X
Y
X
X f Y , Y fi X
X=Y
Y
X BEFORE Y , Y AFTER X
Y
X IBEFORE Y , Y IAFTER X
Y
X
X o Y , Y oi X
TimeML Relation
X overlaps with Y
Y
X BEGINS Y , Y BEGUN_BY X
X
X DURING Y , Y DURING_INV X
Y
X INCLUDES Y , Y IS_INCLUDED X
X
Y
X
X
X ENDS Y , Y ENDED_BY X
X SIMULTANEOUS Y
Y
X IDENTITY Y
10. Expressing Temporal Order
• Temporal anchoring
– John drove back home for 20 minutes.
• Explicit temporal connectives
– John went shopping before he drove back home.
• Implicit (and ambiguous) temporal
connectives
– John arrived at home. He parked the car and saw
his son waiting at the front door.
11. Temporal Relation Extraction
• Common approach dividing the task:
– Identifying the pairs of entities having a temporal link
• Often simplified, rule-based approach:
– Main events of consecutive sentences
– Pairs of events in the same sentence
– An event and a time expression in the same sentence
– An event and the document creation time
– Determining the relation types
• Often regarded as a classification problem, supervised learning
approach:
– Given an ordered pair of entities (e1, e2), the classifier has to
assign a certain label (temporal relation type)
12. TempEval-3 (2013)
• Shared task on temporal and event processing
• Automatic identification of temporal expressions, events, and
temporal relations within a text annotated with TimeML
Task
F1
Precision Recall
Task A –Temporal Expression
90.30%
93.09%
87.68%
Task B – Event Extraction
81.05%
81.44%
80.67%
Task ABC – Temporal Awareness
30.98%
34.08%
28.40%
Task C1 – Temporal Relations
(identification + classification)
36.26%
37.32%
35.25%
Task C2 – Temporal Relations (only
classification)
56.45%
55.58%
57.35%
Low performances on temporal relation extraction!
14. Causal Relations: Language Resources
• Penn Discourse Treebank (PDTB) 2.0
– Focuses on encoding discourse relations
– “It was approved when a test showed some positive results,
officials said.” CONTINGENCY:Cause:reason
• PropBank
– Annotates verbal propositions and their arguments
– “Five countries remained on that so-called priority watch list
because of an interim reviewARGM-CAU.”
• SemEval 2007 Task 4 “Classification of Semantic Relations between
Nominals”
– Contains nominal causal relations as a subset
– The period of tumor shrinkagee1 after radiation therapye2 is
often long and varied. Cause-Effect(e2,e1) = "true"
15. Causal Relations: Language Resources (2)
between Events
• Bethard et al. (2008)
– 1000 conjoined event pairs (with conjunctive and) are annotated
manually with BEFORE, AFTER, CAUSE, or NO-REL relations
– Build classification model using SVM (697 train pairs)
– Causal relation extraction evaluation: F-score 37.4%
• Do et al. (2011)
– Detection of causality between verb-verb, verb-noun, and noun-noun
triggered event pairs, using PMI (based on probabilistic contrast
model)
– Causal relation extraction evaluation: F-score 46.9%
• Riaz & Girju (2013)
– Identification of causal relations between verbal events (with
conjunctives because and but, for causal and non-causal resp.)
– Resulting in knowledge base containing 3 classes of causal association:
strongly causal, ambiguous, strongly non-causal
16. Causal Relation Extraction
• No standard benchmarking corpus for
evaluating event causality extraction
• Causal relations in TimeML?
– “The rainse1 causede2 the floodinge3.“
– IDENTITY (e1,e2), BEFORE (e1,e3)
17. Temporal and Causal: the Interaction
• Temporal constraint of causal relations:
The cause happened BEFORE the effect
• Bethard et al. (2008) on corpus analysis:
– 32% of CAUSAL relations in the corpus did not have an
underlying BEFORE relation
– “The walls were shaking because of the earthquake."
• Rink et al. (2010) makes use of temporal relations as
a feature for classification model of causal relations
– Causal relation extraction evaluation: F-score 57.9%
18. Overview
• Introduction to Automatic Event Extraction
• Event Relation Extraction
– Problem Statements
– State-of-the-Art
• Research Goals and Plan
– Preliminary Result
19. Research Objectives & Time Plan
1. Temporal Relation Extraction
– Finding ways to improve the current state-of-the-art
performance on temporal relation extraction: 1st year
2. Causal Relation Extraction
– Creating a standard benchmarking corpus for evaluating
causal relation extraction: 2nd year, 4 months
– Building an automatic extraction system for event
causality: 2nd year, 8 months
3. Integrated Event Relation Extraction
– Utilizing the interaction between temporal and causal to
build an integrated system for temporal and causal
relations: 3rd year, 8 months
20. Temporal Relation Extraction
Preliminary Result
• Temporal Relation Classification
“Given a pair of entities (event-event, event-timex or timex-timex*), the
classifier has to assign a certain label (temporal relation type).”
*) timex-timex pairs are so few in the dataset, so they are not included
– Supervised classification approach
– Support Vector Machines (SVMs) algorithm
– Feature engineering: event attributes, temporal signals, event
duration, temporal connectives (disambiguation), etc.
– Bootstrapping the training data: inverse relations and closure
• TempEval-3 task evaluation setup
System
F-Score
Precision
Recall
TRelPro*
58.48%
58.80%
58.17%
UTTime
56.45%
55.58%
57.35%
NavyTime
46.83%
46.59%
47.07%
JU-CSE
34.77%
35.07%
34.48%
*) Paper submitted to EACL 2014
21. Temporal Relation Extraction (2)
Preliminary Result
• TempEval-3 test data annotated by TRelPro
Can be improved by
including causality
as a feature?
22. Causal Relation Extraction
• Create an annotation format for causal relations based on
TimeML, in order to have a unified annotation scheme for
both temporal and causal relations
– Take the same definitions of events and time expressions
– Introduce CLINK tags, in addition to TimeML TLINK tags for temporal
relations
• Map existing resources (e.g. PDTB, PropBank, SemEval-2007
Task 4 nominal causal corpus) to the newly created
annotation scheme
• Build a causal relation extraction system
– Consider the similar approach (and features) for the temporal relation
extraction system
– New features relevant for causality extraction: causal
signals/connectives, lexical information (WordNet, VerbOcean)
23. Expressing Causality
•
Affect verbs (affect, influence, determine, change)
– Age influences cancer spread in mice.
•
Link verbs (linked to, led to, depends on)
– The earthquake was linked to a tsunami in Japan.
•
Causal conjunctives
–
–
–
–
–
•
She fell because she sat on a broken chair.
John drank a lot of coffee. Consequently, he stayed awake all night. (conjunctive adverb)
I will go around the world if I win the lottery. (conditional)
She stopped the car when she saw the runaway goose. (temporal)
Ralph broke the car and his father went ballistic. (coordinating)
Ambiguous!
Causal prepositions
– He likely died because of a heart attack.
– She was tired from running around all day.
•
Periphrastic causative verbs
– The earthquake prompts people to stay out of buildings. (CAUSE)
– The pole restrains the tent from collapsing. (PREVENT)
– The oxygen lets the fire gets bigger. (ENABLE)
27. Expressing Causality: Implicit
• Lexical causatives
– John broke the clock.
• Resultatives
– John hammered the metal flat.
• Implicit
– Max switched off the light. The room became pitch dark.
Hinweis der Redaktion
Event extent is necessary to determine the attributes of the corresponding event expressed by an event trigger.
Events can be punctual or last for a period of time, and therefore are usually anchored to temporal expressions.Hence, it is possible to arrange the events in temporal order. This is useful if we want to build the timeline of events as a very beneficial way of displaying events.Besides temporal relations, there is another type of relations existing between events, causal relations. This type of relations is very useful to build decision making support tools, where we could predict future events based on causation reasoning.
The problem statements are…This will be the main research questions that we will try to answer during my study.
