This document summarizes Valentina Presutti's presentation on using frames for knowledge extraction and linked data. It discusses how frames can be used as units of meaning to reconcile knowledge from different sources. It provides background on frames and examples of how they can represent situations and relationships described in text. The document then outlines several projects from STLab that use a frame-based approach for tasks like knowledge extraction, relation extraction, and sentiment analysis. It discusses tools like FRED and Framester that perform frame-based knowledge extraction and integrate linguistic and factual knowledge through linked data.

1. Knowledge Extraction and Linked Data:
Playing with Frames
Valentina Presutti
STLab, ISTC-CNR
Linked Data For Information Extraction @ ISWC 2016
Tuesday, October 18th 2016

2. STLab team
Valentina Presutti Aldo Gangemi
Andrea Nuzzolese
Diego Reforgiato
Martina Sangiovanni Mario Caruso
Giorgia Lodi
Alessandro Russo
Luigi Asprino
Piero Conca
2

3. 3
• Frames as units of meaning (claim andintuition)
• Background on frames
• From entity-centric to frame-centric knowledge extraction
• Some STLab research projects and results
• Next and open issues
Outline

4. 4
Claim and intuition

5. 5
Frames naturally support knowledge reconciliation,
regardless the logical, conceptual or syntactic
representation of knowledge sources

6. To understand who speaks to us or a text we read
We identify the main entities and how they relate to
each other within a schema (frame)
Frame occurrences + context-dependent reasoning
The intuition
6

7. 7
I went to the disco and I met a friend, who had lost her keys.
We spent the night looking for them.

8. 8
I went to the disco and I met a friend, who had lost her keys.
We spent the night looking for them.

9. 9
I went to the disco and I met a friend, who had lost her keys.
We spent the night looking for them.

10. 10
I went to the disco and I met a friend, who had lost her keys.
We spent the night looking for them.

11. I went to the disco and I met a friend, who had lost her keys.
We spent the night looking for them.
11

12. 12
We want machines to perform this process

13. 13
Background

14. 14
Minsky [1]
“When one encounters a new situation
[…] one selects from memory a
structure called a Frame. This is a
remembered framework to be adapted
to fit reality by changing details as
necessary.”
“A frame is a data-structure for
representing a stereotyped situation,
like being in a certain kind of living room,
or going to a child's birthday party”
“We can think of a frame as a network
of nodes and relations.”
“Collections of related frames are
linked together into frame-systems”
Fillmore [2]
“[…] in characterising a language system
we must add to the description of grammar
and lexicon a description of the
cognitive and interactional
“frames” […]”
“The evolution toward language must have
consisted in part in the gradual acquisition
of a reportory of frames and of
mental processes for operating with them,
and eventually the capacity to create new
frames and to transmit them.”
“[…] in order to perceive something or to
attain a concept, what is […] necessary is
to have in memory a repertoire of
prototypes. The act of perception or
conception being that of recognizing in
what ways an object can be seens as an
instance of one or another of these
prototypes.”

15. 15
Frame definition and representation

16. 16
Cure
Healer
Medication
http://framenet.icsi.berkeley.edu/
Patient

17. 17
N-ary relation f(e, e1,…en)
f is a first order logic relation
e is a variable for any event or situation
described by f
ei is a variable for any of the entity arguments
of f
An OWL n-ary relation pattern
the n-ary relation is the reification of f, i.e. e
the n objects represent the arguments of f
the n argument relations are binary
projections of f including e
co-participation relations are binary
projections of f not including e
Representing frames
“Hagrid rolled up a note
for Harry in Hogwarts”

18. 18
From entity-centric to
frame-centric design and
extraction
Before:
Key terms à
classes/properties
After:
Key situationsà
frames/patterns
Frames as units of meaning [3]

19. 19
From entity-centric to frame-centric
knowledge extraction

20. 20
We want machines to perform this process

21. This requires at least three ingredients:
Knowledge representation
Knowledge extraction
Automated reasoning and learning
21

22. The Semantic Web and Linked Data
Knowledge representation
Knowledge extraction
Automated reasoning
22

23. Mary
marriedWith
John Mary
weddingDate
October 12th, 2016 John
weddingDate
October 12th, 2016
Mary
weddingPlace
Kobe John
weddingPlace
Kobe Mary
weddingPlace
Rome
The Semantic Web and Linked Data
Knowledge representation
Knowledge extraction
Automated reasoning

24. 24
OL & KE tools main focus:
Named Entity extraction
Taxonomy induction,
Relation extraction
Axiom extraction, …
The Semantic Web and Linked Data
Knowledge representation
Knowledge extraction
Automated reasoning

25. 25
This is useful, but it’s not enough
Semantic heterogeneity
Lack of knowledge boundaries
(context) [3]
marriedWith
firstMarriageWith
spousemarriage
spousedate

26. 26
The role of frames in knowledge representation, extraction and
interaction
Performingempirical observations on the web (in line with van
Harmelen’s [4])
Using frames for driving the design of solutions to research
problems andtest their performance
Frames as units of meaning

27. 27
Some projects and results

28. 28
Frame-based knowledge extraction [5]
http://wit.istc.cnr.it/stlab-tools/fred/
From text to linked data

29. 29
Frame-based Linked Data
“Rico Lebrun taught visual arts at the Chouinard Art Institute and at the Disney
Studios. He was influenced by Michelangelo and maintained a lifelong affinity for
Goya and Picasso.”

30. 30
FRED
“The Black Hand might not have decided to barbarously assassinate Franz Ferdinand
after he arrived in Sarajevo on June 28th, 1914”

31. 31
Automatic selection of relevant binary projections of frames
Usable label generation
Formal alignmentbetween frames and binary properties
Binary relations [6]

32. 32
Binary relation assessment
“Rico Lebrun taught visual arts at the Chouinard Art Institute and at the Disney Studios. He
was influenced by Michelangelo and maintained a lifelong affinity for Goya and Picasso.”
Subject
ObjectSubject Object
http://wit.istc.cnr.it/stlab-tools/legalo

33. 33
Binary property generation
vn.role:Actor1 -> “with”
vn.role:Actor2 -> “with”
vn.role:Beneficiary -> “for”
vn.role:Instrument -> “with”
vn.role:Destination -> “to”
vn.role:Topic -> “about”
vn.role:Source -> “from”
Subject
Object
legalo:teachArtAt
teach art at
“Rico Lebrun taught visual arts at the Chouinard Art Institute and at the Disney Studios. He
was influenced by Michelangelo and maintained a lifelong affinity for Goya and Picasso.”
http://wit.istc.cnr.it/stlab-tools/legalo

34. 34
vn.role:Actor1 -> “with”
vn.role:Actor2 -> “with”
vn.role:Beneficiary -> “for”
vn.role:Instrument -> “with”
vn.role:Destination -> “to”
vn.role:Topic -> “about”
vn.role:Source -> “from”
Subject
Object
legalo:teachAbout
teach about
Binary property generation
“Rico Lebrun taught visual arts at the Chouinard Art Institute and at the Disney Studios. He
was influenced by Michelangelo and maintained a lifelong affinity for Goya and Picasso.”
http://wit.istc.cnr.it/stlab-tools/legalo

35. 35
Semantic Web triples and
properties generation
“Rico Lebrun taught visual arts at the
Chouinard Art Institute and at the Disney
Studios. He was influenced by Michelangelo
and maintained a lifelong affinity for Goya
and Picasso.”
dbpedia:Rico_Lebrun s:teachAbout dbpedia:Visual_arts .
s:teachAbout a owl:ObjectProperty ;
rdfs:subPropertyOf fred:Teach;
rdfs:domain wibi:Artist ;
rdfs:range wibi:Art ;
grounding:definedFromFormalRepresentation
fred-graph:a6705cedbf9b53d10bbcdedaa3be9791da0a9e94 ;
grounding:derivedFromLinguisticEvidence s:linguisticEvidence ;
owl:propertyChainAxiom([ owl:inverseOf s:AgentTeach ] s:TopicTeach) .
_:b2 a alignment:Cell ;
alignment:entity1 s:teachAbout ;
alignment:entity2 <http://purl.org/vocab/aiiso/schema#teaches> ;
alignment:measure "0.846"^xsd:float ;
alignment:relation "equivalence" .
domain, range, subsumption
linguistic and formal scope
alignment to existing LOD vocabularies

36. 36
Evaluation tasks [7]
3
6
Tool/Task Topics NER NE-RS TE TE-RS Senses Taxo Rel Roles Events Frames
+SRL
AIDA
– + + – – + – – – – –
Alchemy
+ + – + – + – + – – –
Apache Stanbol
– + + – – + – – – – –
CiceroLite
– + + + + + – + + + +
DB Spotlight
– + + – – + – – – – –
FOX
+ + + + + + – – – – –
FRED
– + + + + + + + + + +
NERD
– + + – – + – – – – –
Ollie – – – – – – – + – – –
Open Calais
+ + – – – + – – – + –
PoolParty KD
+ – – – – – – – – – –
ReVerb
– – – – – – – + – – –
Semiosearch
– – + – + – – – – – –
Tagme – + + + + – – – – – –
Wikimeta
– + – + + + – – – – –
Zemanta
– + – – – + – – – – –

37. 37
Topic detection and Opinion holder detection [8]
Sentiment propagation through frames and roles [9]
Sentiment analysis
“People hope that the President will be
condemned by the judges”

38. 38
50 sentencesfrom MPQA opinion corpus1 and Europarl corpus2
100 Sentence sentiment polarity of open rated hotel reviews
(positive and negative)
Evaluation
Task Measure Value
Holder detection F1 0.95
Topic detection F1 0.68
Sub-topic
detection
F1 0.77
Review sentiment
vs. user scores
Avg. correlation 0.81
2 http://www.statmt.org/europarl/
1 http://mpqa.cs.pitt.edu/corpora/mpqacorpus/
3 http://www.stlab.istc.cnr.it/documents/sentilo/reviewsposneg.zip

39. 39
Frame-basedlinked data shows an effective representation of
discourse
Our ultimate goal is machine understanding, hence
an important issue is the limited coverage of existing resources
and their integration with factual world knowledge
FrameBase [10] partially addresses this problem, starting from
similar principles and intuitions
STLab has develop Framester [11,12]: a general web-scale
integrated resource which integrateslinguistic and world factual
knowledge
(see Aldo’s presentation later)
Coverage and integration of
linguistic and world knowledge

40. 40
Abstract, formalised frame model
generalised model of roles
Represents all resources’ entitiesin terms of its frame semantics
Links linguistic data with ontologies and facts (~43M triples)
Includes FrameBase’s ReDer rules
Framester

41. 41
Word-Frame-Disambiguation (frame detection)
any word, e.g. Shakespeare, write, alone, nicely, etc.
frames evoked by word senses
Outperforms Semafor and FrameBase
details to come in few minutes J
!!!Spoiler Warning!!!
http://lipn.univ-paris13.fr/framester/en/wfd/

42. 42
Helping people with Dementia and their carers
Natural language understanding
questionnaire for cognitive ability
assessment
speech to tag (pictures, music, events, etc.)
reminiscence games and suggestions
suggesting missing words
understanding with partial information
Current project and challenge
http://www.mario-project.eu
Blah blah blah blah
Blah blah blah blah
Blah blah blah blah
Blah blah blah blah
Blah blah blah blah
Blah blah blah blah
Blah blah blah blah
Blah blah blah blah
Blah blah blah blah
User-Robot KB

43. 43
Current work:
To integrate FRED and Framester for normalising results
Framester-driven Ontology Alignment (part of a PhD thesis under dev)
MARIO understanding component and evaluation (with datasets and
PwD)
Open challenge:
How to combine statistical learning with our approaches?
we want FRED to learn from interaction experiences
we want to learn new rules and procedures, not only data
(algorithm learning), and get their formalisation, explicitly
Next and open issues

44. 44
Stupid questions are only those that are not asked (Prof. Paolo Ciancarini)

45. 45
References
[1] Marvin Minsky: A Framework for Representing Knowledge. MIT-AI
Laboratory Memo 306, June, 1974.
[2] Charles J Fillmore. Frame Semantics and the Nature of Language. Annals of
the New York Academy of Sciences, 280(1):20-32, 1976.
[3] Aldo Gangemi, Valentina Presutti: Towards a pattern science for the Semantic
Web. Semantic Web 1(1-2): 61-68 (2010)
[4] Frank van Harmelen: The Web of Data: do we understand what we build?
https://sssw.org/2016/?page_id=386
[5] Aldo Gangemi, Valentina Presutti, Diego Reforgiato Recupero, Andrea
Giovanni Nuzzolese, Francesco Draicchio, Misael Mongiovì: Semantic Web
Machine Reading with FRED. Semantic Web (To appear)
[6] Valentina Presutti, Andrea Giovanni Nuzzolese, Sergio Consoli, Aldo
Gangemi, Diego Regorgiato Recupero: From hyperlinks to Semantic Web
properties using Open Knowledge Extraction pp. 351-378, Semantic Web,
Volume 7, Number 4 / 2016.

46. 46
[7] Aldo Gangemi: A Comparison of Knowledge Extraction Tools for the
Semantic Web. ESWC 2013: 351-366
[8] Aldo Gangemi, Valentina Presutti, Diego Reforgiato Recupero:
Frame-Based Detection of Opinion Holders and Topics: A Model and a
Tool. IEEE Comp. Int. Mag. 9(1): 20-30 (2014)
[9] Diego Reforgiato Recupero, Valentina Presutti, Sergio Consoli, Aldo
Gangemi, Andrea Giovanni Nuzzolese: Sentilo: Frame-Based Sentiment
Analysis. Cognitive Computation 7(2): 211-225 (2015)
[10] Jacobo Rouces, Gerard de Melo, and Katja Hose. Framebase: Representing
n-ary relations using semantic frames. ESWC 2015: 505-521
[11] Aldo Gangemi, Mehwish Alam, Valentina Presutti, Luigi Asprino and Diego
Reforgiato Recupero: Framester: A Wide Coverage Linguistic Linked Data Hub.
In Proceedings of EKAW 2016
[12] Aldo Gangemi, Mehwish Alam, Valentina Presutti: Word Frame
Disambiguation: Evaluating Linguistic Linked Data on Frame Detection.
LD4IE@ISWC 2016: 23-31
References cont.