Knowledge Extraction for the Web of Things (KE4WoT) Challenge: Co-located with The Web Conference 2018 (WWW 2018) New release to upload the latest slides: 2 May Previous slides have been seen 45 times.

1. Knowledge Extraction for the Web of Things
(KE4WoT)
Friday 27 April 2018 (1:40pm-2:20pm), Challenge co-located with The Web
Conference (WWW 2018), 23-27 April, 2018
Ohio Center of Excellence in Knowledge-Enabled Computing
Amelie Gyrard, Manas Gaur,
Swati Padhee, Amit Sheth
Kno.e.sis Research Center
Department of Computer Science and Engineering,
Wright State University, Dayton, Ohio (USA)
Mihaela Juganaru-Mathieu
MINES Saint-Etienne, Institut Henri
Fayol, Saint Etienne, France

2. Challenge Website:
http://wiki.knoesis.org/index.php/KE4
WoTChallengeWWW2018

3. KE4WoT Challenge @ WWW 2018 Co-Chairs
3
Dr. Amelie Gyrard
Dr. Amit Sheth
Swati PadheeManas Gaur
Dr. Mihaela Juganaru-Mathieu

4. Agenda - 40 min Timeslot
• Introduction:
̶ From Internet of Things to Web of Things
̶ Challenge Interest - User feedback
• Challenge Tasks
̶ Tutorials to exploit the datasets
̶ Evaluation of the challenge tasks
• Research Impact
• Conclusion
• Demo and Poster session
4

5. Agenda - Demo and Poster Session
• Demo (Task 2): Neural Machine Translation Approach for Named Entity Recognition
̶ Philips Kokoh PRASETYO, School of Information Systems Singapore Management
University
̶ http://research.larc.smu.edu.sg/health-sense/s/predict
• Poster (Task 1): Semantic Web of Things
̶ Ruta et al., Poliba, Italy
̶ Also a Demo at WWW 2018: A journey from the Physical Web to the Physical
Semantic Web
5

6. Towards Semantic Web of Things
6

7. Connecting the Things to the Web
• Web-based applications are provided to easily monitor
data generated by devices
̶ Example: NetAtmo for smart home, Fitbit for smart
healthcare, Footbot for air quality
7

8. What is the Web of Things (WoT)?
Who is the pioneer of WoT?
8
Quiz time!

9. Web of Things (WoT) Overview
• How to send data produced by sensors/devices to the Web?
̶ Connecting the things/objects to the Web = The Web of Things
• Pioneer of the Web of Things: Dominique Guinard
̶ PhD Thesis: A Web of Things Application Architecture [Guinard
2011] and books
• Real-time Demo: http://devices.webofthings.io/
9

10. Tweet Example
• Tweet Example:
̶ Since taking asthma meds, my Fitbit shows my
heartbeat at >100 even during my nap! I feel like I can
hear my heart in my head #amidying
• How to automatically understand that FitBit and is a
device?
• How to automatically correlate this knowledge with
existing knowledge?
̶ For instance, there is the Fitbit Ontology!
10

11. Tweet Example 2
• IoT and smart cities ontologies
already describe sensors for air
quality!
• Health ontologies already designed
domain knowledge for Asthma and
allergy, etc.
11

12. Do you know what is an ontology?
12
Quiz time!

13. Why this challenge?
• A growing interest within standards to design ontologies for IoT:
̶ W3C Semantic Sensor Networks (SSN)
̶ W3C Web of Things
̶ OneM2M
̶ SAREF for smart building
̶ iot.schema.org
̶ etc.
13
http://wiki.knoesis.org/index.php/KE4WoTChallengeWWW2018#Description_of_the_KE4WoT_Challenge

14. KE4WoT Research Challenge Introduction
• How to exploit domain knowledge in already
designed in WoT applications?
̶ Frequently, models are designed to structure data
produced by devices
̶ Models referenced within the ontology catalogue for
Internet of Things (LOV4IoT)
• How to exploiting tweets related the healthcare
domain and correlate them with WoT?
14

15. Overview of the Challenge Tasks
15
KE4WoTChallenge
Task 1: Exploiting the Web of
Things Knowledge Base
Task 2: Creating a System
for extracting named
entities using Healthcare
Knowledge Sources and a
Q/A System over it
Task 1.1: Extracting the most
popular terms
Task 1.2: Ontology Matching
Task 2.1: Named Entity
Recognition in Healthcare
Unstructured Text
Task 2.2: Q/A System

16. Overall Interest for this Challenge
We design a user feedback form with some of the following questions:
• Q1: Expertise?
• Q2: Challenge task Interests?
• Q3: Why interested in this challenge?
• Q4: IoT applicative domain interest?
• Etc.
You can still fill in the form if you want to stay updated with the future
editions of this challenge: https://goo.gl/forms/iqP22wykRAuamAw43
16

17. Q1: Expertise? - Results
17
You can still fill in the form if you want to stay updated with the future
editions of this challenge: https://goo.gl/forms/iqP22wykRAuamAw43

18. Q2: Challenge task Interests? - Results
18
You can still fill in the form if you want to stay updated with the future
editions of this challenge: https://goo.gl/forms/iqP22wykRAuamAw43
Task 1.1: Extracting the most popular terms
Task 1.2: Ontology Matching
Task 2.1: Named Entity Recognition in
Healthcare Unstructured Text
Task 2.2: Q/A System

19. Q3: Why interested in this challenge? - Results
19
You can still fill in the form if you want to stay updated with the future
editions of this challenge: https://goo.gl/forms/iqP22wykRAuamAw43
Learning new technologies
Solving interesting problems
Compare our systems to other systems

20. Q4: IoT applicative domain interest? - Results
20
You can still fill in the form if you want to stay updated with the future
editions of this challenge: https://goo.gl/forms/iqP22wykRAuamAw43

21. Overview of the Challenge Tasks
21
KE4WoTChallenge
Task 1: Exploiting the Web of
Things Knowledge Base
Task 2: Creating a System
for extracting named
entities using Healthcare
Knowledge Sources and a
Q/A System over it
Task 1.1: Extracting the most
popular terms
Task 1.2: Ontology Matching
Task 2.1: Named Entity
Recognition in Healthcare
Unstructured Text
Task 2.2: Q/A System

22. Google Knowledge Graph
Video: https://www.youtube.com/watch?v=mmQl6VGvX-c
22

23. How to reuse WoT knowledge already designed?
Linked Open Vocabularies for
Internet of Things (LOV4IoT)
23

24. How to exploit the domain knowledge already available on
the Web and make it interoperable?
• Domain knowledge already structured and designed
• Ontologies used to share and reuse the domain knowledge
24
Collect InteroperabilityClassify

25. LOV4IoT in Real-Time - Demo
•
25
LOV4IoT Project: http://lov4iot.appspot.com/

26. LOV4IoT Demo Embedded
LOV4IoT Video on Youtube: https://youtu.be/a1zsdkzE_oY
26

27. Task 1: Exploiting the Web of Things Knowledge Base
• Ontology catalogue the Internet of Things and Web of Things
• More than 400 ontology-based projects for IoT, smart cities,
etc.
• Almost 20 domains relevant to IoT referenced such as
healthcare, building, smart grid, smart agriculture, smart
transportation, etc.
• LOV4IoT Project: http://lov4iot.appspot.com/
• We are aware of ontology catalogues such as LOV, BioPortal,
etc.:
̶ State of the art analysis here
27

28. LOV4IoT - IoT Ontology Web Services or Dumps
Tutorial
• LOV4IoT new functionalities created for the
KE4WoT challenge!
• Creation of web services to get the latest
insertions of ontology-based IoT projects.
• OR dump file of ontologies for a specific
domain
28LOV4IoT Project: http://lov4iot.appspot.com/

29. LOV4IoT - IoT Web Service & Tutorial - Demo
=> To automatically retrieve
the ontology code
29
LOV4IoT IoT Web Service: http://lov4iot.appspot.com/perfectoOnto/getOntoDomain/?domain=IoT

30. LOV4IoT - Healthcare Web Service & Tutorial - Demo
=> To automatically retrieve
the ontology code
30
LOV4IoT Healthcare Web Service:

31. Overview of the Challenge Tasks
31
KE4WoTChallenge
Task 1: Exploiting the Web of
Things Knowledge Base
Task 2: Creating a System
for extracting named
entities using Healthcare
Knowledge Sources and a
Q/A System over it
Task 1.1: Extracting the most
popular terms
Task 1.2: Ontology Matching
Task 2.1: Named Entity
Recognition in Healthcare
Unstructured Text
Task 2.2: Q/A System

32. Task 1.1: Extracting the Most Popular Terms - Evaluation
• Use case with IoT Ontologies:
• Evaluation online Table:
http://lov4iot.appspot.com/?p=OntologyExtractionKE4WoTChallengeWWW2018
32
Too many concepts:
- Neither aligned
- Nor part of core ontologies
like W3C SSN, W3C time

33. Overview of the Challenge Tasks
33
KE4WoTChallenge
Task 1: Exploiting the Web of
Things Knowledge Base
Task 2: Creating a System
for extracting named
entities using Healthcare
Knowledge Sources and a
Q/A System over it
Task 1.1: Extracting the most
popular terms
Task 1.2: Ontology Matching
Task 2.1: Named Entity
Recognition in Healthcare
Unstructured Text
Task 2.2: Q/A System

34. Task 1.2: Ontology Matching
• How to apply ontology matching tools to ontology referenced
with LOV4IoT?
• Why some ontologies cannot be loaded?
• How to share lessons learned and encourage best practices for
better semantic interoperability?
34Online Table: http://lov4iot.appspot.com/?p=OntologyAlignmentKE4WoTChallengeWWW2018

35. Task 1.2: Evaluation
35
Online Table: http://lov4iot.appspot.com/?p=OntologyAlignmentKE4WoTChallengeWWW2018
OAEI: http://oaei.ontologymatching.org/2017.5/
Encouraging to integrate an
IoT track within Ontology
Alignment Evaluation Initiative
(OAEI)

36. Poster (Task 1): Semantic Web of Things
36

37. Overview of the Challenge Tasks
37
KE4WoTChallenge
Task 1: Exploiting the Web
of Things Knowledge Base
Task 2: Creating a
System for extracting
named entities using
Healthcare Knowledge
Sources and a Q/A
System over it
Task 1.1: Extracting the
most popular terms
Task 1.2: Ontology
Matching
Task 2.1: Named Entity
Recognition in Healthcare
Unstructured Text
Task 2.2: Q/A System

38. Task 2: Overview
Named Entity Recognition in Healthcare Unstructured Text
• Definition: Named Entity Recognition (NER) is considered as an important natural
language processing task.
• Focus on health-care domain specific unstructured text obtained from Twitter.
38

39. Overview of the Challenge Tasks
39
KE4WoTChallenge
Task 1: Exploiting the Web of
Things Knowledge Base
Task 2: Creating a System
for extracting named
entities using Healthcare
Knowledge Sources and a
Q/A System over it
Task 1.1: Extracting the most
popular terms
Task 1.2: Ontology Matching
Task 2.1: Named Entity
Recognition in Healthcare
Unstructured Text
Task 2.2: Q/A System

40. Task 2.1: Description
We categorise the entities in healthcare text under following entity types:
■ Disease Entity: It is the name of the disease that is being explicitly stated in the text. Identification of this
entity enables discovery of etiological factors of the disease and its historical information.
■ Severity Entity (a severe form of disease entity): It is a disease entity that of etiological origin from a
relatively mild disease entity.
■ Trigger Entity: It is a disease entity/substance/environmental condition that caused/provoked the disease
entity identified in the text. For example, weather, measure cough rate, respiration patterns, heartbeat,
temperature and other body data.
■ Location Entity: Words listed under human anatomy are location entity. For instance, bones, muscles, nose,
lungs, etc.
■ Procedure/Treatment/Device: These are entities that define a procedure, treatment or device used by the
patient or clinician as an act to cure the disease entity stated in the text. For example, an inhaler is a device
to cure asthma.
■ Control: It is a dichotomous variable whose value is given “yes” when the tweet talks about disease control,
reduction in severity or reduced frequency of asthmatic attacks. This category is created for supporting the
question answering task.
40

41. Tweet Example
Input Tweet Text: Since taking asthma meds, my Fitbit shows my heartbeat at >100 even
during my nap! I feel like I can hear my heart in my head #amidying
Output:
■ Disease Entity: asthma
■ Severity Entity (severity of disease) : Missing
■ Trigger Entity (Findings/Triggers): asthma meds
■ Location Entity: heart
■ Procedure/Treatment/Devices: Fitbit
■ Control (binary variable): (yes/no) : no
41

42. Sample of Annotated Dataset
42

43. Task 2.1: Evaluation
• We expect an ensemble based learning system for identification Disease,
Severity, Triggers, Location qualifier, and Procedure entity types.
• Given an input, our evaluation is based on strict and approximate matching
of the entity type.
• Strict evaluation is performed in cases where the text contain single entity
of each entity types.
• Approximate evaluation is performed in cases where the text contain more
than single entity. In such a evaluation, we look for the “presence” of the
entity in our annotated set for each tweet.
43

44. Task 2.1: Evaluation
44

45. Demo (Task 2.1): Neural Machine Translation
Approach for Named Entity Recognition
45Demo URL: http://research.larc.smu.edu.sg/health-sense/s/predict

46. Overview of the Challenge Tasks
46
KE4WoTChallenge
Task 1: Exploiting the Web
of Things Knowledge Base
Task 2: Creating a
System for extracting
named entities using
Healthcare Knowledge
Sources and a Q/A
System over it
Task 1.1: Extracting the
most popular terms
Task 1.2: Ontology
Matching
Task 2.1: Named Entity
Recognition in Healthcare
Unstructured Text
Task 2.2: Q/A System

47. Task 2.2: Description
• In this Question Answering task, the participant will be required to provide a response
to a natural language question along with the relevant tweet ID and the model that
does the answering.
• In order to complete this task, the participant has to leverage their Named Entity
Recognition (NER) module developed in Task 2.1.
• One can utilize some existing knowledge sources (e.g. SNOMED, DBpedia, etc.) to
enhance the efficiency of their model.
• As a part of this task, the participant will have access to 25 natural language questions
on which they can create their model [1].
47
[1] https://github.com/gyrard/KE4WoT_Challenge_WWW2018/blob/master/Challenge_Dataset/questions.txt

48. 1. Does dust mites causes asthma?
2. Does Prednisone risks obesity?
3. Does Prednisone risks weight gain?
4. Does bronchitis risks pneumonia?
5. Which disease is caused by Exercise-induced Bronchoconstriction?
6. Which drug treats asthma?
7. Which product treats childhood asthma?
8. What is the relation between anxiety and asthma?
9. What is the relation between asthma and COPD?
10. What is the cause of eosinophilic asthma?
11. One procedure for identify bronchitis?
12. Location affected by steroids?
48

49. 13. Location affected by sinus infection?
14. Devices for controlling asthma?
15. Which disease is treated by Vitamin D?
16. Which disease is identified by Lung Bacteria?
17. Which disease is treated by Breastfeeding?
18. Which disease is caused by anabolic steroids?
19. Does eosinophilic pneumonia patients have chronic asthma?
20. What is the cause of sinus venous thrombosis?
21. Which disease is treated by Montelukast?
22. Does Black-mold causes asthma and allergy?
23. Does lung infection causes asthma?
24. Does CBD reduce allergy and asthma?
25. Is the Disease controlled? 49

50. Question Answering Task Example
■ Input Question: Does dupilumab control asthma ?
■ Answer: Yes
■ Relevant Tweet: Patients with severely uncontrolled asthma derive the most benefit
from dupilumab
50

51. Task 2.2: Evaluation
• 3 types of questions:
̶ <Entity><Relation><Entity> : Yes/No
• Does Prednisone risks obesity?
̶ <Entity-Type><Relation><Entity> : One or more entities
• Location affected by steroids?
̶ <Entity><?><Entity> : One or more relations
• What is the relation between asthma and COPD?
51

52. Overall Challenge Research Impact
52
Knowledge
Extraction
Internet of
Things
Natural
Language
Processing
Web of
Things
Medical
Text
Analysis
Semantic
Web

53. Conclusion & Future Work
• A lot of ontologies to build Web of Things applications
̶ More efforts are needed to extend the existing ones
̶ Common patterns are identified
• Future work:
̶ Ontology alignment initiative (OAEI) with an IoT track
̶ Sharing more semantic web best practices [1]
̶ Sharing more ontology catalogs for IoT
̶ How to find ontologies fitting our needs?
• Ontology ranking, etc.
[1] http://perfectsemanticweb.appspot.com/
53

54. Acknowledgments
• This work is partially funded by:
̶ A bilateral research convention with ENGIE Research & Development
̶ The National French ANR 14-CE24-0029 OpenSensingCity project
̶ Hazards SEES NSF Award EAR 1520870
̶ KHealth NIH 1 R01 HD087132-01.
54

55. Questions
55

56. Knowledge Extraction for the Web of Things
(KE4WoT)
Challenge co-located with The Web Conference (WWW 2018)
23-27 April, 2018
Ohio Center of Excellence in Knowledge-Enabled Computing
Amelie Gyrard, Manas Gaur,
Swati Padhee, Amit Sheth
Kno.e.sis Research Center
Department of Computer Science and Engineering, Wright
State University, Dayton, Ohio (USA)
Mihaela Juganaru-Mathieu
MINES Saint-Etienne, Institut Henri Fayol,
Saint Etienne, France

57. Appendix
57

58. Before IoT and WoT
=> Common Goal: Building
smart applications exploiting
sensor data generated by
devices?
=> All domains investigate
semantic web technologies 58
Internet of
Things (IoT)
Web of Things
(WoT)
Ubiquitous
Computing
Pervasive
Computing
Context
Awareness
Mark Weiser,
Kevin Ashton, 1999
Dominique Guinard, 2010
Dey et al., ?
2003

59. 59

60. 60