PhD Thesis defense, 24 April 2015 Designing Cross-Domain Semantic Web of Things Applications

1. Designing Cross-Domain
Semantic Web of Things
Applications
Amelie Gyrard
Christian Bonnet (Eurecom, Mobile Communication)
Karima Boudaoud (I3S, Security)

2. Agenda
 Introduction & Motivation
 State of The Art & Main challenges
 Contributions: M3 framework
 Components
 Use cases
 Evaluations
 Demonstrations
 Conclusion & Future work
2

3. How to interpret Internet of Things (IoT) data?
Thermometer
Sensor data
Applications to visualize data
Interpretation
by humans
How machines can
interpret data?
3
Machine learning?
Reusing domain knowledge?

4. How to combine and reuse IoT data?
How to get
additional
information?
How to combine data from
different domains?
4
How to
combine
domains?
How to describe data?

5. How to describe data?
 Taking inspiration from the Web
Automatically built
by machines
5

6. How to get additional information?
 Agreeing on common
vocabularies to describe data
on the web:
 Semantic search engines
 Web sites
 They built together Schema.org
6

7. How to apply semantic web technologies to
Internet of Things?
 Machine-understandable data
 Describe data with common vocabularies
 Reuse domain knowledge
 Link to other data
 Ease the reasoning
 => How to provide a common description of sensor
data to later reason on it?
7

8. How to combine IoT data from different domains?
8
Innovative
applications
Interoperability on protocols or data?

9. Agenda
 Introduction & Motivation
 State of The Art & Main challenges
 Contributions: M3 framework
 Conclusion & Future work
9

10. “Semantic Web of Things: an analysis of the application semantics for the IoT moving
towards the IoT convergence” [Jara et al. 2014]
Semantic Web of Things: Main challenges (1)
Machine-to-Machine (M2M): no human intervention
Global
interoperability
 How?
 Why?
Common description
Common App. Protocol
Device Abstraction
Common Nwk. Protocol
10

11. Semantic Web of Things: Main challenges (2)
“Semantics for the Internet of Things: early progress and back to the future” [Barnaghi et al.
2012] 11

12. State of the Art: Semantic Sensor Networks
2008
‘Semantic Sensor Web’
‘Linked Sensor Data’
2013 2014
SemSOS,
‘Semantic
Perception’
‘Infer high-level
abstraction’
‘Linked Stream Data’
2015
‘SPARQLStream’
• A) How to design
semantic-based
IoT applications?
• B) Interpret data?
Combine domains ?
Reuse domain knowledge?
• C) Security & IoT?
2011
W3C SSN ontology
Real-time?Use
semantic web
technologies?
Interpret data?
12

13. State of the art: W3C SSN ontology
 Limitations of W3C SSN ontology:
 Interoperability issues to reuse and combine domain ontologies
 Need of a common description to describe sensor measurements
 Need of an approach to share and reuse the reasoning approach
 Need to integrate semantics to IoT and M2M
=> How to extend the W3C SSN ontology to provide a
common description of sensor data to later reason on it
by reusing domain knowledge?
http://www.w3.org/2005/Incubator/ssn/ssnx/ssn#
http://www.w3.org/2005/Incubator/ssn/XGR-ssn-20110628/
http://www.w3.org/2015/spatial/charter
13

14. Three main research challenges to address
 Challenge A: How to design semantic-based IoT
applications?
 Challenge B: How to interpret IoT data?
 Challenge B.1: How to reuse and combine IoT data?
 Challenge B.2: How to reuse and combine domain knowledge?
 Challenge C: How to secure IoT applications?
14

15. Agenda
 Introduction & Motivation
 State of The Art & Main challenges
 Contributions: M3 framework
 Conclusion & Future work
15

16. Our solution: Machine-to-Machine Measurement
Framework (M3)
Challenge A: Design
semantic based IoT
applications
Challenge B.1 &
B.2: Combine
data and domains
Challenge B:
Interpret IoT
data
Challenge C:
Secure IoT
applications
Challenge B.2:
Reuse domain
knowledge 16

17. Agenda
 Introduction & Motivation
 State of The Art & Main challenges
 Contributions: M3 framework
 Conclusion & Future work
17

18. SWoT generator
Template used in 3 steps:
1) Designing phase
2) Development phase
3) Running phase
SWoT template
=> Benefits: No need to learn semantic web technologies
IoT
Application
generate
build
use
IoT
developers
18

19. Designing phase
19
*
* Domain where is deployed the sensor, not the applicative domain
Challenge A: Design
semantic based IoT
applications

20. Development phase
IoT
developers
SWoT
template
1) Load:
- M3 ontologies
- M3 IoT data
- M3 datasets
4) Get M3 suggestions or
high level abstractionsSTEPS BEFORE
Get
template
3) Execute M3 SPARQL query +
SPARQL engine
SemanticWeb
Framework
2) Execute M3 rules +
reasoning engine
20

21. Running phase
21

22. Agenda
 Introduction & Motivation
 State of The Art & Main challenges
 Contributions: M3 framework
 Conclusion & Future work
22

23. M3 language & ontology
23
Challenge B.1 &
B.2: Combine
data and domains

24. M3 language & M3 ontology
 Data is from heterogeneous projects and domains
 Domain (e.g., health, smart building, weather, room, city, etc.)
 Measurement type (e.g., t = temp = temperature)
 Sensor type (e.g., rainfall sensor = precipitation sensor)
 Units (e.g., Celsius, Fahrenheit, Kelvin)
 M3 language implemented in the M3 ontology
 Describe data in an unified way
 Extension of the W3C Semantic Sensor Networks (SSN) ontology
(Observation Value concept)
 Provide a basis for reasoning and cross-domain interlinking
24
http://www.sensormeasurement.appspot.com/documentation/Nomenclat
ureSensorData.pdf

25. M3 language: a hub for cross-domain
ontologies and datasets
25

26. Sensor-based Linked Open Rules (S-LOR)
26
Challenge B:
Interpret IoT
data

27. S-LOR: Deducing new knowledge
 How to deduce new knowledge?
 S-LOR: a dataset of interoperable rules
 Rules example:
 If Domain == Health && MeasurementType == Temperature
then NewType = BodyTemperature
 If BodyTemperature > 38,7°C then “Fever”
 BodyTemperature and Fever are already described in
domain ontologies or datasets!
27
Demo paper: Helping IoT application developers with Sensor-based Linked Open
Rules [Gyrard et al., ISWC 2014, SSN workshop]

28. Linked Open Vocabularies for
Internet of Things (LOV4IoT)
Challenge B.2:
Reuse domain
knowledge 28

29.  A dataset of more than 270 ontology-based projects
relevant for IoT
 Ontologies
 Datasets
 Rules to interpret IoT data
 Technologies used
 Sensors used
 Security mechanisms used
 Domains relevant for IoT
LOV4IoT
http://www.sensormeasurement.appspot.com/?p=ontologies29
A second life for ontologies!

30.  LOV4IoT is used to build the SWoT template
 Used to re-design interoperable ontologies, rules, datasets
 Limitations: Manually and not automatically
LOV4IoT
http://www.sensormeasurement.appspot.com/?p=ontologies
Collect Classify Interoperability
SWoT
template
30
A second life for ontologies!

31. M3 interoperable domain knowledge
 Need to have the set of files generated in the template
compatible with sensor data
 Ontologies + datasets + rules + sensor data
 Domain knowledge structured in the same way
Domain
ontologies
Domain
datasets
Rules
Interoperable
IoT
Application
Provide
sensor data
SWoT templateM3 IoT
data
Produce
31

32. M3 semantic engine
 Enrich data & combine domains
32
Paper: Enrich Machine-to-Machine Data with Semantic Web Technologies for Cross-
Domain Applications [Gyrard et al., WF-IoT 2014]

33. Agenda
 Introduction & Motivation
 State of The Art & Main challenges
 Contributions: M3 framework
 Conclusion & Future work
33

34. Security Toolbox: Attacks & Countermeasures
(STAC)
Challenge C: Help non-
security experts to secure
IoT applications
34

35. The STAC ontology
Paper: The STAC (Security Toolbox: Attacks & Countermeasures) ontology
[Gyrard et al., Poster, WWW 2013] 35

36. STAC Hub
 Reusing security knowledge from LOV4IoT
36
Paper: An ontology-based approach for helping to secure the ETSI Machine-to-
Machine Architecture [Gyrard et al., iThings 2014]

37. Agenda
 Introduction & Motivation
 State of The Art & Main challenges
 Contributions: M3 framework
 Use cases
 Evaluations
 Demonstrations
 Conclusion & Future work
37

38. M3 use cases
 3 Mock-ups: Naturopathy, Tourism, Transport
 Proof of concept: less user-friendly
 Integrating the M3 approach everywhere!
 Cloud, Android-powered devices and Gateway
 Combine domain-specific sophisticated applications
 Not just data visualization
 Suggestions or high-level abstractions
38

39. Use Case: Embedding M3 in smart fridges
M3 suggestions:
Home remedies
Get temperature
measurement
Stop to be sick with M3!
39

40. Use Case: Embedding M3 in smart luggage
M3 suggestions:
Garments & Activities
Get weather
measurement
Stop to forget things with M3!
Smart Luggage
Destination: Mountain in
winter
Destination: Beach in
summer
40

41. Use Case: Embedding M3 in smart cars
Avoid accidents with M3!
41

42. Evaluations: Research hypotheses
 Templates help IoT projects build their scenarios
 The semantic engine is not too resource consuming
 The semantic engine is generic enough to support
various kind of IoT measurement.
 The interoperable knowledge bases built follows
semantic web best practices.
 Our knowledge bases help non-experts in semantic
web or in security
 LOV4IoT is exploited outside of the M3 framework.
42

43. Evaluating the SWoT generator
 Do we have templates covering the most popular IoT
use cases?
http://www.sensormeasurement.appspot.com/?p=m3_scenario
 Adding a new template?
 Less than 1 day
 Depends on whether we already have the interoperable domain
knowledge
43

44. Evaluating M3 software performances
 Goal: The semantic engine is not too resource consuming
 Evaluation:
 Measuring time consumed
 Results:
 Encouraging (16 – 31 ms)
 Could be embedded on
Android-powered device
44

45. Demo
 Demo
http://sensormeasurement.appspot.com
45

46. M3 framework at work
Domain
experts
IoT developers
End users
Design
applications
Need new
applications
Standardize
Design new ontology
matching tools +
Automatic extraction of
domain knowledge
Exploit &
Contribute
46
 Relevant for different communities

47. Agenda
 Introduction & Motivation
 State of The Art & Main challenges
 Contributions: M3 framework
 Conclusion & Future work
47

48. Summary of contributions
48

49. ChallengeA
M3: An entire chain from sensor data to build IoT
cross-domain IoT applications
Sensor
data
Interpret data +
Combine
domains
Interoperable
sensor data
descriptions
Reuse domain
knowledge
Build IoT
applicationsProvide
template
Secure
applications
ChallengeC
ChallengeA
ChallengeB.1
ChallengeB
ChallengeB.2
49

50. Conclusion & Lessons learnt
 M3: an innovative approach to assist users in
designing interoperable cross-domain Semantic Web of
Things applications:
 A uniform language for sensor data descriptions
 An open-source approach to interpret IoT data
 Combine domains
 Semantics is hidden to the users
 Lessons learnt:
 M3 generic enough for other domains than IoT and security
50

51.  Our proposed
approach:
M3 framework
Future work
Sensor Plug & Play
Extract & combine
domain knowledge
Standardizing common descriptions
Merge M3 to existing SWoT projects
Global
interoperability
Common description
Device Abstraction
Common App. Protocol
Common Nwk. Protocol
51
S-LOR with more reasoning

52. Future work: Merge M3 to existing SWoT projects
Use real datasets & scenarios
+ real-time
Suggest machine learning algorithms
to employ for complicated sensors
Connect
new sensors
52
Rewrite
ontologies

53. Future work: Extracting and combining domain
knowledge
 Extracting popular concepts from domain ontologies
 Cloud tag inspired by the W3C SSN validator
 Extracting rules from ontologies
 OWL 2 RL template, DLEJena
 Combining domain knowledge
 Design and combine new ontology matching tools
 Look at ontology alignment ontology & merging tools
 Designing an interoperable domain knowledge
53

54. Thank you!
 gyrard@eurecom.fr
 http://sensormeasurement.appspot.com/
54

55. Relevant Publications
 International Conferences:
 Enrich Machine-to-Machine Data with Semantic Web Technologies
for Cross-Domain Applications (WF-IoT 2014)
 An ontology-based approach for helping to secure the ETSI
Machine-to-Machine Architecture (iThings 2014)
 A machine-to-machine architecture to merge semantic sensor
measurements (WWW 2013, DC)
 International Workshops:
 Standardizing Generic Cross-Domain Applications in Internet of
Things (Globecom , WTS, 2014)
 Demo paper: Helping IoT application developers with Sensor-based
Linked Open Rules (ISWC, SSN 2014)
 See Google Scholar for more publications
 Participation to standardizations:
55