Presentation done at the 9th Summer School on Ontological Engineering and the Semantic Web (SSSW2012, http://sssw.org/) in July 2012. Please do treat references to people (e.g., Manfred Hauswirth) and nationalities (e.g., about Swiss) in the context in which they were done.
1. SSSW2012 – The Ninth Summer School on Ontological Engineering
and Semantic Web
Semantic Sensor Networks
(and Social) Semantic Web
Oscar Corcho
(with the help of Jean Paul Calbimonte)
Ontology Engineering Group
Facultad de Informática, Universidad Politécnica de Madrid
ocorcho@fi.upm.es
2. This is what my talk is going to be about…
Application
Middleware
CQELS REST SPARQL
Linked Streams Linked Data
COAP
Virtual
Sensors
Sensors
3. Was this a Déjà Vu?
• Key differences
• I am younger (as
demonstrated on Tuesday)
• We are both vice-directors
• But I am vice-director
of a most important
organisation (SSSW vs
DERI)
• Lessons learned:
• Never let an invited
speaker speak before you
• Specially if he is the one
giving a great tutorial ;-)
3
4. Was this a Déjà Vu?
• Hence the title of my talk could perfectly be…
4
5. Some work that can be inserted in the picture
Application
Middleware
SPARQL-STR
CQELS REST SPARQL
Linked Streams Linked Data
Data Stream COAP
Virtual
Management
Sensors
Systems Sensors
6. Ingredients for the Semantic Sensor Web
Core ontological model
Additional domain ontologies
Guidelines for generation of identifiers
Sensor Web programming interfaces (e.g.,
lsm.deri.ie)
Query processing engines
http://www.flickr.com/photos/santos/2252824606/
10. Coastal Channel Observatory and other sources
(http://webgis1.geodata.soton.ac.uk/flood.html)
Flood risk alert:
South East England
Emergency
I have to make
planner
sense out of all this
data
wave data Environmental
forecasts defenses
10
11. Should we care? Smart cities
Environmental sensors
Parking sensors
11 SmartSantander Project
12. Some work that can be inserted in the picture
Application
Middleware
SPARQL-STR CQELS REST SPARQL
Linked Streams Linked Data
Data Stream COAP
Virtual
Management
Sensors
Systems Sensors
13. Ingredients for the Semantic Sensor Web
Core ontological model
Additional domain ontologies
Guidelines for generation of identifiers
Sensor Web programming interfaces (e.g.,
lsm.deri.ie)
Query processing engines
http://www.flickr.com/photos/santos/2252824606/
14. Sensor Network Ontologies
Approximately since 2005: Several proposals
State of the art on sensor network ontologies in the report below
Most of them were too project-specific
Not too much reuse
No alignment between them
No ontology design best practices
2009-2011: W3C SSN-XG incubator group
Final report: http://www.w3.org/2005/Incubator/ssn/XGR-ssn-20110628/
SSN Ontology: http://purl.oclc.org/NET/ssnx/ssn
A good number of internal and external references to SSN Ontology
http://www.w3.org/2005/Incubator/ssn/wiki/Tagged_Bibliography
SSN Ontology paper soon at the Journal of Web Semantics
2011-__ : W3C SSN Community Group
16. Overview of the SSN ontology
Deployment deploymentProcesPart only System OperatingRestriction
hasSubsystem only, some hasSurvivalRange only
SurvivalRange
DeploymentRelatedProcess
hasDeployment only
System
OperatingRange
Deployment deployedSystem only hasOperatingRange only
deployedOnPlatform only Process
inDeployment only Device hasInput only
Input
PlatformSite onPlatform only Device Process
Platform Output
attachedSystem only hasOutput only, some
Data Skeleton
isProducedBy some implements some
Sensor
Sensing
hasValue some sensingMethodUsed only
SensorOutput
detects only
SensingDevice observes only
ObservationValue SensorInput
isProxyFor only
Property
includesEvent some isPropertyOf some
observedProperty only
observationResult only
observedBy only hasProperty only, some
Observation FeatureOfInterest
featureOfInterest only
MeasuringCapability ConstraintBlock
hasMeasurementCapability only forProperty only
inCondition only inCondition only
MeasurementCapability Condition
17. SSN Ontology. Sensor and environmental properties
Skeleton
Property
MeasuringCapability Communication
hasMeasurementProperty only
MeasurementCapability MeasurementProperty
Accuracy Resolution Selectivity Frequency Precision Latency
DetectionLimit Drift ResponseTime Sensitivity MeasurementRange
OperatingRestriction EnergyRestriction
hasOperatingProperty only
OperatingRange OperatingProperty
EnvironmentalOperatingProperty MaintenanceSchedule OperatingPowerRange
hasSurvivalProperty only
SurvivalRange SurvivalProperty
EnvironmentalSurvivalProperty SystemLifetime BatteryLifetime
19. A usage example
Upper
DOLCE SWEET
UltraLite
SSG4Env
infrastructure SSN
Schema
Service
External
FOAF Ordnance
Survey
Flood domain
Role Coastal Additional
Defences Regions
García-Castro R, Corcho O, Hill C. A Core Ontological Model for Semantic Sensor Web
Infrastructures. International Journal on Semantic Web and Information Systems Special
Issue on Sensor Networks, Internet of Things and Machine-to-Machine Communications,
Volume 8, Issue 1, 2013
20
21. Let’s now talk about metadata and social stuff
Application
Middleware
SPARQL-STR CQELS REST SPARQL
Linked Streams Linked Data
Data Stream COAP
Virtual
Management
Sensors
Systems Sensors
22. Ingredients for the Semantic Sensor Web
Core ontological model
Additional domain ontologies
Guidelines for generation of identifiers
Sensor Web programming interfaces (e.g.,
lsm.deri.ie)
Query processing engines
http://www.flickr.com/photos/santos/2252824606/
23. (Social) Sensor Metadata
Do we have enough reliable metadata about
sensors and observations?
NO!!!! People are messy!!!
Even if they are scientists who do not use
Twitter
Let’s start with Swiss people (you know, they
are always precise ;-))
24
24. SwissEx
• Global Sensor Networks, deployment for SwissEx.
• 28 Deployments, Aprox. 50 sensors in each deployment
• More than 1500 sensors
• Live updates. Low frequency
• Access to all metadata/not all data
• Distributed environment: GSN Davos, GSN Zurich, etc.
• In each site, a number of sensors available
• Each one with different schema
• Metadata stored in wiki
Sensor observations
• Federated metadata management:
• Jeung H., Sarni, S., Paparrizos, I., Sathe, S., Aberer, K., Dawes, N., Papaioannus, T., Lehning,
M.Effective Metadata Management in federated Sensor Networks. in SUTC, 2010
Sensor metadata
25
25. Sensor Metadata
station
location
sensors
model
properties
Sensors, Mappings and Queries 26
28. Sensor Metadata
Do we have enough reliable metadata about
sensors and observations?
NO!!!! People are messy!!!
Even if they are scientists who do not use
Twitter
Let’s start with Swiss people (you know, they
are always precise ;-))
Swiss are boring… Let’s now move to the rest of
the world (citizen sensing)
29
31. Classifying Sensor Data
• Idea: given a new time series, find similar ones in a
set of classified time-series
• Querying time series
• e.g. find a sub-sequence in a time series database
• Measuring time series similarity
• e.g. are these time series the same?
• Time series classification
• e.g. classify heart beat series: normal, murmur, etc
32
32. Challenges: Data Summaries
• Challenges
• Represent the data
approximating with fewer
linear segments:
• Tradeoff between
Accuracy vs Numerosity
Data buckets
33
33. Data summaries: Linear Approximations
• We care about the angles
π/2 a π/4
a c
d b
a 0
c
-π/4
d
34
34. Use the representation for Classifying
• Linear approximation
• Compute distribution of the slopes
• K-nearest neighbour classification
• Training-Test datasets:
• SwissExperiment
• AEMET
35
36. Conclusions
Identifying Sensor Observations
Take a look at the data
Machine Learning Techniques
Time Series Summarization
Clustering
Normalization
Distance Metrics
Work in progress
Use more of the social tagging information!
Test in Pachube
37. Query processing
Application
Middleware
SPARQL-STR CQELS REST SPARQL
Linked Streams Linked Data
Data Stream COAP
Virtual
Management
Sensors
Systems Sensors
38. Ingredients for the Semantic Sensor Web
Core ontological model
Additional domain ontologies
Guidelines for generation of identifiers
Sensor Web programming interfaces (e.g.,
lsm.deri.ie)
Query processing engines
http://www.flickr.com/photos/santos/2252824606/
39. SPARQL with streaming extensions
Streaming SPARQL
PREFIX fire: <http://www.semsorgrid4env.eu/ontologies/fireDetection#>
SELECT ?WindSpeed
FROM STREAM <http://…/SensorReadings.rdf> WINDOW RANGE 1 MS SLIDE 1 MS
WHERE {
?sensor fire:hasMeasurements ?WindSpeed
FILTER (?WindSpeed<30)
}
C-SPARQL
REGISTER QUERY WindSpeedAndDirection AS
PREFIX fire: <http://www.semsorgrid4env.eu/ontologies/fireDetection#>
SELECT ?sensor ?speed ?direction
FROM STREAM <http://…/SensorReadings.rdf> [RANGE 1 MSEC SLIDE 1 MSEC]
WHERE { …
CQELS
…
40
40. SPARQL-Stream
SELECT ?windspeed ?tidespeed
FROM NAMED STREAM <http://swiss-experiment.ch/data#WannengratSensors.srdf>
[NOW-10 MINUTES TO NOW-0 MINUTES]
WHERE {
?WaveObs a ssn:Observation;
Aggregates
ssn:observationResult ?windspeed;
ssn:observedProperty sweetSpeed:WindSpeed.Static & Streaming
?TideObs a ssn:Observation;
ssn:observationResult ?tidespeed;
ssn:observedProperty sweetSpeed:TideSpeed. Windows
FILTER (?tidespeed<?windspeed)}
Filters, Functions
Disclaimer: some features NYI
In progress: Benchmarking
41
41. Queries to Sensor Data
SNEEql
RSTREAM SELECT id, speed, direction FROM wind [NOW];
Data Stream Management System
Esper QL
SELECT wind_speed FROM wind_sensor.win:time(10 min)
Complex Event Processors
GSN RESTful service
http://montblanc.slf.ch:22001/multidata?vs[0]=wind_sensor&field[0]=wind_speed&
from=15/09/2011+05:00:00&to=15/09/2011+15:00:00
Pachube RESTful service
http://api.pachube.com/v2/feeds/14321/datastreams/4?start=2011-09-
02T14:01:46Z&end=2011-09-02T17:01:46Z
Sensor Data Middleware
Querying through ontologies?
43
42. Differences among all these systems
• Different Query Expressivity
• Windows?
• Union, Filters, Joins?
• Aggregates, Groups?
• Different delivery & query mechanisms
• Pull, Push?
• Continuous queries?, One-off?
• How to merge streaming and static data queries
• Commonalities in RESTful Services
44
43. Where is the Data?
wan7 timed sp_wind
timed: datetime PK 1 3.4
Esper sp_wind: float
2 5.6
3 11.2
GSN
4 1.2
5 3.1
SNEE .. …
SELECT sp_wind FROM wan7 WHERE sp_wind >10
SELECT sp_wind FROM wan7 [NOW -5 HOUR] WHERE sp_wind >10
45
44. Where is the Data?
GSN server instance
.. wan7
sensor1
sensor2 timed: datetime PK
Esper sensor3 sp_wind: float
…
Mappings
ssn:Observation
46
47. Querying the Observations
SELECT ?waveheight
FROM STREAM <www.ssg4env.eu/SensorReadings.srdf>
[NOW -10 MINUTES TO NOW STEP 1 MINUTE]
WHERE {
?WaveObs a sea:WaveHeightObservation; :22001/ multidata ?vs [0]= wan7 &
http://montblanc.slf.ch
field [0]= sp_wind
sea:hasValue ?waveheight; }
Query
:Wan4WindSpeed a rr:TriplesMapClass; Rewriting GSN
rr:tableName "wan7";
SPARQLStream API
rr:subjectMap [ rr:template
"http://swissex.ch/ns#WindSpeed/Wan7/{timed}";
Client
rr:class ssn:ObservationValue; rr:graph
ssg:swissexsnow.srdf ]; Mappings Query
rr:predicateObjectMap [ rr:predicateMap [ Processing
rr:predicate ssn:hasQuantityValue ]; Sensor
rr:objectMap[ rr:column "sp_wind" ] ]; Network
[tuples]
Data
[triples] translation
R2RML Mappings
49
48. Rewriting to different technologies
SELECT ?windspeed
FROM NAMED STREAM <http://swiss-
experiment.ch/data#WannengratSensors.srdf>
[NOW-10 MINUTE TO NOW-0 MINUTE]
WHERE { Query
?WaveObs a ssn:Observation; Rewriting
ssn:observationResult ?windspeed;
Algebra
ssn:observedProperty sweetSpeed:WindSpeed.
} representation
SELECT wind_speed_scalar_av, timed FROM
wan7.win:time(10 min)
Esper (CEP)
SELECT wan7.wind_speed_scalar_av AS windspeed,
wan7.timed AS windts FROM wan7[FROM NOW-10
MINUTES TO NOW] SNEE (DSMS)
http://montblanc.slf.ch:22001/multidata?vs[0]=wan7&
field[0]=wind_speed_scalar_av&
from=15/05/2011+05:00:00&to=15/05/2011+15:00:00
GSN (Middleware)
http://api.pachube.com/v2/feeds/14321/datastreams/4?st
art=2011-09-02T14:01:46Z&end=2011-09-02T17:01:46Z
Pachube (Middleware)
50
49. Query rewriting
SELECT ?waveheight
FROM STREAM <www.ssg4env.eu/SensorReadings.srdf>
[NOW – 5 HOUR TO NOW]
WHERE {
?WaveObs a ssn:ObservationValue;
qudt:numericalValue ?waveheight;
FILTER (?waveheight>10) }
SELECT sp_wind FROM wan7 [NOW -5 HOUR] WHERE sp_wind >10
50. Also for RESTful services
• GSN Web Services
• GSN URL API
• Compose the query as a URL:
http://montblanc.slf.ch :22001/ multidata ?vs [0]= wan7 &
field [0]= sp_wind &
from =15/05/2011+05:00:00& to =15/05/2011+10:00:00&
c_vs [0]= wan7 & c_field [0]= sp_wind & c_min [0]=10
SELECT sp_wind FROM wan7 [NOW -5 HOUR] WHERE sp_wind >10
?
52
51. Algebra expressions
π timed, http://montblanc.slf.ch :22001/ multidata ?vs [0]= wan7 &
field [0]= sp_wind &
from =15/05/2011+05:00:00& to =15/05/2011+10:00:00&
sp_wind c_vs [0]= wan7 & c_field [0]= sp_wind & c_min [0]=10
σ sp_wind>10
ω 5 Hour
SELECT sp_wind FROM wan7 [NOW -5 HOUR] WHERE sp_wind >10
wan7
53
52. Using the Mappings
π timed,
sp_wind
SELECT ?waveheight
σ
FROM STREAM <www.ssg4env.eu/SensorReadings.srdf>
[NOW – 5 HOUR TO NOW]
sp_wind>10
WHERE {
?WaveObs a ssn:ObservationValue;
qudt:numericalValue ?waveheight; ω 5 Hour
FILTER (?waveheight>10) }
wan7
wan7 ssn:ObservationValue
http://swissex.ch/data#
timed: datetime PK qudt:numericalValue Wan7/WindSpeed/ObsValue{timed}
sp_wind: float
xsd:datatype
sp_wind
54
55. The whole picture
Application
Middleware
SPARQL-STR CQELS REST SPARQL
Linked Streams Linked Data
Data Stream COAP
Virtual
Management
Sensors
Systems Sensors
56. In Summary, and for the future
• Use ontologies to query sensor data
• Using R2RML mappings (as in what I have been presenting)
• Using native RDF streams (as what Manfred presented on
Tuesday)
• Use extensions to SPARQL to handle data streams
• However, there is a need to standardise all these query
languages
• There is also a need for good benchmarks
• Different underlying systems can provide support to
data streams
• DSMS, CEP, RESTful services
• What is the best one for each purpose? Again, we need to
have good benchmarks
60
57. Some references
• Sheth A, Henson C, and Sahoo S, Semantic Sensor Web, IEEE
Internet Computing, 2008.
• Sequeda J, Corcho O. Linked Stream Data: A Position Paper.
Proceedings of the 2nd International Workshop on Semantic Sensor
Networks, 2009.
• Le-Phuoc D, Parreira JX, Hauswirth M. Challenges in Linked
Stream Data Processing: A Position Paper. Proceedings of the
3rd International Workshop on Semantic Sensor Networks, 2010.
• García-Castro R, Corcho O. Five challenges for the Semantic
Sensor Web. Semantic Web Journal 1, 2011
• García-Castro R, Corcho O, Hill C. A Core Ontological Model for
Semantic Sensor Web Infrastructures. International Journal
on Semantic Web and Information Systems Special Issue on Sensor
Networks, Internet of Things and Machine-to-Machine
Communications, Volume 8, Issue 1, 2013
• Calbimonte JP, Jeung H, Corcho O, Aberer K. Enabling Query
Technologies for the Semantic Sensor Web. International
Journal on Semantic Web and Information Systems Special Issue
on Sensor Networks, Internet of Things and Machine-to-Machine
Communications, Volume 8, Issue 1, 2013
58. SSSW2012 – The Ninth Summer School on Ontological Engineering
and Semantic Web
Semantic Sensor Networks
(and Social) Semantic Web
Oscar Corcho
(with the help of Jean Paul Calbimonte)
Ontology Engineering Group
Facultad de Informática, Universidad Politécnica de Madrid
ocorcho@fi.upm.es
59. Instructions for hands-on session
• It will be focused on understanding how sensor data can be
retrieved
• Disclaimer: a bit of cheating (working on historical data from
weather stations in Spain, and no continuous SPARQL querying)
• For those interested in hardcore stuff, talk to me later
• Quick instructions
• Based on the material that you already used for session 2
• Remember: Javascript, SPARQL queries, etc.
• Download a zip file from http://bit.ly/LHsqLx
• In fact, you will find it already in your desktop as sssw12-ho-
s7.zip
• Unzip it into /var/www (remember ‘sudoing’ and ‘chowning’ or
‘chmoding’)
• Go to http://localhost/sssw12-ho-s7/s7.html (instructions)
• The results will be at http://localhost/sssw12-ho-s7/index.html
63
Editor's Notes
- A core ontological model that can be used to describe sensor data streams, including the metadata about the sensor data sources and their observations. We take into account here the ontology developed for this purpose in the context of the W3C Semantic Sensor Network Incubator Group, which can be considered the current standard to be followed. - A set of additional domain ontologies in the area in which sensors are applied and for which they generate measurements (e.g., if we deal with environmental sensors related to water, then ontologies about different aspects of water will be needed). These ontologies must be aligned with the previous core ontology.Guidelines for the generation of identifiers (in the form of URIs, since this is one of the key ingredients of Linked Data) for sensors, their observations and the features of interest that they observe.Supporting Sensor Web programming interfaces (APIs) that make use of the HTTP protocol for serving the corresponding data whenever the previous URIs are dereferenced. Query processing engines that support extended versions of SPARQL (the query language used for Linked Data) and handle some of the most characteristic aspects of data streams, such as time and/or tuple windows. The management of spatio-temporal extensions of this query language may be also useful in this context.
- A core ontological model that can be used to describe sensor data streams, including the metadata about the sensor data sources and their observations. We take into account here the ontology developed for this purpose in the context of the W3C Semantic Sensor Network Incubator Group, which can be considered the current standard to be followed. - A set of additional domain ontologies in the area in which sensors are applied and for which they generate measurements (e.g., if we deal with environmental sensors related to water, then ontologies about different aspects of water will be needed). These ontologies must be aligned with the previous core ontology.Guidelines for the generation of identifiers (in the form of URIs, since this is one of the key ingredients of Linked Data) for sensors, their observations and the features of interest that they observe.Supporting Sensor Web programming interfaces (APIs) that make use of the HTTP protocol for serving the corresponding data whenever the previous URIs are dereferenced. Query processing engines that support extended versions of SPARQL (the query language used for Linked Data) and handle some of the most characteristic aspects of data streams, such as time and/or tuple windows. The management of spatio-temporal extensions of this query language may be also useful in this context.
- A core ontological model that can be used to describe sensor data streams, including the metadata about the sensor data sources and their observations. We take into account here the ontology developed for this purpose in the context of the W3C Semantic Sensor Network Incubator Group, which can be considered the current standard to be followed. - A set of additional domain ontologies in the area in which sensors are applied and for which they generate measurements (e.g., if we deal with environmental sensors related to water, then ontologies about different aspects of water will be needed). These ontologies must be aligned with the previous core ontology.Guidelines for the generation of identifiers (in the form of URIs, since this is one of the key ingredients of Linked Data) for sensors, their observations and the features of interest that they observe.Supporting Sensor Web programming interfaces (APIs) that make use of the HTTP protocol for serving the corresponding data whenever the previous URIs are dereferenced. Query processing engines that support extended versions of SPARQL (the query language used for Linked Data) and handle some of the most characteristic aspects of data streams, such as time and/or tuple windows. The management of spatio-temporal extensions of this query language may be also useful in this context.
- A core ontological model that can be used to describe sensor data streams, including the metadata about the sensor data sources and their observations. We take into account here the ontology developed for this purpose in the context of the W3C Semantic Sensor Network Incubator Group, which can be considered the current standard to be followed. - A set of additional domain ontologies in the area in which sensors are applied and for which they generate measurements (e.g., if we deal with environmental sensors related to water, then ontologies about different aspects of water will be needed). These ontologies must be aligned with the previous core ontology.Guidelines for the generation of identifiers (in the form of URIs, since this is one of the key ingredients of Linked Data) for sensors, their observations and the features of interest that they observe.Supporting Sensor Web programming interfaces (APIs) that make use of the HTTP protocol for serving the corresponding data whenever the previous URIs are dereferenced. Query processing engines that support extended versions of SPARQL (the query language used for Linked Data) and handle some of the most characteristic aspects of data streams, such as time and/or tuple windows. The management of spatio-temporal extensions of this query language may be also useful in this context.
- A core ontological model that can be used to describe sensor data streams, including the metadata about the sensor data sources and their observations. We take into account here the ontology developed for this purpose in the context of the W3C Semantic Sensor Network Incubator Group, which can be considered the current standard to be followed. - A set of additional domain ontologies in the area in which sensors are applied and for which they generate measurements (e.g., if we deal with environmental sensors related to water, then ontologies about different aspects of water will be needed). These ontologies must be aligned with the previous core ontology.Guidelines for the generation of identifiers (in the form of URIs, since this is one of the key ingredients of Linked Data) for sensors, their observations and the features of interest that they observe.Supporting Sensor Web programming interfaces (APIs) that make use of the HTTP protocol for serving the corresponding data whenever the previous URIs are dereferenced. Query processing engines that support extended versions of SPARQL (the query language used for Linked Data) and handle some of the most characteristic aspects of data streams, such as time and/or tuple windows. The management of spatio-temporal extensions of this query language may be also useful in this context.
The where clasue for both SPARQL extensions is the same
Addingsemanticsallowsthesearch and exploration of sensor data withoutany prior knowledge of the data sourceUsingtheprinciples of Linked Data facilitatestheintegration of stream data totheincreasingnumber of Linked Data collections