Semantic Metadata to Support Device Interaction in Smart Environments

Semantic Metadata to Support Device Interaction in Smart Environments 1|
Simon Mayer
http://people.inf.ethz.ch/mayersi
Simon Mayer and Gianin Basler, ETH Zurich
Fourth International Workshop on the Web of Things (WoT 2013)
September 9th 2013, Zurich, Switzerland
Semantic Metadata to Support Device
Interaction in Smart Environments

Simon Mayer
Myself
Simon Mayer, PhD Student at ETH Zurich, WoT since 2009

Realize the full potential of smart environments
by integrating functionality across services
I know the
temperature!
What’s the
temperature?
Is it going to rain
today?
Do I need to
be watered?

I can find song
files for you!
I want to listen to
“Yellow Submarine”
I can play song files!
Realize the full potential of smart environments
by integrating functionality across services

 Enabling services to talk to each other and form mashups
 Enabling people to talk to service mashups

Simon Mayer
 Heterogeneous services from different providers should
be dynamically composable
 Services don’t know anything about how to interact with other
services when they are created (i.e., no “standards”)
 Interaction metadata discoverable on the fly!
 Flexibility
 System should work in the user’s home, office, hotels, etc.
Challenges

Simon’s Office
- Contains 2 services
Think about a smart environment…
I have a song title,
“Yellow Submarine”.
I’d like to play that song.
Soodle S.A
Songs Search
Machine
Method: GET
Input: Song Title
Output: MP3 URL
Sapple, Inc.
Media Player
Method: PUT
Input: MP3 URL
Output: Plays song
Not possible! No Link!
I can do
that!
:-(

The songs search machine doesn’t provide any
hypermedia controls that link to the media player
Nor any information that helps the client find out that it could play a song using the services
I can find that
out, actually
YES. I GOT IT!
Our goal is to enable it for this guy!

Simon Mayer
 Crowd-sourced Service Integration
 Web Linking, “Globalized” HATEOAS
 Semantics-based Service Integration
 Web Linking + Functional metadata
Service Integration: “Self-aware” Things?
Techniques for Semantic Service Integration

Simon Mayer
 Rely on developers or users to publish links
Semantic Service Integration
Crowd-sourced Service Integration
Simon’s Office
Soodle S.A
Songs Search
Machine
Method: GET
Input: Song Title
Output: MP3 URL
Sapple Inc.
Media Player
Method: PUT
Input: MP3 URL
Output: Will play music
I know that you can
connect these services!
Let me show you!

Simon Mayer
Approach: Service Marketplaces
S. Mayer, D. Karam. A Computational Marketplace for the Web of Things, 2012
 Mashup developers find links manually, but publish their
mashups in a “WoT Marketplace”
 Clients can ask the marketplace for linking information
 These links make sense, because they are used in a mashup!
 Clients only need to choose which link to follow
 We called these «Computational» and «WoT»
marketplaces…

Simon Mayer
Service Marketplaces
Marketplace
Soodle S.A
Songs Search
Machine
Method: GET
Input: Song Title
Output: MP3 URL
Sapple Inc.
Media Player
Method: PUT
Input: MP3 URL
Node A
Node B
Node C
Node D
Node E

Marketplace
Node A
Node B
Node C
Node D
Node E
I know that you can connect
the services at A and E!
Let me show you!

POST wot_marketplace.org/nodes/A?link=nodes/E&name=play_song
202 Created

Marketplace
Node A
Node B
Node C
Node D
Node E
Now, others can use
that information!
play_song

GET wot_marketplace.org/nodes/A
{
“service” : “www.songsearch.org”
“forward_paths” : {
“play_song” : “nodes/E”
}
}
I can follow that
link now, too!

Issue 1: Availability and reliability of the crowd?
Issue 2: Clients are compiled against the link names (e.g., “play_song”)!
This didn’t enable fully automatic service usage!
Marketplace
Node A
Node B
Node C
Node D
Node E
play_song

Can we automate the publishing of links and the traversal itself?
Simon’s Office
Soodle S.A
Songs Search
Machine
Method: GET
Input: Song Title
Output: MP3 URL
Sapple Inc.
Media Player
Method: PUT
Input: MP3 URL

Simon Mayer
Idea: Semantic metadata to find potential links!
Functional Semantic Metadata
Simon’s Office
Soodle S.A
Songs Search
Machine
Method: GET
Input: Song Title
Output: MP3 URL
Sapple Inc.
Media Player
Method: PUT
Input: MP3 URL
R.Verborgh et al. RESTdesc - A Functionality-Centered Approach to Semantic Service Description and Composition, 2012
S. Mayer and G. Basler. Embedded Semantic Metadata to Support Device Interaction in Smart Environments, 2013

Simon Mayer
Simon’s Office
Soodle S.A
Songs Search
Machine
Method: GET
Input: Song Title
Output: MP3 URL
Sapple Inc.
Media Player
Method: PUT
Input: MP3 URL
Based on what I know
about these services, there
could be a link here!

Embedding of semantic functional
service descriptions
How?
Connect that
information to its API
What?
Describe what a
service does

Based on what I know
about these services, …
Use of a reasoning engine to
process these descriptions

Simon Mayer
 Usability as one major constraint
 It should not be too hard to create service descriptions
 It should be easy to create user goals
 Pragmatism as another
 Make use of previous research in semantic technologies
 Use existing ontologies. Avoid creating new ones!
 Make it work for real use cases
 Format: Notation3 (user-friendly RDF with inference rules)

The title of the resource people.org/person1/, is “Person 1”, and it knows
another resource, people.org/person2/
@prefix dc: <http://purl.org/dc/elements/1.1/>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
<people.org/person1>
dc:title "Person 1";
foaf:knows "people.org/person2/".
Notation3
Brief N3 Primer: Statements

Brief N3 Primer: Rules
𝑹𝒖𝒍𝒆: 𝑃𝑟𝑒𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 → 𝑃𝑜𝑠𝑡𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛
“people.org/person1/ knows people.org/person2/ ”
implies
“people.org/person2/ knows people.org/person1/ ”
If one person knows another person, this second person also knows the first.
(“foaf:knows is symmetric”)

Notation3
{
Precondition.
}
=>
{
Postcondition.
}
is shorthand for=> log:implies

Notation3
{
?x a foaf:Person;
foaf:knows ?y.
}
=>
{
Postcondition.
}
Superb tutorial at n3.restdesc.org (by Ruben Verborgh)

Notation3
{
?x a foaf:Person;
foaf:knows ?y.
}
=>
{
?y foaf:knows ?x.
}
Superb tutorial at n3.restdesc.org (by Ruben Verborgh)

Simon Mayer
Simon’s Office
Soodle S.A
Songs Search
Machine
Method: GET
Input: Song Title
Output: MP3 URL
Sapple Inc.
Media Player
Method: PUT
Input: MP3 URL
Now: Semantic metadata for the Songs Search Machine

Simon Mayer
Precondition: “What facts does a meaningful execution of the service require?”
We have a Song Title
Postcondition: “What additional facts hold after an execution of the service?”
We have the URL of a song with that title
Soodle S.A
Songs Search
Machine
Method: GET
Input: Song Title
Output: MP3 URL

Notation3
The Songs Search Machine
{
Precondition.
}
=>
{
Postcondition.
}
If we know a song title, we also know the URL of a song with that title.

Notation3
{
?songTitle a dc:title.
}
=>
{
Postcondition.
}

Notation3
{
}
=>
{
?songURL a dc:Song;
dc:title ?songTitle.
}
Store this as ssm_rule.n3

“Yellow Submarine” a dc:title.
“Yellow Submarine” a dc:Title.
?songURL a dc:Song;
dc:title “Yellow Submarine”.
[+ proof]
ssm_rule.n3
But I still don’t know
how to call that
service!!

How to include the request? Basically two options…
𝑹𝒖𝒍𝒆: 𝑃𝑟𝑒𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 ∧ 𝑅𝑒𝑞𝑢𝑒𝑠𝑡 → 𝑃𝑜𝑠𝑡𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛
𝑹𝒖𝒍𝒆: 𝑃𝑟𝑒𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 → 𝑃𝑜𝑠𝑡𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 ∧ 𝑅𝑒𝑞𝑢𝑒𝑠𝑡
This works, but is impractical (knowledge about all requests required from the start…)
It also messes up the quantification!
“Given the Preconditions, all GET requests give the Postcondition”
This works too, and it is sound!
“Given the Preconditions, there exists a GET request that gives the Postcondition”
R.Verborgh et al. RESTdesc - A Functionality-Centered Approach to Semantic Service Description and Composition, 2012
The second option
sounds better to me!

Notation3
{
}
=>
{
?songURL a dc:Song;
Request.
}
If we know a song title, we also know the URL of a song with that title. Plus, we
know about an HTTP request.

Notation3
@prefix http: <http://www.w3.org/2011/http#>.
{
}
=>
{
?songURL a dc:Song;
Request.
}

Notation3
@prefix http: <http://www.w3.org/2011/http#>.
{
}
=>
{
?songURL a dc:Song;
_:request http:methodName “GET”;
http:requestURI (<www.ssm.org?title=>?songTitle);
http:resp [ http:body ?songURL ].
}
Store this as ssm_rule.n3
We’ll refer to it as a “RESTdesc” description

“Yellow Submarine” a dc:Title.
?songURL a dc:Song;
dc:title “Yellow Submarine”.
_:request http:methodName “GET”;
http:requestURI (<www.ssm.org?title=>?songTitle);
http:resp [ http:body ?songURL ].
[+ proof]
ssm_rule.n3

Simon Mayer
Now, we only need to link that to the service!
Multiple ways to do that…
We use HTTP OPTIONS
“This method allows the client to determine the options and/or requirements
associated with a resource, or the capabilities of a server, without implying a
resource action or initiating a resource retrieval.” (RFC 2616)
OPTIONS www.ssm.org
Link: <ssm_rule.n3>; rel=“describedBy“
And the Link Header (RFC 5988) to return the description
Soodle S.A
Songs Search
Machine
ssm_rule.n3

How do I say that I
want a song URL?

Goal: ?song a dc:Song; hasTitle “Yellow Submarine”.
A proof that is parsed to an internal HTTP request description format
Send a GET request to www.ssm.org?title={songTitle} and substitute “songTitle” with
“Yellow Submarine”. Then, unpack the response. Its body contains the song.
GET ?title=YellowSubmarine
www.songs.org/yellowsub.mp3
ssm_rule.n3
Soodle S.A
Songs Search
Machine
ssm_rule.n3

And how do I play “Yellow
Submarine” now??

Simon Mayer
Missing: Semantic metadata for the Media Player!
Simon’s Office
Soodle S.A
Songs Search
Machine
ssm_rule.n3
Sapple Inc.
Media Player
Method: PUT
Input: MP3 URL

Simon Mayer
Perfect!
Simon’s Office
Soodle S.A
Songs Search
Machine
ssm_rule.n3
Sapple Inc.
Media Player
mp_rule.n3

I want to play “Yellow
Submarine” now!!
Then formulate your goal.
But be precise!
?state a st:State;
log:includes {
:SimonsOffice myOnt:musicPlaying [ dc:title “Yellow Submarine” ]
}
I want Simon’s office to be in a state
where “Yellow Submarine” is playing.

Client Goal
A proof that is parsed to an internal HTTP request description format
“Yellow Submarine”. Then, unpack the response. Its body contains the songURL. Take this
variable and send a PUT to www.mp.org?song={songURL}
ssm_rule.n3
Execute the requests!
mp_rule.n3

“Yellow Submarine”. Then, unpack the response. Its body contains the songURL. Take this
variable and send a PUT to www.mp.org?song={songURL}
Sapple Inc.
Media Player
mp_rule.n3
GET ?title=YellowSubmarine
Soodle S.A
Songs Search
Machine
ssm_rule.n3
PUT ?song=www.songs.org/yellowsub.mp3
Yay!

Sounds complicated. Could you
sum up what I have to do?
1. Create your goal and inputs
2. Find a reasoner
3. Ask the reasoner how the goal is reachable
4. Parse the proof to extract the requests
5. Execute the requests

I don’t believe this!
Let’s see it!

http://192.168.0.21:8000/proof?
q=http://people.inf.ethz.ch/mayersi/testdescriptions/n3goals
/user-goal-music.n3&
r=http://people.inf.ethz.ch/mayersi/testdescriptions/n3goals
/user-input.n3
user-input.n3 contains “Yellow Submarine” a dc:title.”
Reasoner has access to ssm_rule.n3, mp_rule.n3, and a states ontology.
Do Proof!

Oh, it actually works…
But I can’t understand a thing!

Do Proof!
http://192.168.0.21:8000/proofHumanReadable?
q=http://people.inf.ethz.ch/mayersi/testdescriptions/n3goals
/user-goal-music.n3&
r=http://people.inf.ethz.ch/mayersi/testdescriptions/n3goals
/user-input.n3
user-input.n3 contains “Yellow Submarine” a dc:title.”
Reasoner has access to ssm_rule.n3, mp_rule.n3, and a states ontology.

But isn’t it too hard
to create the goal, for
a client like me?
That’s tricky. But we’re working on it and have come up with some good
ideas… one of those ideas would even allow for integration with Clickscript !
And you know how to use that, right?
Now I get it!

Hm. So I should
just state my goals.
And I can find and
use services, too!

Simon Mayer
 Localization + stream transmitter+receivers in all rooms
 First version took 10s to switch songs (mostly loading .n3 files… described in the paper)
 Cut that by actively discovering services and pre-loading descriptions
This is in the paper…
Prototype: A music stream that follows you!

Simon Mayer
Our Goal

Simon Mayer
 User defines preferences
(temperature, preferred song, etc.)
 User agent negotiates with the user’s environment to set
these preferences
 User gets feedback regarding success/failure
Our Goal

Thank you!
Simon Mayer and Gianin Basler: Semantic Metadata to Support Device Interaction in
Smart Environments. Proceedings of the 4th International Workshop on the Web of Things
(WoT 2013). Zurich, Switzerland, September 2013

Simon Mayer
Credits
 Nadine Inhelder, Gianin Basler
 Ruben Verborgh

 http://andrewbleakley.com
 http://www.webmarchand.com
 http://www.patentspostgrant.com/
 http://www.orkin.com
 http://sweetclipart.com
 http://la-matrice.org
 http://learn.randolph.k12.ma.us
 http://theparisreview.org
 http://bestclipartblog.com
 http://freepik.com
 http://http://www.rugdots.com
 http://www.productdose.com
 http://westwoodblinds.co.uk
 http://www.ambientdevices.com
 http://www.pats.ua.ac.be
 http://www.vesternet.com
 http://www.rovingnetworks.com
 http://www.qualitymechanicalservices.com
 http://chantalteakettle.org
 http://blog.eeye.com
 http://cartographersguild.com
 http://haverford.edu
 http://clker.com
 http://stickers-addict.fr
 http://en.wikipedia.org
Image Sources

Semantic Metadata to Support Device Interaction in Smart Environments

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Semantic Metadata to Support Device Interaction in Smart Environments

Ähnlich wie Semantic Metadata to Support Device Interaction in Smart Environments (20)

Mehr von Simon Mayer

Mehr von Simon Mayer (7)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Semantic Metadata to Support Device Interaction in Smart Environments