Presentation I Gave at I-Semantic 2010 in Graz on September 3rd. The related paper is available on my website at http://iricelino.org/publications.

1. Towards the Formalization of
Interaction Semantics
Paper Authors:
Irene Celino Francesco Corcoglioniti
CEFRIEL – ICT Institute Dip. Di Elettronica e Informazione
Politecnico di Milano Politcnico di Milano
Via Fucini 2 Via Ponzio 34/5
20133 Milano, Italy 20133 Milano, Italy
irene.celino@cefriel.it corcoglioniti@elet.polimi.it
Presenter: Irene Celino
http://swa.cefriel.it http://iricelino.org
Towards the Formalization of Interaction Semantics
I-SEMANTICS – 6th International Conference on Semantic Systems – Graz, 1st-3rd September 2010 – © CEFRIEL 2010

2. Agenda
Background
Problem statement
Our approach
STAR:ship and STAR:dust ontologies
STAR:chart framework
Case study and evaluation
Challenges and evolutions
Towards the Formalization of Interaction Semantics 2 Graz, September 3rd 2010 – © CEFRIEL 2010

3. Background
Two trends:
Semantic Portals
Long-lasting trend in Semantic Web research (from
early 2000)
Reuse data semantics to create hyperlinks and
navigation paths
Increasing attention on Visualization
Engaging, emotional and/or natural information
visualization
“User-friendly” interfaces and intuitive representations
Towards the Formalization of Interaction Semantics 3 Graz, September 3rd 2010 – © CEFRIEL 2010

4. Problem statement and motivation
I know what my data
are about
I know how to design a
Data manager Web application Web designer
Data knowledge Web knowledge
Mmmh… I think the Web design doesn’t reflect
the meaning of my data. Can you make my data
Sigh… I made my best to present the data, but they
Web presentation more effective and meaningful?
are hard to understand for me. Can you better explain
to me what they mean and your expectations?
I don’t want to deal with
technical details…
And I don’t want to
learn all data details…
Towards the Formalization of Interaction Semantics 4 Graz, September 3rd 2010 – © CEFRIEL 2010

5. Cherry picking from today solutions
Web 2.0 and user experience
Interaction Patterns categorizations
“Fruition modalities”, i.e. how interaction patterns on the Web impact
modalities
user navigation
Web engineering
Web Information Architecture and “sitemaps”
Model-driven Web engineering and Semantic Web
Semantic Web technologies and knowledge modeling
Ontologies, mapping and rules
Towards the Formalization of Interaction Semantics 5 Graz, September 3rd 2010 – © CEFRIEL 2010

6. Domain knowledge vs. Interaction knowledge
Traditional approaches
?!? domain some portal ?!?
ontology ontology
M
Data manager
Web designer
Our approach to “fill the gap”
domain STAR:ship
ontology ontology
M1
Data manager
Web designer
M2
STAR:dust
ontology Conceptualization of Web
Conceptualization of applications structure
interaction patterns
Towards the Formalization of Interaction Semantics 6 Graz, September 3rd 2010 – © CEFRIEL 2010

7. STAR:chart foundations – high level
Data knowledge
STAR:chart Web knowledge
framework
STAR:chart is a framework to develop knowledge-intensive Web applications
that reduces the gap between data managers’ point of view and Web designers’ one,
by letting data managers express their requirements in a more straightforward way
and by helping Web designers to deal only with design-related issues.
issues
domain STAR:ship
ontology ontology
M1
Data manager
Web designer
Declaratively defined M2
by the Data Manager STAR:dust
ontology
(by using STAR:dust) Semi-automatically generated
by STAR:chart based on
“fruition modalities” patterns
Towards the Formalization of Interaction Semantics 7 Graz, September 3rd 2010 – © CEFRIEL 2010

8. STAR:ship ontology
Conceptualization of the Web structural and visual elements that
constitute the elementary building blocks of a Web page organization
Specification of the primitives to model the structure of a Web
application, i.e. its sitemap in terms of areas, pages, "units" or
"widgets" and links
It reflects the Web designer mindset
# STAR:ship namespace declaration
@prefix ss: <http://swa.cefriel.it/ontologies/starship#> .
# definition of the widget class
ss:Widget a owl:Class ;
rdfs:subClassOf ss:Unit .
# definition of the event class
ss:Event a owl:Class .
# definition of the triggersEvent property
ss:triggersEvent a owl:AnnotationProperty .
# example of widget definition
ss:SearchFormWidget a owl:Class ;
rdfs:subClassOf ss:Widget ;
ss:hasJavaClass "it.cefriel.swa.star.widgets.SearchFormWidget"^^xsd:string ;
ss:triggersEvent ss:submitSearchEvent .
http://swa.cefriel.it/ontologies/starship#
Towards the Formalization of Interaction Semantics 8 Graz, September 3rd 2010 – © CEFRIEL 2010

9. STAR:dust ontology
Specification of the different roles the data should play in a Web
application
Definition of the "fruition modalities", i.e. the possible interaction styles
of the final users with the presented information (searching, browsing,
detail viewing, tagging, editing, rating, etc.)
It expresses the best practices to access and interact with the data
# STAR:dust namespace declaration
@prefix sd: <http://swa.cefriel.it/ontologies/stardust#> .
# definition of the PresentationProperty class
sd:PresentationProperty a rdfs:Class ;
rdfs:subClassOf rdf:Property .
# definition of some presentation properties
sd:RelevantProperty a sd:PresentationProperty .
sd:SortableProperty a sd:PresentationProperty .
sd:EditableProperty a sd:PresentationProperty .
http://swa.cefriel.it/ontologies/stardust#
It represents the data expert's point of view it can be used in the
mapping to domain-specific ontologies
Towards the Formalization of Interaction Semantics 9 Graz, September 3rd 2010 – © CEFRIEL 2010

10. Mapping M1
Mapping between domain ontology and STAR:dust whenever a Web
user interface is built on top of a new kind of dataset
It does not require dealing with fine-grained specification of data
selectors and content formatting
It can be directly operated by the data expert, without the need of a
specific knowledge about Web design and development
The structure of this mapping is part of the STAR:dust ontology
# definition of mapping in STAR:dust
domain
sd:PresentationMapping a owl:Class . ontology
M1
# class and properties connected via a mapping
sd:onClass a owl:ObjectProperty ;
rdfs:range rdfs:Class .
STAR:dust
ontology
sd:mappingSource a owl:ObjectProperty ;
rdfs:domain sd:PresentationMapping ;
rdfs:range rdf:Property .
sd:mappingDestination a owl:ObjectProperty ;
rdfs:domain sd:PresentationMapping ;
rdfs:range sd:PresentationProperty .
http://swa.cefriel.it/ontologies/stardust#
Towards the Formalization of Interaction Semantics 10 Graz, September 3rd 2010 – © CEFRIEL 2010

11. STAR:chart framework foundations
interaction patterns
STAR:dust STAR:dust is the Web presentation and
ontology interaction conceptual model
The mapping definition is the artifact – by the
mapping data manager – that expresses the role of the
definition
M1 data in the final Web application; it puts in relation
the data ontology with the STAR:dust ontology
The widgets are the software components that
widgets implement the presentation and interaction
patterns defined in the STAR:dust ontology
Web appl. patterns
STAR:ship is the Web sitemap structure
STAR:ship conceptual model
ontology
The sitemap specification is the artifact – semi-
automatically produced by the framework itself –
sitemap that specifies the actual structure of the Web
specification
M2 application in terms of widgets; it is expressed in
terms of the STAR:ship ontology
Towards the Formalization of Interaction Semantics 11 Graz, September 3rd 2010 – © CEFRIEL 2010

12. STAR:chart framework architecture
STAR:dust widgets STAR:ship
ontology ontology
**
framework portal pages
core generation
mapping
definition * sitemap
*
specification
*
**
config. time
run-time
domain
ontology datasource
Towards the Formalization of Interaction Semantics 12 Graz, September 3rd 2010 – © CEFRIEL 2010

13. The project
“Service-Finder is a platform for service discovery in which Web Services
“Service-Finder is a platform for service discovery in which Web Services
are embedded in a Web 2.0 environment”
are embedded in a Web 2.0 environment”
http://demo.service-finder.eu
The domain ontologies describe Web services and their
characteristics
The portal allows users to search and navigate through the
service data and interact with the data in a Web 2.0 fashion
We mapped Service-Finder (SF) ontologies to the fruition modalities
of the STAR:dust ontology
We expressed the sitemap in terms of the basic elements of the
STAR:ship ontology to describe the pages and their navigation
We employed the STAR:chart framework to realize the SF portal
Towards the Formalization of Interaction Semantics 13 Graz, September 3rd 2010 – © CEFRIEL 2010

14. “Behind the scenes” of the SF Interface
Mapping between SF ontologies and STAR:dust
Service-Finder mapping STAR:dust
ontologies definition ontology
C sfo:Service dc:title P
P sfo:hasName
P sfo:rating
SD:RelevantProperty P
C sfo:Endpoint SD:DetailProperty P
P sfo:availabilityLastMonth
:sampleServiceMapping a SD:PresentationMapping ;
SD:onClass sfo:Service ;
SD:mappingSource sfo:hasName ;
SD:mappingDestination dc:title ;
mappings.n3
SD:mappingDestination SD:RelevantProperty ;
SD:isMultiValue "false" ;
rdfs:label "Name" ;
SD:noValueText " - " .
Towards the Formalization of Interaction Semantics 14 Graz, September 3rd 2010 – © CEFRIEL 2010

15. “Behind the scenes” of the SF Interface
Service-Finder portal sitemap
Service details page
TabWidget
LoginWidget
PropertyWidget RatingWidget
AddTagWidget AddCategoryWidget
Towards the Formalization of Interaction Semantics 15 Graz, September 3rd 2010 – © CEFRIEL 2010

16. “Behind the scenes” of the SF Interface
Widgets’ configuration
STAR:dust
SELECT ?r
WHERE{ SD:RatingProperty P
<x> a Class ;
RatingProperty ?r. RatingWidget
(Java class)
}
HTML
mapping definition template
:m1 a SD:PresentationMapping ; Rating:
SD:onClass sfo:Service ;
SD:mappingSource sfo:rating ;
SD:mappingDestination SD:RatingProperty .
SELECT ?r
instance of WHERE{
RatingWidget <s> a sfo:Service ;
on sfo:Service sfo:rating ?r .
}
Towards the Formalization of Interaction Semantics 16 Graz, September 3rd 2010 – © CEFRIEL 2010

17. “Behind the scenes” of the SF Interface
Widgets’ execution
RatingWidget
(Java class)
HTML
instance of SELECT ?r template
RatingWidget WHERE{
on sfo:Service <s> a sfo:Service ; Rating:
sfo:rating ?r .
}
?r
Triples in the datasource
“3.5”
HTTP Request :ServiceX a sfo:Service ;
sfo:rating “3.5” ;
...
Service details page
on :ServiceX
HTTP Response
generated Service details page
HTML Rating:
fragment on :ServiceX
Towards the Formalization of Interaction Semantics 17 Graz, September 3rd 2010 – © CEFRIEL 2010

18. Evaluation
Usability questionnaires
Usability survey about the Service-Finder portal
http://service-finder.cefriel.it/
Average score: 73.3 (range: 0-100, average “good” values: 65-70)
Implicit user feedbacks
Portal logs leads to insights on usage wrt the fruition modalities
A special kind of widgets logs users’ click stream
Internal evaluation
6 persons involved in the portal development
Flexibility and extendibility (beta approach)
Missing “user-friendly” support to mapping
Towards the Formalization of Interaction Semantics 18 Graz, September 3rd 2010 – © CEFRIEL 2010

19. Challenges and evolutions (1/2)
Open Linked Data movement and Linked Data Cloud
open and decentralized knowledge-intensive sources
new links and connections emerge over time
data schemata are never complete, but "fluid" and possibly
fluid
continuously changing
the more the data grows, the easier to find data inconsistencies
Challenge: presenting linked data to the human user
Linked Data Browsers generic but with limited capabilities
need for a broader approach keeping into account the basic rules of
Interaction Design to improve the usability of such tools
Towards the Formalization of Interaction Semantics 19 Graz, September 3rd 2010 – © CEFRIEL 2010

20. Challenges and evolutions (2/2)
Challenge: unavailability of a complete schema of data to
be displayed
define a "default" visual representation
default
retrieve not only the (linked) data, but also their expected
visualization (metadata about the linked data), similarly to the vOID
vocabulary or the SPARQL endpoint Service Description
Challenge: editing and cleansing of linked data
“collective intelligence”: engaging the final user in editing,
editing
correction and enrichment
linked data are not enough, other context data are needed (e.g.
source or timestamp)
consequent challenges: what is the best and more natural way to
present provenance or time information to the user? what are the
correct interaction patters to be employed to support user editing?
Towards the Formalization of Interaction Semantics 20 Graz, September 3rd 2010 – © CEFRIEL 2010

21. Thanks for your attention! Any question?
Towards the Formalization of Interaction Semantics
Paper Authors: Irene Celino, Francesco Corcoglioniti
Contact: Irene Celino – Semantic Web Practice
CEFRIEL – ICT Institute, Politecnico di Milano
email: irene.celino@cefriel.it – web: http://swa.cefriel.it
personal website: http://iricelino.org
phone: +39-02-23954266 – fax: +39-02-23954466
slides available at: http://www.slideshare.net/iricelino
Towards the Formalization of Interaction Semantics
I-SEMANTICS – 6th International Conference on Semantic Systems – Graz, 1st-3rd September 2010 – © CEFRIEL 2010