The talk was delivered by Dorothea Tsatsou at the 7th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP 2012) from December 3-4, 2012 in Luxembourg, Luxembourg. More info: http://bit.ly/VN77sB
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
Semantic personalisation in networked media: determining the background knowledge
1. Television Linked To The Web
Dorothea Tsatsou, Vasileios Mezaris, Ioannis Kompatsiaris
Semantic personalisation in networked media:
determining the background knowledge
Centre of Research and Technology Hellas
Information Technologies Institute
SMAP 2012 • 3-4 December 2012 • Luxembourg www.linkedtv.eu
2. Towards networked media…
www.linkedtv.eu
Social TV
Second Screen content
push – breakthrough
with HTML5 mobile
http://www.designbynotion.com/metamirror-next-generation-tv/
The rise of
Smart TVs
Second screen apps show related
content without disturbing the TV view
LG SmartTV, pic courtesy http://www.wired.com/gadgetlab/2011/01/lg-smart-tv/
2 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
3. …to LinkedTV
www.linkedtv.eu
TV v ie w
Interweaving TV and Web
content into a single
experience
Se c o nd
s c re e n
Web enrichment of TV
content: a personal
activity that will not
disrupt TV viewing
3 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
4. Networked Media personalisation challenge
www.linkedtv.eu
Digital information overload
Most crucial:
management of data
Start: Background
knowledge
Digital information heterogeneity
4 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
5. Linked Open Data – Overview
www.linkedtv.eu
Current trend for content interpretation, therefore gaining popularity for
user preference representation
DBPedia (different languages support)
Broad coverage, shallow and inconsistent
Freebase
Google Knowledge Graph – content augmentation
Community contributed
YAGO
schema.org
…
Domain-specific ontologies
Geonames, MusicBrainz, …
5 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
6. Linked Open Data – Advantages
www.linkedtv.eu
Broad coverage
Structure over big data
Community contribution,
minimizing manual engineering needs
evolving knowledge
Interlinking of LOD vocabularies
6 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
7. Linked Open Data – Drawbacks
www.linkedtv.eu
Inconsistency
Shallowness
Lacking user-pertinent information
User: Football, Manchester United
Content: Chelsea FC
LOD KB: Chelsea FC Football
User(Football fan)-pertinent KB:
disjoint Manchester United, Chelsea FC
“Unnecessary” volume in knowledge itself and mappings
Scalability
Privacy
Server-bound storage and processing
7 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
8. Formal Ontologies
www.linkedtv.eu
General upper level ontologies
DOLCE, SUMO, BFO, PROTON, …
Advantages:
Rich expressivity
Consistency and well-defined structure
Efficient alignment support over middle and domain ontologies for cross-domain
data handling
Disadvantages:
Generic
Voluminous
Ontologies for the media (TV) super-domain
BBC Programmes ontology
AVATAR ontology
8 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
9. User-specific ontologies & vocabularies
www.linkedtv.eu
Cognitive Characteristics Ontology
Upper characteristics schema (skills, interests, competences…)
OCUM (Ontological Cognitive User Model)
Imager, verbalizer, …
GUMO
Personality, facial expressions, motion, social environment, topics
FOAF & FOAF-WI
Relationships between people, social networking attributes, interest
expression schema
9 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
10. Aligning LOD and formal ontologies
www.linkedtv.eu
Purpose:
Unify relevant schemata under a more lightweight core ontology
Unify individuals under a single type and mapping them to the uniform schema
Individuals classification
NERD ontology
LOD mappings: Alchemy, DBPedia Spotlight, Extractiv, OpenCalais, Zemanta
Schema alignment
S-Match
Class labels information
AROMA
Association rule mining
BLOOMS
Wikipedia & DBPedia for LOD schema alignment, can map to upper level formal ontologies
BLOOMS+: contextual information to map LOD to PROTON
LogMap
Scalable automatic mapping & mappings consistency checking
10 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
11. Hypervideo content enrichment and linking
hypervideo to the web in LinkedTV www.linkedtv.eu
Cubism
Fauvism
Expressionism
CONTENT ENRICHMENT
11 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
12. The LinkedTV ontology (LUMO) design principles
www.linkedtv.eu
Homogenization of multidisciplinary and diverse information
Scalable modeling and matchmaking
Context modeling
Reuse of existing vocabularies and schemata
Use only the knowledge that means something to the user and for the
user of a smart TV environment
Expressive enough to make efficient inferencing
Selective (automatic) domain specific knowledge incorporation
Concept space based on more granular DBPedia subsets
Mappings to classify content to the lightweight LUMO and not to
represent preferences per se
12 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
13. Towards LUMO: a snapshot
www.linkedtv.eu
13 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
14. Minimizing the concept space
www.linkedtv.eu
Voluminous cross-domain information
Multilingual content
Single conceptualisation
Avoid redundant steps in the
inferencing process
dbpedia:’Building’ ≡ de.dbpedia:Fußbal ≡
schema.org:’Civic Structure’ ≡ dbpedia:AssociationFootball ≡
lumo:Building lumo:Football
14 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
15. Mappings
www.linkedtv.eu
Annotation-indexed fuzzy mappings
Separate Mappings
<EquivalentClasses> To not only concepts
<Class abbreviatedIRI="dbpedia:LOCATION"/>
<Class abbreviatedIRI="schema:Place"/>
but possibly plain
<Class abbreviatedIRI="lumo:Location"/> keywords, labels
</EquivalentClasses>
15 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
16. Future work:
Knowledge pulling and learning www.linkedtv.eu
Complexity minimization and scalability maximization upon
matchmaking:
Contextual knowledge pulling
Location and time-based context
Current viewing context
Evolving knowledge and group-specific knowledge
User clustering and learning cluster-specific knowledge (rules)
Expressivity to support rules
16 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
17. Conclusions
www.linkedtv.eu
A more lightweight user-pertinent ontology: LUMO
First version to be released by the end of the year under
data.linkedtv.eu/lumo
Encompassing content related and context (user characteristics,
sensor-extracted data) related knowledge
Emphasis on minimizing concept space yet maintaining sufficient
expressivity and meaningful information
Multilinguality support
17 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas
18. Thank you!
www.linkedtv.eu
Any questions?
www.linkedtv.eu
www.iti.gr
18 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas