SlideShare a Scribd company logo
1 of 39
ConnectME: connecting content
for future online TV & video
Lyndon Nixon
lyndon.nixon@modul.ac.at
Last update: September 25,2013
More at www.connectme.at
Contact : info@connectme.at
Connected Media Experiences
• New services deriving added value for video by
combining it with complementary content from
the Web
• Linking of video objects to the concepts they
represent, and hence related content about
those concepts
• Provisioned as an enhanced in-video experience
• Delivered via an end-to-end service platform
hosting the added value services over different
networks
Growth of Internet on the TV

Below: Boxee, ConnectedTV, Samsung Smart TV (http://thenextweb.com/gadgets/files/2011/05/5msamsung-apps-1.jpeg)
Growth of TV on the Internet
Growth of mobile video
Video streaming accounts
for 37% of all mobile traffic
Of all video streaming
traffic, YouTube accounts
for 45%
A Cisco study* on mobile
traffic growth expects
• 66% of all traffic by 2014
will be video
• having increased 66-fold
from 2009 to 2014
* Cisco‘s Visual Networking Index Global Mobile Data Forecast
2010-2015
Trending towards connected
media
Associating content to video

http://notube.open.ac.uk/sugartub

http://webmademovies.etherworks.ca/popcorndemo/
Emerging hypermedia

Asterpix‘s Asterbot
discontinued!
Cf. Videoclix, Overlay.TV, Klickable, ClickVideo

Wirewax
http://www.wirewax.com
What could be a
connected media experience?
Where is Obertauern?

Which peak is that
Can I ski there? Ho

What is the buildin
If it is a hotel, how
the accommodat
How to experience connected media?
The business case for hypervideo
• Offering an added value service
– Solely to win more visitors & keep them using the
service
– Encourage upgrades to premium packages (by
content provider, by end consumer)

• Monetarization of the links
– E-commerce (drive traffic to electronic purchases)
– Content partnerships (drive traffic to others‘ content)
– Ad inserts
The technical case for
connected media
• Channel annotation, enrichment and delivery
are all services of the ConnectME framework
Content workflow
• Upload video in partner CMS
• Fill in basic information and „commit“ video to
the ConnectME Framework (CMF)
• CMF includes video instance and preannotates it from the basic information and
video analysis components
• Annotation tool provides preview of preannotation and means to finalise annotation
• CMF handles links to content enrichments
• Preview in ConnectME player with widgets
rendering different content enrichments
Relationship Linked media framework (LMF) <->
Connected media framework (CMF)
• CMF wraps the LMF reusing its inherent media & metadata
handling functionality and realising the ConnectME workflow
on top.
Analysis, annotation
Visual analysis

LMF

Speech to text

Workflow
Linked
Media
Store

Concept
extraction

REGISTRATION

Interlinking, enrichment

Annotation
views

ConnectME ontology based

Linked Media
engine

rul
es

LOD

REST

Annotation
tool

Interlinked media generation

Presentation
engine
Enrichment
playout

Presentation, playout
REST

AUTENTICATION

Hypervideo
browser
Channel annotation
• Grouping of videos into a „channel“
• Easy to use tools for content owners or hosts
to annotate video segments with links to
semantic concepts
– Manual annotation tool
– Supported by
• automatic classifiers over visual features, audio,
subtitling/transcripts, or other annotations incl.
crowdsourced tagging
• tool tips for choosing concepts from pre-selected
sources
Yoovis use case: ConnectME
channel

Courtesy Yoovis GmbH, www.yoovis.tv
Integration of annotation
Pre-annotation in Yoovis use case
• Yoovis CMS calls a Web service on CMF which
accepts the mediaRSS format
– „registers“ a video instance in the CMF
– Includes video metadata directly extractable from
the mediaRSS, e.g. locator(s)
– Generates video annotations from title,
description and tags from the author
• Attachs these „pre-annotations“ as 'dct:subjects' in the
video metadata with local Yoovis terms, mapped where
possible to LOD such as DBPedia instances
Pre-annotation in the PS Media
use case
• PS Media CMS pushes to CMF a RDF
description generated in Drupal
– RDF Module maps Drupal node data to RDF
– LMF Module pushes this RDF into our server
– Generates video annotations from title,
description and subtitles from the EPG
• Attachs the „pre-annotations“ as in Yoovis use case
• Entity extraction from subtitles can associate entities
already to a temporal segment (store as annotation)
Pre-annotation (generic)
• Any client could directly call a Web service on
CMF which takes the parameters title,
description, URL & subtitles
– „registers“ a video instance in the CMF
– Includes locator(s) from URL parameter
– Generates video annotations from title,
description and subtitles
• Attachs the „pre-annotations“ as in Yoovis use case
• Entity extraction from subtitles can associate entities
already to a temporal segment (store as annotation)
Annotation suite
- HTML 5, CSS 3,
responsive layout, based
on Twitter Bootstrap
- jQuery, JS based UI
- Timeline based interface
- Allows adding/editing/
deleting annotations in/to
the CMF
- Allows registering videos
to and loading video
annotations from the CMF
- Multilanguage support
(currently EN, DE)
- Access via OpenID
Annotation suite
- Annotations are RDF and
concepts are using LOD
identifiers
- Annotation export in N3,
RDF/XML and Turtle
- Manual annotation with LOD
concepts (URIs) and HTML
content (URLs)
- Currently Plugins for
dbpedia and geonames
search APIs
- Plugin based architecture
allows adding further LOD
sources easily
Annotation model

published at www.connectme.at/ontology
Re-uses a subset of W3C
Media Ontology and
Open Annotation Model
Extends
mao:hasKeyword
to subproperties of
explicitly/implicitly
seeing/mentioning
concepts.
Pre-annotations attached
to video using
dct:subject and
skos:Concepts
cmo:preferredLabel
overrides concept labels
cmo:hasContent points
to static Web content
ConnectME annotation instance
<
http://connectme.at/yoovis_video1.ogv
> a ma:MediaResource;
ma:hasSubtitling <
http://connectme.at/yoovis_video1.srt
>;
ma:hasLanguage [ rdfs:label "de" ] ;
ma:hasContributor <
http://www.yoovis.at>;
ma:creator http://
www.yoovis.at/imprint#me
ma:date "2011-09-13"^^xsd:date;
ma:hasRelatedLocation "27.14,
16.83"^^geo:point;
ma:isCopyrightedBy <
http://www.yoovis.at> ;
ma:hasPolicy <
http://creativecommons.org/licenses/
...>;
ma:hasFragment http://
connectme.at/fragment/1.

http://connectme.at/fragment/1 a
ma:MediaFragment;
ma:locator <
http://connectme.at/yoovis_video1.ogv#t=
>;
cma:explicitlyMentions <
http://dbpedia.org/resource/AntiBaby%20P
>.

<http://connectme.at/annotation/1> a
oac:Annotation;
oac:target <
http://connectme.at/fragment/1>;
oac:body <
http://dbpedia.org/resource/AntiBaby%20P
>;
dcterms:creator http://
open.id/person1;
dcterms:created "2013-0219T16:20:38+00:00"^^xsd:dateTime .
Annotation suite
Production annotation suite
http://annotator.connectme.at
Annotation via the CMF
• Video metadata is turned into ConnectME
annotations on the CMF
– Ingestion services generate top level metadata
such as video title, description, author, license...

• Analysis of textual metadata to extract
keywords a task of the Apache Stanbol
Enhancer
– A generic enrichment chain for entities->DBPedia
– Specific enrichment chains for Yoovis and PS
Media (entities->keywords(->LOD))
Enrichment via the CMF
• Each concept needs to be linked to related
content items
– The player widgets could query this directly but
– The CMF can cache relevant links so that the
player can make simple queries for content

• Common data model for links regardless of
source, type etc.
– See the Media Ontology for Linked Media
Direct & indirect links
Direct links in the concept metadata to related
content can be transformed to the common
model by extending CMF reasoning rules
(„Annotation views“)
($c http://xmlns.com/foaf/0.1/depiction $m) -> ($m
http://connectme.at/ontology#explicitlyShows $c)

- Indirect links must be derived for types of concept
using other Web based content sources (APIs)
Linked Services Infrastructure
• http://production.sti2.org/lsi
– Web interface and REST API
– Input: a LOD URI, output: RDF for matching media
resources
Linked Services Infrastructure
Middleware to different Web APIs (YouTube, Flickr, Ookaboo, Foursquare
Venues, Instagram)
» „lowers“ the input concept to query parameters such as label, long/lat ...
» „lifts“ the API response to RDF links for the CMF, where a content item is
described and connected to the original concept by ma:hasKeyword

LMF

API query
LSI module
RDF answer

Linked
Media
Store

Linked Services
Infrastructure

New module
To-beinterpreted
answer

(wrapped)
query

„lifting“ and „lowering“
handled by LSI

Some online
content source
Linked Services Infrastructure
RDF response format based on
W3C Media Ontology
Types are e.g. Image, VideoTrack
Locator points to the actual
resource URL
hasKeyword links the resource to
LOD URIs related to the resource
We can include further metadata
as provided from the Web API
(e.g. when uploaded, no of
views, duration, size...)
Channel enrichment
• ConnectME player with optional styling
(customer branding)
• Pre-selection of content sources and
presentation (widgets) by administrator
(administration interface)
• On-the-fly video enrichment by selected
widgets at play-out (user interface)
• Administrator access to player analytics –
playback, click-through, in-widget actions
Hypervideo playout & browsing
Based on open source
LIME player project
HTML5 & CSS3 based
Video format support as
per browser support
- MPEG-4 (IE, Safari,
Android)
- WebM and OGV
(Firefox, Chrome,
Android, Opera)
Production player http://player.connectme.at
Development player http://tkurz.github.io/lime
Software project http://github.com/tkurz/lime (Apache 2.0 license)
Player implementations
• Desktop
– Administrative & user interface
– Widgets as extra webpage
regions („standard“ player)
– Fullscreen (with widgets as
overlays)
• TV
– GoogleTV (fullscreen, no video
pause, supports remote control)
• Mobile
– Tablet version (fullscreen
adapted to touch, min 7“
screen)
„The“ player incorporates both
standard and fullscreen impl.
Player widget model
Standard widget model
is tiles-based* and abstracts the
Interaction model from
• Desktop: point and click
• Mobile: touch
• TV: remote control
* A tile is 300x300px and a typical
widget is max 600x600px
„Core“ player widgets
• Info (DBPedia)
• Media (LSI)
• Map
„Extended“ player widgets
Extended widget model
needs definition of:
- Annotation trigger (concept type)
- Content to display (query CMF)
- How to display it (tiling)
„Live TV“ player widgets
PS Media use case
defines these additional widgets
- Actor
- Character
- Director
Demos
Production workflow has demos of ConnectME,
at http://player.connectme.at
IPTV example:
Choose „Fahrenheit 9/11“ and TV player
Mobile tourism example:
Choose „Schladming“ and tablet player

More Related Content

Similar to ConnectME: connecting content for future TV & video

Kimia General Presentation v5.0
Kimia General Presentation v5.0Kimia General Presentation v5.0
Kimia General Presentation v5.0kimia
 
How Open Data Can Enhance Interactive Television
How Open Data Can Enhance Interactive TelevisionHow Open Data Can Enhance Interactive Television
How Open Data Can Enhance Interactive TelevisionLinkedTV
 
Ajax, rss, feeds, web service,
Ajax, rss, feeds, web service, Ajax, rss, feeds, web service,
Ajax, rss, feeds, web service, jmradha krishnan
 
Wiring the Web
Wiring the WebWiring the Web
Wiring the Webgoodfriday
 
Kimia General Presentation v3.0
Kimia General Presentation v3.0Kimia General Presentation v3.0
Kimia General Presentation v3.0kimia
 
Taking Control of Your Future: Own Your Service Platforms
Taking Control of Your Future: Own Your Service PlatformsTaking Control of Your Future: Own Your Service Platforms
Taking Control of Your Future: Own Your Service PlatformsAlan Quayle
 
MNAssociationEnterpriseArchitectsCloudFoundryJuly2017
MNAssociationEnterpriseArchitectsCloudFoundryJuly2017MNAssociationEnterpriseArchitectsCloudFoundryJuly2017
MNAssociationEnterpriseArchitectsCloudFoundryJuly2017Andrew Ripka
 
Neev capabilities in building video and live streaming apps
Neev capabilities in building video and live streaming appsNeev capabilities in building video and live streaming apps
Neev capabilities in building video and live streaming appsNeev Technologies
 
Linking Services and Linked Data: Keynote for AIMSA 2012
Linking Services and Linked Data: Keynote for AIMSA 2012Linking Services and Linked Data: Keynote for AIMSA 2012
Linking Services and Linked Data: Keynote for AIMSA 2012John Domingue
 
Annual Project Scientific Report
Annual Project Scientific ReportAnnual Project Scientific Report
Annual Project Scientific ReportLinkedTV
 
XC partner slides
XC partner slidesXC partner slides
XC partner slidesmhruska
 
Beyond the Theme - Using WordPress as an API
Beyond the Theme - Using WordPress as an APIBeyond the Theme - Using WordPress as an API
Beyond the Theme - Using WordPress as an APIDavid Tufts
 
Plunet Summit 2018: Plunet 7.3 Unboxed! From Refactoring to Automation
Plunet Summit 2018: Plunet 7.3 Unboxed! From Refactoring to AutomationPlunet Summit 2018: Plunet 7.3 Unboxed! From Refactoring to Automation
Plunet Summit 2018: Plunet 7.3 Unboxed! From Refactoring to AutomationPlunet BusinessManager
 

Similar to ConnectME: connecting content for future TV & video (20)

Connected Media Experiences
Connected Media ExperiencesConnected Media Experiences
Connected Media Experiences
 
Kimia General Presentation v5.0
Kimia General Presentation v5.0Kimia General Presentation v5.0
Kimia General Presentation v5.0
 
How Open Data Can Enhance Interactive Television
How Open Data Can Enhance Interactive TelevisionHow Open Data Can Enhance Interactive Television
How Open Data Can Enhance Interactive Television
 
Presentation
PresentationPresentation
Presentation
 
Ajax, rss, feeds, web service,
Ajax, rss, feeds, web service, Ajax, rss, feeds, web service,
Ajax, rss, feeds, web service,
 
Semantic browsing
Semantic browsingSemantic browsing
Semantic browsing
 
Wiring the Web
Wiring the WebWiring the Web
Wiring the Web
 
Kimia General Presentation v3.0
Kimia General Presentation v3.0Kimia General Presentation v3.0
Kimia General Presentation v3.0
 
Taking Control of Your Future: Own Your Service Platforms
Taking Control of Your Future: Own Your Service PlatformsTaking Control of Your Future: Own Your Service Platforms
Taking Control of Your Future: Own Your Service Platforms
 
MNAssociationEnterpriseArchitectsCloudFoundryJuly2017
MNAssociationEnterpriseArchitectsCloudFoundryJuly2017MNAssociationEnterpriseArchitectsCloudFoundryJuly2017
MNAssociationEnterpriseArchitectsCloudFoundryJuly2017
 
Arneb
ArnebArneb
Arneb
 
Neev capabilities in building video and live streaming apps
Neev capabilities in building video and live streaming appsNeev capabilities in building video and live streaming apps
Neev capabilities in building video and live streaming apps
 
Sirio
SirioSirio
Sirio
 
A Complete Guide to Python Web Development
A Complete Guide to Python Web DevelopmentA Complete Guide to Python Web Development
A Complete Guide to Python Web Development
 
Linking Services and Linked Data: Keynote for AIMSA 2012
Linking Services and Linked Data: Keynote for AIMSA 2012Linking Services and Linked Data: Keynote for AIMSA 2012
Linking Services and Linked Data: Keynote for AIMSA 2012
 
Annual Project Scientific Report
Annual Project Scientific ReportAnnual Project Scientific Report
Annual Project Scientific Report
 
XC partner slides
XC partner slidesXC partner slides
XC partner slides
 
Beyond the Theme - Using WordPress as an API
Beyond the Theme - Using WordPress as an APIBeyond the Theme - Using WordPress as an API
Beyond the Theme - Using WordPress as an API
 
Plunet Summit 2018: Plunet 7.3 Unboxed! From Refactoring to Automation
Plunet Summit 2018: Plunet 7.3 Unboxed! From Refactoring to AutomationPlunet Summit 2018: Plunet 7.3 Unboxed! From Refactoring to Automation
Plunet Summit 2018: Plunet 7.3 Unboxed! From Refactoring to Automation
 
Dcms Datasheet V1
Dcms Datasheet V1Dcms Datasheet V1
Dcms Datasheet V1
 

Recently uploaded

activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
The Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementThe Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementNuwan Dias
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...Daniel Zivkovic
 
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideIEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideHironori Washizaki
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimizationarrow10202532yuvraj
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 

Recently uploaded (20)

activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
The Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementThe Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API Management
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
 
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideIEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization100+ ChatGPT Prompts for SEO Optimization
100+ ChatGPT Prompts for SEO Optimization
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 

ConnectME: connecting content for future TV & video

  • 1. ConnectME: connecting content for future online TV & video Lyndon Nixon lyndon.nixon@modul.ac.at Last update: September 25,2013 More at www.connectme.at Contact : info@connectme.at
  • 2. Connected Media Experiences • New services deriving added value for video by combining it with complementary content from the Web • Linking of video objects to the concepts they represent, and hence related content about those concepts • Provisioned as an enhanced in-video experience • Delivered via an end-to-end service platform hosting the added value services over different networks
  • 3. Growth of Internet on the TV Below: Boxee, ConnectedTV, Samsung Smart TV (http://thenextweb.com/gadgets/files/2011/05/5msamsung-apps-1.jpeg)
  • 4. Growth of TV on the Internet
  • 5. Growth of mobile video Video streaming accounts for 37% of all mobile traffic Of all video streaming traffic, YouTube accounts for 45% A Cisco study* on mobile traffic growth expects • 66% of all traffic by 2014 will be video • having increased 66-fold from 2009 to 2014 * Cisco‘s Visual Networking Index Global Mobile Data Forecast 2010-2015
  • 7. Associating content to video http://notube.open.ac.uk/sugartub http://webmademovies.etherworks.ca/popcorndemo/
  • 8. Emerging hypermedia Asterpix‘s Asterbot discontinued! Cf. Videoclix, Overlay.TV, Klickable, ClickVideo Wirewax http://www.wirewax.com
  • 9. What could be a connected media experience? Where is Obertauern? Which peak is that Can I ski there? Ho What is the buildin If it is a hotel, how the accommodat
  • 10. How to experience connected media?
  • 11. The business case for hypervideo • Offering an added value service – Solely to win more visitors & keep them using the service – Encourage upgrades to premium packages (by content provider, by end consumer) • Monetarization of the links – E-commerce (drive traffic to electronic purchases) – Content partnerships (drive traffic to others‘ content) – Ad inserts
  • 12. The technical case for connected media • Channel annotation, enrichment and delivery are all services of the ConnectME framework
  • 13. Content workflow • Upload video in partner CMS • Fill in basic information and „commit“ video to the ConnectME Framework (CMF) • CMF includes video instance and preannotates it from the basic information and video analysis components • Annotation tool provides preview of preannotation and means to finalise annotation • CMF handles links to content enrichments • Preview in ConnectME player with widgets rendering different content enrichments
  • 14. Relationship Linked media framework (LMF) <-> Connected media framework (CMF) • CMF wraps the LMF reusing its inherent media & metadata handling functionality and realising the ConnectME workflow on top. Analysis, annotation Visual analysis LMF Speech to text Workflow Linked Media Store Concept extraction REGISTRATION Interlinking, enrichment Annotation views ConnectME ontology based Linked Media engine rul es LOD REST Annotation tool Interlinked media generation Presentation engine Enrichment playout Presentation, playout REST AUTENTICATION Hypervideo browser
  • 15. Channel annotation • Grouping of videos into a „channel“ • Easy to use tools for content owners or hosts to annotate video segments with links to semantic concepts – Manual annotation tool – Supported by • automatic classifiers over visual features, audio, subtitling/transcripts, or other annotations incl. crowdsourced tagging • tool tips for choosing concepts from pre-selected sources
  • 16. Yoovis use case: ConnectME channel Courtesy Yoovis GmbH, www.yoovis.tv
  • 18. Pre-annotation in Yoovis use case • Yoovis CMS calls a Web service on CMF which accepts the mediaRSS format – „registers“ a video instance in the CMF – Includes video metadata directly extractable from the mediaRSS, e.g. locator(s) – Generates video annotations from title, description and tags from the author • Attachs these „pre-annotations“ as 'dct:subjects' in the video metadata with local Yoovis terms, mapped where possible to LOD such as DBPedia instances
  • 19. Pre-annotation in the PS Media use case • PS Media CMS pushes to CMF a RDF description generated in Drupal – RDF Module maps Drupal node data to RDF – LMF Module pushes this RDF into our server – Generates video annotations from title, description and subtitles from the EPG • Attachs the „pre-annotations“ as in Yoovis use case • Entity extraction from subtitles can associate entities already to a temporal segment (store as annotation)
  • 20. Pre-annotation (generic) • Any client could directly call a Web service on CMF which takes the parameters title, description, URL & subtitles – „registers“ a video instance in the CMF – Includes locator(s) from URL parameter – Generates video annotations from title, description and subtitles • Attachs the „pre-annotations“ as in Yoovis use case • Entity extraction from subtitles can associate entities already to a temporal segment (store as annotation)
  • 21. Annotation suite - HTML 5, CSS 3, responsive layout, based on Twitter Bootstrap - jQuery, JS based UI - Timeline based interface - Allows adding/editing/ deleting annotations in/to the CMF - Allows registering videos to and loading video annotations from the CMF - Multilanguage support (currently EN, DE) - Access via OpenID
  • 22. Annotation suite - Annotations are RDF and concepts are using LOD identifiers - Annotation export in N3, RDF/XML and Turtle - Manual annotation with LOD concepts (URIs) and HTML content (URLs) - Currently Plugins for dbpedia and geonames search APIs - Plugin based architecture allows adding further LOD sources easily
  • 23. Annotation model published at www.connectme.at/ontology Re-uses a subset of W3C Media Ontology and Open Annotation Model Extends mao:hasKeyword to subproperties of explicitly/implicitly seeing/mentioning concepts. Pre-annotations attached to video using dct:subject and skos:Concepts cmo:preferredLabel overrides concept labels cmo:hasContent points to static Web content
  • 24. ConnectME annotation instance < http://connectme.at/yoovis_video1.ogv > a ma:MediaResource; ma:hasSubtitling < http://connectme.at/yoovis_video1.srt >; ma:hasLanguage [ rdfs:label "de" ] ; ma:hasContributor < http://www.yoovis.at>; ma:creator http:// www.yoovis.at/imprint#me ma:date "2011-09-13"^^xsd:date; ma:hasRelatedLocation "27.14, 16.83"^^geo:point; ma:isCopyrightedBy < http://www.yoovis.at> ; ma:hasPolicy < http://creativecommons.org/licenses/ ...>; ma:hasFragment http:// connectme.at/fragment/1. http://connectme.at/fragment/1 a ma:MediaFragment; ma:locator < http://connectme.at/yoovis_video1.ogv#t= >; cma:explicitlyMentions < http://dbpedia.org/resource/AntiBaby%20P >. <http://connectme.at/annotation/1> a oac:Annotation; oac:target < http://connectme.at/fragment/1>; oac:body < http://dbpedia.org/resource/AntiBaby%20P >; dcterms:creator http:// open.id/person1; dcterms:created "2013-0219T16:20:38+00:00"^^xsd:dateTime .
  • 25. Annotation suite Production annotation suite http://annotator.connectme.at
  • 26. Annotation via the CMF • Video metadata is turned into ConnectME annotations on the CMF – Ingestion services generate top level metadata such as video title, description, author, license... • Analysis of textual metadata to extract keywords a task of the Apache Stanbol Enhancer – A generic enrichment chain for entities->DBPedia – Specific enrichment chains for Yoovis and PS Media (entities->keywords(->LOD))
  • 27. Enrichment via the CMF • Each concept needs to be linked to related content items – The player widgets could query this directly but – The CMF can cache relevant links so that the player can make simple queries for content • Common data model for links regardless of source, type etc. – See the Media Ontology for Linked Media
  • 28. Direct & indirect links Direct links in the concept metadata to related content can be transformed to the common model by extending CMF reasoning rules („Annotation views“) ($c http://xmlns.com/foaf/0.1/depiction $m) -> ($m http://connectme.at/ontology#explicitlyShows $c) - Indirect links must be derived for types of concept using other Web based content sources (APIs)
  • 29. Linked Services Infrastructure • http://production.sti2.org/lsi – Web interface and REST API – Input: a LOD URI, output: RDF for matching media resources
  • 30. Linked Services Infrastructure Middleware to different Web APIs (YouTube, Flickr, Ookaboo, Foursquare Venues, Instagram) » „lowers“ the input concept to query parameters such as label, long/lat ... » „lifts“ the API response to RDF links for the CMF, where a content item is described and connected to the original concept by ma:hasKeyword LMF API query LSI module RDF answer Linked Media Store Linked Services Infrastructure New module To-beinterpreted answer (wrapped) query „lifting“ and „lowering“ handled by LSI Some online content source
  • 31. Linked Services Infrastructure RDF response format based on W3C Media Ontology Types are e.g. Image, VideoTrack Locator points to the actual resource URL hasKeyword links the resource to LOD URIs related to the resource We can include further metadata as provided from the Web API (e.g. when uploaded, no of views, duration, size...)
  • 32. Channel enrichment • ConnectME player with optional styling (customer branding) • Pre-selection of content sources and presentation (widgets) by administrator (administration interface) • On-the-fly video enrichment by selected widgets at play-out (user interface) • Administrator access to player analytics – playback, click-through, in-widget actions
  • 33. Hypervideo playout & browsing Based on open source LIME player project HTML5 & CSS3 based Video format support as per browser support - MPEG-4 (IE, Safari, Android) - WebM and OGV (Firefox, Chrome, Android, Opera) Production player http://player.connectme.at Development player http://tkurz.github.io/lime Software project http://github.com/tkurz/lime (Apache 2.0 license)
  • 34. Player implementations • Desktop – Administrative & user interface – Widgets as extra webpage regions („standard“ player) – Fullscreen (with widgets as overlays) • TV – GoogleTV (fullscreen, no video pause, supports remote control) • Mobile – Tablet version (fullscreen adapted to touch, min 7“ screen) „The“ player incorporates both standard and fullscreen impl.
  • 35. Player widget model Standard widget model is tiles-based* and abstracts the Interaction model from • Desktop: point and click • Mobile: touch • TV: remote control * A tile is 300x300px and a typical widget is max 600x600px
  • 36. „Core“ player widgets • Info (DBPedia) • Media (LSI) • Map
  • 37. „Extended“ player widgets Extended widget model needs definition of: - Annotation trigger (concept type) - Content to display (query CMF) - How to display it (tiling)
  • 38. „Live TV“ player widgets PS Media use case defines these additional widgets - Actor - Character - Director
  • 39. Demos Production workflow has demos of ConnectME, at http://player.connectme.at IPTV example: Choose „Fahrenheit 9/11“ and TV player Mobile tourism example: Choose „Schladming“ and tablet player

Editor's Notes

  1. WireWax on iPad video: http://www.youtube.com/watch?v=KmILrnoc8bk
  2. Dev CMF: http://connectme.salzburgresearch.at:8080/CMF Prod CMF: http://connectme.salzburgresearch.at/CMF
  3. ConnectME videos: Login to http://www.yoovis.tv/administrator/video/as connectme@yoovis.tv - password: connectme. Currently there are 18 videos.
  4. CURRENTLY NEITHER PLAY (IN PLAYER) OR ANNOTATE (IN ANNOTATION TOOL) WORKS
  5. A sample mediaRSS file: What can be created in CMF as video metadata: (NOTES: we should be able to include video creator, owner, license...)
  6. A sample TV Anytime file: What can be created in CMF as video metadata:
  7. API: http://connectme.salzburgresearch.at/CMF/video/resource?url= Doc: see http://connectme.salzburgresearch.at/CMF/connectme/doc/rest/video/resource/url/index.html accepts video url, title, description, keywords HOW TO INCLUDE MULTIPLE LOCATORS HOW TO INCLUDE SUBTITLES FILE HOW TO INCLUDE OTHER METADATA, e.g. owner, creator, license
  8. Getting started: Open video → get a list of videos from the CMF the tool is configured to use „Enter a video URL“ option from „Open video“ also uses the Web service to add a new video instance to the CMF From Yoovis, the tool is opened directly using a special URL, e.g. http://annotator.connectme.at/yoovis.php?channel=1802&amp;video=7112&amp;partner=yoovis&amp;user=519 From PS Media, TBD If video is pre-annotated, these are loaded into the add new annotation interface If video is annotated, these are loaded into the timeline interface
  9. Annotation interface: - enter a concept or thing uses DBPedia - enter a place uses GeoNames - choose a suggestion uses pre-annotations (if any)
  10. Published at http://connectme.at/ontology
  11. TO BE CORRECTED; USING mao:locator
  12. Both tools are STABLE and production is FROZEN
  13. CMF can be configured to use different Stanbol enrichment chains WHAT IS IMPL IN PRODUCTION AND WHAT IS IMPL IN DEVELOPMENT CURRENTLY - for ingestion via Web service the default is DBPedia entity extraction using a local DBPedia dump (subset) from the title, descriptions and keywords Same for subtitle files, but generates annotations NOT pre-annotations - for ingestion from Yoovis CMS (mediaRSS) The default is keyword extraction using a local specific taxonomy of SKOS concepts (http://connectme.salzburgresearch.at/CMF/skos/index.html, open http://yoovis.tv/resource/context) Yoovis taxonomy looks completely flat, intended? Where are the mappings to LOD? OPTIONALLY I still need to see evidence of use of Stanbol giving us the best (pre-)annotations Online NER services (like NERD) could be used, NERD also supports subtitle files (SRT)
  14. AFAIK no annotation views are programmed in any CMF (widgets encapsulate this functionality if required by SPARQL querying cached LOD) LSI integration has been made in the DEVELOPMENT CMF?
  15. 1. https://github.com/tkurz/lime/wiki/DBPedia%20Info%20plugin 2. https://github.com/tkurz/lime/wiki/Map%20plugin 3. https://github.com/tkurz/lime/wiki/LSI%20Video%20Plugin
  16. Currently only enabled on the test player at cbara.github.io