A new approach for adding metadata to online Media - TNC2012

A new approach for adding metadata to online Media

The results of two MediaMosa technology-scouting projects

Frans Ward
TNC 2012 - Reykjavik, May 22 2012

Tuesday, May 22, 12

About SURFnet...

• Development and exploitation of
- the Dutch National Network for
Higher Education and Research
- innovative ICT platforms & services

• By and for the Dutch Higher
Education and Research
community
- 160+ connected organizations,
serving 1 million users
- Not for proﬁt, 85 employees

A new approach for adding metadata to online Media - TNC2012 - Reykjavik - Frans.Ward@surfnet.nl 2

Tuesday, May 22, 12

About MediaMosa....

• Open Source (GPLv2)
software to build a
Digital Asset Management
platform (DAM)

• Since 2008, MediaMosa is the
basis for various successful
video services that SURFnet
provide to the Dutch
educational sector

• Many adopters and large
community

• Proven to be an excellent
‘vehicle for innovation’


Tuesday, May 22, 12

MediaMosa is based on a Service Oriented Architecture

Video Frontend Video Backend

REST calls
in XML

i.e. SURFmedia MediaMosa

• Flexibility: combining components
• Scalable for future expansion
• Open source and using open
source components ➡ Common base for
MediaMosa is Drupal 7


Tuesday, May 22, 12

MediaMosa API - http://mediamosa.org/api

REST based webservices
over 185, for services like... • Play video
• Authentication
• Authorization (Play restriction)
• Upload (PUT, POST, FTP)
• Transcoding
• Media Management: media
ﬁles, assets and collections
• Search
• Jobs
• Metadata and OAI/PMH
• Notiﬁcation
• Logging and Statistics
• Stills
• ....


Tuesday, May 22, 12

Introducing MediaMosa SiteBuilder


Tuesday, May 22, 12

MediaMosa SiteBuilder...

...a Drupal distribution for a media site connected to MediaMosa

Key features:
• federative login (SAML2)
• upload and play media
• add play restrictions to media
• transcode media
• metadata
• skinnable
• easy installable


Tuesday, May 22, 12

MediaMosa SiteBuilder...

...a Drupal distribution for a media site connected to MediaMosa

Key features:
• federative login (SAML2)
• upload and play media
• add play restrictions to media
• transcode media
• metadata
• skinnable
• easy installable

An example site can be viewed at: https://sitebuilder.mediamosa.surfnet.nl/


Tuesday, May 22, 12

Community Website & Online Demo

http://mediamosa.org


Tuesday, May 22, 12

Community Website & Online Demo


http://demo.mediamosa.org


Tuesday, May 22, 12

A growing MediaMosa Community


Tuesday, May 22, 12

MediaMosa Status & Roadmap

Latest stable release on Github: Release 3.2.1
https://github.com/mediamosa/

Continued Development in 2012:

• July 2012
MediaMosa 3.5 ‘Flexible workflows, Speech-to-Text
functionality, support for: Postgres, BagIt, JPEG2000, PDF
generation, Command-line using Drush (Drupal Shell)'

• December 2012
MediaMosa 4.0 'Better Drupal integration, Drupal Rules,
REST call caching, Document oriented database support
(ie. Mongo DB with Grid FS) offering even more scalability
and flexibility’


Tuesday, May 22, 12

MediaMosa Status & Roadmap

Latest stable release on Github: Release 3.2.1
https://github.com/mediamosa/

Continued Development in 2012:

• July 2012
MediaMosa 3.5 ‘Flexible workflows, Speech-to-Text
functionality, support for: Postgres, BagIt, JPEG2000, PDF
generation, Command-line using Drush (Drupal Shell)'

• December 2012
MediaMosa 4.0 'Better Drupal integration, Drupal Rules,
REST call caching, Document oriented database support
(ie. Mongo DB with Grid FS) offering even more scalability
and flexibility’

Concrete steps are taken to make MediaMosa sustainable by
forming an independent MediaMosa Foundation in June 2012


Tuesday, May 22, 12

MediaMosa is ‘Networking to Services’

Services for Education File Storage
Video Services & Online Content Services

Two service providers will oﬀer MediaMosa based video
services to the R&E market using SURFconext
Realtime
Collaboration Generic Online
Services Services

Infrastructure
Services Services for Research


Tuesday, May 22, 12

It’s all about metadata...

Categories of Metadata:

• User Generated Metadata
(title, description, ..)

• Technical Metadata
(ﬁle analysis by FFMPEG)

• Social Metadata
(comments, tags..)

• Automatic Generated
Metadata
(product of ASR and OCR)

• ...and more


Tuesday, May 22, 12





• Social Metadata
➡ Video Annotation Project
(comments, tags..)

Metadata

• ...and more


Tuesday, May 22, 12





• Social Metadata
➡ Video Annotation Project
(comments, tags..)

Metadata
➡ Transcripting Project
(Speech-to-Text)
• ...and more


Tuesday, May 22, 12

Video Annotation Project

Can you combine diﬀerent video annotation
techniques to enrich video?
Why?
• adding metadata makes the video better searchable and usable
• adding user interaction will make the video more enjoyable to
watch

Ergo: to enhance usability of video in education


Tuesday, May 22, 12

Video Annotation 1: store user comments

• Proof of
Concept using
HTML5 player

• User can
comment on
video content

• Comments
made visible
using SubRip
subtitle format

• and stored in


Tuesday, May 22, 12

Video Annotation 2: social interaction

• Using Twitter to
enrich video
with comments

• Comments are
stored as tweets


Tuesday, May 22, 12

Video Annotation 3: make video interactive

• ‘Hotspots’ in
video can be
added

• Activating this
hotspot can
start a new
event

• Events can be:
start a new
video, show
additional
information etc.

• Using WebVTT
format


Tuesday, May 22, 12

Transcripting Technology Project

UK National Film and Television Archive, Berkhamsted
http://www.ﬂickr.com/people/footage/


Tuesday, May 22, 12

Transcripting Technology Project

...is all about disclose of audiovisual archives

UK National Film and Television Archive, Berkhamsted


Tuesday, May 22, 12

Disclosure of audiovisual archives

• The number of AV-archives on the
Internet increases rapidly

• Archiving is not enough: disclosure
and reusing is required!

• Adding Metadata is the key
component here

• The use of speech technology is
needed (to reduceUKhumanTelevision Archive, Berkhamsted
National Film and
eﬀort)


Tuesday, May 22, 12

Metadata, the traditional approach

Adding Manually

Huge amount of work
and no time-coded relations with video


Tuesday, May 22, 12

Metadata, the new approach

Audio Extraction

Speech Recognition
(Speech-to-Text)

Indexing and Search:
Search on fragment level

using Speech-2-Text technology is key here


Tuesday, May 22, 12

Transcripting Technology Use Case

Web lecture • Recording of Teacher
recording App • Recording of Slides
• Reference material

Transcription Multi-Source
MediaMosa
by Spraak Player
• Recognize the Speech • Transcode into audio • Enhanced Search
• Produce time-coded • Store all into an asset • Optional Subtitles
Transcript • Mashup info
• CMU Sphinx for English
Partners:


Tuesday, May 22, 12

In fact, this is all we need...


Tuesday, May 22, 12

In fact, this is all we need...

a timecoded transcript


Tuesday, May 22, 12

There is a bonus: subtitles in SubRip format


Tuesday, May 22, 12

Demo Transcripting Technology

Online demo: http://spraak.mediamosa.surfnet.nl/


Tuesday, May 22, 12

MediaMosa Directions and Q&A

WWW
sa
MediaMo
Online Demo MediaMosa
http://demo.mediamosa.org
MediaMosa

Forum
http://mediamosa.org/forum

Issue Tracker
http://mediamosa.org/trac

Source Code
https://github.com/mediamosa

Slideshare
http://www.slideshare.net/MediaMosa

Twitter
http://twitter.com/mediamosa


Tuesday, May 22, 12

Frans.Ward@surfnet.nl

Tuesday, May 22, 12

A new approach for adding metadata to online Media - TNC2012

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (7)

Ähnlich wie A new approach for adding metadata to online Media - TNC2012

Ähnlich wie A new approach for adding metadata to online Media - TNC2012 (20)

Mehr von SURFconext

Mehr von SURFconext (13)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

A new approach for adding metadata to online Media - TNC2012