The MediaMixer project and community promote the use of semantic technologies for media mixing through real use cases and demos that showcase them. A typical MediaMixer demo will involve fragmenting media assets, annotating them using semantic descriptions and exposing these descriptions to customers, for fragment level search and selection. Fragments will be also linked to rights information based on a copyright ontology, which integrates licenses, policies and rights expressions based on existing standards like DDEX, ODRL or MPEG-21.

1. MediaMixer !
facilitating media fragments mixing and its rights
management using semantic technologies!
!"#$%&"'()%*+),'-./0$%1/&)&'2$'34$/2),'56)/.'
!"#$%!$&"'!"#$%&''()**+,-./0*12-3*-41501+6!
789!:+,1!;<=>!
?@AB!C1,1D3!

2. 06/06/13 Slide 2 of 28
MediaMixer"
Community set-up and networking for the !
reMIXing of online MEDIA fragments !
•! FP7 Coordination Action!
–!Started November 2012 !
–!Create an European market for media
fragments re-purposing and re-selling!
E!#-F.)5/!)G!CH!D-)I1,(1!"J!K1+.15F6!!#$%&''FI-21F#3510,1.'.5),(/'38F1*3,L(8*+IL*12-38M1N8(513.183,,).3.18%51F1,.83,28F#3518/)+58*12-3!!

3. 06/06/13 Slide 3 of 28
Introduction!
•! The MediaMixer project and community promote
the use of semantic technologies for media mixing!
•! Real use cases and demos that showcase these
technologies!
–! Fragmenting media assets!
–! Annotating them using semantic descriptions!
–! Exposing these descriptions for fragment level search
and selection!
–! Representing rights information using a copyright
ontology that integrates licenses, policies and rights
expressions languages!

4. 06/06/13 Slide 4 of 28
Coordination. Semantic Web technologies R&D
IT solutions for broadcasters (WebTV, EPG,…)
Asset Management solutions
videolectures.net academic content platform
Semantic multimedia research
Semantic copyright management research
Partners!

5. 06/06/13 Slide 5 of 28
!"#$%$&'
!"#$%$&'
!"#$%$&'
!"#$%&'" (%)*+,
-#.&'"
(&)$*+
!,")-%.$"/
!"#&/0'"
!"#$%$&'
!"#$%$&'
01,&&2&/.'
!,")-%&)+
(&)$*
(&)$*+
3,*12&/.'
4"/!$%.'
Producer checks reused media
licenses vs. internal policies and
agreements (with providers and
rights holders)!
Policy example: “…avoid media
fragments showing violence when
producing content for children”. !
Problems:!
1.) Semantic annotation!
a) For instance, automatically tag"
violent content!
2.) Rights Integration!
a) Policies and agreements
(unstructured documents) !
b) Media fragments licenses "
(rights expression languages)!
3.) Intelligently detect conflicts!
Example Use Case!

6. 06/06/13 Slide 6 of 28
Current Situation!
!"!#"$
%&'()*
+",$
-./""0"#,
122"/$
3/4.0"#,
-./""0"#,
%&'()*
!"4'
5"6(4$
3/4.0"#,
%/&67)"/
8'".4'9
:(.;,<$=&'6"/
%/&>(6"/
+"4/);$
3/4.0"#,
?;")@
A"/0<
:"<&'>"$
?&#"(),
%/&67)"/
8)&#,"#,$
04#4."/9
Issues:!
•! scalability "
(mostly a manual process)!
•! conflicts detected too late
(added costs)!
•! missed opportunities "
(media already available)!
•! …!
!

7. 06/06/13 Slide 7 of 28
Proposals:"
Assisted Policy and Agreement Modelling!
•! Model policies and
agreements!
•! Formal model that allows
automatic integration and
intelligence support!
•! Semantic model based on
Copyright Ontology!
•! Editor with assistance
guided by the ontology!

9. 06/06/13 Slide 9 of 28
Media Fragments Ingestion!
Provider offers media
fragment:!
!
•! Content: "
automatic annotation!
•! Detect topics: Violence?
Outdoors? Cars?...!
•! Rights:"
automatic translation of
rights language to
Copyright Ontology!
•! DDEX, ODRL, MPEG-21,
Creative Commons…!

10. 06/06/13 Slide 10 of 28
Semantic Annotation!
10!
•! Transform whole videos to sets of meaningful,
indexable and re-usable video fragments!
O15F),B!P,)MB!Q511FB!!
@+-I2-,RB!PS-B!T!
@3F1N3IIB!Q#5)M-,RB!P%)5.FB!!
OI3,.B!K+,,-,RB!T!
U-.(#1,B!V,2))5B!W3S1B!T!
T!

11. 06/06/13 Slide 11 of 28
Semantic Annotation"
Key Technologies!
11!
•! Temporal video fragmentation to!
–! Shots (elementary video segments)!
–! Scenes (groups of consecutive shots)!
•! Video annotation (to make video fragments
indexable / searchable)!
–! Concept detection!
–! Event detection!
–! Event recounting!
•! Video object re-detection (to create spatio-
temporal video fragments)!
V,%+.&!),1!*12-3!
-.1*!
X+.%+.&!F1.!)G!
*12-3!G53R*1,.F!
V,%+.&!),1!*12-3!
-.1*!Y!143*%I1!)G!
F)+R#.!)NZ1(.!
X+.%+.&!F1.!)G!
*12-3!G53R*1,.F!
V,%+.&!),1!.1*%)53I!
*12-3!G53R*1,.!
X+.%+.&!F1.!)G!I3N1IF!
"(),(1%.B!1D1,.6!

12. 06/06/13 Slide 12 of 28
Semantic Annotation"
Metadata!
•! Metadata for semantic annotation:!
–!Descriptive metadata for media characteristics!
–!Provenance metadata to credit source and
specify rights!
–!Conceptual metadata to reflect what media is
perceived to represent!

13. 06/06/13 Slide 13 of 28
Conceptual Metadata!
• Globally unambiguous
identifiers for any
concept needed
–! Ideally more information
about the concept itself is
available via the identifier!
–! Use of Web URIs
Linked Data concept
space!
–! Would allow for inference
of concept type and
relationship to other
concepts!

14. 06/06/13 Slide 14 of 28
Producer Proposed Workflow!
!"#$%&"'(
)*+$,-
."&(
/#'""0"1&
)'*23,"'
4+"#5+6
."5',7(
8'5#0"1&
97",:
;%"
)'*23,"'
4,*1&"1&(
0515#"'6
/#'""0"1&
<=
)*+$,-
<=
9*>-'$#7&
!"5%*1"'
)*+$,-(512(
/#'""0"1&(
?*2"++$1#(
/%%$%&51&
."051&$,(
?"2$5(!">*%$&*'-
/#'""0"1&(<= @"5+(?"2$5(
8'5#0"1&(<=
A1&"12"2(;%"
)*+$,-(<=
?"2$5(
8'5#0"1&
<=
."051&$,(
?"&525&5
/++*B"2(3%"%
@$%++*B"2(3%"%

15. 06/06/13 Slide 15 of 28
Implementation!
•! Rights, License and Policies "
implemented as Classes!
–! Reproduction Right, Copy, "
Copy Pattern for fragment #1,...!
•! Uses implemented as Instances!
–! u: Peter copies media fragment #1 "
in the US on 2013-06-03 using…!
•! if u Copy Pattern then"
pattern authorises u !
Reproduction
Right
Copy
Copyright
Copy
Pattern
u
?

16. 06/06/13 Slide 16 of 28
Pattern Building"
Actions (“verbs”)!
!"#$%"&'
(')%$'*+
,$'"-+)%$%"&'
.&/0
1+/-&/2$'*+
2$'"-+)%
!"#$%&' ()*%$''$'
3&224'"*$%"&'
+"'&),%&-*.'
%/$')-&/2
"25/&6")+
7")%/"84%+
*&59
*&59
-"#
5+/-&/2
*&224'"*$%+
/+%/$')2"%
!"#$%&'()*%+,''
-.,'/",0&123.,'
Copyright
EconomicRights
RelatedRights
MoralRights
DistributionRight
ReproductionRight
PublicPerformanceRight
FixationRight
CommunicationRight
AttributionRight
TransformationRight
IntegrityRight
DisclosureRight
WithdrawalRight
PermorfersRights
ProducersRights
BroadcastersRights
RentalRight
ImportationRight
SoundRecordRight
MotionPictureRight
BroadcastingRight
MakingAvailableRight
AdaptationRight
TranslationRight

17. 06/06/13 Slide 17 of 28
Pattern Building"
Case Roles (from linguistics)!
•! Connect verbs to the action participants!
–!Example: “The dog ate food”!
initiator resource goal essence
Action agent, instrument result, patient,
effector recipient theme
Process agent, matter result, patient,
origin recipient theme
Transfer agent, instrument, experiencer, theme
origin medium recipient
Spatial origin path destination location
Temporal start duration completion pointInTime
Ambient reason manner aim, condition
consequence
Example: “The dog
P+NZ1(.B!5)I1![3R1,.!
food”
XNZ1(.B!5)I1![.#1*1!

18. 06/06/13 Slide 18 of 28
DDEX Sample-08.04.xml!
<Deal>!
!<DealTerms>!
! !<ValidityPeriod>!
! ! !<StartDate>2013-01-01</StartDate>!
! !</ValidityPeriod>!
! !<Usage>!
! ! !<UseType>!
! ! ! !OnDemandStream"
! ! !</UseType>!
! ! !<DistributionChannelType>!
! ! ! !Internet!
! ! !</DistributionChannelType>!
! !</Usage>!
! !<TerritoryCode>ES</TerritoryCode>!
! !<TerritoryCode>US</TerritoryCode>!
!</DealTerms>!
</Deal> !
Copyright Ontology Model!
<http://media.com/agreement#1> owl:Class;!
!co:start "2013-01-01" ;!
!owl:intersectionOf (!
! !ddex:OnDemandStream !
! ![ a owl:Restriction ;!
! ! !owl:onProperty co:medium ;"
! ! !owl:someValuesFrom ddex:Internet ]!
! ![ a owl:Restriction ;!
! ! !owl:onProperty co:location ;!
! ! !owl:someValuesFrom "
! ! ![ a !owl:Class ;!
! ! ! !owl:oneOf (territory:ES territory:US) ]!
! ![ a owl:Restriction ;!
! ! !owl:onProperty co:theme ;!
! ! !owl:hasValue!
! ! ! !<http://my.tv/video.ogv#t=60,100> ] !
!
! !]!
!) . !
DDEX to RDF
<http://
co:start
owl:intersectionOf
1</StartDate> ddex:
[ a[ a
[ a owl:Restriction
[ a
[ a
m
DistributionChannelType>
owl:onProperty
TerritoryCode>
TerritoryCode>
owl:someValuesFrom
[ a
[ a owl:Restriction
owl:onProperty
owl:hasValue
[ a
[ a owl:Restriction
[ a

19. 06/06/13 Slide 19 of 28
])R-(3I!-,.15%51.3L),!)G!3!I-(1,F1!*)21I!
Reasoning with DDEX

20. 06/06/13 Slide 20 of 28
Reasoning with DDEX
Policy #1
Agreement #1

21. 06/06/13 Slide 21 of 28
!"#$%&"!'('&")'!"#$%"*&")'&'()*+!,-./0*++'
!"#$%&"!,-%#./&0%-%12"$*+'
3'4%#.".5'2$%6788#9,%)-'*0"!!20%!2"$+:'..";,%)-:'<%=>!21?0%&0%,%)-'
!"#$%&"!,-%#.@%-2<>*AB%C0!"#D2&1E&FC,%)-A+'
!"#$%&"!,-%#.42<"&$"*AGGHIJC#D=-"KLM,LN,%)-A+'
O$"7'('&")'1#"*O$"7FPE:'AO$"!L7A:'A/&G"D#&.C0!"#DA:'AE&0"!&"0A:''
Ahttp://my.tv/video.ogv#t=60,100A:'AHCA:'A9L76KLNK7QA+'
!"#$%&"!,-%#.*O$"7+'
!"$=%&$"'('!"#$%&"!,2$F$"@"!D200".*O$"7FPE+'
=!2&0'!"$=%&$"'
3'RO0?%!2$".5'?00=5SSD".2#D2;"!,"OS<%=>!21?0S";#D=-"$S#1!""D"&0T7'
O$"9'('&")'1#"*O$"9FPE:'AO$"!L7A:'A/&G"D#&.C0!"#DA:'AE&0"!&"0A:'
Ahttp://my.tv/video.ogv#t=60,100A:'AFCA:'A9L76KLNK7QA+'
!"#$%&"!,-%#.*O$"9+'
!"$=%&$"'('!"#$%&"!,2$F$"@"!D200".*O$"9FPE+'
=!2&0'!"$=%&$"'
3'@!%?2U20".5'?00=5SSD".2#D2;"!,"OS<%=>!21?0S";#D=-"$S=%-2<>T7'
License Checking Implementation

22. 06/06/13 Slide 22 of 28
@prefix co: <http://rhizomik.net/ontologies/2013/05/copyrightonto.owl#> .
@prefix odrlv: <http://w3.org/ns/odrl/vocab#> . !
@prefix owl: <http://www.w3.org/2002/07/owl#> . !
!
<http://example.com/policy> a co:Agree ;!
co:agent <http://example.com/billie>, <http://example.com/sony> ;!
co:theme [ a owl:Class ;!
co:end "2012-12-31" ;!
co:start "0001-01-01" ;!
owl:intersectionOf (!
[ owl:intersectionOf (!
odrlv:play [ owl:complementOf odrlv:mobile_ringtone ] ) ]
[ a owl:Restriction ;!
owl:hasValue <http://example.com/billie> ;
owl:onProperty co:agent ]
[ a owl:Restriction ;
owl:hasValue <http://example.com/music> ;
owl:onProperty co:theme ] ) ] .
!
<http://example.com/sony:10> a co:Person .!
<http://example.com/music> a co:Creation .!
<http://example.com/billie> a co:Person .!
ODRL Example from LCC
Rights Reference Model Spec!
()*+'$,-./01.)2),+/2$3"14"/.)/."&.5/
[F),/!3,2!N-II-1!ER511!.#3.!!
N-II-1!%I3/!3,2!,).!*)N-I1!5-,R.),1!!
*+F-(0!

23. 06/06/13 Slide 23 of 28
Creative Commons "
BY-SA Model!
!"#$% %
&'()*'+,)-% %
!"..,/'01)-% %
213-451'61+6-
2$21/'7-()1)'"/
!"#$#
.$8"*3
4))*'+,)-
%&'()!)&'
.-
!"#$#
*#%)+)#'!
49*--
!"#$#
&-*'5-
!"#$#
!"#$#
:-;21/'7-()1)'"/
/-;8"*3
*#,-.!
),/0')1#,!0!)&'21
),3#*)40!)&'21
!"#$% %
&'()*'+,)-% %
!"..,/'01)-% %
213-451'61+6-
!"#$#
!"#$#%&'()!)&'
),/0')1#,!0!)&'21
! "
#

24. 06/06/13 Slide 24 of 28
•! Following Linked Data principles, "
starting from just content ID (URI)…!
Long term vision…!
24!
W),.1,.!
!
AKV!
AKV!3!1N+()51&V*3R1!
!!!2(.&L.I1![?@A!^_!
!!!2(.&()%/5-R#.!678/
!!!T!
!!!!
HTTP GET
image/jpeg

25. 06/06/13 Slide 25 of 2806/06/13 Slide 25 of 28
Linked Data for Worldwide "
Copyright Management?!
25!
W),.1,.!
!
AKV!
W),.1,.
AKV!3!1N+()51&V*3R1!
!!!2(.&L.I1![?@A!^_!
!!!2(.&()%/5-R#.!AKV!
!!!T!
!!!!
HTTPGET
ER511*1,.!
!
AKV!
ER511*1,.
AKV!3!()&ER511*1,.!
!!!()&3R1,.!678!
!!!()&.#1*1!T/
!!!T!
!!!!
HTTPGET
O15F),!
!
AKV!
URI a foaf:Person
foaf:name “…”
vcard:address …
vcard:country URI
HTTPGET
ER511*1,.ER511*1,.
AKV
ER511*1,.
URI a foaf:Person
foaf:name “…”
vcard:address …
vcard:country
HTTPGET

26. 06/06/13 Slide 26 of 28
Get involved!
•! Join MediaMixer community to know more "
(http://community.mediamixer.eu):!
–! discussions, use cases, demonstrators, tutorials,
presentations, software,…!
06/06/13 Slide 26 of 28

27. 06/06/13 Slide 27 of 28
Get more involved!
•! Want to experience semantic technologies?
Apply to become core partner"
(http://mediamixer.eu/contact) :!
–!Contribute use case, sample data and just
some time…!
–!…selected partners get a demo showcasing
semantic technologies for their problem!

28. More details:
http://community.mediamixer.eu
http://rhizomik.net/copyrightonto/
Roberto García
Universitat de Lleida, Spain
http://rhizomik.net/~roberto
Thank you for your attention