Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Linking media, data,
and services
Ruben Verborgh
Ghent University – iMinds
The problem
The solution
The problems
The problem
The solution
The problems
How can semantic technologies
assist with multimedia analysis?
analysis

algorithms
Linked

Data
planning
knowledge

bases
reasoning
goal-driven
data-driven
• subrequest 1
…
• subrequest n Engine
EXIFJPEG
metadata
JPEG
inputrequest
The problem
The solution
The problems
multimedia
data
+
metadata
Supervisor
requested
metadata
description
Blackboard
requested
metadata
description
gathered
me...
multimedia
data
+
metadata
Supervisor
requested
metadata
description
Blackboard
requested
metadata
description
gathered
me...
query request
input
Collaborator
output
result
2.
3.
5.
1.
4.
Collaboratorinput output
Toolkit
Collaborator
Wrapper
input output
start
<Loft.jpg> a foaf:Image.
<Loft.jpg> foaf:depicts ?person.
input
goal
composition
{
?image a foaf:Image.
}
=>
{
?regionList a rdf:List.
_:region owl:oneOf ?regionList;
rdfs:subClassOf face:FaceRegion,
[ o...
{
?region a face:FaceRegion.
}
=>
{
?region imreg:regionDepicts ?face.
?face a face:Face;
face:isFaceOf ?person.
?person a...
{
?image imreg:region ?region.
?region imreg:regionDepicts ?face.
?face face:isFaceOf ?person.
}
=>
{
?image foaf:depicts ...
Example 7.8 Composition from image input to person request
8
>>>>>>><
>>>>>>>:
Iæ := (param1 = ) √ ( )
IFD := (regionList ...
execution
<Loft.jpg#xywh=45,121,51,51> a face:FaceRegion;
imreg:regionOf <Loft.jpg>.
<Loft.jpg#xywh=221,91,56,56> a face:FaceRegion;...
Example 7.8 Composition from image input to person request
8
>>>>>>><
>>>>>>>:
Iæ := (param1 = ) √ ( )
IFD := (regionList ...
_:face owl-s-sparql:bindsVariable "face";
owl-s-sparql:boundTo
[face:isFaceOf dbpedia:Koen_De_Bouw].
_:person owl-s-sparql...
dbpedia:Koen_De_Bouw ???
??? dbpedia:Bruno_Vanden_Broucke
failure
alternative
plan
CONSTRUCT WHERE {
dbpedia:Koen_De_Bouw ?p ?o.
}
CONSTRUCT WHERE {
?s ?p dbpedia:Koen_De_Bouw.
}
dbpedia:Koen_De_Bouw a dbpedia-owl:Actor;
rdfs:label "Koen De Bouw"@nl.
dbpedia:Bruno_Vanden_Broucke a dbpedia-owl:Actor;
...
{
?image foaf:depicts ?person
?person foaf:knows ?acquaintance.
}
=>
{
?image face:maybeDepicts ?acquaintance.
}.
knowledg...
recomposition
acquaintances to the parameter. Note that the actual process is slightly more
complex, but some details were omitted for b...
{
?image imreg:region ?regionA, ?regionB.
?regionA owl:differentFrom ?regionB.
?regionA imreg:regionDepicts [face:isFaceOf...
execution
dbpedia:Koen_De_Bouw
dbpedia:Bruno_Vanden_Broucke
dbpedia:Matthias_Schoenaerts
dbpedia:Bruno_Vanden_Broucke
dbpedia:Koen_D...
_:face owl-s-sparql:bindsVariable "face";
owl-s-sparql:boundTo [face:isFaceOf ?person].
_:person owl-s-sparql:bindsVariabl...
solution
dbpedia:Koen_De_Bouw dbpedia:Matthias_Schoenaerts
dbpedia:Koen_De_Graeve dbpedia:Bruno_Vanden_Broucke
The problem
The solution
The problems
ἀπὸ μηχανῆς θεός
deus ex machina
machina ex deo
reasoner
What are the research questions?
!
How can semantic technologies
assist with multimedia analysis?
What are the hypotheses?
!
This solution does analysis

better than… what?
How to evaluate this solution?
!
Is our solution

a good one or a bad one?
How can this solution fail?
!
Can (existing) knowledge

always fill the gap?
The problem
The solution
The problems
Can we link images like we link data?
Can we have a Linked Media Cloud?
Linking media, data,
and services
@RubenVerborgh

ruben.verborgh.org
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Linking media, data, and services
Nächste SlideShare
Wird geladen in …5
×

von

Linking media, data, and services Slide 1 Linking media, data, and services Slide 2 Linking media, data, and services Slide 3 Linking media, data, and services Slide 4 Linking media, data, and services Slide 5 Linking media, data, and services Slide 6 Linking media, data, and services Slide 7 Linking media, data, and services Slide 8 Linking media, data, and services Slide 9 Linking media, data, and services Slide 10 Linking media, data, and services Slide 11 Linking media, data, and services Slide 12 Linking media, data, and services Slide 13 Linking media, data, and services Slide 14 Linking media, data, and services Slide 15 Linking media, data, and services Slide 16 Linking media, data, and services Slide 17 Linking media, data, and services Slide 18 Linking media, data, and services Slide 19 Linking media, data, and services Slide 20 Linking media, data, and services Slide 21 Linking media, data, and services Slide 22 Linking media, data, and services Slide 23 Linking media, data, and services Slide 24 Linking media, data, and services Slide 25 Linking media, data, and services Slide 26 Linking media, data, and services Slide 27 Linking media, data, and services Slide 28 Linking media, data, and services Slide 29 Linking media, data, and services Slide 30 Linking media, data, and services Slide 31 Linking media, data, and services Slide 32 Linking media, data, and services Slide 33 Linking media, data, and services Slide 34 Linking media, data, and services Slide 35 Linking media, data, and services Slide 36 Linking media, data, and services Slide 37 Linking media, data, and services Slide 38 Linking media, data, and services Slide 39 Linking media, data, and services Slide 40 Linking media, data, and services Slide 41 Linking media, data, and services Slide 42 Linking media, data, and services Slide 43 Linking media, data, and services Slide 44 Linking media, data, and services Slide 45 Linking media, data, and services Slide 46 Linking media, data, and services Slide 47 Linking media, data, and services Slide 48 Linking media, data, and services Slide 49 Linking media, data, and services Slide 50 Linking media, data, and services Slide 51 Linking media, data, and services Slide 52 Linking media, data, and services Slide 53 Linking media, data, and services Slide 54 Linking media, data, and services Slide 55 Linking media, data, and services Slide 56 Linking media, data, and services Slide 57 Linking media, data, and services Slide 58 Linking media, data, and services Slide 59 Linking media, data, and services Slide 60 Linking media, data, and services Slide 61 Linking media, data, and services Slide 62
Nächste SlideShare
Lime recommendation
Weiter
Herunterladen, um offline zu lesen und im Vollbildmodus anzuzeigen.

1 Gefällt mir

Teilen

Herunterladen, um offline zu lesen

Linking media, data, and services

Herunterladen, um offline zu lesen

Keynote at the 4th International Workshop on Linked Media (LIME) http://www.mico-project.eu/pages/lime-workshop-2016/

Ähnliche Bücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Linking media, data, and services

  1. 1. Linking media, data, and services Ruben Verborgh Ghent University – iMinds
  2. 2. The problem The solution The problems
  3. 3. The problem The solution The problems
  4. 4. How can semantic technologies assist with multimedia analysis?
  5. 5. analysis
 algorithms Linked
 Data planning knowledge
 bases reasoning
  6. 6. goal-driven data-driven
  7. 7. • subrequest 1 … • subrequest n Engine EXIFJPEG metadata JPEG inputrequest
  8. 8. The problem The solution The problems
  9. 9. multimedia data + metadata Supervisor requested metadata description Blackboard requested metadata description gathered metadata Collaborator 3 semantic capability description semantic requirements description Collaborator 2 semantic capability description semantic requirements description Collaborator 1 semantic capability description semantic requirements description multimedia data + metadata r requested metadata description metadata Blackboard requested metadata description gathered metadata n 1 tic ity ion multimedia data + metadata requested metadata description metadata Blackboard requested metadata description gathered metadata
  10. 10. multimedia data + metadata Supervisor requested metadata description Blackboard requested metadata description gathered metadata Collaborator 3 semantic capability description semantic requirements description Collaborator 2 semantic capability description semantic requirements description Collaborator 1 semantic capability description semantic requirements description multimedia data + metadata r requested metadata description metadata Blackboard requested metadata description gathered metadata c y on r 1 ntic bility ption multimedia data + metadata visor requested metadata description metadata Blackboard requested metadata description gathered metadata or3 mantic ability cription tor2 semantic capability description rator1 semantic capability description
  11. 11. query request input Collaborator output result 2. 3. 5. 1. 4.
  12. 12. Collaboratorinput output Toolkit
  13. 13. Collaborator Wrapper input output
  14. 14. start
  15. 15. <Loft.jpg> a foaf:Image. <Loft.jpg> foaf:depicts ?person. input goal
  16. 16. composition
  17. 17. { ?image a foaf:Image. } => { ?regionList a rdf:List. _:region owl:oneOf ?regionList; rdfs:subClassOf face:FaceRegion, [ owl:hasValue ?image; owl:onProperty imreg:regionOf ]. } service rule
  18. 18. { ?region a face:FaceRegion. } => { ?region imreg:regionDepicts ?face. ?face a face:Face; face:isFaceOf ?person. ?person a foaf:Person. }. service rule
  19. 19. { ?image imreg:region ?region. ?region imreg:regionDepicts ?face. ?face face:isFaceOf ?person. } => { ?image foaf:depicts ?person. }. knowledge rule
  20. 20. Example 7.8 Composition from image input to person request 8 >>>>>>>< >>>>>>>: Iæ := (param1 = ) √ ( ) IFD := (regionList = ) √ (image = ) IFR := (f ace = , per son = ) √ (region = ) I≤ := ( ) √ (per son = ) Iæ and I≤ are the virtual start and end services, respectively. Iæ IFD IFR I≤ 62
  21. 21. execution
  22. 22. <Loft.jpg#xywh=45,121,51,51> a face:FaceRegion; imreg:regionOf <Loft.jpg>. <Loft.jpg#xywh=221,91,56,56> a face:FaceRegion; imreg:regionOf <Loft.jpg>. <Loft.jpg#xywh=535,118,43,43> a face:FaceRegion; imreg:regionOf <Loft.jpg>. <Loft.jpg#xywh=734,83,69,69> a face:FaceRegion; imreg:regionOf <Loft.jpg>.
  23. 23. Example 7.8 Composition from image input to person request 8 >>>>>>>< >>>>>>>: Iæ := (param1 = ) √ ( ) IFD := (regionList = ) √ (image = ) IFR := (f ace = , per son = ) √ (region = ) I≤ := ( ) √ (per son = ) Iæ and I≤ are the virtual start and end services, respectively. Iæ IFD IFR I≤ 62
  24. 24. _:face owl-s-sparql:bindsVariable "face"; owl-s-sparql:boundTo [face:isFaceOf dbpedia:Koen_De_Bouw]. _:person owl-s-sparql:bindsVariable "person"; owl-s-sparql:boundTo dbpedia:Koen_De_Bouw. <Loft.jpg#xywh=221,91,56,56>
  25. 25. dbpedia:Koen_De_Bouw ??? ??? dbpedia:Bruno_Vanden_Broucke
  26. 26. failure
  27. 27. alternative plan
  28. 28. CONSTRUCT WHERE { dbpedia:Koen_De_Bouw ?p ?o. } CONSTRUCT WHERE { ?s ?p dbpedia:Koen_De_Bouw. }
  29. 29. dbpedia:Koen_De_Bouw a dbpedia-owl:Actor; rdfs:label "Koen De Bouw"@nl. dbpedia:Bruno_Vanden_Broucke a dbpedia-owl:Actor; rdfs:label "Bruno Vanden Broucke"@nl. ! dbpedia:Loft_%282008_film%29 dbpprop:director dbpedia:Erik_Van_Looy; dbpprop:name "Loft"@nl; dbpprop:producer dbpedia:Woestijnvis; dbpprop:starring dbpedia:Bruno_Vanden_Broucke, dbpedia:Filip_Peeters, dbpedia:Jan_Decleir, dbpedia:Koen_De_Bouw, dbpedia:Koen_De_Graeve, dbpedia:Matthias_Schoenaerts, dbpedia:Veerle_Baetens.
  30. 30. { ?image foaf:depicts ?person ?person foaf:knows ?acquaintance. } => { ?image face:maybeDepicts ?acquaintance. }. knowledge rule
  31. 31. recomposition
  32. 32. acquaintances to the parameter. Note that the actual process is slightly more complex, but some details were omitted for brevity. Example 7.14 Adapted composition, recovering from the face recognition failure 8 >>>>>>>< >>>>>>>: ... I0 æ := (param1 = ) √ ( ) I0 FR := (f ace = , per son = ) √ (region = , candidates = ) Iæ IFD IFR I≤ I0 æ I0 FR 67
  33. 33. { ?image imreg:region ?regionA, ?regionB. ?regionA owl:differentFrom ?regionB. ?regionA imreg:regionDepicts [face:isFaceOf ?personA]. ?regionB imreg:regionDepicts [face:isFaceOf ?personB]. } => { ?personA owl:differentFrom ?personB. }. knowledge rule
  34. 34. execution
  35. 35. dbpedia:Koen_De_Bouw dbpedia:Bruno_Vanden_Broucke dbpedia:Matthias_Schoenaerts dbpedia:Bruno_Vanden_Broucke dbpedia:Koen_De_Graeve
  36. 36. _:face owl-s-sparql:bindsVariable "face"; owl-s-sparql:boundTo [face:isFaceOf ?person]. _:person owl-s-sparql:bindsVariable "person"; owl-s-sparql:boundTo ?person. ({?person = dbpedia:Koen_De_Graeve.} {?person = dbpedia:Bruno_Vanden_Broucke.}) e:disjunction [a e:T]. <Loft.jpg#xywh=45,121,51,51>
  37. 37. solution
  38. 38. dbpedia:Koen_De_Bouw dbpedia:Matthias_Schoenaerts dbpedia:Koen_De_Graeve dbpedia:Bruno_Vanden_Broucke
  39. 39. The problem The solution The problems
  40. 40. ἀπὸ μηχανῆς θεός
  41. 41. deus ex machina
  42. 42. machina ex deo reasoner
  43. 43. What are the research questions? ! How can semantic technologies assist with multimedia analysis?
  44. 44. What are the hypotheses? ! This solution does analysis
 better than… what?
  45. 45. How to evaluate this solution? ! Is our solution
 a good one or a bad one?
  46. 46. How can this solution fail? ! Can (existing) knowledge
 always fill the gap?
  47. 47. The problem The solution The problems
  48. 48. Can we link images like we link data? Can we have a Linked Media Cloud?
  49. 49. Linking media, data, and services @RubenVerborgh
 ruben.verborgh.org
  • Wikier

    Jun. 3, 2016

Keynote at the 4th International Workshop on Linked Media (LIME) http://www.mico-project.eu/pages/lime-workshop-2016/

Aufrufe

Aufrufe insgesamt

1.910

Auf Slideshare

0

Aus Einbettungen

0

Anzahl der Einbettungen

664

Befehle

Downloads

4

Geteilt

0

Kommentare

0

Likes

1

×