Multimedia content is extracted automatically using MPEG-7 visual descriptors. The MPEG-7 uses an extended XML standard for defining structural relation between descriptors allowing creation and modification of description schemes. MPEG-7 visual descriptors are numerical representations of features - such as: texture, shape and color - extracted from an image. In this paper, the MPEG-7 is conceived as a set of services for extracting and storing visual descriptors. The MPEg-7 text-annotation tool is used for semantic descriptions. Semantic descriptions are linked to images content and conceived as a service for annotating and storing. A framework using service oriented architecture for mapping semantic descriptions and MPEG-7 visual descriptors into a pure-relational model is proposed.

1. MPEG-7 SERVICE
ORIENTED SYSTEM
MPEG-7 SOS
Margaret Florian
Maria Trujillo

2. Outline
MPEG-7
Database Models
 XML Native Database
 Relational Database
Service Oriented System
 MPEG-7 SOS Architecture
 MPEG-7 SOS Client
 MPEG-7 SOS Server
Slide 2 / 23

3. Introduction
Metadata – data that describe media – are the
key to effective media management
 Rich Metadata address different aspects of media
• description of media content – semantic descriptions –
• administrative aspects – such as the location of media
and copyright information –
• preservation aspects – the technology with which media
where produced and the physical condition of storage
media –
• technical aspects – encoding formats and encoding
parameters – and
• usage aspects – users’ behavioural information.
Slide 3 / 23

4. MPEG-7
 The Multimedia Content Description Interface (MPEG-
7) is an ISO metadata standard
 The MPEG-7 standardizes the Description Definition
Language (MPEG-7 DDL) and a comprehensive set
of media description schemes
 The MPEG-7 DDL is a superset of XML Schema
 XML Schema was added to support array, matrix and
temporal data types
Slide 4 / 23

5. MPEG-7 Documents
 The MPEG-7 DDL can be considered as a definition
language for XML documents
 The MPEG-7 does not provide a model for mapping
MPEG-7 descriptors into a database system
 Multimedia content is difficult to manage due to
MPEG-7 descriptors are often XML documents
 MPEG-7 descriptors require user technical knowledge
Slide 5 / 23

6. MPEG-7 Descriptors
 MPEG-7 storage specification
XML
document
Images Binary
document
Sound Text
Movie
Multiple Media Extracting
Binary
descriptors Format for
process MPEG-7
Slide 6 / 23

7. MPEG-7 Descriptors
 MPEG-7 storage specification
XML document per image XML document per descriptor
<?xml version='1.0' encoding='ISO-8859-1' ?> <?xml version='1.0' encoding='ISO-8859-1' ?>
<Mpeg7 xmlns = "urn:mpeg:mpeg7:schema:2001" <Mpeg7 xmlns = "urn:mpeg:mpeg7:schema:2001"
xmlns:namespace = "http://www.w3.org/XML/1998/namespace" xmlns:namespace = "http://www.w3.org/XML/1998/namespace"
xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance"
xmlns:mpeg7 = "urn:mpeg:mpeg7:schema:2001" xmlns:mpeg7 = "urn:mpeg:mpeg7:schema:2001"
xsi:schemaLocation = "urn:mpeg:mpeg7:schema: 2001 Mpeg7-2001.xsd"> xsi:schemaLocation = "urn:mpeg:mpeg7:schema:
<DescriptionUnit xsi:type = "DescriptorCollectionType"> 2001 Mpeg7-2001.xsd">
<Descriptor xsi:type = "DominantColorType"> <DescriptionUnit xsi:type = "DescriptorCollectionType">
Dominant-color
<SpatialCoherency>0</SpatialCoherency> <Descriptor xsi:type = "EdgeHistogramType">
Image 1
<Value><Percentage>8</Percentage> <BinCounts>3 6 3 4 3 0 5 4 3 4 1 5 1 0 2 4 3
<Index>4 4 5 </Index> 5 6 2 4 5 6 5 2 4 5 5 7 3 4 5 6 5 4 2 6 6 4
</Value> 4 5 5 5 3 5 3 5 5 4 2 5 6 5 3 5 2 3 4 2 4 2
<Value><Percentage>17</Percentage> <Index>12 12 12 </Index> </BinCounts>
</Value> </Descriptor>
<Value><Percentage>4</Percentage> <Index>27 26 27 </Index> </DescriptionUnit>
</Value> <DescriptionUnit xsi:type = "DescriptorCollectionType">
</Descriptor> <Descriptor xsi:type = "EdgeHistogramType">
Image 2
</DescriptionUnit> <BinCounts>4 3 4 1 5 1 0 2 4 3 5 5 4 2 5 6 5 3 5
2 3 5 6 2 4 5 6 3 6 3 4 3 0 55 2 4 5 5 76 6 4
Edge-histogram
<DescriptionUnit xsi:type = "DescriptorCollectionType">
<Descriptor xsi:type = "EdgeHistogramType"> 4 5 5 5 3 5 3 4 2 4 23 4 5 6 5 4 2
<BinCounts>3 6 3 4 3 0 5 4 3 4 1 5 1 0 2 4 3 </BinCounts>
5 6 2 4 5 6 5 2 4 5 5 7 3 4 5 6 5 4 2 6 6 4 </Descriptor>
4 5 5 5 3 5 3 5 5 4 2 5 6 5 3 5 2 3 4 2 4 2 </DescriptionUnit>
</BinCounts> </Mpeg7>
</Descriptor>
</DescriptionUnit>
</Mpeg7>
Slide 7 / 23

8. MPEG-7 SOS
Efficient database schema for
MPEG-7 visual descriptors
Web services for
providing functionality to
non-technical users
Slide 8 / 23

9. Databases for Managing XML documents
Physical
XML Native storing
Database unit
XML Documents
Logical
Extended storing
Relational unit
Database
Slide 9 / 23

10. XML Native Databases
Hard Disk
i_1
i_2 Retrieval speed performance
Database
If all documents are stored
i_4 physically together and physical
pointers are used between parts
i_3 of the document.
 Physical pointers to each document
 Physical pointers to inside sections
Relational model XML native model
A collection could store
Relation  collection of documents documents without a
schema that relates them.
Row  document
Slide 10 / 23

11. Relational Databases
Extended for managing XML documents
XML
document
mapping
structured-mapping through XML model-mapping
meta-model
document
Extended Relational Extended Relational
database database
table-based object-based
No limitations (generic)
same schema for all XML documents
Slide 11 / 23

12. Relational vs. Native XML
Relational Native XML
databases databases
XML structure is  Manage XML Schemas yes yes
static and well
defined  Structured data yes yes
Data-centric  Querying data efficiently yes Low performance
documents  Aggregated functions yes no
 Manage large data yes Low performance
Number of images
grows exponentially  Distribution capability yes Low performance
Multi-user access
 User concurrency yes Low performance
support
Slide 12 / 23

13. Model Vs. Structured mapping
 Actual Database Mapping Features
Model-Mapping Structured-Mapping
All XML documents are stored in five Database schema is descriptor dependent
tables
XML scheme centered Stores XML document structure
All data are stored as string data-type A lot of tables for representing descriptors
 A lot of joins
Allows to reconstructs the XML document Allows to reconstruct the XML document
 Some DBMS
Grows fast
Slide 13 / 23

14. Proposed Database Schema
Static-mapping sos_DedgeHistogram
Imagen_id
DedgeHistogram_group1
DedgeHistogram_group2
1 1 DedgeHistogram_group3
sos_DcolorStructure 1 DedgeHistogram_group4
Imagen_id sos_image DedgeHistogram_group5
DcolorStructure_values Imagen_id DedgeHistogram_group6
1 Imagen_nombre 1
Imagen_fecha
Imagen_url/file
Imagen_tamano
….
1
sos_DcolorLayout
Imagen_id
1 DcolorLayout_YDCCoeff
1
DcolorLayout_CbDCCoeff
sos_DdominantColor DcolorLayout_CrDCCoeff
DcolorLayout_YACCoeff5
Imagen_id DcolorLayout_CbACCoeff2
DdominantColor_ColoursNumber DcolorLayout_CrACCoeff2
DdominantColor_descriptor
Slide 14 / 23

15. Proposed Database Schema
Theoretical Validation
Structured-mapping Static-mapping
π Mpeg7PK
π image_url/file
(dist(|Iinput - π ColorValueIndex, Percentage, ColorVariance |)<umbral) image_id=image_id
DescriptorFK=DescriptorPK
(dist(|Iinput- dc.Idb|)<umbral) sos_image
π DescriptionUnitFK,xsi:type π DescriptorFK,xsi:type sos_DdominantColor
Xsi:type=‘DominantColorType’
Values
π DescriptionUnitPK Descriptor
DescriptionUnit Π : projection (select) : join (from) : selection (where)
Slide 15 / 23

16. MPEG-7 SOS
 The MPEG-7 User options
Output: search result
SOS, as a
provider, offers Images storage Retrieval information
Multimedia Query by Example View-Based
Web Services for resource User input an image search
storing and User interface (WEB Portal)
retrieving images HTTP
through automatic Annotation Web Services
Feature Information
and manual extraction retrieval
Automatic – Textual Web Services
annotation MPEG-7 descriptions
Storage Web Services
Mapping to Database
database querying
MPEG-7 Manual
descriptors Annotation
Web Services Infrastructure
Slide 16 / 23

17. MPEG-7 SOS
For using the MPEG-7 SOS, it is necessary to:
 Create a link between resources – web
services – and a client web application
 Create a client account
 Login in MPEG-7 SOS for storing images
 Configure the web application with a valid
credential
Slide 17 / 23

18. MPEG-7 SOS Web-Services
Input Form
Browse
Storing Manual
information form
MPEG-7 XM mapping DB
Save Descriptors Algorithms Proposed DB schema
Retrieving Color Layout
Dominant Color link
Example Image
Browse
Search
form Image 1: 90%
Image 2: 88%
link
Color Structure Image 3: 80%
Edge Histogram Search …
Query by Example Selected Color descriptor
Selected Image
User visible steps Example Image
Hidden for user executing
MPEG-7 XM store-procedure
DB
Descriptors Algorithms
Proposed DB schema
View based search
Search
Name Image 1 link
View-based Description
form
Image 2
Image 3
Key words
…
Search
Selected Image
Slide 18 / 23

19. Client using MPEG-7 SOS Web-Services
Client link input
form web service
Retrieved web service is a
XML document
Slide 19 / 23

20. Client using MPEG-7 SOS Web-Services
Client link query
by example form
web service
Client redirect
web service
search result
Slide 20 / 23

21. Client using MPEG-7 SOS Web-Services
Client link view
based search
form web service
Client redirect
web service
search result
Slide 21 / 23