1. Metadata Harvesting and
Validation
Bram Vandeputte
K.U.Leuven
1

2. slideshare
• http://www.slideshare.net/bramvandeputte
2

3. Overview
• Validation Service
• Integration
• Online Validation Service
• Harvesting Component
3

4. Validation Service
• Interoperability : Application Profile (AP)
• Manual check : very time consuming
• Need a tool for enforcing an AP =>
validation scheme
Best practices derived
from previous projects
• A set of validation rules such as MELT and MACE
Reusable : modular +
• Reusable & extendable inheritance possible
4

5. Validation Service
• Components :
• XML schema : structure
• schematron :
• mandatory/conditional elements
• empty fields
• vocabularies (auto generated)
• ...
• Vcard component
5

6. Validation Service
component : atomic
block which does specific
validation checking
scheme : collection of
components that
• Terminology : ensures validity against
a whole AP
• Validation Component URI : unique identifier of
a scheme
• Validation Scheme
• Validation Scheme URI :
• http://aspect-project.org/validation/ASPECTv1.0/core
6

7. Validation Service
7

8. Validation Service
ASPECTv1.0/ ASPECTv1.0/
LOM loose
recommended core
recommended lomloose.xsd
vocabulary bank
schematron rules
core schematron vcard validator
Legend
rules
uses empty attribute
fields
extends IMS ILOX
ASPECT validationScheme
vcard validator validation component
8

9. Validation Service
9

10. Integration
• When do we validate ?
• Possible integration points :
• Mapping
• Harvesting
• Enriching
10

11. Online Validation
Service demo
11

12. 12

13. validation to lre AP
refer to lre ap document
13

14. invalid : discarded or
identifier recorded for
next harvesting
14

15. The Harvest component
invalid : discarded or
identifier recorded for
next harvesting
14

16. ARIADNE Harvester
invalid : discarded or
identifier recorded for
harvester log
next harvesting
14

17. ARIADNE Harvester
ASPECT Repository
SPI SQI
invalid : discarded or
identifier recorded for
harvester log
validation service
next harvesting
14

18. ARIADNE Harvester
ASPECT Repository
SPI SQI
External Repository
OAI
OAI-PMH
LOM
LOM
LOM
invalid : discarded or
identifier recorded for
harvester log
validation service
next harvesting
14

19. ARIADNE Harvester
ASPECT Repository
OAI-PMH
SPI SQI
External Repository
LOM
LOM
LOM
OAI
OAI-PMH
LOM
LOM
LOM
invalid : discarded or
identifier recorded for
harvester log
validation service
next harvesting
14

20. ARIADNE Harvester
ASPECT Repository
OAI-PMH
SPI SQI
External Repository
LOM
LOM
OAI
OAI-PMH
LOM
LOM
LOM
LOM
invalid : discarded or
identifier recorded for
harvester log
validation service
next harvesting
14

21. ARIADNE Harvester
ASPECT Repository
OAI-PMH
SPI SQI
External Repository
LOM
LOM
OAI
OAI-PMH
LOM
LOM
LOM
LOM
LOM
invalid : discarded or
identifier recorded for
harvester log
validation service
next harvesting
14

22. ARIADNE Harvester
ASPECT Repository
OAI-PMH
SPI SQI
External Repository
LOM
LOM
OAI
OAI-PMH
LOM
LOM
LOM
LOM
Validation
Msg
invalid : discarded or
identifier recorded for
harvester log
validation service
next harvesting
14

23. ARIADNE Harvester
ASPECT Repository
OAI-PMH
SPI SQI
External Repository
LOM
LOM
OAI
OAI-PMH
Validation
LOM
Msg
LOM
LOM
LOM
invalid : discarded or
identifier recorded for
harvester log
validation service
next harvesting
14

24. ARIADNE Harvester
ASPECT Repository
OAI-PMH
SPI SQI
External Repository
LOM
LOM
OAI
OAI-PMH
LOM
LOM
LOM
LOM
Validation
Msg
invalid : discarded or
identifier recorded for
harvester log
validation service
next harvesting
14

25. ARIADNE Harvester
ASPECT Repository
OAI-PMH
SPI SQI
External Repository
LOM
LOM
OAI
LOM
OAI-PMH
LOM
LOM
LOM
Validation
Msg
invalid : discarded or
identifier recorded for
harvester log
validation service
next harvesting
14

26. ARIADNE Harvester
ASPECT Repository
OAI-PMH
SPI SQI
External Repository
LOM
LOM
LOM
OAI
2 6 LOM
OAI-PMH 1
Validation
LOM
Msg
LOM
LOM
LOM
4
5 3
Validation Validation
LOM
Msg Msg
harvester log
validation service
15

27. Validation Reports
• After harvesting -> report generated and put
online
• report has 4 “levels” :
• full log (incl. metadata)
• reporting log
• Grouped Errors
• Error Summary
16

28. • Questions ?
17

29. References
• SPI : http://ariadne.cs.kuleuven.be/lomi/index.php/
SimplePublishingInterface
• IEEE LOM : http://ltsc.ieee.org/wg12/
• OAI-PMH : http://www.openarchives.org/
• Online Validation :
http://ariadne.cs.kuleuven.be/validationService/
18