WP1 wrap up - - Doron Goldfarb (ONB) at DM2E all WP meeting, 12 December 2014, Pisa

1. co-funded by the European Union
DM2E WP1 – Wrap Up
DM2E All WP Meeting
Pisa, 12 December 2014
Doron Goldfarb, Austrian National Library (ONB)

2. DM2E WP1 Tasks
DM2E WP1 - Wrap Up 212.12.2014
• Task 1.1. Collect metadata formats and relational backend
structures
• Task 1.2 Collect requirements
• Task 1.3 Testing the User Interface for Creating Mapping,
Interlinking Heuristics and for Configuring the Workflow
• Task 1.4 Setup a test scenario for the prototype platform
• Task 1.5 Organise the final content integration

3. T1.1 Collect metadata formats and relational
backend structures
DM2E WP1 - Wrap Up 312.12.2014
0
1
2
3
4
5
6
TEI EAD MARC METS/MODS MAB2 Own format ESE DC(X)
Source metadata formats used in DM2E
• Metadata Questionnaire
• Content Questionnaire

4. Task 1.2 Collect requirements
DM2E WP1 - Wrap Up 412.12.2014
• Additional Metadata properties
• Hierarchical representations
• Administrative Metadata
• Represent authority data (GND, LCSH, etc)

5. Deliverable 1.1
DM2E WP1 - Wrap Up 512.12.2014
• Task 1.1
• Task 1.2
Deliverable 1.1
Requirements Report
Submitted 08/2012
Approved 04/2013

6. Task 1.3 Testing the User Interface for Mapping,
Interlinking and for Configuring the Workflow
DM2E WP1 - Wrap Up 612.12.2014
• Evaluation of MINT, OMNOM and SILK

7. DM2E WP1 - Wrap Up
• Visual interfaces provide intuitive starting point for novice
users
• Simple tasks can be managed successfully
• The more complex the tasks get, the more users switch to
alternative, „under-the-hood“ ways of fulfilling their tasks
• Visual, web based interfaces sometimes have problems
with logging and progress feedback
12.12.2014
Task 1.3 Testing the User Interface for Mapping,
Interlinking and for Configuring the Workflow
7

8. Task 1.4 Setup a test scenario for the prototype
platform
DM2E WP1 - Wrap Up
• Assess the scholarly research platform
• Focus on usability and usefulness for DH scholars
12.12.2014 8

9. DM2E WP1 - Wrap Up
Results Wittgenstein Pilot Group
DH scholars of the test group could
imagine using the platform for a variety
of scenarios such as
• Propaedeutic uses
• Close reading
• Creating citation networks
• Collaborative research scenarios
• Teaching
• Refining ontologies
DH solutions in the LOD context
• Discussions about the publication
status of semantic annotations
• Credibility
• Intellectual property rights
• Characteristics of ontologies
12.12.2014
Results Burckhardtsource Group
Pundit systematically used for
proofreading and semantic annotation
• The reader is facilitated in his
understanding of the sources
• More experienced users are also able
to create "intellectual mappings" of
an author
• Topics and interesting fields can be
highlighted
• Semantic annotations are both useful
for the scholars working with the
material and for others who may
want to re-use the semantically
enriched data
9
Task 1.4 Setup a test scenario for the prototype
platform

10. Task 1.5 Final Content Integration
DM2E WP1 - Wrap Up 1012.12.2014
• Scheduled to end 02/2014

11. Task 1.5 Final Content Integration
DM2E WP1 - Wrap Up 1112.12.2014
• Scheduled to end 02/2014

12. Task 1.5 Final Content Integration
DM2E WP1 - Wrap Up 1212.12.2014
• Scheduled to end 02/2014
• Still ongoing 12/2014

13. Task 1.5 Final Content Integration
DM2E WP1 - Wrap Up 1312.12.2014
• Still ongoing 12/2014
• Reasons:
• Waited too long for data model to be „perfect“
• This delayed implementation of data model in MINT
• Data model too complex for using MINT only
• MINT not suited for some metadata formats
• Providers had to code their own mappings
• Associate providers have no resources for that
• Late data deliveries

14. Task 1.5 Final Content Integration
DM2E WP1 - Wrap Up
• Providers started their mappings with MINT, switched
to manual mappings throughout the process
• Various „interpretations“ of the DM2E model.
• Collaborative effort between WP1 and WP2 in order to
set up guidelines how to represent DM2E metadata in
RDF
• Result: “Recommendations for the RDF
representation of DM2E metadata”
12.12.2014 14

15. Task 1.5 Final Content Integration
DM2E WP1 - Wrap Up
• WP1, WP2 and WP3 cooperated to define additional
options for content providers to contribute
annotatable content
• Result: „Annotatable Content Specification“
• Types of content that can be annotated with the tools
provided in DM2E and how they should be provided
12.12.2014 15

16. Deliverable 1.2
DM2E WP1 - Wrap Up 1612.12.2014
• Task 1.3
• Task 1.4
• Task 1.5
Deliverable 1.2
Final Integration Report
Submitted 02/2014
Approved 04/2014
Report about missing content submitted 04/2014

17. Task 1.5 Final Content Integration
DM2E WP1 - Wrap Up
• Europeana ingestion
– Samples delivered 05/2014
– 1st delivery 09/2014
12.12.2014 17

18. Task 1.5 Europeana ingestion
DM2E WP1 - Wrap Up
• 1st delivery ONB 09/2014
UIB
UBFFM
MPIWG
UBER
BBAW
SBB
12.12.2014 18

19. Task 1.5 Europeana ingestion
DM2E WP1 - Wrap Up
• 1st delivery ONB 09/2014
UIB
UBFFM
MPIWG
UBER
BBAW
SBB
• Summer 2014:
Europeana enforces strict Public Domain policy
12.12.2014
LICENSING
ISSUES
19

20. Task 1.5 Europeana licensing issues
DM2E WP1 - Wrap Up
• Affected providers 09/2014
ONB
UIB
MPIWG
UBER
BBAW
SBB
CJH
NLI
12.12.2014
UBFFM
BRANDEIS
JDC
PIM
BAS
ECORR
20

21. Task 1.5 Europeana licensing issues
DM2E WP1 - Wrap Up
• Negotiations with Providers & Europeana until
11/2014
ONB
UIB
MPIWG
UBER
BBAW
SBB
CJH
NLI
12.12.2014
UBFFM
BRANDEIS
JDC
PIM
BAS
ECORR
21

22. Task 1.5 DoW content delivered to date
DM2E WP1 - Wrap Up12.12.2014
UBFFM
BRANDEIS
JDC
PIM
ECORR
BAS
Provider Dataset Pages / DoW Pages delivered 12/14 Difference Europeana Pages in Europeana 12/14
ONB
ABO - Books 15000000 15596190 596190 1 15596190
CODICES - Manuscripts 42400 46104 3704 1 46104
UIB Wittgenstein Autographs 5000 5000 0 1 5000
UBER Polytechnisches Journal 205000 203000 -2000 1 203000
CJH (EAJC)
LBI / YIVO Archival
400000
315000
490000
1 315000
LBI / YIVO Library 575000 1 575000
SBB
Hauptmann Collection 128000 54547 -73453 1 54547
Chamisso Collection 12000 34567 22567 1 34567
Western Manuscripts 0 49880 49880 1 49880
Gebauer & Schwetschke 0 80000 80000 0 0
MPIWG
RaRa Rare Books 500000 532379 32379 1 532379
Harriot Autographs 1200 10427 9227 0 0
ISMI Manuscripts 100000 152600 52600 0 0
BBAW Deutsches Textarchiv 470000 461161 -8839 0 0
NLI
Books 2800000 829133 -1970867 0 0
Manuscripts 102000 78000 -24000 0 0
Archival Material 0 2953 0 0 0
DFGA/CRNS Nietzsche Manuscripts 19330 0 -19330 0 0
19789930 19025941 -763989 17411667
22

23. Task 1.5 DoW content delivered to date
DM2E WP1 - Wrap Up12.12.2014
UBFFM
BRANDEIS
JDC
PIM
ECORR
BAS
Provider Dataset Pages / DoW Pages delivered 12/14 Difference Europeana Pages in Europeana 12/14
ONB
ABO - Books 15000000 15596190 596190 1 15596190
CODICES - Manuscripts 42400 46104 3704 1 46104
UIB Wittgenstein Autographs 5000 5000 0 1 5000
UBER Polytechnisches Journal 205000 203000 -2000 1 203000
CJH (EAJC)
LBI / YIVO Archival
400000
315000
490000
1 315000
LBI / YIVO Library 575000 1 575000
SBB
Hauptmann Collection 128000 54547 -73453 1 54547
Chamisso Collection 12000 34567 22567 1 34567
Western Manuscripts 0 49880 49880 1 49880
Gebauer & Schwetschke 0 80000 80000 0 0
MPIWG
RaRa Rare Books 500000 532379 32379 1 532379
Harriot Autographs 1200 10427 9227 0 0
ISMI Manuscripts 100000 152600 52600 0 0
BBAW Deutsches Textarchiv 470000 461161 -8839 0 0
NLI
Books 2800000 829133 -1970867 0 0
Manuscripts 102000 78000 -24000 0 0
Archival Material 2953 2953 0 0 0
DFGA/CRNS Nietzsche Manuscripts 19330 0 -19330 0 0
19789930 19025941 -763989 17411667
23

24. Task 1.5 DoW content delivered to date
DM2E WP1 - Wrap Up12.12.2014
UBFFM
BRANDEIS
JDC
PIM
ECORR
BAS
Provider Dataset Pages / DoW Pages delivered 12/14 Difference Europeana Pages in Europeana 12/14
ONB
ABO - Books 15000000 15596190 596190 1 15596190
CODICES - Manuscripts 42400 46104 3704 1 46104
UIB Wittgenstein Autographs 5000 5000 0 1 5000
UBER Polytechnisches Journal 205000 203000 -2000 1 203000
CJH (EAJC)
LBI / YIVO Archival
400000
315000
490000
1 315000
LBI / YIVO Library 575000 1 575000
SBB
Hauptmann Collection 128000 54547 -73453 1 54547
Chamisso Collection 12000 34567 22567 1 34567
Western Manuscripts 0 49880 49880 1 49880
Gebauer & Schwetschke 0 80000 80000 0 0
MPIWG
RaRa Rare Books 500000 532379 32379 1 532379
Harriot Autographs 1200 10427 9227 0 0
ISMI Manuscripts 100000 152600 52600 0 0
BBAW Deutsches Textarchiv 470000 461161 -8839 0 0
NLI
Books 2800000 829133 -1970867 0 0
Manuscripts 102000 78000 -24000 0 0
Archival Material 2953 2953 0 0 0
DFGA/CRNS Nietzsche Manuscripts 19330 0 -19330 0 0
19789930 19025941 -763989 17411667
24

25. Task 1.5 DoW content delivered to date
DM2E WP1 - Wrap Up12.12.2014
UBFFM
BRANDEIS
JDC
PIM
ECORR
BAS
Provider Dataset Pages / DoW Pages delivered 12/14 Difference Europeana Pages in Europeana 12/14
ONB
ABO - Books 15000000 15596190 596190 1 15596190
CODICES - Manuscripts 42400 46104 3704 1 46104
UIB Wittgenstein Autographs 5000 5000 0 1 5000
UBER Polytechnisches Journal 205000 203000 -2000 1 203000
CJH (EAJC)
LBI / YIVO Archival
400000
315000
490000
1 315000
LBI / YIVO Library 575000 1 575000
SBB
Hauptmann Collection 128000 54547 -73453 1 54547
Chamisso Collection 12000 34567 22567 1 34567
Western Manuscripts 0 49880 49880 1 49880
Gebauer & Schwetschke 0 80000 80000 0 0
MPIWG
RaRa Rare Books 500000 532379 32379 1 532379
Harriot Autographs 1200 10427 9227 0 0
ISMI Manuscripts 100000 152600 52600 0 0
BBAW Deutsches Textarchiv 470000 461161 -8839 0 0
NLI
Books 2800000 829133 -1970867 0 0
Manuscripts 102000 78000 -24000 0 0
Archival Material 2953 2953 0 0 0
DFGA/CRNS Nietzsche Manuscripts 19330 0 -19330 0 0
19789930 19025941 -763989 17411667
25

26. Task 1.5 DoW content delivered to date
DM2E WP1 - Wrap Up12.12.2014
UBFFM
BRANDEIS
JDC
PIM
ECORR
BAS
Provider Dataset Pages / DoW Pages delivered 12/14 Difference Europeana Pages in Europeana 12/14
ONB
ABO - Books 15000000 15596190 596190 1 15596190
CODICES - Manuscripts 42400 46104 3704 1 46104
UIB Wittgenstein Autographs 5000 5000 0 1 5000
UBER Polytechnisches Journal 205000 203000 -2000 1 203000
CJH (EAJC)
LBI / YIVO Archival
400000
315000
490000
1 315000
LBI / YIVO Library 575000 1 575000
SBB
Hauptmann Collection 128000 54547 -73453 1 54547
Chamisso Collection 12000 34567 22567 1 34567
Western Manuscripts 0 49880 49880 1 49880
Gebauer & Schwetschke 0 80000 80000 0 0
MPIWG
RaRa Rare Books 500000 532379 32379 1 532379
Harriot Autographs 1200 10427 9227 0 0
ISMI Manuscripts 100000 152600 52600 0 0
BBAW Deutsches Textarchiv 470000 461161 -8839 0 0
NLI
Books 2800000 829133 -1970867 0 0
Manuscripts 102000 78000 -24000 0 0
Archival Material 2953 2953 0 0 0
DFGA/CRNS Nietzsche Manuscripts 19330 0 -19330 0 0
19789930 19025941 -763989 17411667
26

27. Task 1.5 DoW content delivered to date
DM2E WP1 - Wrap Up12.12.2014
UBFFM
BRANDEIS
JDC
PIM
ECORR
BAS
Provider Dataset Pages / DoW Pages delivered 12/14 Difference Europeana Pages in Europeana 12/14
ONB
ABO - Books 15000000 15596190 596190 1 15596190
CODICES - Manuscripts 42400 46104 3704 1 46104
UIB Wittgenstein Autographs 5000 5000 0 1 5000
UBER Polytechnisches Journal 205000 203000 -2000 1 203000
CJH (EAJC)
LBI / YIVO Archival
400000
315000
490000
1 315000
LBI / YIVO Library 575000 1 575000
SBB
Hauptmann Collection 128000 54547 -73453 1 54547
Chamisso Collection 12000 34567 22567 1 34567
Western Manuscripts 0 49880 49880 1 49880
Gebauer & Schwetschke 0 80000 80000 0 Technical
MPIWG
RaRa Rare Books 500000 532379 32379 1 532379
Harriot Autographs 1200 10427 9227 0 Licensing
ISMI Manuscripts 100000 152600 52600 0 Licensing
BBAW Deutsches Textarchiv 470000 461161 -8839 0 Ready for ingestion
NLI
Books 2800000 829133 -1970867 0 Technical
Manuscripts 102000 78000 -24000 0 Licensing
Archival Material 2953 2953 0 0 Technical
DFGA/CRNS Nietzsche Manuscripts 19330 0 -19330 0 0
19789930 19025941 -763989 17411667
27

28. Task 1.5 Additional content delivered to date
DM2E WP1 - Wrap Up12.12.2014
Provider Dataset Pages / DoW Pages delivered 12/14 Difference Europeana Pages in Europeana 12/14
UBFFM Medieval Manuscripts 238760 238760 1 238760
BRANDEIS (EAJC) Spanish Civil War Posters 112 112 0 Ready for ingestion
BAS Codex Suprasliensis 570 570 0 Ready for ingestion
JDC (EAJC) JDC Archive NY 1914-18 20000 20000 0 technical
ECORR Eur. corresp. Jakob Burckhardt 2834 2834 0 technical
262276 262276 238760
19789930 19288217 -501713
UBFFM Medieval Manuscripts 68442 68442 0 0
GEI Textbooks 456324 456324 0 0
28

29. DM2E WP1 Achievements
DM2E WP1 - Wrap Up 2912.12.2014
• Managed to specify DM2E data model together with WP2

30. DM2E WP1 Achievements
DM2E WP1 - Wrap Up 3012.12.2014
• Managed to specify DM2E data model together with WP2
• Managed to set up transformation workflows

31. DM2E WP1 Achievements
DM2E WP1 - Wrap Up 3112.12.2014
• Managed to specify DM2E data model together with WP2
• Managed to set up transformation workflows
• For rich and contextualized metadata descriptions

32. DM2E WP1 Achievements
DM2E WP1 - Wrap Up 3212.12.2014
• Managed to specify DM2E data model together with WP2
• Managed to set up transformation workflows
• For rich and contextualized metadata descriptions
• For structural metadata descriptions enabling annotations of
underlying content

33. DM2E WP1 Achievements
DM2E WP1 - Wrap Up 3312.12.2014
• Managed to specify DM2E data model together with WP2
• Managed to set up transformation workflows
• For rich and contextualized metadata descriptions
• For structural metadata descriptions enabling annotations of
underlying content
• Provided best-practice experience for mapping to EDM/DM2E

34. DM2E WP1 Achievements
DM2E WP1 - Wrap Up 3412.12.2014
• Managed to specify DM2E data model together with WP2
• Managed to set up transformation workflows
• For rich and contextualized metadata descriptions
• For structural metadata descriptions enabling annotations of
underlying content
• Provided best-practice experience for mapping to EDM/DM2E
• Published DM2E metadata as Linked Open Data

35. DM2E WP1 Achievements
DM2E WP1 - Wrap Up 3512.12.2014
• Managed to specify DM2E data model together with WP2
• Managed to set up transformation workflows
• For rich and contextualized metadata descriptions
• For structural metadata descriptions enabling annotations of
underlying content
• Provided best-practice experience for mapping to EDM/DM2E
• Published DM2E metadata as Linked Open Data
• Delivered high quality content to the scholarly research platform
and to Europeana

36. co-funded by the European Union
Thank You !