MCN2014 Panel presentation on Accurator: Acquisition of Qualitative Annotations from Niche Communities.

1. Henrike Hövelmann
Accurator
Acquisition of Qualitative Annotations
from Niche Communities
Chris Dijkshoorn

2. Accurator
Collaboration between museum
and research institutes
Nichesourcing of annotations
Accurator system sneak preview
Datasets used in Accurator
Evaluation
Future work

3. Rijksmuseum Amsterdam
Collection
‣ Collection of ~1,000,000 objects
‣ Print collection of ~550.000 prints
‣ Artworks on display ~8.000
Online Collection
‣ Accessible through API
‣ Published as Linked Data
‣ 548,785 object records
‣ 424,098 prints
‣ 212,517 works are digitised

4. Print Room Online Project
Make artworks accessible online
Professional catalogers and
photographers
Process
‣ Register artwork
‣ Provide annotations
‣ Digitise artwork
‣ Publish online
~40,000 new object records a
year

5. Annotating Artworks
Focus on art-historical
information
Occasional lack of expertise
regarding subject matter
annotations
This print is described as:
‣ “Bird with blue head”
‣ “Branch with red leaves”

6. research project
Socially-Enriched Access to Linked
Cultural Media
Diverse collaboration
‣ 4 Research institutes
‣ 5 Cultural heritage institutions
‣ 5 Industry partners
4 PhDs are working on Accurator
‣ Crowdsourcing the annotation
process
‣ Finding niche groups with required
expertise

7. Tagging
State of the art
‣ Steve.museum
‣ YourPaintings
Rijksmuseum allowed tagging
on website

8. Beyond crowdsourcing
Use nichesourcing to gather
annotations
‣ Find niche groups with required
expertise
‣ Use the knowledge of expert laymen
‣ Cluster tasks based on topic

9. Niche
Experts
Cluster Trust
Search
User
Modelling
Crowd
Annotations
Cultural
Heritage
Collection
Niche
Experts
Niche
Experts
Accurator System

10. Nichesourcing Annotations
Niche
Expert
Japanese
Redstart

11. Accurator screencast

12. Which niche group to
address?
Base choice on gaps in collection
Started with two topics
‣ Castles
‣ Flowers
Contacted multiple groups, but
low response
Change the focus

13. Focus on birds

14. Naturalis biodiversity data
Specimen Collection
‣ Songbirds 3.210 specimen
Print Collection
‣ Koekkoek 406 prints
Thesaurus
‣ IOC World Bird List 34,197 concepts

15. Semantic Annotation
Open Annotation Data Model for
storing information
Semantic support for annotators
‣ Autocompletion from structured
vocabularies
‣ IOC world bird list in Accurator

16. Semantic Annotation - Data model
dc:title skos:prefLabel
oa:hasTarget
oa:hasTarget
#xywh=percent:
19,40,39,27
rdf:value
hdl:
504055 ico: 25F34
"Eagle owl in
magnolia"@en
"owls"@en
edm:
aggregated
CHO
agg:COLLEC
T.504055
edm:
isShownBy
dc:
subject
ann:id_6111
fedf002e9
birds:species
-bubo_bubo
oa:
hasBody
genid:target
_44478b9e
oa:hasSource
ann:h "0.27"^^xsd:float
ann:w "0.38"^^xsd:float
ann:x "0.18"^^xsd:float
ann:y
"0.40"^^xsd:float

17. Evaluation results
Compare provided annotations
to correct answers
‣ Define correct answer
‣ Collect “ground truth”
Experts tend to disagree
Obtained a set of Naturalis bird
prints with scientific names
“Sexy ladies”
“Women”
“The frog indicates the
profession of the women”
“The frog stands for
the rejected lover”

18. Future work
Run experiment with ground
truth
Incorporate automatic trust
assessment in Accurator
Show effects of annotating by
leveraging it for collection
integration

19. Possibilities for integration - Rijksmuseum Website

20. Possibilities for integration - Dutch Species
Print
Kono Bairei

21. Accurator
Acquisition of Qualitative Annotations from Niche Communities
Henrike Hövelmann h.hovelmann@rijksmuseum.nl
Chris Dijkshoorn c.r.dijkshoorn@vu.nl http://chrisdijkshoorn.nl