Maarten Brinkerink and Johan Oomen (Netherlands Institute for Sound and Vision, NL) will talk about Waisda?, an open source video labeling game framework developed by Sound and Vision[3], which is currently being developed further in the context of Europeana.[4] Sound and Vision has collaborated with several public broadcasters in the Netherlands to enable fans of certain programmes to contribute fine-grained description of this content. In the latest edition called ‘Spotvogel’ (Mockingbird) Sound and Vision collaborated with the nature TV programme ‘Vroege Vogels’ (Early Birds, by the VARA) to mobilize the online community around the programme for identifying flora, fauna and locations within specific segments of the broadcasts. To support the tagging of the flora and fauna the game utilized a controlled vocabulary that is maintained by Naturalis. Players are awarded points when their tag entries match with other players, and they can score bonus points for using ‘professional’ terms from the controlled vocabulary. Players can also earn badges for certain achievements within the game, for instance for identifying a certain number of birds. Up until now the game managed to gather over 240,000 tags.

1. Crowdsourcing Descriptions for
Nature Recordings
Maarten Brinkerink
Netherlands Institute for Sound and Vision
November 22nd – MCN2014 Dallas

2. Netherlands Institute
for Sound and Vision

3. Our Mission
“As guardian of Dutch audiovisual heritage,
we keep Dutch history, as documented in
moving images, alive. We enable everyone to
utilize the collections to learn, experience and
create.”
2014

4. R&D at Sound and Vision

5. Audiovisual
broadcasts
Professional annotation
Search engine
General
public
Academics
Television
makers

6. General
public
Professional annotation Academics
Machine analysis Data gathering
Television
makers
Audiovisual
broadcasts
Search engine

7. Europeana (Awareness)
Europeana is the trusted source of cultural heritage.
Explore millions of items from a range of Europe's leading
galleries, libraries, archives and museums. Books and
manuscripts, photos and paintings, television and film, sculpture
and crafts, diaries and maps, sheet music and recordings,
they’re all here.
Europeana Awareness is a Best Practice Network, led
by the Europeana Foundation, designed to - among other things
- promote its use by a broad public for a variety of purposes
including recreation and hobbies, research, learning, genealogy
and tourism – engaging users via user generation of content,
creation of digital stories and social networking.

8. Three core WP2 objectives
Research in end-user involvement that will help
define opportunities and challenges for Europeana
Launch a two thematic campaigns that each cover
a specific challenge for gathering and linking UGC to
Europeana
Establish close collaborations with the Wikipedia
Community

9. WP2 – End-user Engagement
“This WP implements support for the meaningful
inclusion of User Contributed Content (UCC)
content in Europeana and of the distribution of
Europeana content in external environments.”
[1] Contextualisation – users adding context to heritage objects in the
form of stories and descriptions;
[2] Contribution – gather digital objects from end-users that can help to
enrich and compliment the collection on Europeana;

10. Task 2.1 Tools used to enable end user contributions to
Europeana content
Oxford University
Used to contribute stories in the context of 1914-1918
Existing tools (Waisda)
Spild af Tid, NTUA
Digital Storytelling Platform
We Are What we Do and PSNC,
Used to upload and publish content for 1989

11. Digital Storytelling Platform:
Editing a Story

13. Historypin
http://www.europeana1989.eu

15. Wikipedia Edit a thons, 10 countries
Sweden (WW1) – November 7, 2012
Sweden (Fashion) – March 22, 2013
Poland (1989) – June 9, 2013
Denmark (1894) – June 8, 2013
Netherlands, Greece, Australia,
Belgium, Germany, Serbia,
Sweden and UK (WW1 Edit-a-thons)
– June 29, 2013
Sweden (Fashion) – November 12,
2013
Europeana Fashion Editathon at Nordiska museet in Stockholm
https://commons.wikimedia.org/wiki/File:Europeana_Fashion_Editatho
n_2013_11.jpg

16. Wiki Loves Public Art photo competition
• Executed in May 2013
• Sweden, Spain, Austria, Finland and Israel joined the contest in
2013
• 9,250 images were uploaded as part of the contest by 225
uploaders, of which 57 percent were first time contributor
• The articles with photos from the contest have been shown a
total of 1,353,909 times between May-October 2013.

17. Classification of Crowdsourcing
Projects
Correction
and
Transcription
Classification and
Tagging
Collection
acquisition
Co-curation Contextualisation
Europeana Awareness: D2.1 User requirements and IPR implications for User Contributed Content in
Europeana
Johan Oomen & Lora Aroyo
http://www.iisi.de/fileadmin/IISI/upload/2011/p138_oomen.pdf

18. Classification of Crowdsourcing
Projects
Correction
and
Transcription
Classification and
Tagging
Collection
acquisition
Co-curation Contextualisation
Europeana Awareness: D2.1 User requirements and IPR implications for User Contributed Content in
Europeana
Johan Oomen & Lora Aroyo
http://www.iisi.de/fileadmin/IISI/upload/2011/p138_oomen.pdf

19. Video Labeling Game – What’s That?
(Waisda?)
Allows internet users to annotate
audiovisual archive material in the form of a
(serious) game
The goal of the game is consensus
between players
Fun and competition as motivation

20. Why?
Investigate the added value of social
tagging
Experimenting with new forms of services
for the public (serious games)
Which results in:
• Time-related metadata
• Social tags (bridging the semantic gap)
• Interaction between the archive/broadcaster and
the public

21. Spotvogel (‘Mockingbird’)
The Third Installment of Waisda?
Based on the Vroege Vogels (‘Early Birds’)
nature series by NL public broadcaster
VARA (also a partner in the project)
Collaboration with Naturalis, utilizing their
Dutch Species Catalogue for matching the
social tags to an authoritative taxonomy
Targeted the online community of interest
associated with the series (thousands of
active online forum contributors, on the
programme website)

22. Game Mechanics
Homepage

23. Game Mechanics
Tagging and scoring

24. Game Mechanics
Tagging and scoring (zoomed in)

25. Game Mechanics
Match with user
Tagging and scoring (zoomed in)

26. Game Mechanics
Vocabulary match
Match with user
Tagging and scoring (zoomed in)

27. Game Mechanics
Vocabulary match
Match with user
Tagging and scoring (zoomed in)
Potential match

28. Game Mechanics
Game recap

29. Game Mechanics
User profile

30. Results
Three implementations resulted in over a
million social tags, by thousands of players
On average 50% of the social tags consists
of matched tags, and 25% corresponds to
controlled vocabularies
On average 10-20% of the social tags are
unique
‘Super taggers’ are responsible for the vast
majority of the social tags that are added

31. Results
The extent to which expert cataloguers
deem the social tags to be useful, heavily
depends on the type of content
The balance between social tags that
correspond with terms from a controlled-vocabulary
and terms invented by users
themselves, also depends heavily on the
type of content
First experiments suggest that the social
tags enable high recall fragment retrieval

32. Lessons Learned
Don’t try to reach a broad audience, but
find an active niche
Open knowledge structures provide a way
to structure the data that is gathered, and –
at the same time – provide great
possibilities for linking collections
Crowdsourcing means accepting and
respecting multiple authorities and
perspective in regards to your collection

33. Related: eCreative – Sound
Connections
-Enrich sounds with Europeana materials and
other websources
-Invite communities to interact
-http://www.historypin.com/en/explore/birdlife/

34. Related: eSounds – Wikipedia Editathon
-Enrich Wikipedia with bird recordings
-Contextualize sound recordings in a relevant
knowledge environment
-Bring together Wikipedians & birders

35. Thanks for your attention!
Maarten Brinkerink
Netherlands Institute for Sound and Vision
mbrinkerink@beeldengeluid.nl
@mbrinkerink
@benglabs
http://labs.europeana.eu/apps/Waisda/
https://github.com/beeldengeluid/waisda
Many thanks to:
Johan Oomen & Lizzy Komen, for their input
Just Vervaart, Cyril Snijders, Sander Pieterse, Michiel Hildebrand &
Martijn van Steenbergen, for their involvement in ‘Spotvogel’
Let’s continue to think big, y’all!!!