Talk at I Annotate 2016 conference.

1. The State of Image
Annotations
Robert Casties
Max Planck Institute for the History of Science, Berlin

2. What we do
• Work with historical sources
• scanned books, manuscripts, notebooks
• digital photos
• scanned drawings and paintings
• digital texts
• data

3. Galileo Galileis MS72 "Notes on motion" (1998)
http://www.mpiwg-berlin.mpg.de/Galileo_Prototype/INDEX.HTM

4. Musawwarat Graffiti Archive (2011)
http://musawwaratgraffiti.mpiwg-berlin.mpg.de

5. digilib client-side point annotation (since 2002)

6. HyperImage annotated presentation (2009?) of
James Gillray: „Political-Dreamings!” (1801)

7. Status quo
• lots of digital editions and projects
• nice and diverse features and tools
• re-inventing lots of wheels
• data silos
• solution: annotations and standards

8. Our vision
• "Weaving a web of knowledge" (Jürgen Renn, 2003?)
• close and distant reading of all kinds of sources (works
best with Open Access)
• create comments, relations, narratives
• share the results during the process or the result with
collaborators or everybody
• bi-directional links (Xanadu) - collect research results
related to sources creating a "semantic network" of
sources

9. references to networks

10. Annotations as bi-directional
links
• standardized (open, stable) access to source
documents (image, text,…)
• standardized annotation data linking comment to
source
• relevant segment needs to be referenced
• annotation server needs to be shared
• relation can be reversed

11. Annotation standards
• Open Annotation Data Model (http://openannotation.org)
• general annotation data model
• last spec 1.0 of 2013
• W3C Web Annotation Working Group (since 2014)
• Abstract Annotation Data Model (spec)
• Data Model Vocabulary
• Data Model Serializations
• HTTP API (spec)
• Client-side API (FindText API spec)

12. Image annotation in W3C
web annotation
• Target of annotation can be an image
• Segment of image can be specified by
• FragmentSelector with W3C media fragment
(rectangle specified in pixel or percent)
• SVGSelector with SVG

13. Annotation standards
• SharedCanvas (http://shared-canvas.org)
• data model for ”virtual books“
• uses Open Annotation
• last spec 1.0 in 2013
• IIIF (http://iiif.io)
• Image API standard
• Metadata API standard
• spec 2.1 released May 12, 2016

14. IIIF (International Image
Interoperability Framework)
• ARTstor
• Bayerische
Staatsbibliothek
(Bavarian State
Library)
• La Bibliothèque
nationale de
France
• Biblissima
• British Library
• British Museum
• Brown University
• Centre de
Recherche et de
Restauration des
Musées de France
(C2RMF)
• Cogapp
• Columbia
University
• e-codices – Virtual
Manuscript Library
of Switzerland
• Cornell University
• DPLA
• Digital Image
Archive of
Medieval Music
(DIAMM)
• Digirati Ltd
• Europeana
• The J. Paul Getty
Trust
• Ghent University
• Gottingen State
and University
Library
• Harvard University
• Indiana University
• Internet Archive
• Johns Hopkins
University
• Klokan
Technologies
• Leiden University
• MIT Libraries
• National Gallery of
Art
• National Library of
Austria
• Nasjonalbiblioteke
t (National Library
of Norway)
• National Library of
Denmark
• National Library of
Egypt
• National Library of
Israel
• National Library of
New Zealand
• National Library of
Poland
• National Library of
Scotland
• National Library of
Serbia
• National Library of
Wales
• New York
University
Libraries
• Ohio State
University
• Oxford University
(Bodleian Library)
• Princeton
University Library
• Qatar National
Library
• Sirma Group
• St. Louis
University
• Stanford University
• TextGrid
• text & bytes
• University College
Dublin
• University of
Edinburgh
• University of Hong
Kong
• University of
Illinois at Urbana-
Champaign
• University of Notre
Dame
• University of
Pennsylvania
• University of
Toronto
• Vatican Library
• The Walters Art
Museum
• Wellcome Trust
• Wikipedia
(Wikimedia
Foundation)
• World Digital
Library
• Yale Center for
British Art
• Yale University

15. {scheme}://{server}
{/prefix}/
{identifier}/
{region}/{size}/
{rotation}/
{quality}.{format}
Image API Presentation API

16. Software
IIIF Shapes
Annotorious - (+)
HyperImage / Yenda - / + + / +
digilib + / - +
SemToNotes - +
Mirador + (+)
Diva + -
T-PEN (+) -
AnnoTate - -

17. Annotorious

18. Annotorious
• http://annotorious.github.io
• Javascript client image annotation toolkit
• annotates images in HTML pages
• can integrate with Annotator.js
• can use scaling image server (OpenSeadragon)
• can do polygons

19. HyperImage

20. HyperImage / Yenda
• http://hyperimage.ws
• “like hypertext for images”
• Java editor and server, Flash/HTML5 client
• free shapes in layers, linking between images, texts and layers
• used in a number of digital humanities projects
• Yenda (http://yenda.tools)
• new fully web architecture using IIIF
• under development

21. digilib

22. digilib
• http://digilib.sourceforge.net
• scaling image server (Java)
• HTML + Javascript client
• view state and client-side annotations encoded in URL (since
2002)
• server-side annotations using Annotator.js (since 2012)
• polygon shapes (since 2015)
• IIIF API server

23. digilib

24. SemToNotes

25. SemToNotes
• https://hkikoeln.github.io/SemToNotes
• JavaScript API to develop browser-based image
annotation tools
• topological queries on eXist and EXPath Geo
Module backend
• under development

26. Mirador

27. Mirador
• http://projectmirador.org/
• IIIF reference viewer
• Javascript client, can use any IIIF Presentation
API feed
• deep zoom, metadata display, book reading,
bookmarking, comparison view
• shape annotations (under development)

28. Diva.js

29. Diva.js
• http://ddmal.github.io/diva.js
• Javascript client
• uses IIIF Image API
• deep zoom, fast scrolling through documents,
synchronized information views
• annotation overlays

31. T-PEN
• http://t-pen.org
• Transcription for Paleographical and Editorial
Notation
• Java server, online version free to use
• new version 3 with IIIF support under
development

32. AnnoTate

33. AnnoTate
• https://anno.tate.org.uk , https://github.com/
zooniverse/AnnoTate
• Zooniverse crowdsourcing project for Tate
Archive holdings
• transcription of artist’s personal papers
• zoomable view, simple annotations

34. where are we now
• IIIF image server standard incredibly successful
• Annotation use cases growing
• Annotation standards on the way…
• Annotation software development scattered
• Discovering and sharing annotations not yet
there

36. ceterum censeo…

37. Standards for annotating
image areas
• something between very simple (W3C Media Fragments)
"selector": {
"type": „FragmentSelector",
"conformsTo": "http://www.w3.org/TR/media-frags/",
"value": "xywh=135,16,25,53"
}
• and very complex (SVG)
"selector": {
"type": ["SvgSelector", "Content"],
"text": "<svg:svg> …
<polygon xmlns="http://www.w3.org/2000/svg"
points="5315,5639 5444,5529 5505,5593 5346,5691 5315,5639" />
… </svg:svg>“
}

38. Standards for annotating
image areas
• Proposition
• use resolution independent coordinates (e.g.
0≤x≤1, fraction of image width)
• use GeoJSON / WKT
• points, lines, polygons
• only point coordinates

39. Annotations as "micro-
publications"
• stable URLs for annotations
• provenance information
• copyright and license information
• versioning?
• authority / reputation?