Community web sites: small pieces loosely joined

ViBRANT
Virtual Biodiversity
Community web sites: small pieces loosely joined
Dave Roberts, David King, Simon Rycroft, David Morse,
Lyubomir Penev, Donat Agosti & Vince Smith

SEVENTH FRAMEWORK
PROGRAMME
-infrastructure

ViBRANT

SEVENTH FRAMEWORK
PROGRAMME -infrastructure

ViBRANT

Small pieces loosely joined
Has many potential meanings:

Joining contributors together to form
communities
Joining the data together that go towards
forming a Scratchpad
Joining Scratchpad content with the landscape of
biodiversity informatics data on the web

SEVENTH FRAMEWORK

ViBRANT

Addressing the challenges of taxonomy
Goal ...
Inventory the Earth’s species
Document their relationships
“Publish” & apply these data

Data set ...
1.8 M described spp. (10M names)
300M pages (over last 250 years)
1.5-3B specimens

People ...
4-6,000 taxonomists
30-40,000 “pro-amateurs”
Many more citizen scientists?

SEVENTH FRAMEWORK

ViBRANT

I The technology must largely embody the cause–effect
relationship connecting problem to solution.
II The effects of the technological fix must be assessable using
relatively unambiguous or uncontroversial criteria.
III Research and development is most likely to contribute
decisively to solving a social problem when it focuses on
improving a standardized technical core that already exists.

Sarewitz and Nelson (2008) Three rules for technological fixes. Nature, 456: 871-872

SEVENTH FRAMEWORK

ViBRANT

Biodiversity - a kind of washing powder?

When 2010 was named as the "year of biodiversity" by
the UN, it began with a plea to save the world's
ecosystems.

UN Secretary-General Ban Ki-moon said: "Biological
diversity underpins ecosystem functioning... its
continued loss, therefore, has major implications for Recently, members of the public
current and future human well-being." were asked what biodiversity is.
The most common answer was
"some kind of washing powder".

http://www.bbc.co.uk/news/science-environment-11546289 15 October 2010

SEVENTH FRAMEWORK

ViBRANT

Addressing the challenges of biodiversity informatics
“…the field [of biodiversity informatics] appears to be growing in
a void of overarching, motivating questions, effectively making it
a set of technologies in search of questions to address.”

Peterson et al, Syst. & Biodiv. 2010

SEVENTH FRAMEWORK

ViBRANT

Scratchpads Hosted websites for taxonomists
http://scratchpads.eu Research & publication platform
Modular (Drupal) & flexible
Supports the taxonomic workflow
Bottom-up design, agile dev.
Ecosystem of communities (185)
2,350+ users (unpaid) from 2007
ViBRANT follow on, €4.75M

SEVENTH FRAMEWORK

ViBRANT

Taxonomy & Literature DNA, Phylogeny & Specimens
2.3k users, 58 countries,
268k pages
185 "Virtual Research
Communities"
EDIT, GBIF, NHM, & EOL
Platform for biodiversity
research & data publication
eBooks eJournals
Changing the nature of
collaboration
Expanding opportunities to
participate in science

Image Galleries Societies & Organizations

SEVENTH FRAMEWORK

ViBRANT

A website for you & your community

Magic
Your data Your web site
SEVENTH FRAMEWORK

ViBRANT

Taxonomy import,
management and
navigation

SEVENTH FRAMEWORK

ViBRANT

Reference manager /
Endnote support for
bibliographies

SEVENTH FRAMEWORK

ViBRANT

Image galleries,
image upload &
annotation

SEVENTH FRAMEWORK

ViBRANT

Nexus / Newick import for
visualizing phylogenies

SEVENTH FRAMEWORK

ViBRANT

Molecular &
morphological character
matricies
(discrete, morphometric
and text characters)

SEVENTH FRAMEWORK

ViBRANT

Presence / absence
country maps

SEVENTH FRAMEWORK

ViBRANT

Specimen & location
records (DwC)

SEVENTH FRAMEWORK

ViBRANT

Static web
pages

Web fora
with e-mail
integration
Newsletters with
User blogs e-mail integration

SEVENTH FRAMEWORK

ViBRANT

Import from CSV text file to any
content type

SEVENTH FRAMEWORK

ViBRANT

ViBRANT Products
A Virtual Research Environment (Scratchpads) where users can
safely store, share and manage their research information.
Analytical services for users to build identification keys and
phylogenetic trees.
A publication platform for users to automatically compile taxonomic
manuscripts from their research database.
A portal for users to centrally access publicly accessible biodiversity
research information and literature.
Training, support & sociological study, helping research communities
to use these tools and services.
A standards compliant technical architecture that can be sustained by
biodiversity research community.

SEVENTH FRAMEWORK

ViBRANT
Training Biodiversity
& outreach data
programme standards

Networking
User Controlled
feedback vocabulary

The “chromosome” systems platform

WP3. Training
User
sociology
Data
aggregation
WP4. Standards
study portal
WP8. Mobilisation
Field GBIF
recording integration
support activities

Citizen
Biodiversity
science
visualisation
programme
layers

Scratchpads
Virtual Research

Service Research
Environment Distributed
Phylogenetic
Scratchpad
analysis
hosting
Bioclimatic Software
WP5. Data modelling
& metrics
module
integration
WP2. Architecture
WP6. Publishing WP7. Literature
Identiﬁcation Sustainability
tools plan
Communal
Matrix data
biodiversity
editor
literature

Biodiversity Biodiversity
data literature
publishing markup
Scholarly
Biodiversity
manuscript
datamining
publishing

SEVENTH FRAMEWORK

ViBRANT

Biodiversity literature
looks like this

Cues
Indented text
UPPER CASE TEXT
Bold text
Italic text
Latin
Keywords
Symbols

SEVENTH FRAMEWORK

ViBRANT

Adobe Reader has this
M
BRITISH MUSEUM
(NATURAL HiSi
26JU
PRESENTED
GENERAL UC.-lARY
Bulletin ofthe
BritishMuseum (Natural History)
The ichneumon-fly genus Banchus
in the OldWorld
(Hymenoptera)
M. G. Fitton
series
Entomology
Vol51 Nol 25 July 1985

SEVENTH FRAMEWORK

ViBRANT

Lura (BHL) has this
M
BRITISH MUSEUM
(NATURAL HiSi
26 JU
PRESENTED
GENERAL UC.-lARY
Bulletin of the
British Museum (Natural History)
The ichneumon-fly genus Banchus
(Hymenoptera) in the Old World
M. G. Fitton
Entomology series
Vol51 Nol 25 July 1985

SEVENTH FRAMEWORK

ViBRANT

But choice of XML schema is important
ABBYY XML is very detailed
This line of text has 202 bytes:
The Bulletin of the British Museum (Natural History), instituted in
1949, is issued in fourscientific series, Botany, Entomology,
Geology (incorporating Mineralogy) and Zoology,and an
Historical series.
To encode in ABBYY XML format this line requires 45,533 bytes.
There are 84,263 lines in the document from which this example
was taken.

SEVENTH FRAMEWORK

ViBRANT

Look for taxon names
Used uBio FindIT web service
Overall excellent
Especially as add Namebank ID
But still some oddities
Genus = ‘The’
The scutellum
The primitive
Species or Author = ‘and’
Exetastes and
B[anchus] falcatorius and

SEVENTH FRAMEWORK

ViBRANT

Look for paragraph types

Simple keyword matching
Surprisingly effective!
Issue – can identify start, but not end…
Follow up work
Punctuation
Concepts

SEVENTH FRAMEWORK

ViBRANT

Look for other proper names
Biologia Centrali-Americana has a gazetteer
Most journals do not
Generic solution = OpenCalais
Good accuracy
Old countries
D.D.R.
West Germany
Continents
America

SEVENTH FRAMEWORK

ViBRANT

Ambiguities and Mis-identifications
New York Other Oddities
City Persons
State Surname only
Washington Two part names
City Van Veen
State van Veen
Lake George Regions and Continents
City East Africa
Lake Victoria Africa
City

SEVENTH FRAMEWORK

ViBRANT

Negative spell checking

Go beyond stop words
Remove everything not in a spell dictionary
Check:
Minor
Vulgar
Bulletin 27 from the Zoology Series reduced
From 139,034
to 5,219 words

SEVENTH FRAMEWORK

ViBRANT

Ligatures
INTRODUCTION.
Volume, one of five required for the enumeration of the Rhynchophora, was
THIS
commenced by Dr. Sharp in 1889 and is now concluded by myself. The study of the
" Otiorhynchinœ Alatse " has unfortunately been delayed for many years, during the
publication of Vol. IV. parts 4, 5, and 7, all of which are devoted to the Family
Curculionidœ. The present Volume, IV. part 3, includes the Subfamilies Attelabinae,
Pterocolinœ, Allocoryninee, Apioninœ, Thecesterninae, and Otiorhynchinre. The
Attelabinae are represented by 104 (88 new), the Pterocolinse by three (all new), the
Allocoryninse (a new subfamily) and Thecesterninse each by one, the Apioninae by
88 (84 new), and the Otiorhynchinae by 419 (340 new) species respectively; the total
number for the six subfamilies being 616 species, with 516 new, and forty new
genera. Amongst the 419 Otiorhynchinae, the apterous and winged forms are almost
equal in number, there being a preponderance of apterous terrestrial species
(Eupagoderes, Epicœrus, Epayriopsis, &c.) in the arid portions of Mexico and the
winged forms ÇExophthalmuS) &c.) becoming relatively more numerous in the forest
regions southward. Taking the Curculionidœ as a whole—the subfamilies
Curculioninae and Calandrinse, in addition to those worked out in the present
Volume,—the number of species enumerated altogether from Central America is as
follows :— Vol. IV. part 3, 616; IV. part 4, 1365; IV. part 5, 908; IV. part 7, 344 : total
3233. The three other families of Rhynchophora—the Brenthidae, Scolytidae, and

SEVENTH FRAMEWORK

ViBRANT

Ligatures
Otiorhynchinæ => Otiorhynchinœ Thecesterninæ => Thecesterninse
Alatæ => Alatse Apioninæ => Apioninae
Curculionidæ => Curculionidœ Otiorhynchinæ => Otiorhynchinae
Attelabinæ => Attelabinae Otiorhynchinæ => Otiorhynchinae
Pterocolinæ => Pterocolinœ Curculionidæ => Curculionidœ
Allocoryninæ => Allocoryninee Curculioninæ => Curculioninae
Apioninæ => Apioninœ Calandrinæ => Calandrinse
Thecesterninæ => Thecesterninae Brenthidæ => Brenthidae
Otiorhynchinæ => Otiorhynchinre Scolytidæ => Scolytidae
Attelabinæ => Attelabinae Anthribidæ => Anthribidae
Pterocolinæ => Pterocolinse Hispidæ => Hispida
Allocoryninæ => Allocoryninse Cassididæ => Cassididae

For the 24 æ there are: 11 ae; 5 œ; 5 se; 1 ee; 1 re; 1 a?;

So not a single correct rendering of the ligature, æ.

By contrast, the only example of œ in the page, Epicœrus, was correctly rendered.

SEVENTH FRAMEWORK

ViBRANT

Soundex
831 elytra E436 831 elytra E436
639 prothorax P636 509 Elytra E436
637 Hab H100 294 elytris E436
616 punctate P523 125 elytral E436
578 millim M450 36 elytron E436
12 elytrisque E436
9 elytrorumque E436
8 Elytral E436
7 elytrorum E436
2 elytro E436
1 Elytrorum E436
1 Elytris E436

SEVENTH FRAMEWORK

ViBRANT

Similar words?

denticulate => denticulata
Levenshtein distances of 1: 0,0,1

denticulate => reticulate

denticulate => geniculate

SEVENTH FRAMEWORK

ViBRANT

What did we achieve?
Marked up 11 volumes, i.e. 4,504 pages
Have robust workflow, can mark up a Bulletin in about 10-15
minutes. Choke point is call to OpenCalais web service
No manual intervention or review required: workflow is
scalable
Recognising taxon names:
Well uBio gives us a goods start, and we have techniques to
cluster ALL mis-spellings and variants with a valid taxon; but
not perfect, eg BanchusFabricius ends up in more than one
cluster

SEVENTH FRAMEWORK

ViBRANT

“making the Scratchpads better”
More reliable (e.g., distribute the servers)
More functional (e.g., phylogenetic & publication services)
Easier to use (better workflows)
Prettier (better graphical design - more intuitive)
More integrated (for data stored inside & outside the Scratchpad framework)
More sustainable (simple administration, distribute developers, development sandbox)

“making natural history better”
Easier to compile, manage and reuse your data
Easier to find and reuse other peoples data
Promoting your data inside & outside the taxonomic community
Getting people to work for you (crowdsourcing)

SEVENTH FRAMEWORK

ViBRANT

Author
Manuscript Public Enhanced
preparation on HTML
a Scratchpad
PDF

Submit as XML
Enhanced XML Printed paper

Produce PDF

Send to Register with
reviewers ZooBank,
Publisher GBIF, EoL etc.

SEVENTH FRAMEWORK

ViBRANT

Thank you for your
attention.

Any questions

SEVENTH FRAMEWORK

Community web sites: small pieces loosely joined

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (14)

Andere mochten auch

Andere mochten auch (8)

Ähnlich wie Community web sites: small pieces loosely joined

Ähnlich wie Community web sites: small pieces loosely joined (20)

Mehr von Vince Smith

Mehr von Vince Smith (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Community web sites: small pieces loosely joined