The presentation gives an overview of what metadata is and why it is important. It also addresses the benefits that metadata can bring and offers advice and tips on how to produce good quality metadata and, to close, how EUDAT uses metadata in the B2FIND service.
November 2016
Introduction of Human Body & Structure of cell.pptx
Introduction to Metadata
1. EUDAT receives funding from the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures. Contract No. 654065 www.eudat.eu
Introduction to metadata
Version 2
August 2016
This work is licensed under the Creative
Commons CC-BY 4.0 licence
2. What is metadata and why do we need it?
How to produce good quality metadata?
EUDAT and metadata
Overview
3. WHAT IS
METADATA?
Image CC-BY ‘Metadata is a love note to the future’ by
Cea+ www.flickr.com/photos/ centralasian/8071729256
4. Commonly defined as ‘data about data’, metadata helps to
make data findable and understandable
Metadata can be:
Descriptive: information about the content and context
of the data
Structural: information about the structure of the data
Administrative: information about the file type, rights
management and preservation processes
What is metadata?
5. Comprehensive metadata will:
Facilitate data discovery
Help users determine the applicability of the data
Enable interpretation and reuse
Allow any limitations to be understood
Clarify ownership and restrictions on reuse
Offer permanence as it transcends people and time
Provide interoperability
Why use metadata?
6. Metadata and documentation
Think about what will be needed in order to find, evaluate,
understand, and reuse the data.
Have you documented what you did and how?
Did you develop code to run analyses? If so, this should
be kept and shared too.
Is it clear what each bit of your dataset means? Make
sure the units are labelled and abbreviations explained.
Record all the information needed for you and others to
understand the data in the future
8. Create metadata at the time of data creation
Information will be forgotten and there won’t be time or
effort left to capture it later.
Metadata benefits from quality control at an early stage
too.
Time matters!
Image CC-BY-SA ‘egg timer – hour glass running out’ by OpenDemocracy
www.flickr.com/photos/opendemocracy/523438942
9. GOOD QUALITY METADATA
Image CC-BY ‘Quality’ by Elizabeth Hahn www.flickr.com/photos/128185330@N03/17517769750
10. Use of standards
Controlled vocabularies for unambiguous keywords
Simple, complete and consistent information
Appropriate description
Explanation of limitations to support reuse
Avoid special characters e.g. !@<~ etc...
Provide persistent identifiers such as DOIs
What makes metadata good?
11. The good and the bad
Metres / seconds
2015-09-10T15:00:01+01:00
Longitudinal wind speed
PDF 1.7
2008 US Population statistics
Barcelona, Venezuela
Furlongs and fortnight
10th Sept. 2015 15:00:01
U
PDF
Population statistics
Barcelona
More precise and
standardised Ambiguous
12. Metadata standards
Metadata standards provide a structured way to describe
the data
Information is presented in a reliable and predictable
format which allows for computer interpretation
Use of standards enables data interoperability
13. Metadata Standards Directory
Catalogue initiated by the Digital Curation Centre (DCC)
now maintained as a community initiative via the
Research Data Alliance
www.dcc.ac.uk/resources/metadata-standards
14. There are a number of factors to consider:
Data type – look for standards to suit your data
Community norms – what is accepted and common
practice in your field?
Organisational policies – is one recommended?
Instruments being used – any automated metadata?
What resources are available? – there are tools to create
metadata in certain standards, more instructional
materials and support
How to choose a metadata standard?
15. How to write quality metadata
Organise your information and reuse where possible e.g.
project abstracts, lab notebooks, citations
Write your metadata using a metadata tool
Review for accuracy and completeness
Have someone else read your record
Revise based on comments from your reviewer
Review once more before you publish Draft
ReviewRevise
Review
16. Tips to follow when creating metadata
Do not use jargon
Define technical terms and acronyms:
– CA, LA, GPS, GIS : what do these mean?
Clearly state data limitations
– E.g. data set omissions, completeness of data
– Express considerations for appropriate re-use
Use “none” or “unknown” meaningfully
– None usually means that you knew about data and nothing
existed (e.g., a “0” cubic feet per second discharge value)
– Unknown means that you don’t know whether that data
existed or not (e.g., a null value)
17. Dataset titles
Titles are critical in helping readers find your data
– While individuals are searching for the most appropriate
data sets, they are most likely going to use the title as the
first criteria to determine if a dataset meets their needs.
– Treat the title as the opportunity to sell your dataset.
A complete title includes: What, Where, When, Who, and
Scale
An informative title includes: topic, timeliness of the data,
specific information about place and geography
18. Which is the better title?
Rivers
OR
Greater Yellowstone Rivers from 1:126,700 U.S. Forest
Service Visitor Maps (1961-1983)
Greater Yellowstone (where) Rivers (what) from 1:126,700
(scale) U.S. Forest Service (who) Visitor Maps (1961-
1983) (when)
19. Write for machines, not just humans
Remember: a computer will read your metadata
Do not use symbols that could be misinterpreted:
Examples: ! @ # % { } | / < > ~
Don’t use tabs, indents, or line feeds/carriage returns
When copying and pasting from other sources, use a
text editor (e.g., Notepad) to eliminate hidden characters
20. Could someone use an automatic search to locate the
data?
Can others assess the usefulness of the data?
Could a novice understand it?
Is the metadata specific enough?
Is there enough information to re-use the data?
Is the information unambiguous – are all codes,
abbreviations and variables explained?
Remember to review your metadata!
21. EUDAT AND METADATA
Image CC-BY ‘University of Michigan Library Card Catalog’ by David Fulmer
www.flickr.com/photos/annarbor/4350629792
22. B2FIND is based on a comprehensive joint metadata
catalogue of research data collections stored in EUDAT
data centres and other repositories
It allows researchers or data users to find relevant data,
and supports communities and data providers to increase
visibility of their data
B2FIND provides a simple and user-friendly discovery
service on metadata steadily harvested from a wide
range of research communities
The B2FIND service
b2find.eudat.eu
23. The same term can be used by different disciplines
Species for chemists and zoologists
Andromeda for astronomers and historians
Some domain knowledge is therefore necessary
The EUDAT B2FIND service needs to suit a wide range of
different communities
The interdisciplinary problem
24. Metadata is harvested from different communities,
usually using the OAI-PMH protocol
The metadata (in a wide variety of standards) are
processed to map and transform them to the B2FIND
schema
How the B2FIND service works
INPUT
Metadata in community
standards e.g. DDI,
Dublin Core, CMDI, ISO
19115
OUTPUT
Homogenised metadata
in the B2FIND schema
25. Metadata records in B2FIND
http://b2find.eudat.eu/dataset/3a063891-6952-5bcf-a5ed-46f8a681c1c9
26. For more info: https://eudat.eu/services/b2find
User documentation: https://www.eudat.eu/services/userdoc/b2find-
integration
b2find.eudat.eu
27. www.eudat.eu
Authors Contributors
This work is licensed under the Creative Commons CC-BY 4.0 licence
EUDAT receives funding from the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures.
Contract No. 654065
Sarah Jones, Digital Curation Centre
Shaun de Witt, STFC
Sara Garavelli, Trust-IT
Thank you
Content has also been repurposed from the DataONE Educational
modules, ‘Metadata’ and ‘How to Write Good Quality Metadata’ Retrieved
from https://www.dataone.org/education-modules
Hinweis der Redaktion
This presentation will give an introduction to the concept of metadata, why it is important and how to address this in research projects.
There are three main topics that we will discuss:
What is metadata and why is it important. Here we will think about the benefits that metadata can bring to you and others.
Secondly, we will think about how to produce good quality metadata and offer some advice and tips
To close, we’ll explain how EUDAT uses metadata in the B2FIND service
So let’s begin by thinking about what metadata is.
The quote in the image here that ‘metadata is a love note to the future’ really gets at the meaning.
Metadata is critical to ensure data can be found, understood and reused. If you don’t create metadata, it’s unlikely you will still understand your data in a few years time. The act of creating metadata opens up possibilities for future use.
Metadata is commonly defined as ‘data about data’. By creating metadata, you will ensure that others can find and understand your data.
There are different types of metadata:
Descriptive metadata includes things like the title, author, date, location, coverage and subjects. It’s all the basic information people would need to find the data and understand the main content and context.
Structural metadata explains how data interrelate. For example, if a book has been digitised, you want to understand which set of images form each chapter.
Administrative metadata may include information added by others e.g. preservation metadata added by a repository to note what processes have been performed on the data
There are lots of reasons to create metadata. It:
Facilitates data discovery so others can find your data and your research gains more recognition and impact
Good metadata helps potential users determine whether the data meet their needs, and enables them to interpret and reuse the data
Metadata should outline any limitations and clarify data ownership and restrictions on reuse. This ensures others use the data appropriately
Without metadata, your data will become meaningless over time as others can’t understand and reuse them. Providing associated metadata will give your data permanence and ensure they live on.
By creating high quality metadata and using standards, you can also make your data interoperable
When you create metadata, it is useful to think broadly. You may come across the concept of ‘documentation’ to explain all the details you need to capture and share too.
You should aim to provide all the information a third-party would need to understand and reuse your data. This may include a description of what you did, your workflows, any code created and data dictionaries or clarification of all terms and abbreviations.
This diagram from Bill Michener from DataOne in the United States shows how much information is lost over time.
Metadata is a way to formalise this knowledge so your data retain meaning.
Time really matters when creating metadata – you should create metadata at the time of data creation as information is forgotten quickly. This also gives you an opportunity to do quality control early on.
We have explained why metadata is so important. Let’s now think about how to create good quality metadata.
There are lots of things you can do to improve the quality of your metadata:
Primary among these is to use standards. There are lots out there so look for something relevant to your data type and discipline.
Metadata standards don’t always prescribe how the information should be completed. For this you want to use controlled vocabularies or thesauri for keywords, and recognised ISO standards for common elements like languages or dates
Be consistent in the information provided and ensure the description is appropriate – enough information to avoid being ambiguous but also simple and concise
Any limitations with the dataset should be explained to ensure others reuse it appropriately and don’t make false assumptions
You should avoid using special characters, particularly in file names or column headers in spreadsheets as some software may interpret these symbols as an operator
Also provide persistent identifiers so others can reliably link to and locate your data. This helps with citation and tracking impact too.
Let’s look at some good and bad examples. You can see what we’re looking for in terms of metadata is more precise and standardised entries rather than information that could be ambiguous.
Metres and seconds are universally accepted units of measurement as opposed to furlongs or fortnights
For clarity, provide dates and times in the ISO standard, specifying the timezone
In the third example we can see a properly described variable as opposed to an abbreviation which others may not understand
When stating file formats, it is always useful to specify the version too
The final two examples show the need to be specific so others can understand the coverage properly
It is highly recommended to use metadata standards. It enables interoperability and ensures the information is presented in a predictable way to allow it to be processed by computers.
There are lots of standards that can be used and you can search for them by discipline.
The DCC started a catalogue of disciplinary metadata standards which is now being taken forward as an international initiative via an RDA working group
When choosing a metadata standard, you should consider:
Your data type
What is accepted practice for your field
Whether your organisation or the tools and instruments being used suggest using one format over another (for example if one is recommended or if some metadata is created automatically in a given standard)
Also think about what resources you have available. Some standards have associated tools and more comprehensive instructional materials, so they may be preferred.
When writing metadata, reuse information where possible rather than starting from scratch. For example, you may be able to use a project abstract written for your proposal, information from lab notebooks or citations for data you are reusing.
Where possible, write your metadata using a tool to make the processes easier and more consistent.
Think about metadata creation as an iterative process. It’s best to ask somebody else to read your record to make sure it makes sense to others and then revise and review it again before publishing.
Some general tips to follow include:
Avoid using jargon
Define any abbreviations, acronyms or technical terms
Clearly state limitations and express considerations for appropriate reuse
Use terms like ‘none’ or ‘unknown’ properly
Be comprehensive when writing your dataset title as this is how others will determine whether to look into your data further.
A complete title should explain what the data relates to, a location, time period, subject and scale.
This example illustrates the importance of descriptive titles in metadata records.
The second title gives enough detail for a reader to discern whether they might like more information about your data.
When you are writing your metadata, remember that it will be read by machines as well as people.
You should avoid using symbols that could be misinterpreted, and tabs/indents/breaks that may be stripped out. Using a text editor for copying data will ensure hidden characters and formatting are removed.
The final point to reiterate is the need to review your metadata.
It’s always useful to get a second opinion to make sure others can understand it and feel it’s clear and specific enough.
To close we want to explain how EUDAT is approaching metadata and how services like B2FIND can help you
B2FIND provides a simple and user friendly service that allows the users to discover a wide range of metadata from a variety of research communities. It is based upon a comprehensive metadata catalogue of data collections stored in the EUDAT data centres and harvested from other data repositories.
The B2FIND service helps researchers to find relevant data to reuse, and helps data providers to increase the visibility of their data.
Since EUDAT is a pan-European infrastructure supporting a wide range of disciplines, we have to think about how terms are used differently by different communities.
Chemical species for examples are atoms, molecules, ions etc, whereas for zoologists, ‘species’ denotes different families of animals
The B2FIND service works by harvesting metadata from different communities. This is done on a regular and incremental basis, usually using the OAI-PMH protocol.
The metadata is provided in a range of community standards. It is then processed to transform it to the generic B2FIND schema to allow cross-search.
This is what a record looks like in the B2FIND catalogue. There’s a basic description, a number of keyword tags and some additional information to note the source, creator, language etc
To find out more about B2FIND or use the service, please follow the links provided.