What would the Millennium Development Goals look like for digital heritage information? New metrics are needed to understand consumer behavior and improve the social impact potential of heritage information.

1. Use of Indicators to Improve
Digital Heritage Consumption
Trilce Navarrete
ACEI
25 June 2014

2. Digital Heritage Indicators 2
Today
 How to measure digital heritage activities to improve consumption?
 The case of the Netherlands
1. Why do we need new indicators?
2. Current practice
 Museums at national level
 Missing indicators
1. Ideal data sets
 Proposal for metrics and evaluation: What and how to measure?
1. Policy implications

3. Digital Heritage Indicators 3
1. Why do we need new indicators?
Cultural indicators have supported decision-making in the use of resources
allocation in heritage, since the 1970s. There has been a significant investment
in digital technology within the heritage field, since the 1980s.
Indicators for digital heritage activities have changed, new metrics are being
established in response to new technologies and their applications.
Decisions on what to measure (using what data and how) is defined by policy.
Data on digital heritage activities has focused on production and increasingly on
distribution. What about consumption?
A new data collection framework is of essence to understand consumption.

4. Digital Heritage Indicators 4
1. Why do we need new indicators?
Focus has gone from building networks > producing digital content > coordinate
efforts > participate in information market > wide evaluation > social impact.
This is evident from the historic data sets available.

5. Digital Heritage Indicators 5
2. Current practice
The EU funded NUMERIC (2008) and ENUMERATE (2012, 2013) to define the
current state of digitization: production, costs and distribution of collection
information from LAMs.
Surveys have refined their approach and have expanded questions to include:
-Scope of dissemination (online publication)
-Monitor consumer behavior
-Inclusion of sustainability in information policy
Despite some methodological changes (definitions, pool of participants) and
project-based approach (multiple managers, one-time financing), much can be
said about digital collection information.
The Netherlands has produced national reports that include digital activities.
MusIP (2008) produced the first national overview (at collection level) and ICT
use in Museums (2002, 2008) had a broad population.

6. Digital Heritage Indicators 6
Digital collection information EU
ENUMERATE 2013

7. Digital Heritage Indicators 7
Digital collection information: museums
Based on NUMERIC and ENUMERATE:
2008 2012
% collections digitized NDL 32 41
EU 23 28
% collections available online NDL 31 25
EU 5 29
% museums with a digitization plan NDL - 55
EU 26 39
% digitization staff of total staff NDL - 11
EU - 6

8. Digital Heritage Indicators 8
Digital collection information: museums
Based on NUMERIC and ENUMERATE:
2008 2012 2013
% collections digitized NDL 32 41 36
EU 23 28 24
% collections available online NDL 31 25 33
EU 5 29 34
% museums with a digitization plan NDL - 55 21
EU 26 39 30
% digitization staff of total staff NDL - 11 10
EU - 6 4

9. Digital Heritage Indicators 9
Digital collection information: museums
Distribution channels (based ENUMERATE):

10. Digital Heritage Indicators 10
What is digitization?

11. Digital Heritage Indicators 11
Consumption enabler
“Many users might not be able to express information needs for a cultural heritage artifact in a
query” Stiller, 2012.

12. Digital Heritage Indicators 12
3. Ideal data sets
Institutions have experimented with metrics to understand use (consumption),
which does not have to be restricted to content in the institution’s website.
Ex.1: The Amsterdam Museum has completed basic digitization of ca. 90,000
objects, of which 60% have an image. All collection information is available
online as open data. Digitization is guided by an information policy with an
annual budget of €50,000.
Ex.2: The Rijksmuseum in Amsterdam digitizes 25,000 objects per year (since
2011), high-resolution and contextual metadata, with an annual budget of ca.
€470,000. An information policy guides selection process and overall
performance. Collections information is published on institutional website (of
which 111,000 objects as open data).

13. Digital Heritage Indicators 13
3. Ideal data sets
Ex.3: The Amsterdam City Archive digitizes 15,000 objects per week (1.5
million per year), low-resolution and limited metadata, with an annual budget of
ca. €200,000 (of which ca. €130,000 are paid for by consumers). Selection is
made with an information policy and by digitization on demand. Collection
information (11m scans online) is fully available (on-site for free).
In 2013:
22,455 orders
(963 users)
11,430 paid orders
(329 users)
823,020 scans viewed
(50% extern IP address)
900,000 website visits
(300,000 image bank)

14. Digital Heritage Indicators 14
3. Ideal data sets
Performance is multidimensional and can be informed by:
•Resources: collections, budget, policy, FTEs
•Services: digital activities, level of engagement
•Access: to information
= Structure
= Process
= Result
ResourcesResources ServicesServices
AccessAccess
efficiency effectiveness
productivity

15. Digital Heritage Indicators 15
3. Ideal data sets
• Resources:
• Collections = 100% basic / 80% image
• Budget = 3% annual (FTEs 10%)
• Policy = yes (incl. earmarked budget, sustainability plan)
• Services:
• Digital activities = 100% own website, aggregators, 50% content sites
• Level of engagement = open linked data 60%
• Access:
• 400% increase users
• 200% increase content used

16. Digital Heritage Indicators 16
Ideal progression
Resource allocation towards digitization becomes efficient when use (and
reuse) increases in time.
Digital collection information growth
Contextualization / linked open data
Distribution
% objects
Onsite Remote: Own website Aggregator Content sites

17. Digital Heritage Indicators 17
Ideal progression
Resource allocation towards digitization becomes efficient when use (and
reuse) increases in time.
Digital collection information growth
Contextualization / linked open data
Distribution
% objects
Onsite Remote: Own website Aggregator Content sites

18. Digital Heritage Indicators 18
Ideal progression
Resource allocation towards digitization becomes efficient when use (and
reuse) increases in time.
Digital collection information growth
Contextualization / linked open data
Distribution
% objects
Onsite Remote: Own website Aggregator Content sites

19. Digital Heritage Indicators 19
Ideal progression
Resource allocation towards digitization becomes efficient when use (and
reuse) increases in time.
Digital collection information growth
Contextualization / linked open data
Distribution
% objects
Onsite Remote: Own website Aggregator Content sites

20. Digital Heritage Indicators 20
4. Policy implications
The Ministry of Culture wants to support, guide and evaluate the digital
heritage infrastructure. Information and ICT are seen as agents of change:
Emerging indicators may include:
1.The digital collection
• Diversity in content
• Quality of content (data, metadata, image, links, standards, open data)
• Sustainability (constructing a digital infrastructure)
2.The network
• Hub growth (constituents linked/consumers)
• Imbed of results (for policy development)
• Communication (informing the public about results and activities)
3.Social change
• New uses, new user groups (B2B, B2C), new services
• User studies

21. Digital Heritage Indicators 21
4. Policy implications
Mechanisms to support data gathering, reporting and dissemination of results
and best practice can reduce effort redundancy.
Incentives (and clarity on benefits) to gather and report data at institutional
level can increase participation.
Long-term goals of the heritage information network involve more than
scanning (or data entry):
• To improve access to heritage information to all user groups
• To reduce information inequalities so that all heritage forms and
perspectives are represented
• To enhance user experience, supporting access / adoption / reuse of
heritage information
• To work efficiently, given the available resources

22. Digital Heritage Indicators 22
Challenges and opportunities
Digital heritage data for analysis is manually gathered, irregularly, with no
assigned institution responsible for reporting. Automated data gathering
methods would facilitate analysis (the “big data” approach).
Harmonization of relevant methods are being developed at EU level. Locally
(at institutional level) policy development requires detailed data –not always
standardized. Effectiveness is dependent on institutional goals.
Definitions on digital heritage consumption and metrics for its measurement
will continue to change as consumer behavior / technology change.
It is about giving access to heritage information while stimulating life long
learning in an engaged critical society.

23. Digital Heritage Indicators 23
Thank you !
Questions ?
T.Navarrete@uva.nl