Kick off to the 2015 Defining New Metrics for Library Success Symposium at the iSchool, University of Toronto

1. Framing
&
Defining
Metrics
for
Libraries
Rebecca Jones
rebecca@dysartjones.com

2. HOT topic

3. Know the
language of
your context
Metrics
Value
Evalua:on
Assessment
Measures

4. Measure: info about what is

5. Assess: test against a target

6. Evaluate: judge the findings

7. h=p://goo.gl/kJrzVw
Symptoms
Underlying causes

8. What’s
below the
water
causing the
symptoms?
• We don’t tend to have
a “culture of
assessment”…
• Most “measures”
convey past
performance
• Difficult and complex
• Our “measures” don’t
align with decision-
makers

9. We’ve learned that:
• Identifying & illustrating value depends on conversations
with your stakeholders
– …the first conversation shouldn’t be when measures are
presented
• “More” isn’t necessarily better
– …in fact, “more” clouds the issue and the message

10. Our frames
& mental
models
h=ps://goo.gl/WOjS7N

11. Why should
we frame
this issue?

12. • No one magic measure
• Successful organizations:
• clarity of purpose
• understand their culture
• performance measurement system that
fits that culture
• “Value” is dynamic, economic,
psychological & relative to alternatives
• Identifying outcomes & especially impact
takes a long time & heavy duty analysis

13. IT’S HARD

14. Center for Investigative Reporting’s
Frame
Non-profit media & their funders have been
working on impact for years
4 “strands of the conversation”:
ü Standards
ü Engagement
ü Online analytics
ü Offline activities

15. Oh, those offline activities
“Because offline activities are
messy and irregular and require
staff and time to track, they are
cast aside as, at best, qualitative
data and, at worst, anecdotes. But
qualitative data is data, and it is as
valuable as quantitative data if it
is collected and analyzed using
rigorous methods.”!

16. Impact
CIR’s definition:
Change in
status quo @
macro, meso or
micro levels
“we want to understand the
relationship of convertibility
among these outcomes. We are
only beginning to think about
how to quantify these
interactions; this is not alchemy,
but science – and maybe a little
art. “
• Dissection
• App in Podio

17. Your frame
Your assumptions &
beliefs about
measurements now:
Ideas,
discomfort,
provoking
Ideas,
discomfort,
provoking
Your assumptions
&
beliefs about
measurements at
noon tomorrow: