Moe Hosseini-Ara, Director of Culture, City of Markham and I lead a discussion of outcomes based measures with the participants at the University of Toronto's iSchool Symposium on Defining New Metrics for Library Success, May 2015.

1. Determining &
Demonstrating Value
-
with the Logic Model
MOE HOSSEINI-ARA, CITY OF MARKHAM
MHOSSE@MARKHAM.LIBRARY.ON.CA
REBECCA JONES, DYSART & JONES ASSOCIATES
REBECCA@DYSARTJONES.COM

2. Demonstrate Value…
TO WHOM?

3. Ivanna U.R. Mony
Climber
Lobbying to have
library budget
reallocated to other
projects
Never used the library
Buys all her books from
Amazon
Why does the library
need so many staff?
And why are they paid
so much?

4. Dr. Ural Doomed
Libraries days are numbered
Libraries aren’t relevant
◦ Everyone has computers
and internet
◦ Need information? It’s all
online
◦ Research happens in
faculties not libraries
With eBooks and research
online believes facilities can
be made smaller – no need
for print material

5. CFO Roi Bottomline, MBA
All about the
bottomline
Revenue and
investments
How does any support
revenue generation?
Services that don’t
generate revenue are
a drain on the system
If it’s not efficient
outsource it!

6. Kee Paper
Influential in many circles
Has deep seated beliefs in
traditional library services
and importance of paper
for legal profession
Libraries are institutions for
reading & research

7. Understand
the Context

8. Know the Context & the
Contexters
How do they
measure their
progress?
Know your
Stakeholders –
who are they?
How is value
defined,
measured &
communicated?

9. Understand the
Context
Align Strategies
& Objectives
Identify
Services &
Programs
Define
Measures
Manage
Measurement
Data
Translate Data
Into Outcomes
& Impacts
Communicate
Results
Measurement
Framework

10. Not everything that counts can be
counted AND not everything that
can be counted counts…
- Einstein
…Wait! What??

11. Our logic isn’t everyone’s logic

12. Value
SatisfactionOperational
Three types of measures

13. Meaningful measures
Matter to you AND your stakeholders
Are conveyed in stakeholders’ language
Focus attention on what is most important for the
organization
 Are critical for managing, planning
& decision-making
 Are organization-dependent

14. What measures
do you keep
now?

15. Measures…
…are, by definition, based on a “beginning” or
monitor results against an agreed-to objective or
value…

16. Operational
or
Usage
Measures
•We’ve long tracked “how much”
•We also need to look at differences:
• Peaks? Dips? Switches?
•What are these statistics really telling
us? Or, not?
•Who needs to see these statistics?
•Are they telling a story that you want
to tell?
Operational

17. Customer
Satisfaction
Measures
Tends to focus on existing products
and/or services
Usually measured through surveys
Are we also measuring importance?
Survey burnout, plus the expectation is
to score all “excellent” or “10 out of
10”
Satisfaction

18. The VALUE
proposition
…
•Are we adding value?
•What difference are we
making?
•What is the impact of our
services and/or programs?
•How are we contributing to the
success of….?
Value

20. Start with stakeholders
Name Position How do they
measure
their
progress?
Goals &
objectives
What do we know about
them? What type of
stakeholder are they?
What do
they know
about our
services?
EXERCISE
Critical that you understand the context in which the measures are
considered.
1. In your organization, how is value defined, measured and
conveyed?
2. Who are the stakeholders who must understand the value of your
services?

21. The Logic Model

22. Logically
…go beyond documenting what
you did
and measure what difference
you made in the life of your
targeted clients…
…how their situation changed

23. Measuring Impact or Value
Change in the human condition?
• Behaviour
• Skill
• Knowledge
• Attitude
• Circumstances
• Awareness
• Motivation
• Condition
• Status

25. IMPACT
Equipment
INPUT
resource
perspective
OUTPUT
operational
perspective
OUTCOME
user
perspective
IMPACT
stakeholder
perspective
It’s logical that:

26. IMPACT
Equipment
INPUT
Resource
Perspective
• Equipment
• Space
• Supplies
• Funding
• Staff

27. IMPACT
Equipment
OUTPUT
Operational
Perspective
• Training module
• Report
• # of Training Attendees
• # of Services Provided

28. IMPACT
OUTCOME
User Perspective
• Increased skill
• Know-how or know-that
• Change in status
• Change in behavior or
attitude

29. IMPACT
IMPACT
Stakeholder
Perspective
• Successful cases
• Increase in targeted clients
• Improved market position

30. Academic Example
Input One liaison librarian
Output Faculty research support for a Grant Application
Outcome Completed application submitted on time
Impact Grant dollars received by faculty. Faculty member is
able to hire 2 grad students to conduct research:
increased university reputation, a number of articles are
submitted and accepted by academic journals,
increased citations for faculty member, university
increases visibility.

31. Government Example
Input $100K resource budget
Output 2500 resources utilized
Outcome Resources used by 75% of employees in policy
development
Impact Usage enabled policy development to:
1. Eliminate department-specific resource purchases
saving the organization $35K
2. Reduce policy duplication or “blind-sides” by at
least 25% (as reported during policy debriefings)

32. Corporate Example
Input •$80K electronic content contract; info pro; technology
Output Customized research portfolio in response to request for
background on deal between XYZ company & ABC
government agency, including specific data on terms of deal
Outcome Client extrapolated how ABC agency likes to see deals
structured
Impact Organization successfully completed a deal with ABC
government agency for $10M, the 1st in 5 years

33. Public Library Example
Input
Output
Outcome
Impact Children attending programs are able to maintain the
skills taught in school verified by parent follow-up.
Library recognized for helping children develop and
maintain literacy skills.
Board of Education partnership & recognition based on
improvements in standardized testing results.

34. Public Library Example
Input
Output
Outcome Number of program registrations increased by 25% year
to year. Reduced cancellation rates. Reduced
operating costs.
Impact Children attending programs are able to maintain the
skills taught in school verified by parent follow-up.
Library recognized for helping children develop and
maintain literacy skills.
Board of Education partnership & recognition based on
improvements in standardized testing results.

35. Public Library Example
Input
Output 10 new programs each offered 5 times per year
attended by 1,600 students ages 6-10
Outcome Number of program registrations increased by 25% year
to year. Reduced cancellation rates. Reduced
operating costs.
Impact Children attending programs are able to maintain the
skills taught in school verified by parent follow-up.
Library recognized for helping children develop and
maintain literacy skills.
Board of Education partnership & recognition based on
improvements in standardized testing results.

36. Public Library Example
Input -Funding
-Staffing
-Supplies
=$10,000 to develop learning place programs
Output 10 new programs each offered 5 times per year
attended by 1,600 students ages 6-10
Outcome Number of program registrations increased by 25% year
to year. Reduced cancellation rates. Reduced
operating costs.
Impact Children attending programs are able to maintain the
skills taught in school verified by parent follow-up.
Library recognized for helping children develop and
maintain literacy skills.
Board of Education partnership & recognition based on
improvements in standardized testing results.

37. BEGIN WITH
THE
IN MIND
END
Covey 1989

38. Understand the
Context
Align Strategies
& Objectives
Identify
Services &
Programs
Define
Measures
Manage
Measurement
Data
Translate Data
Into Outcomes
& Impacts
Communicate
Results
Measurement
Framework

39. Align
Strategies &
Objectives

40. Align Strategies & Objectives
Stakeholder’s Goals &
Objectives
Your Goals & Objectives:
How they Contribute

41. Align objectives
Articulate & document your goals & objectives
◦ What are you trying to accomplish?
Clarify how these contribute towards your
stakeholders’ objectives & desired outcomes
EXERCISE

42. Identify
Services
&
Programs

43. Identify Services
What services, programs or products contribute to
your realizing an objective?
Stakeholder Goals &
Objectives
Your Goals & Objectives
Your Services
EXERCISE

44. Define
Measures

45. Define success
What will success look like?
What indicators or measures will you use?
◦ Qualitative? Quantitative?
◦ What data needs to be collected?
◦ Keep it key
◦ Keep it simple
◦ Be practical
◦ Don’t get mired – or overwork frontline staff

46. Service Impact
Inputs
Person power, money,
content, etc.
Activity
What you do
Outputs
What the activity
produces
Outcomes
How the client uses the output
Impact
What changes for the client
Outcome Measures Impact Measures
EXERCISE

47. Manage
Measurement
Data

48. Manage collection
Determine a collection schedule
Work out agreements with other departments that
have needed data
Are you currently collecting data or indicators
that are no longer relevant?
How long do you need to keep
data?
Who is responsible?
EXERCISE

49. Translate
Data into
Outcomes
& Impacts

50. Interpret data So…..what?
◦ What does the data
say? What doesn’t it
say?
◦ Examine it from various
angles
◦ What is the progress
towards the goals &
objectives?
EXERCISE

51. Define outcomes & impact
What will success look like?
And how will you “get” there?
OUTCOME IMPACT
INPUT OUTPUT

52. Define outcomes & impact
• Increased skill
• Know-how or know-that
• Change in status
• Change in behaviour or
attitude
• Increased cases
• Increased profits
• Increased recovery
rates
OUTCOME
User
perspective
IMPACT
Stakeholder
perspective
EXERCISE

53. Communicate
Results

54. To…
Remember
us?

55. Communicate results
Actually begins when you are understanding the
context
Your message: “we have contributed towards
your goals by……”
EXERCISE

56. Are they meaningful?
If your goals & objectives are meaningful for your
planning & decision-making, your measures will be
indispensable

57. What do you need to stop
doing?

58. Value
SatisfactionOperational
Sweet Spot

59. Thank you!
MOE
&
mhosse@markham.ca
REBECCA
rebecca@dysartjones.com