Dr. Ross Todd's presentation to librarians at the NYC DOE Spring Summit, May 13, 2014.

1. School Libraries and Learning Outcomes:
Measuring your Impact
Dr Ross J Todd
Center for International Scholarship in School Libraries (CISSL)
School of Communication & Information
Rutgers, The State University of New Jersey
rtodd@rutgers.edu
www.cissl.rutgers.edu
www.twitter.com/RossJTodd
www.facebook.com/RossJTodd

2.  Report published March
2011, tabled at Parliament 23
May 2011.
• “6.10 - The Committee has
been struck by the breadth
of anecdotal evidence that it
received demonstrating the
significant contribution to
learning outcomes in primary
and secondary schools that a
fully resourced school
library, when staffed by a
fully qualified and active
teacher librarian, can make.
This supports the findings of
Australian and international
research in this area.”
(p.118)

6. Developing Capacity as Evidence-Based
Practitioners
“We recommend that the profession as
a whole needs to develop the capacity
to articulate needs from
research-based evidence
and local evidence
collected in the school.”

7. https://www.facebook.com/groups/EBPforSL/
https://www.facebook.com/groups/EBPforSL/

8. Evidence-Based Practice
Gathering evidence in YOUR local school
You are able to provide convincing evidence that
answers these questions:
“What differences do my school library and its learning
initiatives make to student learning outcomes?
“What are the differences, the tangible learning
outcomes and learning benefits of my school library”?
EVIDENCE = ADDING VALUE: Learning, Living, Life

9. Holistic Model of Evidence-Based Practice for School Libraries
Evidence FOR
Practice
FOUNDATION
INFORMATIONAL
Existing formal research provides the essential
building blocks for professional practice:
Evidence IN Practice
Applications /
Actions
PROCESS
TRANSFORMATIONAL
Locally produced evidence;
Data generated by practice is meshed with
research-based evidence to provide a dynamic
decision-making and learning environment:
Librarian’s evidence / thinking / wisdom
Evidence OF
Practice
Results – impacts &
outcomes
OUTCOMES
FORMATIONAL
user-reported evidence
learner changes as result of inputs, interventions,
activities, processes

10. Evidence-Based Strategies:
Emphasis on Information
• Number of classes in the library
• Number of library items borrowed
• Number of students using the library at lunch times
• Number of items purchased annually
• Number of web searches
• Number of books lost
And who can do this?

11. Emphasis on Knowledge
Understanding how school libraries help kids
learn: Learning outcomes in terms of
– Knowledge outcomes – deep mastery of content
through inquiry; mastery of CCSS
– Information processes: transformation of
information into new knowledge through inquiry:
key to CCSS
– Information technology: using tools to construct
representations of knowledge and to present new
knowledge: ICT impact
– Reading for learning in digital and print formats
– Independent learning: knowledge transfer
– Attitudes and values of information, learning
– Self concept and personal agency
And who can do this?

12. Mission Statements & Policy Documents are
Contexts for Evidence
School Library Policy and Mission Statements:
•Redefine school libraries from Place to Experience:
Learning Commons; iCenters; Knowledge Place
•Shift in focus from School Library as collections,
access, information literacy, to inquiry, thinking,
knowledge development and learning outcomes
Test criteria:
“Celebrate the Understood, not the Found”
Does your mission and policy statements do this?

14. What is a School Library?
The school library is the school’s physical and virtual
learning commons where reading, inquiry, thinking,
imagination, discovery, and creativity are central to
students’ information-to-knowledge journey, and to their
personal, social and cultural growth.

15. School Libraries as Verbs
"Libraries are the verbs in the content standards. Wherever verbs
such as read, research, analyze, explore, examine, compare, contrast,
understand, interpret, investigate, and find appear in the standards,
Teacher Librarians and library resources are involved."
(Oxnard Union High School District)
http://www.ouhsd.k12.ca.us/lmc/ohs/stronglib/StrongSLMP.ppt

16. NJ Research Study 2009-2012
• School libraries as pedagogical centers
• Key role of the school librarian as co-teacher
implementing curriculum
• Creative pedagogies centering on inquiry, critical
thinking and development of knowledge
• School library as connector: school and community
• Digital and ethical citizenship

17. EBP
Knowledge
Outcomes

18. Intellectual Quality
Intellectual Agency
Deep knowledge
Deep understanding
Problematic knowledge
Critical thinking
Meta-language
Substantive communication
Personal Agency
Self Confidence
Willingness to take risks
Trying new ideas and practices
Independence
Autonomy
Social and Cultural Agency
Respect for different values, cultural knowledges and viewpoints
Team building, collaboration, negotiation and decision making:
inclusivity
Knowledge integration: conceptual coherence and integration
Connect with current and future lives
Social and ethical values: digital citizenship
Gore, J.M. et al.Towards better teaching: productive pedagogy as a framework for teacher education. Teaching and Teacher
Education 20 (2004) 375–387
Curriculum: Dimensions of Evidence

19. Examples of Knowledge Claims
• Following an inquiry-based unit of work, 70% students
showed improved ability in formulating essential
questions that directed their inquiry as compared to their
previous research task
• As a result of instructional interventions focusing on the
development of arguments, students showed stronger
ability to identify claims, provide supporting evidence and
to identify rebuttals
• Students’ final products showed improved ability to
analyze and synthesize information
• Students’ research reports showed improved ability to
draw conclusions and state implications of their findings
• Students’ presentations showed ability to present
different viewpoints and to provide a strong and
supported case for their own position

20. Example of Knowledge Claims
• 93% of the students showed mastery of strategies for
evaluating websites for misinformation and bias
• Following instructional interventions that focused on
establishing the quality of websites, 100% of the students’
bibliographies showed use of high quality websites
• Based on a pre-test of initial and final knowledge of the
Grade 8 science curriculum topic, students’ knowledge of
their topics changed from descriptive and random listing of
facts to statements that showed explanations, conflicting
knowledge and making predictions
• Students’ products showed increasing complexity of
language used to describe their knowledge, and the
ordering of this knowledge into conceptually coherent units
• 83 % of the class show improved ability in thoughtfully
analyzing and evaluating alternative points of view

21. STUDENTS AS GOOD RESEARCHERS
• Resource-based
capabilities
• Knowledge-based
capabilities
Reading-to-learn
capabilities
• Thinking-based
capabilities
• Learning
management
capabilities
• Personal and
interpersonal
capabilities

22. Tools and measures for charting
and documenting evidence

23. NJ Schools Study: Did they learn anything?
•10 New Jersey public schools
•Experienced and expert school librarians
•10 teacher-school librarian teams working on
curriculum inquiry projects with 17 classroom teachers
•574 students in Grades 6 – 12
•Data collected over four weeks:
1. Writing Task 1 (at initiation of inquiry unit)
2. Writing Task 2 (at midpoint of inquiry unit)
3. Writing Task 3 (at conclusion of inquiry unit)
4. Search Journal Log

24. Writing Tasks
Writing task 1 and 2 consisted of the following questions
1. Write the title that best describes your research project at this
time.
2. Take some time to think about your research topic. Now write
down what you know about this topic.
3. What interests you about this topic?
4. How much do you know about this topic? Check (✓) one box
that best matches how much you know. Nothing, Not much, Some,
Quite a bit and A great deal
5. Write down what you think is EASY about researching your topic.
6. Write down what you think is DIFFICULT about researching your
topic.
7. Write down how you are FEELING now about your project. Check
(✓) only the boxes that apply to you. Confident, Disappointed,
Relieved, Frustrated, Confused, Optimistic, Uncertain, Satisfied,
Anxious or Other.

25. Additional Questions at Writing Task 3
1. What did you learn in doing this research project?
(This might be about your topic, or new things you
can do, or learn about yourself)
2. How did the SCHOOL LIBRARIAN help you?
1. How did the TEACHER help you?

26. CISSL WEB SITE
• http://cissl.rutgers.edu/joomla-license/impact-
studies/57-impact-studies-slim
• Student Learning through Inquiry Measure (SLIM)
SLIM Handbook
•
SLIM Reflection Instruments and Scoring Guidelines
SLIM Scoring Sheet

27. Analysis of student bibliographies
• Diversity of choice of sources
• Depth / levels of knowledge
• Accuracy of citations
• Relevance to learning task
• Use of multiple formats
• Engaging with state-of-the art
knowledge – recency / accuracy
• Digital citizenship: ethics

28. Rubric Strategies
• Students’ performance in final products are scaled according to a set
of criteria that clearly define what is the range of acceptable to
unacceptable performances and/or information products look like
• Compare with previous assignments where no instructional
intervention took place See Kathy Schrock’s Guide to Everything
• http://www.schrockguide.net/

29. CriticalThinking:MakeThinkingVisible
www.insightassessment.com/pdf_files/​Rubric%20HCTSR.pdf

30. Evidence through formal
feedback strategies
Examples
• simple feedback survey every term on what the
library does “best” and “least” to help students with
their school work
• Feedback at end of instructional unit: what helped
and did not help
• Feedback on quality of resources
• Feedback on what students could do better at
• Use of “How School Libraries Help” Survey instrument
(Ohio / Australia)
• Judicious use of evidence: How learning is enabled
and hindered

31. Student Learning Through Ohio School
Libraries
“Now, remember one time when the
school library really helped you. Write
about the help that you got, and what
you were able to do because of it”.
TODD, R. & KUHLTHAU, C. (2005). Student learning through Ohio school
libraries, Part 1: How effective school libraries help students. School
Libraries Worldwide, 11(1), 89-110.

32. Students’ Voices
•1015 I I would have never have found the sources I
needed for the paper if not for the school library, the
public library, and the helpful people who staff those
places. They even showed me steps to work through
to do the research and complete it. They ran some
classes specifically for us and they were very very very
helpful
•1075 Well one time was when we had to do a report
on Animals and I had no clue how to find information
about my animal. So Mrs. X helped me find the
information on the computer. On the internet if its
true or false – to learn that is very important at school.

33. Students’ Voices
•3532 I was working on History project and we had to
have several sources (primary documents) and the
librarians instructed the students on how to go about
finding the information we needed and compiling it into
something worthwhile. I was able to combine
everything together and earn a good grade.
•100 I needed help doing a project for government that
had to do with presidents and they had so many books
and then the librarian helped me find web sites. But
then they gave me ways of sorting through all the ideas
to extract the key points so I could get my head around
it all

34. •433 It helped me find info on racism for a 10th grade
project, and made me really think about that, especially I
didn’t realize how racist some of my ideas were
•6256 Sometimes I argue with my parents about things
and use the library to check if my opinions are true
•1408 One time, I wanted books on Teen Suicide and they
were able to get some for me. It was helpful of them as
my cousin died that way and I could figure it out a bit
more for me.
•6110 I guess I’ve discovered one thing. When I do my
research well, and do the proper thing with note cards
and writing in my own words, I seem to just get to know
the stuff and that makes a big help when I talk about the
stuff in class.
Students’ Voices

35. Personal
Reflection
•What did I learn? What worked well?
•What pleased me?
•How do I know that it was successful?
•Who can/did I share my success with?
•What did I do that helped me to:
- prepare for the task
- create new ideas
- practise new skills
- improve existing skills
- modify my learning habits
- find relevant information or materials
- organize information or materials
- correctly summarize information
- understand unfamiliar ideas
- take relevant notes
- use my existing knowledge or skills
- represent information in meaningful ways?
•What could I do differently next time?
•What factors influenced my ability to learn?
•What might help me learn more about this?

37. Teacher-Related Strategies
• Number of collaborations
• Instructional focus of collaborations
(curriculum standards and inquiry-
based interventions
• Subject / grade levels
• Teacher observations of student
engagement
• Teacher summary of outcomes
• Teacher summary of benefits

38. Planning a change
Acting & observing the
process & consequences
of the change
Reflecting on these
processes &
consequences
Replanning
Acting and observing
again
Reflecting again
Replanning...
ACTION RESEARCH
(Kemmis & McTaggart, 2005)

39. TOOLS FOR DATA GENERATION
collection & analysis

40. http://www.zoomerang.com/

41. Sticky note & collaborating apps
• http://en.linoit.com/ to collect student
feedback, capture data and publish as is
• Padlet http://padlet.com/
• Thoughtboxes https://www.thoughtbox.es/
• Google Docs (Drive) https://docs.google.com/
• Qikpad http://qikpad.org/
• Zoho http://www.zoho.com/
• Edmodo http://www.edmodo.com/

42. Blogs & wikis & pics
• Edublogs http://edublogs.org/
• Wordpress http://wordpress.com/
• Blogger http://www.blogger.com/
• Wikispaces http://www.wikispaces.com/
• Glogster http://www.glogster.com/
• Pbworks http://pbworks.com/
• Flickr http://www.flickr.com/
• Photobucket http://photobucket.com/

43. presentation &
publication
Tools for
reporting

44. LibGuides
http://www.theunquietlibrary.libguides.com/may2011

45. Pdf in slideshare
http://www.slideshare.net/jlagarde/mgms-annual-report-2010-11 by Jennifer
LaGarde

46. Issuu.com Self publishing
http://issuu.com/joycevalenza/docs/sept11nopw3

47. Powerpoint slides as pdf
http://www.hopkintonschools.org/hhs/library/2012AnnualReport.pdf

48. Infographics
http://informania.wordpress.com/2012/04/15/infographic-monthly-library-
report/ using http://piktochart.com/
http://piktochart.com/
http://visual.ly/

49. https://www.facebook.com/groups/EBPforSL/

50. Shout it out loud!
The essentials of dissemination
•School library website
•Reports
•Newspaper articles
•Research journals, featured research
columns
•Teacher journals, discipline-based
newsletters
•Exhibitions
•Photo voice & digital story telling

51. “If living is
seeing
I’m holding my
breath
In wonder –
I wonder
What happens
next?
A new world, a
new day to see”