Amy Hofer & Margot Hanson. From Loex-of-the-West 2010 conference program: "We need evidence of our value to justify the library budget, gain buy-in from faculty, and please accreditors, but the process of gathering meaningful evidence can be quite a challenge. After all, were you trained to do this kind of research? Neither were we, but the Assessment Coordinator that our university hired in preparation for a WASC visit guided us through our study design. Drawing on this relationship with a neutral expert outside the library, two business librarians at Golden Gate University designed an effective learning assessment of our instruction to our school’s intensive ESL program. We’ll use the valuable lessons we learned as a starting point for small-group exploration of some real-world assessment territory."

1. TERRA INCOGNITA WITH NO GPS:
A case study approach to mapping assessment
Amy Hofer, Distance Learning Librarian, Portland State University
Margot Hanson, Web Services Librarian, Golden Gate University
LOEX-of-the-West 2010, Mount Royal University, Calgary, Alberta

2. ABOUT US!
‣Golden Gate University
‣The IL landscape
‣Our assessment needs

3. ABOUT YOU!
‣Hello my name is...
‣Your school
‣Pressing assessment issues
(past, present, or future)

5. PROGRAM EVALUATION
Study your program to find out whether it’s meeting your goals

6. LEARNING ASSESSMENT
Study whether students gained knowledge, skills, and abilities as
a result of your instruction

7. PLEASE DISCUSS
‣Think about an assessment
project at your organization.
‣Learning assessment or
program evaluation?
‣Right tool for the job?

9. LESSONS FROM LISA
‣ For what general purpose is the evaluation being
conducted?
‣ What are the specific goals of the study?
‣ Who is the audience for the research findings?
‣ What kinds of information are needed to make the
decisions you need to make?
‣ What data collection method will be appropriate?

10. OUR STUDY

11. POPULATIONS: FALL 2009
PLUS Undergraduate
‣ English language learners ‣ 43% ELL in our sample
‣ 8 librarian visits ‣ 3 librarian UG class visits out of
61 UG classes offered Fall 09
‣ 2 librarians per visit (none in our sample)
‣ 18.5 hours total ‣ 30-60 minutes per visit

12. METHODS
PLUS Students Undergraduate
Students
‣ 43 completed tests before
research instruction (paper) ‣ 51 completed tests (paper and
online)
‣ 40 completed tests after
research instruction (paper) ‣ Sample drawn from 9 classes
with agreeable faculty
‣ 39 writing samples

13. OUR AREAS OF INQUIRY
‣ A. Dostudents identify appropriate sources for their
research need?
http://tinyurl.com/lotw2010 of the
‣ B. Do students ask a great question that fits the scope
assignment and requires research and analysis to answer?
‣ C. Do students demonstrate academic integrity by correctly
citing sources?
‣ D. Do students use the library’s resources and services as
part of the research/writing process?
‣ E. Qualitative info about students

14. Didn't Get It Got It Got It & Ran With It!
•Uses a range of formats,
including reports, articles and
•Uses only one format •Uses a range of formats, websites
including reports, articles and
Appropriate •Not relevant or authoritative websites
•Includes resources only available
Sources •Heavy reliance on one or two •Uses only resources presented
from the library, but went beyond
You don’t have to read this -
sources resources presented in class
in class
•Sources are relevant and
substantive
it’s online!
•Topic too narrow or too broad •Research question is of
•Uses critical thinking and
creative approach to topic, which
is of appropriate scope
for assignment appropriate scope for assignment
Topic
• Uses report format, as a simple •Includes original analysis •Research question requires
Development original analysis
statement of facts, missing original •Question can be answered with
analysis resources presented in class •Question requires additional
sources beyond what was
presented in class
•Errors in APA formatting •APA correctly and consistently
•Incorrect use of paraphrasing, •Consistent APA formatting in- used for in-text and reference list
quotations, or summarizing
Citation •Quotes are dumped into paper text and reference list citations
and not incorporated into analysis • •Citations are distributed evenly
Originality report reveals less
than 10% exact match throughout paper, contribute to
•Originality report reveals analysis, and support conclusions
problems with exact matches

15. FINDINGS

16. OMG, data!

17. Most Improved Players
PLUS pretest PLUS posttest Undergrad
70%
60%
66%
60%
50% 54%
Test Score
40%
30%
20%
10%

18. What the PLUS work samples told us
Got it & ran with it!
Got it
Didn’t get it
Appropriate sources Topic development Citation
39
10 8 8
Student Papers
31
23 13
18 21
16
8 18
11 10

19. Undergrads: How many times during the semester
do you get help from a librarian?
2% 6%
22% More than 5 times
2-5 times
46% One time
Never
I don’t know
24%

20. Do you think that the library and its services
contribute to your academic success at GGU?
Yes Somewhat Maybe No I don’t know
Undergrad PLUS
10% 2% 5%
13%
18%
51%
20% 83%

21. Do students know what they know?
20
15
Students
10
5
0
0 1 2 3 4 5 6 7 8 9 10
Test Scores
My skills are excellent
My skills are good
My skills are fair
My skills are poor
I don’t know

22. NEXT STEPS
‣Connections to your own
situation or small-group
discussions?
‣Strategic outreach - getting a
foot in the door?
‣Questions for us?

23. Image sources:
http://www.flickr.com/photos/telmo32/2967379555
http://www.flickr.com/photos/rhiannonstone/3782706061
http://www.wordle.net
http://www.flickr.com/photos/princessashley/1546575032
http://www.flickr.com/photos/gadl/320300354
http://www.flickr.com/photos/mcaven/4319012152
http://www.flickr.com/photos/huffstutterrobertl/3269126061/
http://www.flickr.com/photos/quinnanya/3183174451/

24. Thank you!
Amy Hofer Margot Hanson
ahofer@pdx.edu mhanson@ggu.edu