In 2008, the University of Southern Indiana began a new Doctor of Nursing Practice (DNP) program offered via distance education. Prior to its beginning, USI librarians collaborated with nursing faculty to identify ways to support the program. As a result of these discussions and using grant funding provided by the nursing department, the library added relevant electronic journals and open access journal collections Providing electronic access to the journals would enable distance education students to easily use these materials. In addition to existing full-text databases, the library also made available two open access journal collections, including Bentham Open Access Journals and PubMed Central Open Access Journals. The library also expanded its interlibrary loan (ILL) services by providing an article delivery (AD) service on a trial basis to all students enrolled in the program living more than 50 miles from campus. This three-year study sought to answer the question, “How well do the library’s new and existing resources and services support DNP student research?” The methods used by this study included a citation analysis of references listed in formal papers, an examination of usage statistics generated by the electronic resources supporting the program, a student satisfaction survey, and a review of students’ use of ILL and AD services. The citation analysis provided data indicating the sources used (number, type, currency, etc.) by the students and the library’s ability to supply them. Reviews of the usage statistics for the online journals and databases provided a comparison of how the use of these resources has been affected by the DNP enrollment. The student satisfaction surveys have given feedback on the students’ opinions and behaviors regarding the resources and services made available by the library. Presenters: Peter Whiting and Philip Orr, David L. Rice Library, University of Southern Indiana

1. Evaluating Library Support for
a New Graduate Program:
Finding Harmony with a Mixed
Method Approach
Philip Orr Peter Whiting
Distance Learning Librarian Serials Technical Services Librarian
David L. Rice Library David L. Rice Library
University of Southern Indiana University of Southern Indiana
porr@usi.edu pwhiting@usi.edu

2. Background / Significance of
Research

3. USI Doctor of Nursing Practice (DNP)
Program
• Begun in August 2008
• University’s first doctoral program
• Program requires a minimum of 36 hours beyond
the MSN and involves both coursework and clinical
practice
• Courses use a hybrid method involving on-campus
intensives and distance education delivery
• Curriculum designed to be completed within 2
(full-time) or 3 (part-time) years

4. The DNP Student
• Enrolled 64 students, 2008-2010
• 78% lived more than 50 miles from campus
• 49% lived outside Indiana

5. The DNP Student
Based on student satisfaction survey (35 responses):
• Majority are more than 40 years old
• Last educational degree program:
< 5 Years – 14
5-10 Years – 8
> 10 Years -- 13
• Employed as:
Nurse professional – 15 Nurse educator – 16
Nurse administrator – 2 Other – 2

6. DNP Student Participation in Research Project
Summary
2008 2009 2010 Total
Cohort Cohort Cohort
Number of Students
Enrolled 19 21 24 64
Number of Students
Participating in 10 5 15 30
Citation Analysis
Number of Students
Surveyed 16 13 20 49
Number of Students
Responding to Survey 14 10 11 35
Number of Students
Graduating 15 15 18 (proj.) 48

7. Library Collaboration with DNP Program
• Provided access to nine online journals – “Nursing
Nine”
• Provided access to two open access journal
collections:
PubMed Central
Bentham Open Access Journals
• Expanded interlibrary loan (ILL) service to include
trial article delivery (AD) service
• Provided an intensive face-to-face orientation to
Rice Library’s resources and services
• Maintained a “Library Support” section within the
Doctoral Student Virtual Center (Blackboard Site)

8. Health care professionals are users of
information
Accurate
Current
Understandable
Timely

9. Research Question
This study sought to answer the following question:
How well do Rice Library’s resources and services support
the information needs (study and research) of students
enrolled in the University of Southern Indiana’s DNP
program?

10. Research Methods

11. Four Methods Used To Evaluate the
Library’s Performance:
1. Examination of Database Usage Statistics
2. Survey of Student Satisfaction
3. Examination of Interlibrary Loan and Article
Delivery Usage
4. Analysis of Reference Citations in Student
Research Papers

12. Findings

13. Use of Nursing Databases
All Users
2006 - 2011
2006/2007 2007/2008 2008/2009 2009/2010 2010/2011
Database TOTAL
Searches Searches Searches Searches Searches
CINAHL with Full Text 37,763 68,672 57,141 62,665 85,606 311,847
Health Source: Nursing 25,946 37,531 35,526 45,858 48,080 192,941
MEDLINE 17,912 22,430 22,461 28,103 34,415 125,321
ProQuest Health and
48,839 61,952 60,500 69,203 78,694 319,188
Medical
Statistics
PubMed Central Not
Available
Statistics
Bentham Open Access
Not
Journals
Available
Total per year 130,460 190,585 175,628 205,829 246,795

14. College of Nursing and Health Professions
Enrollment
2,500
2,218
1,968
2,000
1,832
1,743
1,500 Undergraduates
Masters
1,000 Doctorate
357 410
500
285 317
19 31 48
0
2007 2008 2009 2010
Source: USI Fact Book --
http://www.usi.edu/opra/factbook/

15. Use of Nursing Databases
All Users
2006 - 2011
90,000
80,000
70,000
60,000
CINAHL with Full-Text
50,000
Health Source: Nursing
40,000
30,000 MEDLINE
20,000
ProQuest Health &
10,000 Medical Complete
-

16. Use of Nursing Databases
Percent of Change
2006 - 2011
2006/2007 2007/2008 2008/2009 2009/2010 2010/2011
Database
Searches Searches Searches Searches Searches
CINAHL with Full Text 37,763 68,672 57,141 62,665 85,606
Percent Change 82% -17% 10% 37%
Health Source: Nursing 25,946 37,531 35,526 45,858 48,080
Percent Change 45% -5% 29% 5%
MEDLINE 17,912 22,430 22,461 28,103 34,415
Percent Change 25% 0% 25% 22%
ProQuest Health & Medical 48,839 61,952 60,500 69,203 78,694
Percent Change 27% -2% 14% 14%

17. Use of “Nursing Nine” Journals
Ranked by Usage
All Users
2008 - 2011
7/08 – 7/11
Journal Title
Usage
1 Journal of Nursing Administration (JONA) 1,863
2 Nursing Research 531
3 Nurse Educator 456
4 Evidence Based Nursing 443
5 Applied Nursing Research 385
6 Journal of Professional Nursing 349
7 Nursing Outlook 266
8 Western Journal of Nursing Research 184
Clinical Scholars Review: The Journal of Doctoral
9 29
Nursing
TOTAL 4,506

18. Use of “Nursing Nine” Journals
All Users
2008 - 2011
1,876
1,570
1,056
7/08 - 7/09 8/09-7/10 8/10-7/11

19. Student Satisfaction Survey
• Distributed electronically by university’s Office of
Planning, Research, and Assessment
• Distributed at the end of the students’ second semester in
the program
• 49 students received survey / 35 students responded
(Response rate: 71%)
• Included 28 questions – 23 content questions + 5
demographic questions

20. Overall DNP Student Satisfaction
Satisfied /
Service Very Satisfied
Reference Services 96%
(Ex: Library orientation, e-mail reference, etc.)
Interlibrary Loan / Article Delivery Services 100%
Online Services 97%
(Ex: Library web pages, Journal Finder, etc.)
Resources 94%
(Ex: Electronic course reserves, e-books, etc.)

21. Q18: Databases Used
Database No. of
Responses
Most Frequently Used:
CINAHL w/FT / CINAHL+ w/FT 34
ProQuest Health and Medical Complete 34
MEDLINE 33
Health Source: Nursing/Academic Edition 31
Least Frequently Used:
Health Business FullTEXT 12
WorldCat 8
Health Source: Consumer Edition 6

22. Q19: Most Useful Databases
Database No. of
Responses
CINAHL w/FT / CINAHL+ w/FT 15
Health Source: Nursing/Academic Edition 10
ProQuest Health and Medical Complete 6
ProQuest Research Library 1
PsycINFO 1

23. Q20: Least Useful Databases
Database No. of
Responses
MEDLINE 7
PsycINFO 5
Health Source: Nursing/Academic Edition 4
ProQuest Health and Medical Complete 4
WorldCat 4
ProQuest Research Library 3
Health Source: Consumer Edition 3
Academic Search Premier 2
Health Business Full Text 2
CINAHL w/FT, CINAHL+ w/FT 1

24. Q21: Information Providers Used
Information Provider No. of
Responses
Internet 30
Rice Library 28
Journal Articles from Publishers’ Websites 21
Other Academic Libraries 15
Workplace Library 15
Other Corporate/Special Libraries 10
Other Information Providers 5
Public Libraries 4
Commercial Document Delivery Services 4

25. Q22: Action Taken When Article Was Not
Available in Full Text
Action No. of
Responses
Requested it through Rice Library’s interlibrary 19
loan service
Did not use the article / Chose another article 19
Checked on its availability in / used from another 15
library
Used a copy from personal journal collection 8
Did something else 7
Requested it from a publisher’s Website 5
Borrowed a copy from a colleague’s personal 2
journal collection
Requested it from a commercial document 0
delivery service
Did not have this problem 0

26. Q5: Interlibrary Loan / Article Delivery Use
Used Have Not Not Aware Of
Used
Interlibrary
Loan 22 (63%) 13 (37%) 0 (0%)
Article
Delivery 12 (34%) 23 (66%) 0 (0%)
Q6: Interlibrary Loan / Article Delivery Usefulness
Very Useful Somewhat Not at all
Useful Useful
Interlibrary
Loan 19 (86%) 2 (9%) 1 (5%)
Article
Delivery 12 (100%) 0 (0%) 0 (0%)

27. Interlibrary Loan / Article Delivery Usage
Submitted Cancelled Filled
Interlibrary
Loan 746 183 563 (75%)
Requests
Article
Delivery 17 2 15 (88%)
Requests

28. Citation Analysis
Overview
• Thirty students participated, 2008-2011
10 2008 cohort
5 2009 cohort
15 2010 cohort
• Examined a total of 229 papers involving 4,399
citations

29. Number and Percentage of Cited Sources by
Format
Percentage
Articles 67%
Web sites 18%
Books 13%
Other 1%

30. Number and Percentage of Cited Sources by
Date Range
N=4,391 Number Percentage
0 – 5 Years 2,895 66%
6 – 10 Years 1,020 23%
11+ Years 476 11%
Number and Percentage of Articles Cited by
Date Range
N=2,954 Number Percentage
0 – 5 Years 1,824 62%
6 – 10 Years 793 27%
11+ Years 337 11%

31. Number and Percentage of Cited Sources Which
Rice Library Made Available
(Excluding Web Sites)
Number
Available / Percentage
Number Cited
Articles 2,390 / 2,954 81%
Books 232 / 672 35%
Other 4 / 54 7%
TOTAL 2,626 / 3,680 71%

32. Number and Percentage of Cited Sources Freely
Available on the Web
Percentage
Articles 24%
TOTAL 25%

33. Journals Most Frequently Cited
Some
Ulrich’s
Citation Avail. in Content Ulrich’s Subject
Journal Title Doc.
Freq. RL Free on Classification
Type
Web
JONA (Journal of Nursing 1986- Med. Sci- Nurses
1 171 N J, A/S
Administration) present and Nursing
1983- Med. Sci- Nurses
2 Nursing Economic$ 124 N J, A/S
present and Nursing
1981 to
Med. Sci- Nurses
3 Journal of Advanced Nursing 93 1 year Y J, A/S
and Nursing
ago
Online Journal of Issues in 2002- Med. Sci- Nurses
4 61 Y A/S
Nursing present and Nursing
Nursing Administration 1986- Med. Sci- Nurses
5 50 N J, A/S
Quarterly 2005 and Nursing
Ulrich’s Document Type Codes: A/S = Academic/Scholarly; J = Journal; T =
Trade

34. Journals Most Frequently Cited
Some
Ulrich’s
Citation Avail. in Content Ulrich’s Subject
Journal Title Doc.
Freq. RL Free on Classification
Type
Web
1993- Med. Sci.- Nurses
6 Nursing Management 44 N J, A/S
2007 and Nursing
Journal for Nurses in Staff Med. Sci.- Nurses
7 44 n/a N J, A/S
Development and Nursing
New England Journal of 1961-
8 39 Y J, A/S Medical Science
Medicine present
1986- Med. Sci.- Nurses
9 Critical Care Nurse 38 Y J, A/S
present and Nursing
Journal of the American 1986- Gerontology and
10 37 N J, A/S
Geriatrics Society present Geriatrics
Ulrich’s Document Type Codes: A/S = Academic/Scholarly; J = Journal; T =
Trade

35. Journals Most Frequently Cited
Some
Ulrich’s
Citation Avail. in Content Ulrich’s Subject
Journal Title Doc.
Freq. RL Free on Classification
Type
Web
Journal of Continuing 1997-
11 Education in the Health 35 one year N J, A/S Medical Sciences
Professions ago
American Journal of Critical 1998- Medical Sci.-Nurses
12 33 Y J, A/S
Care present and Nursing
Journal of Gerontological 1986- Gerontology and
13 32 N J, A/S
Nursing present Geriatrics
1986- Med. Sci.- Nurses
14 Nursing Research 30 N A/S
present and Nursing
1994- Public Health and
15 Health Affairs 29 N J, A/S
present Safety
Ulrich’s Document Type Codes: A/S = Academic/Scholarly; J = Journal; T =
Trade

36. Frequency of Citation
“Nursing Nine”
Citation
Journal Title Frequency
1 Journal of Nursing Administration (JONA) 171
2 Nursing Research 30
3 Nursing Outlook 17
4 Journal of Professional Nursing 16
5 Applied Nursing Research 14
6 Western Journal of Nursing Research 8
7 Nurse Educator 1
8 Evidence Based Nursing 0
9 Clinical Scholars Review: The Journal of Doctoral
Nursing 0
TOTAL 257

37. Discussion / Implications

38. Evaluation of Library Performance

39. Library Orientation

40. Promote the Use of Interlibrary Loan
and Article Delivery Services

41. Emphasize Proper Use of APA
Citation Style

42. Promote the Use of “Nursing Nine”
Electronic Journals

43. Adjust Journal Offerings Supporting
the Program

44. Expand the Nursing Databases
Available

45. Extend Article Delivery Services to
Other Graduate Programs

46. Thank You For Attending
Today’s Session
Questions?
Philip Orr Peter Whiting
Distance Learning Librarian Serials Technical Services Librarian
Rice Library, USI Rice Library, USI
porr@usi.edu pwhiting@usi.edu