Radcliff, S. & Wong, E. Librarians Leading the Charge: Collaborating with Faculty to Design Evidenced-Based Instruction. Presented at California Academic and Research Libraries (CARL) 2014 conference. Librarians at Saint Mary’s College of California will present part two of their study, following their 2012 CARL conference presentation: “English Composition Students: How Are They Using Their Sources?” Having discovered through this research that students do have problems paraphrasing, quoting, integrating and citing their sources, Librarians, in collaboration with English Composition faculty, designed a new study to test out three instructional methods (behaviorist, cognitive and social constructivist) on teaching integration and citing of sources in six sections of advanced English Composition classes. Results of the three methods will be evaluated through pre/post test scores and correlated with a content analysis of the students’ research papers. The results of the content analysis will also be used to compare past studies’ results and presented to English Composition faculty in part three of the librarians’ study. All three methods and the lesson plans will be made available for faculty to use with the knowledge of how effective the methods are in relation to specific student learning outcomes.

1. Librarians Leading the Charge:
Collaborating with Faculty to Design
Evidenced-Based Instruction
Presenters:
Sharon Radcliff and EliseY Wong
Saint Mary’s College of California
1

2. What are the Effects ofVaried
Instructional Methods?
 Behaviorist, cognitivist, and social
constructivist instructions
 Pre/post tests comparison
 Impact on student performance on citing
and integrating sources
2

3. How Well Composition Students
Cite and Integrate Sources?
 In-text citations and bibliographies
 Types of sources used in bibliographies
 Paraphrasing vs. quoting
 How quotations are integrated
3

4. Our Study Objectives
 Assess the effectiveness of different
instructional methods
 Students' citing behaviors
 Examine how sources are integrated
 Evaluate students' citation performance
 Students’ writing proficiency and faculty
expectation
4

5. About Saint Mary’s College
 Catholic, Lasallian, liberal arts college
 Undergraduate and graduate schools
 Total enrollment: 4257
 Total full time students: 3746
 Number of full-time faculty: 213
 Student-faculty ratio: 13:1
 Average class size: 20
5

6. Who are SMC Students?
 Female: 59%; Male: 41%
 White: 43%, Latino: 25%,Asian: 14%,
African-American/Black: 2%, others: 16%
 87% freshmen from California
 99% of freshmen living on campus
 Tuition and fees: $39,890
 % of full-time undergraduate receiving
financial aid: 86%
6

7. SMC New Core Curriculum
Information Evaluation and
Research Practices
Students will learn to judge the
authenticity, validity, reliability, and
originality of the sources of information
they use
7

8. SMC Composition Program
Learning outcomes for ENG 5 includes:
 Develop search strategies and use library
resources to locate relevant materials
 Practice evaluating sources critically
 Evaluate and synthesize evidence
 Properly integrating and citing sources
8

9. Learning Theories Overview
 Gredler, M. E. (2009). Learning and instruction:
Theory into practice (4th ed.). New Jersey:
Merrill.
 Hergenhahn, B. R. & Olson, M. H. (2005). An
introduction to theories of learning (7th ed.).
New Jersey: Prentice.
 Leonard, D. C. (2002). Learning theories,A to
Z. Connecticut: Greenwood.
 Schunk, D. H. (2008). Learning theories :An
educational perspective (5th ed.). New Jersey:
Merrill/Prentice.
9

10. Behaviorist Models
 Learning is shaped by a change in behavior
 E.Thorndike, J.Watson, & B. F. Skinner
 Classical and operant conditioning
theories
 Stimulus-Response, positive/negative
reinforcement, rewards & punishment,
behavior modification
10

11. Cognitivist Models
 Learning progresses in stages
 Component display theory (D. Merrill);
Social cognitivist theory (A. Bandura);
Stage theory of cognitive development (J.
Piaget);
Elaboration theory (Reigeluth);
Conditions of learning (R. Gagne)
 Information processing schemas, mastery
learning, concept mapping, scaffolding
11

12. Social Constructivist Models
 Learning is interactive, reciprocal, and
collaborative
 L.Vygotsky, J. Bruner, D. Kolb, C. Rogers
 Problem based learning; Discovery
learning; Social development theory
 Inquiry based, active learning, learner-
centered, experiential/situated learning
12

13. Studies on LearningTheories and
Instruction
 Blummer, B., Kenton, J. M., & Liyan, S. (2010).The Design and
Assessment of a Proposed Library Training Unit for
Education Graduate Students. Internet Reference Services
Quarterly, 15(4), 227-242.
 Miranda, M.V. (2009). Creating the Successful Community
College Student: Using Behaviorism to Foster
Constructivism. Community College Enterprise, 15(1), 21-38.
 Sulaiman, J. J., & Dwyer, F. F. (2002).The Effect ofVaried
Instructional Text Design Strategies on the Achievement of
Different Educational Objectives. International Journal Of
Instructional Media, 29(2), 215-224.
 Yilmaz, K. (2008). Social Studies Teachers'Views of Learner-
Centered Instruction. European Journal OfTeacher
Education, 31(1), 35-53.
13

14. Highlights from the Studies
 A collaboration between faculty and librarian in a
library training unit for information literacy using
direct instruction and inquiry-based approach
(Blummer, 2010)
 Constructivism is not always better than
Behaviorism (Miranda, 2009)
 Strategies have varying influences on student
achievement depending on types of learning
objectives (Sulaiman, 2002)
 Faculty participants identify their teaching style
more with cognitive and constructivist approach
than the behaviorist approach (Yilmaz, 2008)
14

15. Research Background
 SMC librarians have done two major bibliographic
studies and two internal citation study over the
past 10 years.
 The first bibliographic study was done on 9
Composition sections in 2004; the second was
done on 13 sections in 2006.
 The first internal citation study was a pilot study
of 25 papers done in 2008. In the second internal
citation study (2010), 85 papers were analyzed.
15

16. Library Instruction Classes
 SMC librarians traditionally do an
instruction session for every section of
ENG 5 (25-28 sections)
 Library session is now required for ENG
4
 Library instruction for ENG 4 and 5
includes group work and tutorials on
search strategies, evaluating and citing
sources.
16

17. Benefits of Collaboration
 Curriculum design
 Instruction
 Research
17

18. Research Design
 IRB approval
 Composition faculty survey
 Lesson plan/instructional methods
 Data collection (Spring 2013)
◦ Pre/post tests
◦ Student papers
18

19. Faculty Survey
 15 questions regarding their classroom
practice on MLA citing and integrating
sources (12/28 responses)
 Instruction via group work, lecture, and
individual exercises
 All devote class time on integrating sources;
most on MLA; some also give take-home
assignments
 All require students to submit work-cited
page; most require students to consult
specific types of sources
19

20. Survey highlights
 All respondents devote class time on
integration of sources; 10 give take home
assignments
 11 devote class time on MLA; 8 give take home
assignments
 7 give at least one research paper
 11 require students to consult specific types of
sources
 9 include quality of work cited page into grading
rubric
 11 devote class time on ethical use of sources
20

21. Instructional Methods
 Integrating sources, quoting, paraphrasing, MLA
citation (in-text and bibliographic)
 Six participating faculty members randomly assigned
to teach one of the following:
◦ Behaviorist: Mostly direct lecture; students complete
exercises individually in class; candies were used as
incentives
◦ Cognitivist: Series of mini-lectures/discussions, inter-
woven with students working on examples with
increasing levels of difficulty
◦ Social constructivist: Brief introductory lecture;
students then work on examples in groups
21

22. Pre/Post Tests
 Tests were given before and after the
instruction during one class period
 10 questions per test
 Learning objectives of both tests:
◦ Understand the purpose of citing
◦ Identify the key components of a citation
◦ Identify basic format in MLA for in text and
bibliographic citations
◦ Learn how to identify good uses of paraphrasing
and quoting as part of integrating material from a
source into a research paper
22

23. Pre-Test Score Results by Learning
Theory Group
23

24. Post-Test Scores by LearningTheory
Group
24

25. Results: Behaviorist Pre/Post Test
Scores
25
Behaviorist
n=33
Pre-Test
Score
Post-Test
Score
Difference
Mean 8.09 8.70 .61
SD 1.55 1.13

26. Results: Cognitivist Pre/PostTest
Scores
Cognitivist
n=35
Pre-Test
Score
Post-Test
Score
Difference
Mean 7.63 9.06 1.43
SD 1.37 .94
26

27. Results: Social Constructivists
Pre/PostTest Scores
Social
Constructivist
n=35
Pre-Test Post-Test Differences
Mean 8.29 8.86 .57
SD 1.78 .810
27

28. Results: Pre/PostTest Scores for all
3 Groups
All groups
n=103
Pre-Test
Score
Post-Test
Score
Difference
Mean 8.0 8.87 .87
SD 1.4 .97
28

29. Whole Group Increase in Scores
from Pre-Test to Post-Test
 After performing a t-test on the two results:
pre and post test score means; it was
determined that this change was statistically
significant at the .05 level.
 So although no difference in performance could
be detected between the learning theory
groups, all the mean test scores improved test
after the treatment (instruction session)
29

30. Analysis of Results
 The post-test scores went up by varying
amounts (behaviorist: .66; cognitivists: 1.43;
social constructivists: .57) for all learning
theory groups.
 The cognitivists group showed the largest
gain; however after a statistical analysis was
performed (2-way ANOVA) it was
determined that the difference between
groups was not statistically significant at
p<.05
30

31. Implications
 Providing students with lessons based in the
behaviorist, cognitivist and social
constructivist methods improves their
performance on tests measuring citation
skills.The test results are somewhat
mitigated because the results for each group
did not show a normal curve but were
negatively skewed, (students tended to score
high), so results must be viewed with
caution.
31

32. Results: Content Analysis of Final
Papers by Learning Theory Groups
We analyzed all papers and reporting:
 Content of bibliographies by item type.
 Percentage of bibliography items cited in paper
 Number of factual items in paper not cited
 Percentages of citations that are paraphrased,
quoted and if quoted, introduced and /or analyzed.
 Rubric scores for format of in-text and
bibliographic citations.
32

33. We collected papers only from
students present during the session
 Behaviorist n=33
 Cognitivists n=33
 Social Constructivists n=30
33

34. Bibliography Content: Behaviorist
34
15.71%
46.23%
3.26%
30.00%
4.20%
0.61%
Behaviorist
Web Journal Media Book Interview Other

35. Cognitivist
35
19.75%
60.19%
1.71%
18.36%
Cognitivist
Web Journal Media Book Interview Other

36. Social Constructivist
36
17.20%
50.98%
8.05%
19.76%
3.00%
1.01%
Social Constructivist
Web Journal Media Book Interview Other

37. Percentage of Bibliography Cited
37
93.02%
85.75%
81.10%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Behaviorist Cognitivist Social Constructivist
Percentage of Bibliography cited in Paper

38. Rubrics for bibliographies:
simple & complex citations
 4 (Excellent): Mostly free of error except for minor
punctuation, and small formatting problems. No
elements should be left out of a citation
 3 (Very Good): Simple citations may have very few
minor punctuation and formatting errors; complex
citations may contain minor errors in punctuation and
order; 1-2 omission of minor elements are allowed
 2 (Fair): Some simple citations may have errors in
punctuation, order, or minor omission of elements;
complex citations may be missing major elements
 1 (Poor):Widespread problems in both simple and
complex citations showing errors in punctuation, order,
and omission of major elements
38

39. Bibliography Quality Rank
39
3.27
3.06
3.44
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
Behaviorist Cognitivist Social Constructivist
Bibliography Quality Rank

40. Rubrics for internal citations
 4 (Excellent):Almost perfect, minor
punctuation problems
 3 (Very Good): Occasional elements
omitted; bib source can still be identified
 2 (Fair): Some citations do not uniquely
identify sources in bibliography; no more
than 1-2 citations completely absent
 1 (Poor): Some or most citations completely
absent
40

41. In-Text Citation Rating
41
2.42
2.65
2.37
0.00
0.50
1.00
1.50
2.00
2.50
3.00
Behaviorist Cognitivist Social Constructivist
In-text Citation Rating

42. Percentage of data not cited
42
13.31%
11.21%
13.64%
0.00%
2.00%
4.00%
6.00%
8.00%
10.00%
12.00%
14.00%
16.00%
Behaviorist Cognitivist Social Constructivist
Percentage of data not cited

43. Percentage Paraphrased
43
49.28% 50.34%
69.77%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
Behaviorist Cognitivist Social Constructivist
Percentage Paraphrased

44. Percentage Quoted
44
50.72%
49.66%
30.23%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
Behaviorist Cognitivist Social Constructivist
Percentage Quoted

45. Percentage Quoted with Intro Only
45
14.84%
2.45%
12.86%
0.00%
2.00%
4.00%
6.00%
8.00%
10.00%
12.00%
14.00%
16.00%
Behaviorist Cognitivist Social Constructivist
QuotedWith Introduction only

46. Percentage Quoted with Analysis
Only
46
14.84%
5.39%
3.57%
0.00%
2.00%
4.00%
6.00%
8.00%
10.00%
12.00%
14.00%
16.00%
Behaviorist Cognitivist Social Constructivist
Percent Quoted with Analysis Only

47. Percentage Quoted with No Intro
or Analysis
47
2.75%
3.92%
1.43%
0.00%
0.50%
1.00%
1.50%
2.00%
2.50%
3.00%
3.50%
4.00%
4.50%
Behaviorist Cognitivist Social Constructivist
Quoted with no Intro or analysis

48. Percentage Quoted with Intro &
Analysis
48
67.58%
88.24%
82.14%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Behaviorist Cognitivist Social Constructivist
Quoted with Intro and Analysis

49. Paraphrased & Quoted with Intro &
Analysis
49
84.68%
94.15% 94.58%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Behaviorist Cognitivist Social Constructivist
Paraphrased and Quoted with Intro and Analysis

50. Discussion
 The most sources cited by all 3 groups
◦ Journal articles
 % bibliography cited in paper
◦ Behaviorist 93% (highest)
 % of claims not cited in paper
◦ Cognitivist 11% (lowest)
 Quality of bibliography
◦ Social Constructivist 3.44 (very good)
 In-text citation rating
◦ Cognitivist 2.65 (fair)
50

51. Discussion
 Paraphrasing vs. quoting
◦ Social Constructivist 70/30
 Quotes with introduction & analysis
◦ Cognitivist 88%
 Paraphrasing and proper integration of
quotes (P + IQA)
◦ Social Constructivist 95%
 The winner…
◦ Cognitivist
◦ and Social Constructivist
51

52. Limitations of This Study
 This study was limited to Saint Mary’s
Students; random selection was not possible
as we used already formed sections of
English Composition. Results may differ for
other groups of students at other
institutions.
 Statistical testing of content analysis of final
papers (between groups) was not analyzed.
 Teacher variability might have impacted the
differences between learning theory groups.
 Transference from knowledge to practice
was not explored.
52

53. Preliminary data comparison:
2010 and 2013
2010 2013
Paraphrasing/
Quoting
60/40 50/50 (B, C)
70/30 (SC)
Bibliography cited 78% 86.8%
Internal citations 14% not cited 12.69% not cited
Proper integration
of quotes (IQA) 53% 79.23%
53

54. Future Analysis
We will compare this whole group to the
papers collected in 2010 to see if there are
statistically significant changes in terms of
using, integrating, and citing sources.
 Radcliff, S., & Wong, E. (2012). English
Composition students: How are they using their
sources? Paper presented at CARL
Conference 2012, San Diego, CA, 5-7 April.
54

55. Direction of Future Studies
 More research could be done in this area by
applying learning theory to instructional design
and assessing the results. Using a standardized
or previously piloted test is desirable to get
stronger results. Correlation to skill level by
learning theory could be explored.
 The ACRL Assessment in Action project
broadens the focus of assessment while
exploring an experimental method focusing on
transferring critical thinking skills to the
composition 5 research process.
55

56. Faculty, Librarians, and Writing
Center Instructors
 Feedback and recommendations on
improving students’ research writing skills
 Create ongoing discussion of learning
theory and designing instruction for
information literacy and learning how to
cite
56

57. Faculty feedback
 Opinion on the 3 methods were generally
favorable
 Most gave positive comments on the quality of
handouts and lesson plan
 More challenging examples are needed
 A "one-shot deal" is not an effective treatment
 A variety of learning strategies “in context”
enhances student learning process
 Future collaborations to design
workshops/tutorials
57

58. Librarians feedback
 Instruction methods vary according to
different learning levels
 A blend of learning theories maybe the most
effective approach in facilitating a diverse
classroom environment
 Library instruction and tutorials should be
embedded and scaffolded in tandem with the
course
 On-going collaboration with faculty and
writing center is essential
58

59. Special thanks to our
participating Composition faculty
 Gabrielle Myers
 Glen Carl Silva
 Katherine Field
 Kathryn S. Koo
 Valerie Sullivan
 Victoria Phillips
59

60. Questions, Comments, Suggestions?
Sharon Radcliff
sharon.radcliff@csueastbay.edu
EliseY Wong
yw3@stmarys-ca.edu
60