Slides from a workshop on the Scholarship of Teaching and Learning at Lakehead University in November 2019. They include an introduction to SoTL and information/activities on getting started with a research question and thinking about which data one might collect to fit that question.
1. Scholarship of
Teaching & Learning
Workshop
Christina Hendricks
University of British Columbia, Vancouver
Lakehead University, November 2019
2. Agenda
□ What is SoTL?
□ Getting started with SoTL
■ Defining a question (activity)
■ Identifying evidence (activity)
■ Other considerations
Slides & worksheets:
is.gd/sotlworkshop2019
2
4. A definition
of SoTL
The “systematic study of teaching and
learning and the public sharing and
review of such work through live or
virtual presentations, performances, or
publications”
(McKinney 2006, p. 39).
4
6. SoTL & Scholarly
Teaching
6
Using teaching
methods that
support student
learning
Grounding
teaching in
scholarly research
on T&L
(consumption)
Creation &
dissemination of
scholarly work
on T&L
(production)
Effective
Teaching
Scholarly
Teaching
SoTL
8. Developing a teaching
commons through SoTL
“Moving teaching from a mostly
private enterprise … to teaching
as ‘community property,’ which is
documented, shared, and built
upon,” for the sake of ongoing
improvement.
(Huber & Hutchings, 2005, p. 19)
8
11. Some types of SoTL questions
□ What works?
■ Seek evidence of the relative effectiveness of particular
teaching approaches (evaluative)
□ What/how is…?
■ Seek to describe, but not evaluate, a phenomenon
observed in the classroom or the consequences of
particular teaching approaches (descriptive, exploratory)
□ Theory building
■ Seek to build theoretical frameworks
Hutchings, “Introduction” in Hutchings (2000)
11
12. Factors in
defining
questions
Practice:
What want to
evaluate?
12
Define
questions
Identify
evidence
Create a
detailed plan
Prepare and
pilot
Collect data
and analyze
Lit review:
What have
others said?
Impact:
What effect
hope to achieve?
Context:
Where does it
take place?
13. Research question
templates
□ What/how is …?
■ What are the factors that influence
_something_ in the context of _context_?
■ How does _something_ look in the context of
_context_?
□ What works?
■ What is the impact of _practice_ on _area of
impact_ in the context of _context_?
13
14. Examples: What/how is..?
□ How do the students in a 100 level philosophy
course at [institution] approach reading a
philosophy text? How does that differ from the
approach of professional philosophers reading
the same text?
□ What factors influence the decision of students at
[institution] to major in computer science or not,
and is there a difference based on gender?
14
15. Examples: What works?
□ What is the impact of pre-lecture videos on students’
understanding of concepts in research methods in a
100-level Sociology class at a large, public university?
□ What is the impact of a “productive failure” method
of instruction vs a traditional active learning method
on student performance on exams in a large
enrollment, first year, cell biology course at a large,
public university?
15
16. Activity 1: Research questions
Practice
□ What ignites your
curiosity about your
teaching? Is there a
particular problem
you’d like to address?
□ What area of your
teaching will you focus
on? What practice
might you try?
16
Impact
□ What effect do you
hope to achieve?
□ What impact
might the practice
have that you
could measure?
Context
□ In what context
do the practice
and desired
impact take
place?
is.gd/sotlworkshop_nov2019
19. Example
19
Research
question
What do you
want to
evaluate?
How will you
evaluate?
What is the
impact of pre-
class videos on
students’
knowledge?
Conceptual
understanding
Students’
confidence
Concept
inventories,
reflective writing
Surveys, interviews
A Compendium of Data Sources for Use in SOTL-Based Inquiries –Doug Hamilton, Royal Roads University
20. Disciplinary approaches
(Philosophy)
Fairness — in grading (Close, 2009); in
two-stage exams (Chan, in process)
Think-alouds — how are students
thinking as they read? (Bloch-
Schulman, 2016)
20
24. Think about:
□ “It’s interesting” ≠ good SoTL research
question
□ “I understand” ≠ participants understand
■ Importance of doing pilot
□ Timing & integration of data collection into
(or outside of) course
24
25. Ethics & Equity
□ Ethics: informed consent, fairness, power
relationships, confidentiality, data security,
etc.
■ Taylor Institute Guide to ethics & SoTL
□ Equity & inclusion in data collection
■ UBC Guide on collecting demographic info
■ UBC Guide on asking about gender
25
27. Credits
Special thanks to the people who made and released
these awesome resources:
□ Presentation template by SlidesCarnival
□ Icons purchased with a subscription to The Noun
Project
□ Slides 5, 10-13, 18-19, 21 & 24 are adapted from slides
created by the Institute for the Scholarship of Teaching
and Learning at University of British Columbia,
Vancouver.
27
28. Free templates for all your presentation needs
Ready to use,
professional and
customizable
100% free for personal
or commercial use
Blow your audience
away with attractive
visuals
For PowerPoint and
Google Slides
28
Hinweis der Redaktion
The quote continues: “SoTL shares accepted criteria of scholarship in general, such as that it is made public, can be reviewed critically by members of the appropriate community, and can be built on by others to advance the field” (McKinney 2006).
“three important attributes of SoTL versus excellent teaching: that the inquiry must be systematic or methodical to gain credible results, be shared in order to advance the goal of improving practice outside one’s own classroom and that the ultimate goal be the students’ learning that results from the faculty member’s teaching” (Kern et al. 2015, p. 2).
Our disciplines affect our pedagogy; we do research in our disciplines, and we could also do research on pedagogy generally. But SoTL often involves research on specific pedagogies used in disciplines, and can involve specific research methods common in the disciplines.
Dimensions of Activities Related to Teaching (DART)
Scholarly teaching
Literature-based teaching
Attending teaching conferences & applying
SoTL
Peer reviewed, presented or published empirical research
Meta-analysis
Case study
Teaching practice
Developing courses, lessons
Curriculum design
Changing teaching based on feedback and reflection
Teaching portfolio for award, promotion
Sharing about teaching
Sharing teaching portfolio
Presentation of teaching tips based on own experience
“I had this problem and here’s what I did”
Evidence of effectiveness is informal and/or anecdotal
Blog about teaching
Newspaper or magazine article on teaching
Huber, M. T., & Hutchings, P. (2005). The Advancement of Learning: Building the Teaching Commons | Wiley. Retrieved from https://www.wiley.com/en-us/The+Advancement+of+Learning%3A+Building+the+Teaching+Commons+-p-9780787981150
“Higher education has long fostered the robust commons created by scientific and scholarly research. This has not been the case with teaching and learning. … As Lee Shulman observed … teaching will not be fully recognized in the academy until its status changes from ‘private to community property’ (1993, p. 6). Without a functioning commons, it is hard for pedagogical knowledge to circulate, deepen through debate and critique, and inform the kinds of innovation so important to higher education today” (p. 5).
We need to “capture the work of teaching and learning in ways that can be built upon--to stop losing ‘the intellectual work that is regularly being done,’ as Dan Bernstein has written, by creating ‘a community of teachers whose decisions about how to teach will be informed by the collective effectiveness of the work’ (2001, pp. 228-229)” (p. 18).
But isn’t that already being done by education researchers? Why do we need more people contributing to the commons?
Partly: SoTL is at the intersection of teaching practice, research, and disciplinary knowledge--comes from practice, contextualized in practice, aimed back at practice. And done in multiple disciplinary methods.
But why is this useful?
Huber, M. T., & Hutchings, P. (2006). Building the Teaching Commons. Change: The Magazine of Higher Learning, 38(3), 24–31. https://doi.org/10.3200/CHNG.38.3.24-31
“The scholarship of teaching and learning invites faculty from all disciplines and fields to identify and explore interesting questions in their own teaching—and, especially, in their students’ learning—and to share what they discover with colleagues who can build on their insights. Such work has the potential to transform higher education by making the private work of the classroom visible, talked about, studied, built upon, and valued—conditions for ongoing improvement in any enterprise” (p. 25).
There is a great deal of edu research, but it’s not a simple matter to directly apply it to practice given how contextualized teaching practice is. Linking research and practice more closely can have value--teaching practitioners engaging in research in their own contexts, and applying it back (28).
“teaching and learning are highly dependent on contextual factors, and faculty often find as much to learn from the situated experience of other faculty as from studies done with methodologies designed to minimize the influence of context on research results.” (28)
““They can identify a good question, a promising investigative strategy, an assignment or assessment design that they might try out or include in their own repertoire, and they are aided in incorporating it into their own work by their understanding of how the original context differs from their own” (29)
“While there is a place for general principles and lists of best practices to guide improvement, what’s also needed are rich representations of teaching and learning—new genres that capture the blooming, buzzing complexity of real students in real settings” (29)
Hutchings, P. (2000). Opening Lines: Approaches to the Scholarship of Teaching and Learning. Retrieved from https://eric.ed.gov/?id=ED449157
Silvia Bartolic: https://isotl.ctlt.ubc.ca/features/sotl-projects/
“Using a flipped classroom approach, the instructor provided online content that will supplement lectures. This allowed students to revisit important information in their own time and created more in-class time for problem-based learning activities and hands-on time with data and data analysis software through individual research projects.”
“Using pre- and post-tests the instructors assessed student understanding of key concepts in class. The instructor also analyzed change in student affect and performance over the course of the semester, comparing results from two different sections of the course – one taught in the regular manner (lectures with some class activities) and the other in the flipped classroom approach.”
Mizuzu Kazama: https://isotl.ctlt.ubc.ca/features/spotlight/
“Engaging in the SoTL process has provided me with the opportunity to find out whether students, especially at an early intermediate level in a Japanese as foreign language classroom, are able to identify the area of improvement in a peer’s oral performance. Our finding suggest that students’ feedback on the use of language such as grammar, pronunciation, and intonation has a more than 90% degree of accuracy!”
3. DIY productive failure: boosting performance in a large undergraduate biology course https://www.nature.com/articles/s41539-019-0040-6
Students in first-year university courses often focus on mimicking application of taught procedures and fail to gain adequate conceptual understanding. One potential approach to support meaningful learning is Productive Failure (PF). In PF, the conventional instruction process is reversed so that learners attempt to solve challenging problems ahead of receiving explicit instruction. While students often fail to produce satisfactory solutions (hence “Failure”), these attempts help learners encode key features and learn better from subsequent instruction (hence “Productive”). Effectiveness of PF was shown mainly in the context of statistical and intuitive concepts, and lessons that are designed and taught by learning scientists. We describe a quasi-experiment that evaluates the impact of PF in a large-enrollment introductory university-level biology course when designed and implemented by the course instructors. One course-section (295 students) learned two topics using PF; another section (279 students) learned the same topics using an active learning approach, which is the standard in this course. Performance was assessed on the subsequent midterm exam, after all students had ample opportunities for practice and feedback, and after some time has elapsed. PF students scored nearly five percentage-points higher on the relevant topics in the subsequent midterm exam. The effect was especially strong for low-performing students. Improvement on the final exam was only visible for low-performing students. We describe the intervention and its potential to transform large introductory university courses.
PF condition:
Pre-readings
Try to address a problem in groups
Formative feedback on work (eg clicker questions)
Walk through: “During this part, the instructor modeled an expert’s way of approaching the activity, while continuing to involve students in this process by prompting and eliciting their contribution to the problem-solving path.”
Active learning condition
Pre-readings
Walk through
Activity—working on the problem
Formative feedback
One section used only AL; other used PF for two topics & AL for the rest
Close, D. (2009). Fair Grades. Teaching Philosophy, 32(4), 361–398. https://doi.org/10.5840/teachphil200932439
Provides a philosophical conception of what grading should do, what a “fair” grade means, and then evaluates various ways of grading as to whether or not they fit that definition
Grading to get students to attend or do something, rather than basing grade only on the merits of the work, is not “fair” in his view. Grades should provide information about mastery of content only.
Bloch-Schulman, S. (2016). A Critique of Methods in the Scholarship of Teaching and Learning in Philosophy. Teaching & Learning Inquiry, 4(1), 1–15. https://doi.org/10.20343/teachlearninqu.4.1.10
“For this study, I devised think alouds to examine schema differences between the reading practices of philosophers and students. In particular, I wanted to investigate whether students were reading philosophic work through a schema driven by plot, which is quite different from how philosophers read, seeing philosophy as argumentation, and thus that students might be utilizing the reading skills they would correctly use reading fiction when reading philosophy, and missing the purpose and structure of philosophical writing” (9).
Comparing how students and philosophers approach texts as they speak aloud what they’re thinking--this can and should inform our teaching methods to help move students more towards reading and thinking practices useful in our discipline.
Qualitative
Often for verbal, visual, textual information; more open-ended questions
Useful for getting deeper explanations of things
Interviews, focus groups, observations, case studies
Quantitative
Broader look at lots of people but often can’t get as deeply into explanations
e.g., Surveys can do likert scales and ask for explanations but not everyone will add an explanation, and they might not go very deep
Surveys, student results on quizzes & exams, student grades, enrolment numbers & patterns
Mixed methods
Often combining different kinds of methods can get best results: different views on same question
e.g., start with a survey, ask ppl if willing to do follow up interview or focus group
Or vice versa; develop survey from pilot focus groups or interviews, observationns
Research question should focus on improving student learning in some way, should be potentially applicable in other contexts.
* e.g., Does amount of participation in an optional online forum differ according to student major?
* e.g., how many students think that emails are actually being written directly by their professor when they are coming instead from an automated email generating program that targets specific messages to students who meet certain criteria? Do they get upset when they find out?