This document summarizes a presentation on implementing evidence-based teaching methods in college classrooms. The presentation discusses how student and faculty expectations often differ, with research showing students have different expectations than professors, especially in introductory courses. The presentation advocates making learning expectations and goals explicit and transparent to students through stating connections between activities, assignments, and outcomes. Specific strategies are provided, such as linking daily lessons to overall learning outcomes and using assignment templates that specify the purpose, skills practiced, and evaluation criteria.

1. 1 collegeclassroom.ucsd.edu
If these students are
thinking anything other
than “I’m bored” what
is it?

2. CIRTL – The College Classroom Meeting 9:
Implementing Evidence-Based Teaching Methods
March 24, 2016
Unless otherwise noted, content is licensed under
a Creative CommonsAttribution- 3.0 License.
Tom Holme
Department of Chemistry, Iowa State University
taholme@iastate.edu
Peter Newbury
Center for EngagedTeaching, UC San Diego
pnewbury@ucsd.edu
collegeclassroom.ucsd.edu

3. Thinking about our own experience
collegeclassroom.ucsd.edu3
One way that teaching practice stays “stuck” in old paradigms is
that we often use our experience as students to inform our teaching
practice. Let’s start out today thinking about what we encountered
as students.

4. Thinking about our own experience
collegeclassroom.ucsd.edu4
In how many classes that you have taken can you remember a
teacher providing explicit guidance about how activities would help
you learn how to think about the topic?
A) none
B) 1 or 2 classes
C) Between 3 and 6 classes
D) More than 6 classes
One way that teaching practice stays “stuck” in old paradigms is
that we often use our experience as students to inform our teaching
practice. Let’s start out today thinking about what we encountered
as students.

5. The role of expectations
collegeclassroom.ucsd.edu5
- It takes some significant effort for college students to
figure out what the expectations are for a course.
- A syllabus is often crammed with so much information
that they don’t really know how to gain anything other than
the most rudimentary facts from it.
- How do we understand expectations?
- How do we mold expectations?

6. Expectation differences
collegeclassroom.ucsd.edu6
- there is data to suggest that students have different
expectations of their courses than the professors have of
them.

7. Expectations research
collegeclassroom.ucsd.edu7
In both physics and chemistry researchers have investigated
the differences in expectations between students and their
professors.
The chemistry project was carried out by Stacey Bretz and
her group[1] and they developed a tool called CHEMX.

8. The Instrument: CHEMX
Cluster Representative Statement
Effort
(α=0.85)
“I read the text in detail and work
through many of the examples given
there.”
Concepts
(α=0.73)
“When I solve most exam or homework
problems, I explicitly think about the
concepts that underlie the problems.”
Math-link
(α=0.82)
“In this course, I do not expect to
understand equations in an intuitive
sense; they just have to be taken as
givens.”
Reality-link
(α=0.86)
“It is unnecessary for me to have to
relate chemistry to the real world.”

9. The Instrument: CHEMX
Cluster Representative Statement
Outcome
(α=0.73)
“Only a very few specially qualified people
are capable of really understanding
chemistry.”
Laboratory
(α=0.85)
“I really don’t expect to understand how
laboratory instruments work – they are just
tools that help me complete the lab.”
Visualization
(α=0.85)
“Solving a chemistry problem may require me
to be able to draw molecules in more than one
way.”

10. Results: Faculty Expectations
0
10
20
30
40
50
60
70
80
90
100
0 20 40 60 80 100
PercentFavorableResponse
Percent Unfavorable Response
Analytical
Biochemistry
Chemistry Education
Inorganic
Organic
Physical

11. 0
10
20
30
40
50
60
70
80
90
100
0 20 40 60 80 100
PercentFavorableResponse
Percent Unfavorable Response
OP Pre GC1
OP Post GC1
OP Post GC2
OP Post OC2
OP Junior
Faculty
Results: Student ExpectationsMedium, Open-admission Public University (n=133)

12. Observations
12
- Student expectations for what they will do in science
classes don’t really match the expectations of professors.
-This is particularly true for introductory courses.
-Taking those introductory courses seems to make the
gap even wider in some (perhaps many) cases!
(We showed chemistry, but physics found similar things
[2])

13. Why do we have to study this?
collegeclassroom.ucsd.edu13
Heading into breakout rooms:
- In your discipline, how many years do undergraduates
take classes before they are mostly in rooms with other
majors? (for example: introductory physics doesn’t have a
majority of physics majors in it.)
- In your experience (as a student or as part of the
teaching team) how well, if at all, do these early (service)
courses help students make connections between their
major interests and the material covered in the course?

14. Embracing the “service” course
collegeclassroom.ucsd.edu14
In some cases, such as the
one-semester chemistry
courses that many
engineering students take,
it is possible to design
relevance into the course.

15. Features we incorporated
collegeclassroom.ucsd.edu15
-To make this book useful to faculty who are often
devising a new course that will use it, we concentrated on
several things.
- First, we start and end each chapter with a connection
between the chemistry and an engineering application.
- Second, we include problems that are more like an
engineering scale.
-Third, we include problems that require students to
think about how the problem is solved, not the final answer.

16. Domain specific vs. domain general
collegeclassroom.ucsd.edu16
- One area of cognitive science that is important in things
like problem solving is the distinction between domain
general and domain specific skills.
- Many of the things we’ve been emphasizing in this
course are domain general methods.
-This doesn’t mean they cannot be applied in domain
specific ways, however.

17. Strategy: Be explicit about cognitive
gains that students can make
collegeclassroom.ucsd.edu17
Many of the things we’ve emphasize that can help students
learn are more successful if students understand why we
are doing them.
What are the best strategies for doing this?

18. Transparency in learning project
collegeclassroom.ucsd.edu18
- one approach is the transparency in learning project that
we had you do some reading on.
The goal is to make the learning expectations explicit for students
in as many ways as possible.
Three key components.
(1) Transparent learning outcomes
(2) Transparent assignments linked to learning outcomes
(3) Transparent classroom activities

19. Transparent Learning Outcomes
collegeclassroom.ucsd.edu19
Problem statement:
- If the only place they appear is on a syllabus, chances are
your learning outcomes are not part of what your students
actively consider in your course.
Way to resolve:
- Before each class starts, include how the material to be
covered that day relates to the course learning outcomes.
- If possible have this about content and skills, not just
content.

20. Example Discussion Breakout Rooms
collegeclassroom.ucsd.edu20
Each Individual by themselves first.
-Think of a pretty generalizable skill that students might
encounter in an introductory course in your discipline.
-You’ll have a white board, so you can put in a textbox and
put the learning outcome up.
Discussion:
- Briefly describe to your roommates how you could
motivate that learning outcome at the start of a class.

21. Transparent assignments
collegeclassroom.ucsd.edu21
- It takes more time, but if you can add learning outcomes
to your assignments, students are more likely to see the
relevance.
- It’s worth noting that newer electronic homework
systems are capable of letting you make these connections
to learning outcomes.

22. Example from the web.
collegeclassroom.ucsd.edu22

23. On the fly -
collegeclassroom.ucsd.edu23
- It may be a challenge to have this level of organization
when a class starts.
- Even if you can’t do a grid at the start, it’s still
important to tell students the connections for each
assignment.

24. Template
collegeclassroom.ucsd.edu24
© 2013 Mary-Ann Winkelmes http://www.unlv.edu/provost/teachingandlearning
Principal Investigator mary-ann.winkelmes@unlv.edu
Transparent Assignment Template
© 2013 Mary-Ann Winkelmes
This template can be used as a guide for developing, explaining, and discussing class activities and out-of-class assignments.
Making these aspects of each course activity or assignment explicitly clear to students has demonstrably enhanced students’
learning in a national study.1
Assignment Name
Due date:
Purpose: Define the learning objectives, in language and terms that help students recognize how this assignment will
benefit their learning. Ideally, indicate how these are connected with institutional learning outcomes, and how the
specific knowledge and skills involved in this assignment will be important in students’ lives beyond the contexts of
this assignment, this course, and this college.
Skills: The purpose of this assignment is to help you practice the following skills that are
essential to your success in this course / in school / in this field / in professional life beyond
school:
Terms from Bloom’s Taxonomy of Educational Objectives may help you explain these skills in language
students will understand. Listed from cognitively simple to most complex, these skills are:
o understanding basic disciplinary knowledge and methods/tools
o applying basic disciplinary knowledge/tools to problem-solving in a similar but unfamiliar context
o analyzing
o synthesizing
o judging/evaluating and selecting best solutions
o creating/inventing a new interpretation, product, theory
Knowledge: This assignment will also help you to become familiar with the following
important content knowledge in this discipline:
1.
2.
Task: Define what activities the student should do/perfom. “Question cues” from this chart might be helpful:
http://www.asainstitute.org/conference2013/handouts/20-Bloom-Question-Cues-Chart.pdf. List any steps or
guidelines, or a recommended sequence for the students’ efforts. Specify any extraneous mistakes to be avoided.
Criteria for Success:
Define the characteristics of the finished product. Provide specific examples of what these characteristics look like in
practice. With students, collaboratively analyze an example of good work before the students begin working. Offer a
critiqued example of excellent work with specific indicators of what makes the work successful. Explain how excellent
work differs from adequate work. It is often useful to provide a checklist of characteristics of successful work to help
the student know if s/he is doing high quality work while s/he is working on the assignment. This enables students to
evaluate the quality of their own efforts while they are working, and to judge the success of their completed work.
Students can also use your checklist to provide feedback on peers’ coursework. Indicate whether this task/product
will be graded and/or how it factors into the student’s overall grade for the course. Later, asking students to reflect
and comment on their completed, graded work empowers them to focus on changes to their learning strategies that
might improve their future work.
1
Winkelmes, Mary-Ann. “Transparency in Teaching: Faculty Share Data and Improve Students’ Learning.” Liberal Education 99,2 (Spring 2013);
Winkelmes et al, “A Teaching Intervention that Increases Underserved College Students’ Success…” Peer Review (Winter 2016).
*
* The author developed an earlier version of this template at the University of Illinois, Urbana-Champaign.

25. Beyond just transparency
collegeclassroom.ucsd.edu25
- As assignments become more transparent, you probably
also need to think in terms of how to have them scaffold
student learning towards deeper understanding.
-This can be done using Bloom’sTaxonomy and asking
questions in more careful ways.

26. Question
template
collegeclassroom.ucsd.edu26
Bloom’s Critical Thinking Cue Questions
Cue Questions Based on Blooms’ Taxonomy of Critical Thinking
Adapted by C. Allen (January 2013) from Public Consulting Group’s Center for Resource Management,
in partnership with the Council of Chief State School Officers, August 2007
LOWER-ORDER THINKING SKILLS
(BASIC THINKING)
3. APPLYING (Using learned knowledge in new
situations or to solve a real life problem)
• How would you use …?
• What examples can you find to …?
• How would you solve _______ using what you
have learned …?
• How would you organize _______ to show …?
• How would you show your understanding of …?
• What approach would you use to …?
• How would you apply what you learned to develop
…?
• What other way would you plan to …?
• What would result if …?
• How can you make use of the facts to …?
• What elements would you choose to change …?
• What facts would you select to show …?
• What questions would you ask in an interview
with…?
2. UNDERSTANDING (Comprehension;
Explaining the meaning of information)
• How would you classify the type of …?
• How would you compare …? contrast …?
• How would you rephrase the meaning …?
• What facts or ideas show …?
• What is the main idea of …?
• Which statements support …?
• How can you explain what is meant …?
• What can you say about …?
• Which is the best answer …?
• How would you summarize …?
1. REMEMBERING INFORMATION
(Knowledge; recalling facts and information)
• What is …?
• How is …?
• Where is …?
• When did _______ happen?
• How did ______ happen?
• How would you explain …?
• How would you describe …?
• What do you recall …?
• How would you show …?
• Who (what) were the main …?
• What are three …?
• What is the definition of...?
HIGHER-ORDER THINKING SKILLS
(ABSTRACT THINKING)
6. CREATING (Putting ideas together to form a new and
different whole)
• What changes would you make to solve …?
• How would you improve …?
• What would happen if …?
• How can you elaborate on the reason …?
• What alternative can you propose …?
• How can you invent …?
• How would you adapt ________ to create a different …?
• How could you change (modify) the plot (plan) …?
• What could be done to minimize (maximize) …?
• What way would you design …?
• What could be combined to improve (change) …?
• How would you test or formulate a theory for …?
• What would you predict as the outcome of ...?
• How can a model be constructed that would change
…?
• What is an original way for the …?
5. EVALUATING (Making judgments about the merits of
ideas, materials, or phenomena based on criteria)
• Why do you agree with the actions? The outcomes?
• What is your opinion of …? (Must explain why)
• How would you prove …? disprove …?
• How can you assess the value or importance of …?
• What would you recommend …?
• How would you rate or evaluate the …?
• What choice would you have made …?
• How would you prioritize …?
• What details would you use to support the view …?
• Why was it better than …?
4. ANALYZING (Breaking down a whole into component
parts; Examining critically)
• What are the parts or features of …?
• How is _______ related to …?
• Why do you think …?
• What is the theme …?
• What motive is there …?
• What conclusions can you draw …?
• How would you classify …?
• How can you identify the different parts …?
• What evidence can you find …?
• What is the relationship between …?
• How can you make a distinction between …?
• What is the function of …?
• What ideas justify …?

27. Question
template
collegeclassroom.ucsd.edu27
Bloom’s Critical Thinking Cue Questions
Cue Questions Based on Blooms’ Taxonomy of Critical Thinking
Adapted by C. Allen (January 2013) from Public Consulting Group’s Center for Resource Management,
in partnership with the Council of Chief State School Officers, August 2007
LOWER-ORDER THINKING SKILLS
(BASIC THINKING)
3. APPLYING (Using learned knowledge in new
situations or to solve a real life problem)
• How would you use …?
• What examples can you find to …?
• How would you solve _______ using what you
have learned …?
• How would you organize _______ to show …?
• How would you show your understanding of …?
• What approach would you use to …?
• How would you apply what you learned to develop
…?
• What other way would you plan to …?
• What would result if …?
• How can you make use of the facts to …?
• What elements would you choose to change …?
• What facts would you select to show …?
• What questions would you ask in an interview
with…?
2. UNDERSTANDING (Comprehension;
Explaining the meaning of information)
• How would you classify the type of …?
• How would you compare …? contrast …?
• How would you rephrase the meaning …?
• What facts or ideas show …?
• What is the main idea of …?
• Which statements support …?
• How can you explain what is meant …?
• What can you say about …?
• Which is the best answer …?
• How would you summarize …?
1. REMEMBERING INFORMATION
(Knowledge; recalling facts and information)
• What is …?
• How is …?
• Where is …?
• When did _______ happen?
• How did ______ happen?
• How would you explain …?
• How would you describe …?
• What do you recall …?
• How would you show …?
• Who (what) were the main …?
• What are three …?
• What is the definition of...?
HIGHER-ORDER THINKING SKILLS
(ABSTRACT THINKING)
6. CREATING (Putting ideas together to form a new and
different whole)
• What changes would you make to solve …?
• How would you improve …?
• What would happen if …?
• How can you elaborate on the reason …?
• What alternative can you propose …?
• How can you invent …?
• How would you adapt ________ to create a different …?
• How could you change (modify) the plot (plan) …?
• What could be done to minimize (maximize) …?
• What way would you design …?
• What could be combined to improve (change) …?
• How would you test or formulate a theory for …?
• What would you predict as the outcome of ...?
• How can a model be constructed that would change
…?
• What is an original way for the …?
5. EVALUATING (Making judgments about the merits of
ideas, materials, or phenomena based on criteria)
• Why do you agree with the actions? The outcomes?
• What is your opinion of …? (Must explain why)
• How would you prove …? disprove …?
• How can you assess the value or importance of …?
• What would you recommend …?
• How would you rate or evaluate the …?
• What choice would you have made …?
• How would you prioritize …?
• What details would you use to support the view …?
• Why was it better than …?
4. ANALYZING (Breaking down a whole into component
parts; Examining critically)
• What are the parts or features of …?
• How is _______ related to …?
• Why do you think …?
• What is the theme …?
• What motive is there …?
• What conclusions can you draw …?
• How would you classify …?
• How can you identify the different parts …?
• What evidence can you find …?
• What is the relationship between …?
• How can you make a distinction between …?
• What is the function of …?
• What ideas justify …?
If you go back to your
breakout rooms and
look at your learning
outcome there.What
is the highest level
area on this template
that you can (fairly
quickly) think of a
homework question
related to the learning
outcome?

28. Next week
collegeclassroom.ucsd.edu28
The topic next week is The First Day of Class
Watch the blog and your email for tasks to complete before
we meet.

29. References
collegeclassroom.ucsd.edu29
[1] Grove, N.P.; Bretz, S. (2007) J.Chem.Educ.,84(9),
1416-1424
[2] Redish, E. F.; Saul, J. M. (1998) Am.J. Phys., 66, 212-
224