Peter Newbury UC San Diego and Tom Holme Iowa State University collegeclassroom.ucsd.edu Center for the Integration of Research, Teaching and Learning (CIRTL) Network - cirtl.net

1. CIRTL – The College Classroom Meeting 3:
Learning Outcomes / Goals / Objectives
February 11, 2016
Unless otherwise noted, content is licensed under
a Creative CommonsAttribution- 3.0 License.
Peter Newbury
Center for EngagedTeaching, UC San Diego
pnewbury@ucsd.edu
Tom Holme
Department of Chemistry, Iowa State University
taholme@iastate.edu
collegeclassroom.ucsd.edu

2. Learning Outcomes - collegeclassroom.ucsd.edu2
Flight Deck byWayanVolta on flickr CC
Today, you’ll need
• pen/paper to write on
• ability to open a separate
window/tab in your browser
Run audio setup wizard or
Tools >Audio >Audio SetupWizard

3. Learning Outcomes - collegeclassroom.ucsd.edu3
Flight Deck byWayanVolta on flickr CC
Do you understand
how to fly an airplane?
Yes
Okay, but let’s look at
this checklist.
[trust and verify]

4. Scholarly approach to teaching
(“Backward Design”)
Learning Outcomes - collegeclassroom.ucsd.edu4
CarlWieman
Science Education Initiative
cwsei.ubc.ca
What should
students
learn?
learning
outcomes
assessment
active
learning
What should
students
learn?
What are
students
learning?
What instructional
strategies
help students
learn?

5. (Image: NASA)Learning Outcomes - collegeclassroom.ucsd.edu5

6. Introductory “Astro 101”
Traditional Course Syllabus
Learning Outcomes - collegeclassroom.ucsd.edu6
This course covers Chapters
1. Mercury
2. Venus
…
8. Neptune
9. other objects
10. Formation of the
Solar System

7. Introductory “Astro 101”
Traditional Course Syllabus
Course with
Learning Outcomes
Learning Outcomes - collegeclassroom.ucsd.edu7
This course covers Chapters
1. Mercury
2. Venus
…
8. Neptune
9. other objects
10. Formation of the
Solar System
 deduce from patterns in the
properties of the planets,
moons, asteroids and other
bodies that the Solar System
had single formation event.
 provide notable examples of
how comets influenced
history, art and science

8. Learning outcomes
Learning Outcomes - collegeclassroom.ucsd.edu8
 completes the sentence,“By this end of this
lesson/unit/course, you will be able to…”
 begins with an action verb (“deduce”) (more later)
 deduce from patterns in the
properties of the planets,
moons, asteroids and other
bodies that the Solar System
had single formation event.

9. Learning outcomes are valuable to…
Learning Outcomes - collegeclassroom.ucsd.edu9
 the students
o reveal what the instructor is looking for: no more
guessing what “understand” means or what will be
on the exam
o big picture of the next part of the course
o allow students to monitor their own progress
metacognition
o allow students to check they’ve mastered the
concept (especially when studying later)

10. What is the Value of Course-Specific
Learning Outcomes[1]
10 Learning Outcomes - collegeclassroom.ucsd.edu

11. What is the Value of Course-Specific
Learning Outcomes[1]
11 Learning Outcomes - collegeclassroom.ucsd.edu

12. How People Learn[2]
Learning Outcomes - collegeclassroom.ucsd.edu12
Which Key Finding(s) do you
think this supports?
 pre-existing knowledge
 expertise
 metacognition
Take 1 minute to write
down your thoughts and
then be prepared to share
with your class mates.

13. How People Learn[2]
Learning Outcomes - collegeclassroom.ucsd.edu13

14. How People Learn[2]
Learning Outcomes - collegeclassroom.ucsd.edu14
Key Finding 3:
A “metacognitive” approach
to instruction can help
students learn to take control
of their own learning by
defining learning goals and
monitoring their progress in
achieving them.
(How People Learn,p.18)

15. Learning outcomes are valuable to…
Learning Outcomes - collegeclassroom.ucsd.edu15
 the instructor
o crystallizes what the instructor cares about
o helps the instructor create assessments
o helps the instructor select instructional strategies
and activities

16. “create assessments”
Learning Outcomes - collegeclassroom.ucsd.edu16
Astro 101 Final Exam
(10 marks) List 3 patterns of the Solar System as a
whole.Then, outline in some detail the current model for
the formation of the Solar System. In particular, make sure
you explain how the observed patterns and regularities are
related to this theory of formation.

17. “select instructional activities”
Learning Outcomes - collegeclassroom.ucsd.edu17 ClassAction http://astro.unl.edu/classaction/
Does this peer
instruction question
support the
learning outcomes?
don’t
use it
use it
yesno

18. A course should have
Course-level LOs
Learning Outcomes - collegeclassroom.ucsd.edu
 several LOs giving big
picture, attitudes,
behaviors:“how you will
be different by the end”
 likely can’t be assessed
with a single exam
question
18

19. A course should have
Course-level LOs Topic-level LOs
Learning Outcomes - collegeclassroom.ucsd.edu
 several LOs giving big
picture, attitudes,
behaviors:“how you will
be different by the end”
 likely can’t be assessed
with a single exam
question
 many LOs defining what
it means to “understand”
at this level (freshman,…)
 should be repeatedly
assessed on homework,
exams
19

20. A course should have
Course-level LOs Topic-level LOs
Learning Outcomes - collegeclassroom.ucsd.edu
 several LOs giving big
picture, attitudes,
behaviors:“how you will
be different by the end”
 likely can’t be assessed
with a single exam
question
 many LOs defining what
it means to “understand”
at this level (freshman,…)
 should be repeatedly
assessed on homework,
exams
20
course-level and topic-level LOs are related

21. Topic-level
LO
Topic-
level LO
Topic-level
LO
Topic-level
LO
Course-level LO #4
Learning Outcomes - collegeclassroom.ucsd.edu21
Course-level LO #2
Course-level LO #3Course-level
learning outcome (LO) #1
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-level
LO
Topic-
level LO
Topic-
level LO
Topic-
level LO
Topic-
level LO Topic-
level LO
Topic-level
LO
Topic-level
LO
Topic-level
learning outcome
How are course-level and topic-level learning outcomes related?
Write down at least 2 rules or patterns you see here.
Topic-
level LO Topic-
level LO

22. A course should have
Course-level LOs Topic-level LOs
Learning Outcomes - collegeclassroom.ucsd.edu
 every course-level LO is
supported by many
topic-level LOs
(not zero, or how will
students learn it?)
 every topic-level LO
supports one or more
course-level LO
(not zero, or why you
talking about it?)
22

23. Writing topic-level LOs
Learning Outcomes - collegeclassroom.ucsd.edu23
Writing learning outcomes is hard because you have to
 recognize (through your expert blindness)
 declare (and be accountable for)
what you want your students to be capable of doing.
A good start is picking the verb describing the
action the students will perform to demonstrate their
mastery of the concept.

24. Bloom’s Taxonomy [3-5]
Learning Outcomes - collegeclassroom.ucsd.edu24
transform or combine ideas to create
something new
think critically about and
defend a position
break down concepts into parts
apply comprehension to unfamiliar
situations
demonstrate understanding of ideas
and concepts
remember and recall factual
information
6 Create
5 Evaluate
4 Analyze
3 Apply
2 Understand
1 Remember

25. Bloom’s Taxonomy [3-5]
Learning Outcomes - collegeclassroom.ucsd.edu25
higher order thinking
lower order thinking
6 Create
5 Evaluate
4 Analyze
3 Apply
2 Understand
1 Remember

26. Good Strategy: Start with the verb
Learning Outcomes - collegeclassroom.ucsd.edu26
6 Create
5 Evaluate
4 Analyze
3 Apply
2 Understand
1 Remember

27. Writing LOs – The Wrong Way
Learning Outcomes - collegeclassroom.ucsd.edu27
Astro 101 Learning Outcomes
1. understand how Saturn’s rings formed
Assess your LOs:
 “understand”? That could be a sentence…or a thesis
 how does a student check that they mastered the Saturn
part of the course?
 how does a student demonstrate to you they
“understands” at this level?
 what and how are you going to teach?

28. Write LOs from course outline
Learning Outcomes - collegeclassroom.ucsd.edu28
Work your way through the list of topics and decide “What
do I want students to be able to do, to demonstrate they
‘get’ this topic?”
Astro 101 Learning Outcomes - Saturn
1. give a detailed description of the size and structure of
Saturn’s rings
2. trace the gravitational feedback cycle that keeps
Saturn’s rings so thin

29. “Back-engineer” LOs from exams
Learning Outcomes - collegeclassroom.ucsd.edu29
Use last year’s (or several years’) final exam. For each good
question, ask yourself
 What is this question assessing?What is the learning
outcome I want students to demonstrate to properly
answer this question?
 Is that the outcome I want for my class?

30. Driver’s Ed 101: How to Drive in CA
Learning Outcomes - collegeclassroom.ucsd.edu30
What should
students
learn?
learning
outcomes
assessment
What should
students
learn?
What are
students
learning?
active
learning
What instructional
strategies
help students
learn?


31. Driver’s Ed 101: How to Drive in CA
Learning Outcomes - collegeclassroom.ucsd.edu31
What should
students
learn?
learning
outcomes
assessment
What should
students
learn?
What are
students
learning?
active
learning
What instructional
strategies
help students
learn?
Write a learning
outcome for the question
with your Room number.
(Room 1 – Question 1, etc.)

Think to yourself:
If a student can correctly
answer the question, what are
they able to do?

32. 6 Create: transform and combine ideas to create something new
develop,create,propose,formulate,design,invent
5 Evaluate: think critically about and defend a position
judge,appraise,recommend,justify,defend,criticize,evaluate
4 Analyze:: break down concepts into parts
compare,contrast,categorize,distinguish,identify,infer
3 Apply: apply comprehension to unfamiliar situations
apply,demonstrate,use,compute,solve,predict,construct,modify
2 Understand: demonstrate understanding of ideas, concepts
describe,explain,summarize,interpret,illustrate
1 Remember: remember and recall factual knowledge
define,list,state,label,name,describe
Learning Outcomes - collegeclassroom.ucsd.edu Adapted from CarlWieman (2007)
www.cwsei.ubc.ca/resources/learn_goals.htm
32
• Follow the link in the Chat window to a shared Google spreadsheet.
• Together, write a learning outcome for the question with your Room
number (Room 1 – Q1, etc.) Select an action verb from Bloom’s taxonomy:

33. Learning Outcomes - collegeclassroom.ucsd.edu33
Sample Class CWrittenTest 5
California Department of MotorVehicles
http://www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm

34. Learning Outcomes - collegeclassroom.ucsd.edu34
Sample Class CWrittenTest 5
California Department of MotorVehicles
http://www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm

35. Learning Outcomes - collegeclassroom.ucsd.edu35
Sample Class CWrittenTest 5
California Department of MotorVehicles
http://www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm

36. Learning Outcomes - collegeclassroom.ucsd.edu36
Sample Class CWrittenTest 5
California Department of MotorVehicles
http://www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm

37. Learning outcomes in your discipline
Learning Outcomes - collegeclassroom.ucsd.edu37
Before today’s meeting, you drafted a Bloom’s Level 1
(“remember”) learning outcome for a lower-division course
in your discipline.
1. Follow the link in the Chat window to a shared Google
document
2. Find your name (alphabetical by first name)
3. Draft a higher-order learning outcome (“analyze”,
“evaluate”, or “create”) for your course

38. 6 Create: transform and combine ideas to create something new
develop,create,propose,formulate,design,invent
5 Evaluate: think critically about and defend a position
judge,appraise,recommend,justify,defend,criticize,evaluate
4 Analyze:: break down concepts into parts
compare,contrast,categorize,distinguish,identify,infer
3 Apply: apply comprehension to unfamiliar situations
apply,demonstrate,use,compute,solve,predict,construct,modify
2 Understand: demonstrate understanding of ideas, concepts
describe,explain,summarize,interpret,illustrate
1 Remember: remember and recall factual knowledge
define,list,state,label,name,describe
Learning Outcomes - collegeclassroom.ucsd.edu Adapted from CarlWieman (2007)
www.cwsei.ubc.ca/resources/learn_goals.htm
38
Draft a higher-order learning outcome (“analyze”,
“evaluate”, or “create”) for the course

39. Share your LOs with your students
Learning Outcomes - collegeclassroom.ucsd.edu39
 Publish them in your syllabus
AND include relevant learning
goals in your lecture slides at the
beginning of each topic, even
each class.
 Be wary of reading them aloud:
your students may not yet have
the knowledge (or jargon) to
appreciate the LOs.The LOs
will be there when they study.
 You’re not “spoon-feeding” them – you’re helping your
students develop expert-like ways of thinking and skills

40. Watch the blog for next week’s
readings and assignments
(I’ll try to get it posted by Friday, February 12.)
Next time: Meeting 4
Fixed and Growth Mindsets and
Assessment that Supports Learning

41. References
Learning Outcomes - collegeclassroom.ucsd.edu41
1. Simon, B., &Taylor, J. (2009).What is theValue of Course-Specific Learning
Goals? Journal of College ScienceTeaching, 39, 2, 52-57.
2. National Research Council (2000). How People Learn:Brain,Mind,Experience,
and School:Expanded Edition. J.D. Bransford,A.L Brown & R.R. Cocking
(Eds.),Washington, DC:The NationalAcademies Press.
3. Bloom B. S. (1956). Taxonomy of Educational Objectives,Handbook I:The Cognitive
Domain. NewYork: David McKay Co Inc.
4. Adapted from edorigami.wikispaces.com/Bloom%27s+Digital+Taxonomy
5. Revised Bloom’sTaxonomy
www.celt.iastate.edu/teaching/RevisedBlooms1.html
6. California DMV Sample Class CWrittenTest 5
www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm
7. Excerpt fromWieman, C. (2007). Slides from theWieman Learning Goals
Workshop. www.cwsei.ubc.ca/resources/learn_goals.htm

42. 6 Create: transform and combine ideas to create something new
develop,create,propose,formulate,design,invent
5 Evaluate: think critically about and defend a position
judge,appraise,recommend,justify,defend,criticize,evaluate
4 Analyze:: break down concepts into parts
compare,contrast,categorize,distinguish,identify,infer
3 Apply: apply comprehension to unfamiliar situations
apply,demonstrate,use,compute,solve,predict,construct,modify
2 Understand: demonstrate understanding of ideas, concepts
describe,explain,summarize,interpret,illustrate
1 Remember: remember and recall factual knowledge
define,list,state,label,name,describe
Bloom’s Taxonomy of the Cognitive Domain
(Levels of Learning)
Learning Outcomes - collegeclassroom.ucsd.edu Adapted from CarlWieman (2007)
www.cwsei.ubc.ca/resources/learn_goals.htm
42