Effective peer instruction requires good "choreography" and a good question. This workshop is part of our UCSD CTD Weekly Workshop series

1. slides available at ctd.ucsd.edu/2012/10/writing-clicker-
questions/
CREATING GOOD CLICKER
QUESTIONS
Peter Newbury
Center for Teaching Development
University of California, San Diego
pnewbury@ucsd.edu
@polarisdotca
#ctducsd
Cynthia Heiner
Department of Physics
Free University of Berlin, Germany
cynthia.heiner@fu-berlin.de
Teaching and Learning Weekly Workshop, November
8, 2012

2. Typical Peer Instruction
Episode poses a conceptually-challenging
1. Instructor
multiple-choice question.
2. Students think about question on their own.
3. Students vote for an answer using clickers,
coloured cards, ABCD voting cards,...
4. The instructor reacts, based on the
distribution of votes.
2 Creating good clicker questions

3. In effective peer instruction
 students teach each other immediately,
students learn
while they may still hold or remember
and practice
their novice misconceptions
how to
 students discuss the concepts in theirthink, commu
own language nicate like
experts
 the instructor finds out what the students know
(and don’t know) and reacts
3 Creating good clicker questions

4. Effective peer instruction
requires key
1. identifying
concepts, misconceptions before
class
2. creating multiple-choice questions that
require deeper thinking and learning
3. facilitating peer instruction episodes during
that spark student discussion class
4. resolving the misconceptions
4 Creating good clicker questions

5. What makes a good clicker
question?
clarity Students should waste no effort trying to
figure out what’s being asked.
context Is this topic currently being covered
in class?
connection to Does the question make students do the
learning goals right thing to demonstrate they grasp the
concept.
distractors What do the “wrong” answers tell you
about students’ thinking?
difficulty Is the question too trivial? too hard?
stimulates Will the question engage the students and
thoughtful spark thoughtful discussions?
discussion Is there potential for you to be “agile”?
(Adapted from Chasteen, CU Boulder)
5 Creating good clicker questions

6. Clicker question (History)
In your opinion, which had the most positive impact on
the modern world?
A) coffee
B) tea
C) chocolate
D) spice
E) sugar
clarity context learning goals distractors difficulty discussion
6 Creating good clicker questions

7. Clicker question (intro
Astronomy)reasons for the seasons?
Which of these are
i. the height of the Sun in the sky during the day
ii. Earth’s distance from the Sun
iii. how many hours the Sun is up each day
A) ii only
B) iii only
C) i and ii
D) i and iii
E) i, ii and iii
clarity context learning goals distractors difficulty discussion
7 Creating good clicker questions

8. Clicker question (intro
Astronomy)are reasons for the seasons?
How many of these
 the height of the Sun in the sky during the day
 Earth’s distance from the Sun
 how many hours the Sun is up each day
A) none of them
B) one
C) two
D) all three
clarity context learning goals distractors difficulty discussion
8 Creating good clicker questions

9. Clicker question (Calculus)
Evaluate: 4
2 3
x 1 x dx
0
3 2 2
A) 16 ( 65 ) C. ( 65
3 2
1)
9
1022
B) 16 D.
9 3
(adapted from Bruff)
clarity context learning goals distractors difficulty discussion
9 Creating good clicker questions

10. Clicker question (Calculus)
Which of the following is an incorrect step when using
the substitution method to evaluate the definite
integral 4
2 3
x 1 x dx
0
3 1 4
A) u 1 x C. u du
3 0
du 2
B) x dx D. none of the above
(Bruff)
3
clarity context learning goals distractors difficulty discussion
10 Creating good clicker questions

11. 11
Types of questions
and when to ask them
Creating good clicker questions

12. Clickers help students learn...
the learning cycle
BEFORE DURING AFTER
setting up developing assessing
instruction knowledge learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen
12 Creating good clicker questions

13. Clickers help students learn...
the learning cycle
BEFORE DURING AFTER
setting up developing assessing
instruction knowledge learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen
13 Creating good clicker questions

14. Clickers help students learn...
the learning cycle
BEFORE DURING AFTER
setting up developing assessing
instruction knowledge learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen
14 Creating good clicker questions

15. Clickers help students learn...
the learning cycle
BEFORE DURING AFTER
setting up developing assessing
instruction knowledge learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen
15 Creating good clicker questions

16. Clickers help teachers teach...
BEFORE DURING AFTER
setting up developing assessing
instruction knowledge learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen
16 Creating good clicker questions

17. Clickers help teachers teach...
Are they ready for the next topic?
What do they already know?
Do they care about this?
What DO they care about, anyway?
BEFORE DURING AFTER
setting up developing assessing
instruction knowledge learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen
17 Creating good clicker questions

18. Clickers help teachers teach...
Where are they in the activity?
Are they getting it?
Do I need to intervene?
Did they notice key idea X?
BEFORE DURING AFTER
setting up developing assessing
instruction knowledge learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen
18 Creating good clicker questions

19. Clickers help teachers teach...
Did they get it?
Can I move to the next topic?
Did that activity work?
How did I do?
BEFORE DURING AFTER
setting up developing assessing
instruction knowledge learning
Adapted from Rosie Piller, Ian Beatty, Stephanie Chasteen
19 Creating good clicker questions

20. predict
Clicker question
A ball is rolling around C
the inside of a circular B D
track. The ball A E
leaves the track
at point P.
P
Which path
does the ball
follow?
(Mazur)
20 Creating good clicker questions

21. What do you think is most discover
interesting about this picture?
A) lunar eclipse
B) solar eclipse
C) cloud blocking
the Sun
D) sunspots
E) other
Astronomy Picture of the Day 2011 January 5 (Newbury, Richer)
21 Creating good clicker questions

22. real-world application
Clicker question predict
An ice cube is floating in a glass of water
that is filled entirely to the brim. As the ice
cube melts, the water level will
A) stay the same, remain at the brim.
B) rise, causing the water to spill.
C) fall to a level below the brim.
D) cannot say without knowing the density of ice.
22 Creating good clicker questions

23. probe
Clicker question misconception
If this is the phase of the Moon when it
rises:
what is the phase of the Moon 12 C
A B hours
later?
D E
(Prather)
23 Creating good clicker questions

24. exercise skill
Clicker question
Susan throws a ball straight up into the air. It goes up
and then falls back into her hand 2 seconds later.
Draw a graph showing the velocity of the ball from the
moment it leaves her hand until she catches it again.
velocity
time
0 2 sec
(CWSEI UBC)
24 Creating good clicker questions

25. Which one is the closest match to your graph? exercise skill
velocity velocity
A B
time time
0 2 sec 0 2 sec
velocity velocity
C D
time time
0 2 sec 0 2 sec
E) some other graph (CWSEI UBC)
25 Creating good clicker questions

26. analysis
Clicker question
John is walking to school. This graph shows his position
as a function of time. When is John moving with the
greatest velocity?
position
time
A B C D E (CWSEI UBC)
26 Creating good clicker questions

27. evaluation
Clicker question
Which of the following is an incorrect step when using
the substitution method to evaluate the definite
integral 4
2 3
x 1 x dx
0
3 1 4
A) u 1 x C. u du
3 0
du 2
B) x dx D. none of the above
3
(Bruff)
27 Creating good clicker questions

28. check knowledge
Clicker question
For the data set displayed in the following
histogram, which would be larger, the mean or the
median?
A) mean
B) median
C) can’t tell from the given histogram
(Peck, mathquest.carroll.edu/resources.html)
28 Creating good clicker questions

29. “big picture”
Clicker question
In your opinion, which had the most positive impact on
the modern world?
A) coffee
B) tea
C) chocolate
D) spice
E) sugar
(Herbst)
29 Creating good clicker questions

30. exit poll
Clicker question
Would you like to learn more about MOOCs next class?
A) yes
B) no
(Newbury)
30 Creating good clicker questions

31. Clicker question (at driving
school)
Learning outcome: [after this lesson, you will be able
to] change lanes from a slower lane into a faster lane.
While you’re writing it, think about
what type of question is it? (predict, assess...)
when would you present it? (before, during, after)
critique your question for
clarity context learning goals distractors difficulty discussion
31 Creating good clicker questions

32. Resources
www.cwsei.ubc.ca/resources/clickers.htm
links to collections of peer instruction questions
ctd.ucsd.edu/2012/10/writing-clicker-questions/
CWSEI Eric Mazur Derek Bruff Doug Duncan
(1996) (2009) (2004, 2005)
32 Creating good clicker questions