1. Black & Wiliam (1998)
Assessment in Education, p. 61
• “As an illustration of just how big these
gains are, an effect size of .70, if it could
be achieved on a nationwide scale, would
be equivalent to raising the mathematics
attainment score of an ‘average’ country
like England, New Zealand or the United
States into the ‘top five’ after the Pacific
rim countries of Singapore, Korea, Japan
and Hong Kong” (Beaton et al, 1996)
100 100
Increase of 49%ile
80 80
Increase of 34%ile to 99%ile
%ile improvement increase
%ile improvement increase
to 84%ile 28%ile increase
13%ile increase to 78%ile
60 60
to 63%ile
Starting percentile Starting percentile Starting percentile Starting percentile
50th 50th 50th 50th
40 40
20 20
0 0
Teacher Student Teacher Student
assessment Achievement assessment Achievement
effectiveness effectiveness
Like most things in education, classroom assessment enhances
John Hattie—reviewed 7,827 studies on learning and instruction.
student achievement under certain conditions only.
Conclusion… “The most powerful single
innovation that enhances achievement • Feedback from classroom assessments should provide students with a clear
picture of
is feedback. The simplest prescription their progress on learning goals and
for improving education must be how they might improve
‘dollops’ of feedback.”
• Feedback from classroom assessment should encourage students to improve.
• Classroom assessment should be formative in nature.
• Formative classroom assessments should be quite frequent.
1

2. Tracking My Own Learning
Student Name______________________ Date__________
Topic
____________________________________________________
____________________________________________________
My score at beginning:_______________ My goal:_________ by ______________
100%
80%
60%
40%
20%
0
a b c d e f g h
Pretest 2/12 (48%)
a________________ e________________
Quiz 2/15 (60%)
b________________ f_________________
Quiz 2/19 (60%)
c________________ g________________
d________________ h________________
2

3. • Feedback from classroom assessments should provide students with a clear picture of
their progress on learning goals and
how they might improve
•Identify one grade level (or course) learning goal per quarter o r per
Bangert-Drowns, Kulik, Kulik, & Morgan, 1991
semester for each of the following subject areas: mathematic, reading,
# of studies Characteristic of Feedback from Percentile
writing, science, and social studies.
Classroom Assessment Gain/Loss
•Construct a rubric, or other type of common scale, for each learning
goal.
6 Right/wrong -3
•Have teachers formally and informally assess each learning goal at least
once every two weeks keeping track of each student’ s score on each 39 Provide correct answers 8.5
learning goal. (Use of appropriate computer software is highly
recommended) 30 Criteria understood by 16
student vs. not understood
•Have students keep track of their progress on each goal and use the data
as the basis for teacher/student interactions about student progress. 9 Explain 20
•Periodically (at least, once per quarter) aggregate the data by grade 4 Student reassessed until 20
level. Have teachers meet to discuss student progress and how it might
correct
be improved
• Feedbackfrom classroom assessments should provide students with a clear A. Items 1-1 0
picture of
Ten items that require recall of
their progress on learning goals and Total for section=
important but simpler content
how they might improve that was explicitly taught
Fuchs & Fuchs 1988
B. Items 11-14
# of studies Characteristic of Feedback Percentile
32
from Classroom Assessment Gain/Loss
Four items that ask for
49 Evaluation by rule application of complex content
Total for section=
that was explicitly taught AND in
89 [uniform way of
Displaying results 26
situations similar to what was
graphically
interpreting results
taught.
49
of classroom
Evaluation by rule 32
C. Item 15-16
assessments using
[uniform way of
interpreting results of
a tight logic) Two items that asks for
Total for section=
classroom assessments application in novel situations
using a tight logic)
that go beyond what was
explicitly taught
Total /100
A. Items 1-1 0 A. Items 1-1 0
Ten items that require recall of Ten items that require recall of
/40 /40
Total for section= Total for section=
important but simpler content important but simpler content
that was explicitly taught that was explicitly taught
+
All correct
B. Items 11-14 B. Items 11-14
Four items that ask for Four items that ask for
application of complex content application of complex content
/40 /40
Total for section= Total for section=
that was explicitly taught AND in that was explicitly taught AND in
situations similar to what was situations similar to what was Two correct +
taught. taught.
C. Item 15-16 C. Item 15-16
Two items that asks for Two items that asks for
/20
Total for section=
Total for section= /20
application in novel situations application in novel situations
None correct
that go beyond what was that go beyond what was
explicitly taught explicitly taught
Total /100 Total /100
3

4. A. Items 1-1 0
Ten items that require recall of
Total for section= 40/40
important but simpler content
that was explicitly taught
All correct +
B. Items 11-14
Four items that ask for
application of complex content
Total for section= 20/40
that was explicitly taught AND in
situations similar to what was Two correct
taught. +
C. Item 15-16
Two items that asks for
Total for section= 0/20
application in novel situations
None correct
that go beyond what was
explicitly taught
Total 60 /100
4
A generic template for 3 The student’s responses
rubric design demonstrate no major errors
or omissions regarding any
of the information and/or
processes (THAT WERE
EXPLICITLY TAUGHT)
2
1
0
4
4
The students responses demonstrate no major errors or
’
3 omissions regarding any of the information and/or
The student’s responses demonstrate no
3 processes
major errors or omissions regarding any of
The students responses indicate major errors or
’
the information and/or processes
2 omissions regarding the more complex ideas and
processes; however they do not indicate major errors or
2 The student’s responses omissions relative to the simpler details and processes
indicate major errors or 1 The student provides
omissions regarding the responses that indicate a
more complex ideas and distinct lack of
processes; however they do understanding of the
not indicate major errors or knowledge. However, with
omissions relative to the help, the student
simpler details and demonstrates partial
processes
1 understanding of some of
the knowledge.
0
0
4

5. 4 In addition to exhibiting level
4
3 performance, the student’s
The students responses demonstrate no major errors or
’
3 responses demonstrate in-
omissions regarding any of the information and/or
processes
depth inferences and
The students responses indicate major errors or
’
2 omissions regarding the more complex ideas and
applications that go beyond
processes; however they do not indicate major errors or
omissions relative to the simpler details and processes
what was taught in class
The student provides responses that indicate a distinct
1 lack of understanding of the knowledge. However, with The students responses demonstrate no major errors or
’
3
help, the student demonstrates partial understanding of omissions regarding any of the information and/or
some of the knowledge. processes
0 The student provides little or The students responses indicate major errors or
’
2 omissions regarding the more complex ideas and
no response. Even with help processes; however they do not indicate major errors or
omissions relative to the simpler details and processes
the student does not exhibit The student provides responses that indicate a distinct
1 lack of understanding of the knowledge. However, with
a partial understanding of help, the student demonstrates partial understanding of
some of the knowledge.
the knowledge. The student provides little or no response. Even with help
0 the student does not exhibit a partial understanding of the
knowledge.
Scale
Three Types of Items
In addition to exhibiting level 3 performance, in-depth inferences and applications
4
that go BEYOND what was taught in class.
• Level 2 items: Simpler details and processes
No major errors or omissions regarding any of the information and/or processes
3
(SIMPLE OR COMPLEX) that were explicitly taught
that have been explicitly taught.
No major errors or omissions regarding the SIMPLER details and processes BUT
2
• Level 3 items: Complex ideas and processes
major errors or omissions regarding the more complex ideas and processes
1 that have been explicitly taught.
With HELP, a partial knowledge of some of the simpler and complex details and
processes
• Level 4 items: Inferences and applications that
Even with help, no understanding or skill demonstrated.
0
go beyond what was taught
Patterns of Responses Patterns of Responses
• Student answers L2 items correctly but not
• Student answers L2 items correctly but not
L3 and L4 items. (2.0)
L3 and L4 items.
• Student answers L2 and L3 items correctly
• Student answers L2 and L3 items correctly
but not L4 (3.0)
but not L4
• Student misses all items, but with help can
• Student misses all items, but with help can
answer some correctly (1.0)
answer some correctly
• Students misses all items even when
• Students misses all items even when
helped (0.0)
helped
5

6. Scale
In addition to exhibiting level 3 performance, in-depth inferences and applications
4
that go BEYOND what was taught in class.
The complete scale allows for No major errors or omissions regarding any of the information and/or processes
3
(SIMPLE OR COMPLEX) that were explicitly taught
half-point scores No major errors or omissions regarding the SIMPLER details and p rocesses BUT
2
major errors or omissions regarding the more complex ideas and processes
(3.5, 2.5, 1.5, .5) 1 With HELP, a partial knowledge of some of the simpler and complex details and
processes
Even with help, no understanding or skill demonstrated.
0
Scale A. Items 1-10 Level 2.0
Ten items that require recall of
In addition to exhibiting level 3 performance, in-depth inferences and applications
4
that go beyond what was taught in class.
important but simpler content
3.5 In addition to exhibiting level 3 performance, partial success at in-depth
that was explicitly taught
+
All correct
inferences and applications that go beyond what was taught in class.
No major errors or omissions regarding any of the information and/or processes
3 B. Items 11-14 Level 3.0
(SIMPLE OR COMPLEX) that were explicitly taught
Four items that ask for
2.5 No major errors or omissions regarding any of the simpler information and/or
processes and partial knowledge of the more complex information and processes. application of complex content
that was explicitly taught AND in
No major errors or omissions regarding the simpler details and processes BUT
2
major errors or omissions regarding the more complex ideas and processes situations similar to what was Two correct +
1.5 Partial knowledge of the simpler details and processes, but majo r errors or taught.
omissions regarding the more complex ideas and processes.
C. Item 15-16 Level 4.0
With help, a partial knowledge of some of the simpler and complex details and
1
processes.
Two items that asks for
.5 With help, a partial knowledge of some of the simpler details and processes but
application in novel situations
not of the more complex ideas and processes.
None correct
that go beyond what was
Even with help, no understanding or skill demonstrated.
0
explicitly taught
Rubric Score:
Averages and Trend
A. Items 1-10 Level 2.0
Scores
Ten items that require recall of Student 1 Student 2 Student 3
important but simpler content 2.0 3.0 2.0
that was explicitly taught 1.5 2.0 1.0
+
All correct
2.0 2.0 1.5
B. Items 11-14 Level 3.0
3.0 2.5 2.0
Four items that ask for 2.5 3.0 2.0
application of complex content 3.0 2.0 2.5
that was explicitly taught AND in 3.0 3.0 3.0
situations similar to what was Two correct +
taught. 2.5 2.5 3.0
3.0 3.0 3.5
C. Item 15-16 Level 4.0
3.0 3.0 3.0
Two items that asks for
application in novel situations Average 2.55 2.60 2.35
None correct
that go beyond what was Trend 3.00 2.71 3.00
explicitly taught Score
Rubric Score:2.5
6

7. In search of the “true score”
Power Law
100
• True Score=Observed Score + Error
90
80
70
• SAT SEM= 33 points
learning
60
50
•GRE SEM = 45 points
40
30
20
10
0
1 rep 10 20 100 1000 2000 3000 4000
reps reps reps reps reps reps reps
1.0 2.5
1.5 2.0
2.5
2.0
2.0
2.0
Making Standards-Based
Reporting Work
• 20 or fewer elements per subject, per
grade level, per year
• a residual category for teacher
supplemental content
• a uniform way of scoring assessments and
assignments that is RIGOROUS
7

8. Language Arts Reporting Topics
If you wanted to teach all of the standards in the
national documents, you would have to change
schooling from K-12 to K-22 .
• Reading
– Comprehension
• 255 standards across 14 subject areas
– Word analysis
• 3,500 benchmarks
– Genre and literary devices
• 13,000 hours of class time available
– The research process
• 9,000 hours of instruction available
– Information gathering and organization
• 15,500 hours of instruction needed to – Technical material
cover the 3,500 benchmarks
Language Arts Reporting Topics Language Arts Reporting Topics
• Writing • Speaking and Listening
– The writing process – Structure and logic of presentations
– Overall logic and complexity of thought – Delivery techniques
– Adaptation to audience and purpose – Listening comprehension
– Conventions – Group discussion
– Use of writing formats
Mathematics Reporting Topics Mathematics Reporting Topics
• Number Operations and Concepts • Geometry
– Basic number concepts and operations – Lines and angles
– Fractions, proportions, decimals,& percents – Shapes and figures
– Exponents, roots, & factors – Motion geometry, transformations,
congruence, & similarity
– Problem solving & mathematical reasoning
8

9. Mathematics Reporting Topics Mathematics Reporting Topics
• Measurement • Algebra
– Units and systems of measurement – Expressions, equations, & functions
– Area, perimeter, circumference,& angles – Graphs and graphing systems
– Capacity, weight, mass, & volume
– Time
Topic Grade 8: Atmospheric Processes & Water Cycle
Mathematics Reporting Topics
4
An understanding of:
3
• Data Analysis and Probability
•How the water cycle processes (condensation,
– Data organization and display precipitation, surface run-off, percolation, evaporation)
impact climate changes
– Central tendency & dispersion
•The effects of temperature and pressure in different layers
– Probability and hypothesis testing
of Earth’ s atmosphere
2
1
0
Topic Grade 8: Atmospheric Processes & Water Cycle Topic Grade 6-8: Assignments & Work Completion
4 4
An understanding of:
•Hand in assignment that meet format requirements
3 3
•How the water cycle processes (condensation, precipitation, surface run-off, percolation, evaporation) impact climate changes
•The effects of temperature and pressure in different layers of Earth’s atmosphere
specified by teacher
•Recognize and recall basic terms such as: climactic
2 •Develop and implement basic time management plan for
patterns, atmospheric layers, stratosphere, troposphere. assignments
•Recognize or recall isolated details such as: •Complete assignments on time and provide acceptable
–Precipitation is one of the processes of the water cycle explanation when assignments not handed in on time
–The troposphere is one of the lowest portions of the earth’s
atmosphere
2
1
1
0
0
9

10. Topic Grade 6-8: Assignments & Work Completion
4
•Hand in assignment that meet format requirements specified by teacher
3 •Develop and implement basic time management plan for assignments
•Complete assignments on time and provide acceptable explanation when assignments not handed in
on time
2 •Be aware of format requirements for assignments
•Be aware of elements of basic time management plans
•Be aware of deadlines for assignments
1
0
Factors Mediating Leadership Leadership for Incremental
Behavior Change
• Emphasize relationships
• Establish strong lines of communication
Focus of the change
• Be an advocate for the school
and • Provide resources
• Maintain visibility
Order of the change • Protect teachers from distractions
• Create culture of collaboration
• Look for and celebrate successes
Leadership for Second Order
Change
• Shake up the status quo
• Expect some things to seem worse
• Propose new ideas
• Operate from strong beliefs
• Tolerate ambiguity and dissent
• Talk research and theory
• Create explicit goals for change
• Define success in terms of goals
10