1. Learning Assessment for the
21st Century
Jonathan E. Martin
www.21k12blog.net
jonathanemartin@gmail.com
ACIS Heads, January 2013

2. “How do we measure what we
value rather than value what we
measure?”

3. My interest because:
• I want to experiment and innovate more, and evaluate
progress on those experiments.
• I want to reassure anxious parents and trustees that we are
not jeopardizing academic excellence as we reinvent 21st
century learning.
• I want to be really sure we are teaching the skills kids need,
21st century capacities and digital literacies.
• I loved my independent school education and yet I see so
many ways it could have been so much more.
• I ran under-enrolled schools and had to show competitive
advantage.

4. Assess What
Matters Most

5. Kevin Mattingly, dean of faculty at Lawrenceville
School (NJ):
“We talk about 21st century skills and
conceptual frameworks, and we say, this is what
our kids need!
How, then, do we find out and evaluate what
they know, probing empathetically and fair-
mindedly?”

6. Less Actually, More Uncertainty
Measurement isn’t going away:
it isn’t likely to diminish in
importance in the coming years.
Governments, Funders, Foundations,
Parents/Consumers, Boards

7. Our Goal Today:
Considering the purposes served and value
provided by a variety of (mostly) new
assessment tools and techniques which might
better evaluate and advance the learning goals
of your mission, and in particular 21st century
capacities and differentiated instruction.

8. I Intro and Overview
II External Measurements of Mission
Short Activity
3 Tools I’ve used
4 Tests on the Horizon
Break
III Developing 21st c. Internal Assessments of Learning
3 Approaches
Short Activity
IV Exploring Non-Cog Assessment
Short Activity
V Digital Portfolios and Demonstrations of Learning
Short Activity
VI Q & A

9. Commission on Accreditation
Criterion 13
The standards require a school to provide
evidence of a thoughtful process, respectful
of its mission, for the collection and use
in school decision-making of data (both
external and internal) about student learning

10. ACIS Standard D9:
The school shall have a curriculum that is
articulated in detailed written form, and
a corresponding process to assess individual
student growth, development and achievement
that reflects the school's mission.

11. • Does the school have a process for assessing student growth,
development, and achievement that is consistent with the
mission? (See related standards A7, B1, D1.)
• Do the assessment methods correspond with the curriculum
so that teachers have a means for tracking student progress in
all areas of the educational program?
• Does the assessment process support effective communication
with parents about their children’s learning? (See related
standard C5, and D1.)
• Does the assessment process give teachers the information
needed to help them provide appropriate instruction that
meets the capacities, styles, and developmental needs of the
students enrolled in the school? (See related standard D1, D4).

12. 21st century capacities: the 7 C’s

13. Twenty-first century readers and writers need to
• Develop proficiency with the tools of technology
• Build relationships with others to pose and solve problems
collaboratively and cross-culturally
• Design and share information for global communities to meet a variety of
purposes
• Manage, analyze and synthesize multiple streams of simultaneous
information
• Create, critique, analyze, and evaluate multi-media texts
• Attend to the ethical responsibilities required by these complex
environments

14. Assessment: Measure What Matters:
– Technology-based assessments that
combine cognitive research and theory
about how students think with
multimedia, interactivity, and
connectivity make it possible to
directly assess these types of skills.
– This can be done within the context of
relevant societal issues and problems
that people care about in everyday
life.
– Systems can be designed to capture
students’ inputs and collect evidence
of their knowledge and problem-
solving abilities.
NETP

15. Will Richardson

17. Often an important decision requires better
knowledge of the alleged intangible, but when
an executive believes something to be
immeasurable, attempts to measure it will not
even be considered.
As a result, decisions are less informed than
they could be. The chance of error increases.
Resources are misallocated, good ideas are
rejected, and bad ideas are accepted.
measurement is a quantitatively
expressed reduction of
uncertainty based on one or
more observations.

18. Longitudinal Data
• The National Clearinghouse has 93% of all US
colleges collecting and providing longitudinal data
• Two independent school associations require
student tracking for accreditation (freshman GPA):
• CAIS Canada ntp@cais.ca
• ISASW www.isasw.org
Lyons/Niebold

19. New and Next Gen. External
Measurements of learning

20. Questions
• Assessment of learning Does the assessment respect
our mission and measure what we value? Does the
assessment measure 21st century skills and/or digital
literacies? Does it provide data we can use for
accountability, communications?
• Assessment as learning Are our students meaningfully
engaged in the experience of the assessment? Is
taking the test itself a learning experience.
• Assessment for learning Does the assessment
effectively provide data and resources we can use for
continuous improvement of learning?

21. Sample CWRA and HSSSE results
How could each of these assessment reports be
used to support and strengthen
the program at your school?
KentGuest

22. Measuring Key Thinking and
Communicating Skills

24. • “CAE is doing truly groundbreaking work in
developing assessments of the skills that
matter most in the 21st century.”
– Tony Wagner

26. Three Metrics
Internal growth at a school
Comparison to other participating schools
College readiness

27. 50
67
97
All College Freshmen St. Gregory Freshmen St. Gregory Seniors
CWRA/CLA median percentile,
College Freshman Normed

31. Assessment
of
Assessment
For
Assessment
As

32. Measuring Student Perspective
on Learning Key 21st c. Skills

33. Not just about Engagement
Engagement Matters- and there is excellent engagement data

34. student, teacher and parent surveys
collect data and generate reports
focused on school climate, classroom
conditions, and student engagement
34

35. What Works in the Classroom? Ask the Students
Teachers whose students described them as skillful at
maintaining classroom order, at focusing their instruction
and at helping their charges learn from their mistakes
are often the same teachers whose students learn the
most in the course of a year, as measured by gains on
standardized test scores, according to a progress report
on the research.

36. II. How do the results inform practice? (cont.)
• HSSSE complements assessment tests and performance data by
identifying school features that affect outcomes
• HSSSE stimulates discussion on effective teaching and learning
– Many schools use HSSSE data during PLCs or other PD activities to identify areas of
improvement to include in School Improvement Plan
– Can help in accreditation process which typically requires self-reflection
• Can be used in the fall and spring to investigate trends and changes in
student engagement over the course of a school year
• Many schools use it from one year to the next to track outcomes over time
• Educators report “aha moments” and “chances to celebrate”

37. I am engaged in School
76
88
90
All HSSSE 2009 STG 2009 STG 2010

38. 86
90
75
STG 2009 STG 2010 All Schools (Averaged)
I have opportunities to be
creative in the classroom
Percentage Strongly Agreeing

39. 72
82
92
All Schools St. Gregory 09 St. Gregory 10
We regularly discuss questions
with no clear answers

40. School emphasizes exploring new ideas
72
92 90
All HSSSE 2009 STG 2009 STG 2010

41. What Excites & Engages Me?
28
65
60
55
Lecture Discussion and
Debate
Group Projects Projects Involving
Technology
All HSSSE students

42. Assessment
of
Assessment
For
Assessment
As

43. Measuring better not what but how
Measurement for differentiation and individual
student learning growth

44. Measures of
Academic
Progress™
(MAP)

46. MAP Testing is a tool that becomes an
extension of our mission, which is to
maximize individual growth.
Pam Shaw, Canton Country Day School (OH)

47. • Traditional/Conventional Standardized Testing
Updated
• Computer Adaptive Assessment
• More frequent (3x) than once a year
• Reporting on not just achievement but growth
• More rapid feedback to teachers
• More detailed analysis of student proficiency
• Curricular resources aligned with achievement

48. “Formative Assessment works…when FA is used,
students learn better—lots better.”
W. James Popham, “Formative Assessment’s
Advocatable Moment.” Edweek, 1.8.13

49. Formative Assessment Integrated

51. Sample Math Question
• Notice the calculator.
• To use it, click the
numbers with your mouse.

52. Sample Reading Question
• Read the passage and
select the best answer
• Click the <Go on> button

55. DesCartes Example
Student Scores a RIT range of 171-180 on the Goal Strand of Estimation and Accurate
Computations

56. Teachers using online portal, grabbing student
reports and class reports– and using curriculum
alignment resources.
Parents and tutors getting these same reports
for supplemental instruction.
Reports correlated with state standardized
testing.

57. Online MAP portals

61. Assessment
of
Assessment
For
Assessment
As

62. On the horizon

63. generating student achievement
international comparison data

64. OECD TEST FOR SCHOOSL
BASED ON PISA
Have you ever wanted to
compare your school’s
effectiveness with the world’s
best national educational
systems?

65. BLOOD DONATION NOTICE
The instruments for taking the blood are sterile and
single-use (syringe, tubes, bags).
There is no risk in giving your blood.
Blood donation is essential.
There is no product that can fully substitute for
human blood. Blood donation is thus irreplaceable
and essential to save lives.
In France, each year, 500,000 patients benefit from a
blood transfusion.
Blood donation:
It is the best-known kind of donation, and takes from 45 minutes to 1
hour.
A 450-ml bag is taken as well as some small samples on which tests and checks will be
done.
- A man can give his blood five times a year, a woman three times.
- Donors can be from 18 to 65 years old.
An 8-week interval is compulsory between each donation.

66. Question 1
Level 2 item – 81.2% of students across OECD can perform tasks at least at
this level

67. Your school’s results are statistically significantly above
Your school’s average score is not significantly different
Your school’s results are statistically significantly below
Compared to Averages
Country or
Economy
Reading
literacy
Mathematical
literacy
Scientific
literacy
Shanghai-
China
Korea
Finland
Canada
United States
United
Kingdom
Germany
Turkey
OECD Average
Reading: 512 Mathematics: 513 Science: 507
Compared to Percentiles (to be developed)
Country or
Economy
Reading
literacy
Mathematical
literacy
Scientific
literacy
Shanghai-
China
Korea
Finland
Canada
United States
United
Kingdom
Germany
Turkey
OECD Average
EXAMPLE: International comparisons of
your school’s performance: PISA 2009
Countries and Economies
Section II. Your School’s Results in an
international context
School ABCD

68. Advantage
PISA Index of socio-economic background
Disadvantage
School performance and schools’ socio-economic background
Student performance and students’ socio-economic background within schools
Private school
Public school in rural area
Public school in urban area
700
EXAMPLE: Your school’s performance in
the context of PISA results for Canada
(with socio-economic background)
Section II. Your school’s results in an
international context
School ABCD
Reading: 512
Index
calculated
from student
questionnaires
200
493
-2 -1 0 1 2
Score
Student
performance
493
Your School
300

69. Will review option for schools to
electronically explore their
confidential results online
Proposed Structure of School Reports (Preliminary)
I. Understanding your school’s results from the PISA-
Based Test for Schools assessment
II. Your school’s results in an international context
III. Additional insights from international PISA results
Annexes
A. Summary of PISA-Based Test for Schools assessment questions (items,
units and response-types)
B. Summary Description of PISA Assessment Frameworks
Contents of print-ready reports will include:

71. OECD- PISA Based Testing for
Schools
The good and the bad:
• Measures valuable
application of knowledge.
• Don‘t think it constrains.
• Extraordinarily
comprehensive reporting.
• Administration not too
onerous.
• Price uncertain.

72. TIMMS

73. CAIS score reports for TIMMS replica test
ABC Country Day School
TIMMS “released item” test results
95% of students scored in the top 1/2 of I.A.
92% of students scored in the top 1/3 of I.A.
90% of students scored in the top 10% of I.A.

74. PISA and TIMMS
Assessment
of
Assessment
For
Assessment
As

75. sustaining competitive position with what
and how the public education “competition”
or alternative will be assessing soon

76. “Today is the day that marks the
beginning of the development of a new
and much-improved generation of
assessments for America’s
schoolchildren. Today marks that start
of Assessment 2.0”
Beyond the Bubble Tests: The Next
Generation of Assessments – Secretary
Arne Duncan’s Remarks to State
Leaders
September 2, 2010

77. “For the first time, many teachers will
have the state assessments they have
longed for—tests of critical thinking
skills and complex student learning
that are not just fill-in-the-bubble tests
of basic skills but support good
teaching in the classroom.”
September 2, 2010

78. “In performance-based tasks, which
are increasingly common in tests
administered by the military and in
other fields, students are given a
problem—they could be told, for
example, to pretend they are a mayor
who needs to reduce a city’s
pollution—and must sift through a
portfolio of tools and write analytically
about how they would use them to
solve the problem.”
September 2, 2010
New York Times

79. PARCC

81. PARCC and Smarter Balanced
2014-15
KEY SIMILARITIES
Summative Assessments:
•Online assessments for grades 3–8
and high school in English language
arts and literacy and in mathematics.
•Use of a mix of item types, including
selected-response, constructed-
response, technology-enhanced, and
complex performance tasks.
•Two required components, both
given during the final weeks of the
school year.
•Use of both electronic and human
scoring, with results expected within 2
weeks.
Other Assessments, Resources, and Tools:
•Optional interim assessments.
•Professional development modules.
•Formative items and tasks for classroom use.
•Model curricular and instructional units.
•Online reporting suite.
•Digital library for sharing vetted resources and
tools.

82. Type I: Tasks assessing concepts, skills and procedures

83. TYPE III: TASKS ASSESSING/MODELING
/ APPLICATIONS

85. Narrative Task (The Narrative Task broadens the way in which students may use this
type of writing. Narrative writing can be used to convey experiences or events, real or
imaginary. In this task, students may be asked to write a story, detail a scientific
process, write a historical account of important figures, or to describe an account of
events, scenes or objects, for example).

86. Research Simulation Task

87. As the public schools move their educational assessment
“beyond the bubble”
– more challenging,
– more authentic/situated performance tasks,
– more evaluation of 21st c. skills and digital literacies,
– more formative assessment
– more integrated teacher PD and collaboration tools
Where do we stand? What must we do to offer value?

88. Assessment
of
Assessment
For
Assessment
As

89. Paralleling some of the tools and resources of PARCC
Expanding 21st c. Capacities and Literacies to K-8

91. CBAL
• Extended, constructed-response tasks that are delivered by
computer and automatically scored.
• Pilot testing occurred in 2010 and 2011, spring of 2012.
• Tests should be available for use in 2012.
• Sample tests available online

94. break

95. Strengthening
assessment of
21st century
skills and digital
literacies in our
programs.

96. More holistic assessment of academic
achievement and 21st century capacities

97. PBL: Holistic Assessment

98. Project Based Learning
Re-conceptualize– not project oriented learning.
When re-conceptualized, rethink assessment:
PBL offers opportunity for a wide array of
assessment for a single, extended project.

99. Final Product:
Creativity and
Craft
Peer
Assessment
Self-
Assessment
Expert
Assessment
Project
Organization
Content
Knowledge
Demonstrated
Teamwork/
Collaboration

100. Source: Work that Matters.

101. Assessment Rubrics
www.bie.org/diy

102. Strengthening Assessment of
Digital Literacies and Deeper Learning

103. Open Computer Testing

105. OCT:
Open Computer Testing
What matters is no longer what one knows, but what
one can do with what one knows and with information
one can access, evaluate, and apply.
Tony Wagner

106. From BBC News, about Denmark
At five to nine, the room falls silent. One of the teachers
stands in front of the class and explains the rules. She tells
the candidates they can use the internet to answer any of
the four questions. They can access any site they like,
even Facebook, but they cannot message each other or
email anyone outside the classroom.
Minister for education in Denmark, Bertel Haarder, says:
“Our exams have to reflect daily life in the classroom and
daily life in the classroom has to reflect life in society. The
internet is indispensable, including in the exam situation.
I’m sure that is would be a matter of very few years when
most European countries will be on the same line.”

107. What do we want for our students?
• 21st c. skills and mindsets
• Authentic, real-world situated problem solving skills
• Lasting understanding of concepts
• Stronger Skills in Using tools they are already most
familiar and comfortable with.
• Preparation for professional careers

108. The Takeaways
• For Schools, OCT provides a vehicle to prioritize and
drive instruction toward 21st century skills, applied
thinking, and better preparation for future careers.
• For Teachers, OCT entails rethinking assessment and
designing “google-proof” questions which demand
higher order thinking and information literacy and
assess more lasting understanding.
• For Students, OCT when well designed is rigorous, and
more authentic, more motivating, and requires greater
information access, analysis, and application skills,
better organization, and stronger real-world
problemsolving skills.

109. Strengthening Assessment of 21st century and
higher order thinking skills

110. Performance Task Assessment
• Handout Sample

111. Read and review the document
Discuss at your table:
1. What would be elements of a quality answer?
2. What would be aspects of a poor answer?
3. What skills would a student need to have to
answer well?

112. Non-Cog

113. Exploring tools and methods for assessing
essential 21st century non-cog capacities

114. Wagner’s 7 Survival Skills
• critical thinking and problem solving
• effective oral and written communication
• accessing and analyzing information
-
• curiosity and imagination
• collaboration across networks and leading by
influence
-
• agility and adaptability
• initiative and entrepreneurship

115. In Eastern cultures, Stigler
says, it's just assumed that
struggle is a predictable part
of the learning process.
Everyone is expected to
struggle in the process of
learning, and so struggling
becomes a chance to show
that you, the student, have
what it takes emotionally to
resolve the problem by
persisting through that
struggle.

117. Measure your grit score–
– http://www.sas.upenn.edu/~duckwort/
» Or
– http://goo.gl/5RRjV
Find, get your grit score, complete, and discuss how
you might use with students.

119. Character and 21st c. skills report cards
www.21k12blog.net

120. St. Gregory’s “egg”
• Character
– Integrity
– Resilience
• Leadership
– Positive role model
– Facilitating Collaboration
• Scholarship
– Inquisitiveness
– Analytic thinking
– Synthetic thinking
– Critical thinking
• Innovation
– Adaptability
– Initiative
– Experimentation
www.21k12blog.net

121. Work ethic and perseverance
–Works hard
–Persists in the face of difficulty
–Takes responsibility for his/her own learning
–Accepts challenges as growth opportunities
–Is dedicated and determined to succeed

123. Robert J. Sternberg
Theory of Successful Intelligence
123
• Individuals are successfully intelligent to the
extent they display
– Creative skills to generate novel ideas
– Analytical skills to ascertain whether the ideas are
good ones
– Practical skills in order to implement their ideas
and persuade others of their value
– Wisdom-based/ethical skills in order to ensure
the ideas help to achieve a common good based
upon positive ethical principles

124. Source for Details
124
Sternberg, R. J. (2010).
College Admissions for the
21st Century. Cambridge,
MA: Harvard University
Press.

125. The Rainbow Project
Robert Sternberg125
• This project was conducted at Yale University in the
early 2000s on roughly 1000 students with freshmen
from 13 colleges and seniors from 2 high schools.
The institutions were geographically very dispersed
and were of greatly varying levels of selectivity, from
community colleges to highly prestigious ones. There
was a high level of ethnic diversity. Assessments
were proctored in the students’ own institutions but
scored at Yale.

126. Predicting GPA: SAT + Analytic, Creative, Practical
Step 1:
SAT-Verbal, SAT-Math
Step 2:
All Rainbow Project Items
(STAT Analytic, Practical,
Creative,
Practical performance,
Creative performance)
9.8
19.9
0
5
10
15
20
Step 1 Step 2
R
squared
(%)
126
Robert J. Sternberg

127. Predicting GPA: All measures (practical before creative)*
Step 1: SAT-M
SAT-V
HSGPA
Step 2: + Analytic
Step 3: + Practical
Step 4: + Creative
*Controlling for school quality in
dependent variable
15.6 15.2 15.9
24.8
0
5
10
15
20
25
30
Step
1
Step
2
Step
3
Step
4
R
squared
127
Robert J. Sternberg

128. In sum
128
• In the Rainbow sample,
–Adding Rainbow measures over SAT roughly
doubles prediction of college success
–Adding Rainbow measures over SAT + High
School GPA increases prediction by roughly
half
Robert J. Sternberg

129. Choate Rosemary Hall Project
129
This battery included a variety of measures to
enhance prediction of academic success in the
environment of Choate Rosemary Hall
Robert J. Sternberg

130. • A good internal locus of control—the ability to
shoulder the blame when things weren’t working, and
also to take credit for one’s own successes.
• Sensible self-confidence. Students who were overly
confident or lacked confidence did not adjust as well.
• A tacit knowledge about how the independent school
environment worked. Students with such knowledge
did a better job of mastering the environment over
time.”

131. Students respond to about 40 statements
describing their own views of themselves and
the world, rating them on a strongly agree-
strongly disagree scale.

132. Statements like:
I like to learn for the sake of learning.
Many people fail to get the recognition they
deserve no matter how hard they try.

133. • The survey is then generated into a report on
each student, explaining where that applicant lies
on a scale normed to their ages of self-efficacy,
locus of control, and intrinsic motivation.
• even brief self-reports forming the PACE SRL
assessments contribute approximately 10% of
unique predictive variance…
• Especially useful for outlier candidates.

134. Exeter
Oregon State
Depaul

135. Continued use after admission

136. Digital Portfolios
• High Tech High Digital Portfolios
• New Tech Network Digital Portfolios

137. Demonstrations of Learning:
“What you do, not what you know, is the
ultimate test of education.” ~PFB Tweet
Conduct a fluent conversation in a foreign language about
of piece of writing in that language.
Declaim with passion and from memory a passage that is
meaningful, of one’s own or from the culture’s literature or
history.
Invent a machine or program a robot capable of performing
a difficult physical task.

138. Demonstrations of Learning for 21st. C. School
By these demonstrations, schools…
• Reunite content and action.
• Backward-design curriculum from desired
outcomes.
• Demonstrate student outcomes recorded in
electronic portfolios.
• Facilitate student-led teacher/parent
conferences.
• Conduct action research and lesson study to
grow professionally.

139. History at Lakeland Prep:
In the four year History sequence at Lakeland Prep, all students will complete the following Demonstrations
of Learning:
• 24 research based position papers (4 to 7 pages) in which an
analysis, synthesis and/ or evaluation of both original and
modern sources is offered in answer to a provocative
question in history.
• 6 research based position papers (10 to 15 pages) focused on
a students original response to one of identified Essential
Questions in American History.
• 12 oral presentations
• 8 collaborative projects,
• 3 projects completed in collaboration with students in other
schools and/ or countries
• 4 interviews with elected officials
• 6 Letters to the Editor written on a current topic in local
and/or state government

140. Discussion
• What would you highlight in your students
portfolios as demonstrations of learning?
– Things your students already do, representative of
mission?
– What might you add?
• How would you highlight, display and measure
them?

141. Q & A

142. www.21k12blog.net
jonathanemartin@gmail.com
Twitter: @JonathanEMartin