د. هانز واقميكر المركز: المدير التنفيذي للمنظمة الدولية لتقيم التحصيل التربوي (IEA) نيوزيلندا

1. IEA’
Assessments:
Assessment
for
Policy
Reform
and
Improvement
Dr
Hans
Wagemaker
INTERNATIONAL
ASSOCIATION
FOR
THE
EVALUATION
OF
EDUCATIONAL
ACHIEVEMENT

2. Impetus
for
ILSA
• Growing
demand
for
data/evidence
based
policy
development.
• Growing
concerns
related
to
quality
of
outcomes,
equity,
efficiency-­‐
move
away
from
focus
on
inputs.
• GlobalizaSon
and
changing
economic
base
from
industrialized
to
knowledge
based
economies.

3. The
Demand
for
evidence
• “At
all
level
in
an
educaSonal
system,
from
the
teacher
in
the
classroom,
through
the
administrator
to
the
policymaker,
decision
have
conSnually
to
be
made,
most
of
the
Sme
on
the
basis
of
very
liVle
factual
informaSon’
(Postlethwaite,
1974).

4. Why
InternaSonal
Comparisons
•
“If
custom
and
law
define
what
is
educaSonally
allowable
within
a
naSon,
the
educaSonal
systems
beyond
one’s
naSonal
boundaries
suggest
what
is
educaSonally
possible”(Foshay
1962:p.
2).

5. The
Growth
of
ILSA
• Decades
since
the
mid
80’s
growth
in
Providers
• IEA:TIMSS,
PIRLS,
pre-­‐PIRLS,
ICCS,
TED’s-­‐M
• OECD
PISA,
TALIS,
PIACC
• UNESCO:
LLECE,
SACMEQ
• CONFENMEN:
PASEC
• Increased
diversity
–
economic,
cultural,
linguisSc

6. CumulaSve
growth
in
unique
IEA
parScipants
over
the
decades
(beginning
of
each
decade)
23
30
53
72
100
4
17
32
0
20
40
60
80
100
120
140
1970
1980
1990
2000
2011
Benchmarking
ParScipants
Regular
ParScipants

7. Design
• Grade
Based
RepresentaSve
Probability
Sample
• Key
TransiSon
Points
(viz.
PIRLS)
• Curriculum
Model-­‐
intended,
implemented,
achieved
curriculum
Knowledge,
Aitudes,
Skills,
DisposiSons-­‐
POLICY
FOCUS
• Comprehensive
background
data,
Country,
School,
Teacher,
Student,
Parent
• Trend
Design.

8.
Understanding
ILSA
Analysis
beyond
the
mean
and
InternaSonal
Rankings.
Intended
to
provide
insights
into
factors
contribuSng
to
our
understanding
of
outcomes.
Needs
to
be
integrated
with
contextual
informaSon.
Needs
to
be
understood
as
both
process
and
product.

9. ReporSng
Strategy
• Achievement
Outcomes-­‐
overall,
sub-­‐
domains,
achievement
benchmarks,
relaSonship
with
background
scales
• Encyclopedia
• ThemaSc
Reports
• Technical
Reports
• InternaSonal
Databases
• Released
Materials

10. Assessment
for
Policy
Reform
and
Improvement

11.
PercentilesofPerformance
95%ConfidenceIntervalforAverage(±2SE)
5th 25th 75th 95th
Botswana 419(3.7) i
Ψ Honduras 396(5.5) i
Ж Yemen 348(5.7) i
BenchmarkingParticipants
12 NorthCarolina,US 554(4.2) h
13 Florida,US 545(2.9) h
Quebec,Canada 533(2.4) h
Ontario,Canada 518(3.1) h
2 Alberta,Canada 507(2.5) h
Dubai,UAE 468(1.6) i
AbuDhabi,UAE 417(4.6) i
h Countryaveragesignificantlyhigherthan
thecenterpointoftheTIMSS4thgradescale
i Countryaveragesignificantlylowerthan
thecenterpointoftheTIMSS4thgradescale
100 200 300 400 500 600 700 800
TIMSS
2011
Grade
4

12. Grade
4
MathemaScs
Advanced
and
Low
InternaSonal
Benchmark
• Advanced
625:
Students
can
apply
their
understanding
in
a
variety
of
relaSvely
complex
situaSons
and
explain
reasoning
…Can
solve
mulS
step
problems
involving
whole
numbers
and
proporSons
…
increased
understanding
of
fracSons
and
decimals
•
Low
400:
Some
basic
mathemaScal
knowledge-­‐
can
add
and
subtract
whole
numbers
read
and
complete
simple
graphs

13.
Percentage
of
G4
mathemaScs
populaSon
reaching
internaSonal
benchmarks-­‐TIMSS
2011
43
39
4
1
78
80
23
9
94
97
58
33
99
100
85
64
1
0
15
26
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Singapore
Korea
New
Zealand
Iran
Below
Low
Low
Intermediate
High
Advanced

14.
Exhibit3.1: AchievementinMathematicsContentDomains
Country
Overall
Mathematics
AverageScale
Score
Number GeometricShapesandMeasures DataDisplay
Average
ScaleScore
Difference
fromOverall
MathematicsScore
Average
ScaleScore
Difference
fromOverall
MathematicsScore
Average
ScaleScore
Difference
fromOverall
MathematicsScore
2Singapore 606(3.2) 619(3.4) 13(0.8) h 589(3.6) –17(1.5) i 588(3.4) –18(1.7) i
Korea,Rep.of 605(1.9) 606(2.0) 1(1.6) 607(1.7) 2(1.4) 603(1.9) –2(2.0)
2HongKongSAR 602(3.4) 604(3.3) 3(1.0) h 605(3.4) 3(0.9) h 593(3.6) –8(2.1) i
ChineseTaipei 591(2.0) 599(2.0) 8(1.2) h 573(2.1) –19(1.3) i 600(2.6) 9(1.6) h
Japan 585(1.7) 584(1.6) –1(0.9) 589(2.0) 4(1.1) h 590(2.9) 4(2.9)
†NorthernIreland 562(2.9) 566(2.9) 4(1.6) h 560(3.3) –2(2.1) 555(3.0) –8(1.5) i
Belgium(Flemish) 549(1.9) 552(2.2) 2(1.4) 552(2.0) 3(1.0) h 536(3.0) –13(2.0) i
Finland 545(2.3) 545(2.3) 0(0.9) 543(2.9) –2(2.2) 551(3.5) 5(3.2)
England 542(3.5) 539(3.7) –3(1.1) i 545(3.9) 3(1.6) 549(4.6) 7(2.9) h
AchievementinMathematicsContentDomains

15.
Exhibit3.4: AchievementinMathematicsCognitiveDomains
Country
Overall
Mathematics
AverageScale
Score
Knowing Applying Reasoning
Average
ScaleScore
Difference
fromOverall
MathematicsScore
Average
ScaleScore
Difference
fromOverall
MathematicsScore
Average
ScaleScore
Difference
fromOverall
MathematicsScore
Korea,Rep.of 613(2.9) 616(2.9) 3(1.9) 617(2.9) 4(1.1) h 612(2.5) 0(1.0)
2Singapore 611(3.8) 617(3.8) 6(1.0) h 613(3.9) 2(0.7) h 604(4.3) –7(1.0) i
ChineseTaipei 609(3.2) 611(3.7) 2(1.4) 614(3.5) 5(1.7) h 609(3.4) 0(1.5)
HongKongSAR 586(3.8) 591(3.9) 6(1.2) h 587(3.7) 2(1.0) 580(3.9) –6(1.1) i
Japan 570(2.6) 558(2.7) –12(1.5) i 574(2.5) 4(1.3) h 579(3.0) 9(1.8) h
2RussianFederation 539(3.6) 548(3.6) 9(1.0) h 538(3.5) –1(1.3) 531(3.7) –8(1.2) i
3Israel 516(4.1) 516(4.1) 0(1.1) 513(4.4) –3(1.4) i 520(4.0) 4(1.7) h
Finland 514(2.5) 508(2.5) –6(1.0) i 520(2.5) 6(1.4) h 512(2.7) –2(1.5)
2UnitedStates 509(2.6) 519(2.7) 10(0.8) h 503(2.8) –6(1.0) i 503(2.7) –6(0.7) i
‡England 507(5.5) 501(5.4) –5(1.1) i 508(5.5) 2(1.2) 510(5.5) 3(2.0)
: AchievementinMathematicsCognitiveDomains

16.
Exhibit4.6: StudentsSpeaktheLanguageoftheTestatHome
ReportedbyStudents
Country
AlwaysorAlmostAlways Sometimes Never
Percent
ofStudents
Average
Achievement
Percent
ofStudents
Average
Achievement
Percent
ofStudents
Average
Achievement
Armenia 94(0.5) 469(2.7) 5(0.4) 440(6.5) 1(0.1) ~~
Australia 93(0.9) 504(5.0) 6(0.8) 516(10.4) 1(0.2) ~~
Bahrain 77(0.7) 405(2.2) 18(0.6) 439(4.2) 5(0.5) 384(9.3)
Chile 96(0.3) 419(2.6) 4(0.3) 362(7.8) 0(0.1) ~~
ChineseTaipei 92(0.7) 616(3.0) 7(0.6) 535(7.7) 1(0.2) ~~
England 95(0.7) 508(5.5) 4(0.6) 491(11.9) 1(0.2) ~~
Finland 97(0.4) 515(2.5) 2(0.3) ~~ 1(0.1) ~~
: StudentsSpeaktheLanguageoftheTestatHome

17.
Exhibit3.11: AchievementinMathematicsCognitiveDomainsbyGender
Country
Knowing Applying Reasoning
Girls Boys Girls Boys Girls Boys
Armenia 463(4.2) 459(4.3) 450(4.5) h 443(4.1) 444(4.6) 441(4.5)
Australia 513(4.0) 520(4.6) 517(3.5) 521(3.7) 509(3.0) 518(3.8) h
Austria 505(2.6) 510(3.1) 499(2.9) 512(3.3) h 505(3.4) 521(4.0) h
2 Azerbaijan 476(7.1) 470(6.5) 461(6.6) 454(6.2) 449(6.5) 441(6.4)
Bahrain 440(4.7) 436(4.8) 436(4.7) 426(4.3) 439(4.5) 440(4.5)
Belgium(Flemish) 562(2.1) 567(2.5) 542(2.6) 549(2.4) h 527(3.4) 537(2.7) h
Chile 452(3.2) 459(2.9) h 459(3.2) 467(3.0) h 461(3.0) 476(3.3) h
ChineseTaipei 599(3.0) 599(2.5) 596(2.6) 591(2.3) 578(3.8) 577(2.5)
2 Croatia 490(3.0) 499(2.5) h 477(2.5) 491(2.4) h 487(2.9) 498(4.1) h
CzechRepublic 497(3.0) 507(3.2) h 505(3.2) 519(3.2) h 520(3.1) 525(3.3)
2 Denmark 527(3.1) 536(3.2) h 537(3.0) 541(3.5) 541(3.5) 544(3.2)
England 550(4.6) 554(5.0) 540(4.1) 544(4.2) 529(5.0) 533(3.8)
Finland 543(2.8) 553(3.1) h 540(3.0) 548(3.1) h 543(3.2) 548(3.2)
1 Georgia 452(3.4) 447(4.7) 452(3.3) h 443(4.5) 452(3.6) 449(4.4)
Germany 518(2.5) 529(2.9) h 525(2.6) 531(2.8) 526(3.1) 538(3.8) h
AchievementinMathematicsCognitiveDomainsbyGender

18.
Exhibit4.10: StudentsAttendedPreprimaryEducation*
CurriculumReportedbyNationalResearchCoordinatorsandPreprimaryAttendanceReportedbyParents
Country
National
Preprimary
CurriculumIncludes
MathematicsSkills
StudentsAttendedPreprimaryEducation
3YearsorMore
Lessthan3Years
butMorethan1Year
1YearorLess DidNotAttend
Percentof
Students
Average
Achievement
Percentof
Students
Average
Achievement
Percentof
Students
Average
Achievement
Percentof
Students
Average
Achievement
Hungary k 86(0.9) 526(3.1) 12(0.7) 473(6.6) 1(0.3) ~~ 0(0.1) ~~
Italy k 74(0.9) 515(2.6) 23(0.8) 497(3.4) 1(0.2) ~~ 1(0.2) ~~
Germany r j 74(0.9) 536(2.3) 23(0.9) 528(3.0) 1(0.2) ~~ 1(0.2) ~~
Sweden k 74(1.1) 513(2.0) 20(1.0) 499(2.7) 2(0.4) ~~ 3(0.4) 485(8.5)
Norway k 72(1.6) 500(3.1) 24(1.4) 486(3.8) 2(0.2) ~~ 3(0.5) 490(11.4)
Austria j 69(1.5) 511(3.0) 27(1.3) 509(3.7) 3(0.7) 500(7.5) 1(0.1) ~~
RussianFederation j 68(1.3) 545(3.5) 14(0.8) 542(5.5) 3(0.3) 530(10.0) 15(1.0) 531(6.4)
HongKongSAR k 68(1.0) 609(2.9) 31(1.0) 604(2.8) 1(0.1) ~~ 0(0.1) ~~
CzechRepublic j 68(1.1) 516(2.8) 28(0.9) 507(3.2) 3(0.4) 508(7.2) 1(0.2) ~~
Spain k 66(1.1) 492(2.8) 28(1.0) 474(4.0) 4(0.4) 466(6.4) 3(0.3) 469(7.9)
SlovakRepublic 65(1.3) 520(3.1) 24(0.8) 497(4.4) 8(0.7) 483(6.7) 4(0.7) 464(16.4)
Exhibit4.10:StudentsAttendedPreprimaryEducation*

19.
The
Challenge
of
Growth
and
enhancing
Impact
•
Detail
is
important-­‐
understand
purpose
• Thinking
globally
analysing/acSng
locally
• ILSA
as
process
and
product
• IntegraSon
of
Framework,
Encyclopaedia,
Background
data,
Technical
reports.
Quality
Assurance
procedures,
Training
• Need
strategic
investment/vision/ownership
• Enhance
basic
template.
• Training
• Policy
-­‐Research
integraSon

21.
Dr
Hans
Wagemaker
INTERNATIONAL
ASSOCIATION
FOR
THE
EVALUATION
OF
EDUCATIONAL
ACHIEVEMENT
TextText
Thank
You!
hanswagemaker@compuserve.com