Please visit my website for more information: http://www.comparative-education.com/

2. Workshop Objectives
2. Identify areas of
1. Identify how large- educational policy
scale data is being interest in order to
used for evidenced- synthesize large-scale
based policymaking data and apply it to
in the Gulf. educational
problems in the Gulf.
3. Develop a draft plan or
recommendations for the
evaluation and investigation of
large-scale data for use in
evidence-based policymaking
among Gulf education leaders.

3. Workshop Objectives
2. Identify areas of
1. Identify how large- educational policy
scale data is being interest in order to
used for evidenced- synthesize large-scale
based policymaking data and apply it to
in the Gulf. educational
problems in the Gulf.
3. Develop a draft plan or
recommendations for the
evaluation and investigation of
large-scale data for use in
evidence-based policymaking
among Gulf education leaders.

4. Workshop Objectives
2. Identify areas of
1. Identify how large- educational policy
scale data is being interest in order to
used for evidenced- synthesize large-scale
based policymaking data and apply it to
in the Gulf. educational
problems in the Gulf.
3. Develop a draft plan or
recommendations for the
evaluation and investigation of
large-scale data for use in
evidence-based policymaking
among Gulf education leaders.

5. The Phenomenon
• Large-scale, international, education
assessment and data are widely available
– TIMSS, PIRLS, PISA, UNESCO, etc.
• Gulf country participation is high
• Arab country participation is high
BUT…

6. The Problem
• Gulf country performance is low
• Arab country performance is low
• Few secondary analyses of data for Gulf
countries
• Little use of data for evidence-based
education policymaking or reform in Gulf
countries

7. Significantly Above International Mean
Not Significantly Different from International Mean
Significantly Below International Mean

8. Significantly Above International Mean
Not Significantly Different from International Mean
Significantly Below International Mean

9. The Need
1. Bottom of Gulf and Arab achievement rankings
2. Significantly belowinternational mean in math and science
achievement
What do rankings tell us?
• Rankings are a crude benchmarking tool, and are “reductionist”
– i.e., they ignore interesting and important variation between
content and cognitive domains.
Where should we look first?
• Comparisons provide important information, but need to be
used in stages: (1) internal comparisons, (2) regional
comparisons, (3) international comparisons.

10. Math Content Domains Math Cognitive Domains

11. Internal comparisons have priority.
CHINESE TAIPEI
• Excelling in algebra
• Struggling in data & chance
• Slightly low, but consistent
across all cognitive domains

12. Math Content Domains Math Cognitive Domains

13. Internal comparisons have priority.
CHINESE TAIPEI SAUDI ARABIA
• Excelling in algebra • Excelling in geometry
• Struggling in data & chance • Struggling in numbers
• Inconsistent across all
• Slightly low, but consistent
cognitive domains, but
across all cognitive domains
strongest in knowing

14. Science Content Domains Science Cognitive Domains

15. Internal comparisons have priority.
SINGAPORE
• Excelling in physics
• Struggling in earth science
• Consistent in knowing and
reasoning, but weaker in
applying

16. Science Content Domains Science Cognitive Domains

17. Internal comparisons have priority.
SINGAPORE SAUDI ARABIA
• Excelling in physics • Excelling in earth science
• Struggling in earth science • Struggling in chemistry
• Consistent in knowing and
• Strong in applying, but
reasoning, but weaker in
weaker in reasoning
applying

18. Why does the Gulf struggle?
It is impossible to improve Gulf education, if educational
policymakers and reformers don’t know where the problems are.
From our Saudi example, the evidence suggests…
Which content areas need the most attention in the KSA?
• Numbers (mathematics)
• Chemistry (science)
Which cognitive areas need the most attention KSA?
• Reasoning (mathematics)
• Reasoning (science)
Why do these content and cognitive areas need attention?
…because they tell us about the teaching and learning process in
Saudi schools and classrooms.

19. The Potential
How does large-scale data help us understand education?
• High-quality data for evidence-based decision-making
• International benchmarks
• Achievement & activity
• Comparative strengths & weaknesses
• Peer groups & target groups
What benefits do large-scale assessments bring?
• Increase understanding of key school and non-school factors
• Resource for identifying areas of concern and action
• Develop and improve capacity for national monitoring &
improvement
• Worldwide community of researchers

20. The Promise (i.e., Target)
1. Significantly above international mean in math and science
achievement
2. National education systems characterized by:
• Consistently high-achieving students
• Motivated, expert teachers
• Strong parent and community support
• International reputation for excellence
• Regional reputation as educational leader
3. Knowledge society and economy fed by Gulf schools

21. How can we make recommendations?
Infra-
structure
Capacity
Sustain-
ability
Large-scale data use
recommendations

22. How can large-scale education data help Gulf
educators and students rise to the top?
Infrastructure + Capacity + Sustainability = Culture
CREATE
DEVELOP TESTING
SUSTAINABLE
& ASSESSMENT
MODELS &
INFRASTRUCTURE
PROCEDURES
BUILD CAPACITY IN
RELEVANT KNOWLEDGE,
SKILLS, ATTITUDES &
VALUES

23. SHORT TERM LONG TERM
OUTCOMES OUTCOMES
• Network of experts • Sustainable
and colleagues collaboration
• Set of policy (grants, pubs,
recommendations reports)
• High-quality • Systematic policy
volume from influence (reports,
Gerlach Press training, evidence-
base)