Lessons from PISA: Strong Preformers and Successful Reformers
1. Programme for International Student Assessment
1
1
Strong performers and successful reformers
Andreas Schleicher
Strong performers and
7 July 2011
successful reformers
Lessons from PISA
PISA
International Student Assessment
OECD Programme for
Andreas Schleicher
Special advisor to the Secretary-General on Education Policy
Head of the Indicators and Analysis Division, EDU
2. Australia
Austria
Czech Republic
A world of change – higher education
Denmark
Finland
Germany
Greece
Expenditure per student at tertiary level (USD)
Hungary
Iceland
Ireland
Italy
Cost per student
Japan
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Republic
Spain
Sweden
United Kingdom
Graduate supply
United States Tertiary-type A graduation rate
3. A world of change – higher education
Expenditure per student at tertiary level (USD)
Cost per student
United States
Finland
Japan
Graduate supply
Tertiary-type A graduation rate
4. A world of change – higher education
Expenditure per student at tertiary level (USD)
Australia
United Kingdom Finland
Tertiary-type A graduation rate
5. A world of change – higher education
Expenditure per student at tertiary level (USD)
Tertiary-type A graduation rate
6. A world of change – higher education
Expenditure per student at tertiary level (USD)
Tertiary-type A graduation rate
7. A world of change – higher education
Expenditure per student at tertiary level (USD)
Tertiary-type A graduation rate
8. A world of change – higher education
Expenditure per student at tertiary level (USD)
Tertiary-type A graduation rate
9. A world of change – higher education
Expenditure per student at tertiary level (USD)
Tertiary-type A graduation rate
10. A world of change – higher education
Expenditure per student at tertiary level (USD)
United States
United Kingdom Australia
Finland
Tertiary-type A graduation rate
11. 11
11
PISA 2009 in brief
PISA countries in 2001
2003
2000
2009
2006
1998
Strong performers and successful reformers
Over half a million of world economy 83%
Coverage students… 87%
86%
85%
81%
77%
representing 28 million 15-year-olds in 74* countries/economies
… took an internationally agreed 2-hour test…
Andreas Schleicher
Goes beyond testing whether students can
reproduce what they were taught…
7 July 2011
65
… to assess students’ capacity to extrapolate Routine what they
from manual
Changes in labour demand
know and creatively apply their knowledge in novel situations
60
PISA
International Student Assessment
OECD Programme for
… and responded to questions on… Nonroutine
manual
their personal background, their schools
55
Routine cognitive
and their engagement with learning and school
50
Parents, principals and system leaders provided data on…
Nonroutine
school policies, practices, resources and institutional factors
analytic
45
that help explain performance differences . Nonroutine
interactive
* 40
Data for Costa Rica, Georgia, India, Malaysia, Malta, Mauritius, Venezuela and Vietnam will be published in December 2011
1960 1970 1980 1990 2002
12. PISA Strong performers and successful reformers
OECD Programme for Andreas Schleicher
International Student Assessment 7 July 2011 14
14
What 15-year-olds can do
13. Shanghai-China
High reading performance
15
15 Average performance
of 15-year-olds in
Strong performers and successful reformers
Korea
540.000
Finland reading – extrapolate
Hong Kong-China
and apply
Singapore
Canada
New Zealand 520.000
Japan
Performance distribution in US
Australia
Andreas Schleicher
18% do not reach baseline Level 2
Netherlands
Norway , when excluding immigrants)
(16%
Suburban schools Belgium Northeast
Poland, Switzerland Midwest Estonia
7 July 2011
Liechtenstein 6%, Canada 9%)
(Finland
United States Iceland
500.000
Germany, Sweden
France, Ireland Chinese Taipei
Hungary, United Kingdom DenmarkEconomic cost: 72 trillion $
Portugal
Macao-China 10% are top performers
Italy
West
Urban schools Latvia
(Shanghai 20%)
PISA
International Student Assessment
OECD Programme for
Slovenia Greece
South
Spain
480.000
Slovak Republic, Czech Republic Croatia
Luxembourg, Israel
Austria Lithuania
Turkey
460.000
Dubai (UAE) Russian Federation
Chile
Serbia
440.000
55 45
… 17 countries perform below25
35
this line
Low reading performance
14. High reading performance
16
16 Average performance
Highof 15-year-olds in
Strong performers and successful reformers
High average performance average performance
Large socio-economic disparities
science – extrapolate
High social equity
and apply
Andreas Schleicher
7 July 2011
Strong socio- Socially equitable
economic impact on distribution of learning
student performance opportunities
PISA
International Student Assessment
OECD Programme for
Low average performance Low average performance
Large socio-economic disparities High social equity
Low reading performance
15. Australia High reading performance
17
17 Belgium 2009 2009
Canada Durchschnittliche
Strong performers and successful reformers
Chile High average performance High average performance
Czech Rep Schülerleistungen im
Large socio-economic disparities High social equity
Denmark Bereich Mathematik
Finland
Germany
Greece
Hungary
Andreas Schleicher
Iceland
Ireland
7 July 2011
Israel
Strong socio- Socially equitable
Italy
economic impact on distribution of learning
Japan
student performance opportunities
Korea
PISA
International Student Assessment
OECD Programme for
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Spain Low average performance Low average performance
Sweden
Switzerland Large socio-economic disparities High social equity
UK 55 45 35 25 15
US Low reading performance
16. High reading performance
18
18 2009
Durchschnittliche
Strong performers and successful reformers
High average performance High average performance
Schülerleistungen im
Large socio-economic disparities High social equity
Bereich Mathematik
Andreas Schleicher
7 July 2011
Strong socio- Socially equitable
economic impact on distribution of learning
student performance opportunities
PISA
International Student Assessment
OECD Programme for
Low average performance Low average performance
Large socio-economic disparities High social equity
Low reading performance
17. 19
19 High performing systems often prioritize the
Strong performers and successful reformers
quality of teachers over the size of classes
Contribution of various factors to upper secondary teacher compensation costs
Salary as % of student as a percentage of GDP per capita (2004) time
per GDP/capita Instruction time 1/teaching 1/class size
Difference with OECD average
Percentage points
15
Andreas Schleicher
7 July 2011
10
5
PISA
International Student Assessment
OECD Programme for
0
-5
-10
Ireland
Spain
Poland
Korea
Norway
Mexico
Finland
Belgium
Denmark
Italy
Austria
Sweden
Greece
New Zealand
Germany
France
Iceland
Luxembourg
Hungary
Australia
Switzerland
Japan
Portugal
Netherlands
Czech Republic
United States
United Kingdom
18. High reading performance
20
20 2009
Durchschnittliche
Strong performers and successful reformers
High average performance High average performance
Schülerleistungen im
Large socio-economic disparities High social equity
Bereich Mathematik
Andreas Schleicher
7 July 2011
Strong socio- Socially equitable
economic impact on distribution of learning
student performance opportunities
PISA
International Student Assessment
OECD Programme for
Low average performance Low average performance
Large socio-economic disparities High social equity
Low reading performance
19. High reading performance
21
21 2000
Durchschnittliche
Strong performers and successful reformers
High average performance High average performance
Schülerleistungen im
Large socio-economic disparities High social equity
Bereich Mathematik
Andreas Schleicher
7 July 2011
Strong socio- Socially equitable
economic impact on distribution of learning
student performance opportunities
PISA
International Student Assessment
OECD Programme for
Low average performance Low average performance
Large socio-economic disparities High social equity
Low reading performance
20. High reading performance
22
22 2000
Durchschnittliche
Strong performers and successful reformers
High average performance High average performance
Schülerleistungen im
Large socio-economic disparities High social equity
Bereich Mathematik
Andreas Schleicher
7 July 2011
Strong socio- Socially equitable
economic impact on distribution of learning
student performance opportunities
PISA
International Student Assessment
OECD Programme for
Low average performance Low average performance
Large socio-economic disparities High social equity
Low reading performance
21. 23 School performance and socio-economic background
23
United States
Strong performers and successful reformers
Private school
Public school in rural area
Public school in urban area
643
700
Andreas Schleicher
7 July 2011
Student performance
PISA
International Student Assessment
OECD Programme for
350
-2 -1 0 1 2
Disadvantage PISA Index of socio-economic background Advantage
22. 25
25 Policy
Policies and practices
R
System
R
School
E
Equity
What students know and can do
Learning climate
Discipline
Teacher behaviour
Andreas Schleicher
7 December 2010
Parental pressure
Teacher-student
relationships
Dealing with heterogeneity
PISA
International Student Assessment
OECD Programme for
Grade repetition
Prevalence of tracking
Expulsions
Ability grouping
(all subjects)
Standards /accountability
Nat. examination
23. PISA Strong performers and successful reformers
OECD Programme for Andreas Schleicher
International Student Assessment 7 July 2011 26
26
Does it all matter?
24. 27 Increased likelihood of postsec. particip. at age 19/21
27
associated with PISA reading proficiency at age 15 (Canada)
Strong performers and successful reformers
after accounting for school engagement, gender, mother tongue,
place of residence, parental, education and family income
(reference group PISA Level 1)
Odds ratio
higher
Andreas Schleicher
education
entry 20
18
7 July 2011
16
14
12
10
PISA
International Student Assessment
OECD Programme for
8
6
4
2
Age 19
0
Age 21
Level 5
Age 21 Level 4
Level 3
Level 2
25. PISA Strong performers and successful reformers
OECD Programme for Andreas Schleicher
International Student Assessment 7 July 2011 28
28
What does it all mean?
26. 29
29
Strong performers and successful reformers
Tools
Standards
Processes
Curricula
Selection
People
Andreas Schleicher
Technology
Teachers
Preparation
Practices
Principals
7 July 2011
Assessments
Instruction
Recruitment/induction
Design, Student
implementation
Support personnel
Intervention
Data systems
andWork learningof policies
organisation
alignment
Families
Support systems
PISA
International Student Assessment
OECD Programme for
Development
Supervision
Retention
27. 30
30 A commitment to education and the belief
that competencies can be learned and
Strong performers and successful reformers
therefore all children can achieve
Universal educational standards and
personalisation as the approach to
heterogeneity in the student body…
Andreas Schleicher
… as opposed to a belief that students have
different destinations to be met with different
Lessons from PISA
7 July 2011
expectations, and selection/stratification as
on successful
the approach to heterogeneity
Clear articulation who is responsible for
education systems
PISA
International Student Assessment
OECD Programme for
ensuring student success and to whom
28. 32
32
Clear ambitious goals that are shared across
Strong performers and successful reformers
the system and aligned with high stakes
gateways and instructional systems
Well established delivery chain through which
curricular goals translate into instructional
systems, instructional practices and student
Andreas Schleicher
learning (intended, implemented and achieved)
Lessons from PISA
7 July 2011
High level of metacognitive content of
instruction
on successful
education systems
PISA
International Student Assessment
OECD Programme for
29. 33
33
Strong performers and successful reformers
Capacity at the point of delivery
Andreas Schleicher
Attracting, developing and retaining high quality
Lessons from PISA
7 July 2011
teachers and school leaders and a work
organisation in which they can use their
on successful
potential
education leadership and human resource
Instructional systems
PISA
International Student Assessment
OECD Programme for
management in schools
Keeping teaching an attractive profession
System-wide career development
30. 34
34
Teacher in-service development
Strong performers and successful reformers
No matter how good the pre-service education for teachers is
… it cannot prepare teachers for rapidly changing challenges throughout their
careers
High-performing systems rely on ongoing professional to…
Andreas Schleicher
… update individuals’ knowledge of a subject in light of recent advances
7 July 2011
… update skills and approaches in light of new teaching techniques, new
circumstances, and new research
… enable teachers to apply changes made to curricula or teaching practice
PISA
International Student Assessment
OECD Programme for
… enable schools to develop and apply new strategies concerning the curriculum
and teaching practice
… exchange information and expertise among teachers and others
… help weaker teachers become more effective .
Effective professional development is on-going…
… includes training, practice and feedback, and adequate time and follow-up
support
31. 36
36
OECD Teaching and Learning
Relatively few teachers participate in the kinds of professional
development which they find has the largest impact on their work
Comparison of teachers participating in professional
International Study (TALIS)
development activities and teachers reporting
moderate or high level impact by types of activity
% TALIS Average
100
90
80
70
60
50
40
Creating Effective Teaching
and Learning Environments
30
20
10
0
Participation
Participation
Impact
Participation
Impact
Participation
Impact
Impact
Participation
Impact
Participation
Impact
Participation
Impact
Participation
Impact
Impact
Participation
Individual Qualification Informal Reading Courses and Professional Mentoring Observation Education
and programmes dialogue to professional workshops development and peer visits to other conferences
collaborative improve literature network observation schools and seminars
research teaching
32. 37
37
OECD Teaching and Learning
Relatively few teachers participate in the kinds of professional
development which they find has the largest impact on their work
Comparison of teachers participating in professional
International Study (TALIS)
development activities and teachers reporting
moderate or high level impact by types of activity
Creating Effective Teaching
and Learning Environments
33. 38
38
Strong performers and successful reformers
Incentives, accountability, knowledge management
Aligned incentive structures
For students
How gateways affect the strength, direction, clarity and nature of
the incentives operating on students at each stage of their education
Andreas Schleicher
Degree to which students have incentives to take tough courses and
study hard
Lessons from PISA
7 July 2011
Opportunity costs for staying in school and performing well
For teacherson successful
Make innovations in pedagogy and/or organisation
Improveeducation systems
PISA
International Student Assessment
OECD Programme for
their own performance
and the performance of their colleagues
Pursue professional development opportunities
that lead to stronger pedagogical practices
A balance between vertical and lateral accountability
Effective instruments to manage and share knowledge and
spread innovation – communication within the system and
with stakeholders around it
A capable centre with authority and legitimacy to act
34. School autonomy, accountability
41
41 and student performance
Strong performers and successful reformers
Impact of school autonomy on performance in systems with and without
accountability arrangements
PISA score in reading
500
Andreas Schleicher
7 July 2011
495
490
PISA
International Student Assessment
OECD Programme for
School autonomy in resource
allocation
480
Schools with less autonomy
Systems with more
accountability Systems with less
accountability
35. 42
42
Local responsibility
and system-level prescription
Strong performers and successful reformers
Trend in OECD countries
Andreas Schleicher
System-level prescription
7 July 2011
‘Tayloristic’ work organisation
Schools today Schools Finland today
The industrial tomorrow? Every school an
PISA
International Student Assessment
OECD Programme for
model, detailed Building capacity effective school
prescription of
what schools do
Schools leading reform
Teachers as ‘knowledge workers’
36. 44
44
Strong performers and successful reformers
Andreas Schleicher
Lessons from PISA
7 July 2011
on successful
education systems
PISA
International Student Assessment
OECD Programme for
Investing resources where they can make
most of a difference
Alignment of resources with key challenges (e.g.
attracting the most talented teachers to the
most challenging classrooms)
Effective spending choices that prioritise high
quality teachers over smaller classes
37. 45
45
Strong performers and successful reformers
Andreas Schleicher
A learning system
Lessons from PISA
An outward orientation of the system to keep
7 July 2011
the system learning, international benchmarks
as the ‘eyes’ and ‘ears’ onthe system
of successful
education systems
Recognising challenges and potential future
PISA
International Student Assessment
OECD Programme for
threats to current success, learning from them,
designing responses and implementing these
38. 47 Coherence of policies and practices
47
Strong performers and successful reformers
Alignment of policies
across all aspects of the system
Coherence of policies
over sustained periods of time
Consistency of implementation
Andreas Schleicher
Fidelity of implementation
Lessons from PISA
7 July 2011
(without excessive control)
on successful
education systems
PISA
International Student Assessment
OECD Programme for
39. 49
49 Education reform trajectories
Strong performers and successful reformers
The old bureaucratic system Student inclusion The modern enabling system
Some students learn at high levels All students need to learn at high levels
Andreas Schleicher
Curriculum, instruction and assessment
Routine cognitive skills, rote learning Learning to learn, complex ways of
7 July 2011
thinking, ways of working
Teacher quality
PISA
International Student Assessment
OECD Programme for
Few years more than secondary High-level professional knowledge workers
Work organisation
‘Tayloristic’, hierarchical Flat, collegial
Accountability
Primarily to authorities Primarily to peers and stakeholders
40. 55
55
Strong performers and successful reformers
Find out more about PISA at…
OECD www.pisa.oecd.org
– All national and international publications
Andreas Schleicher
– The complete micro-level database
U.S. White House www.data.gov
Thank you !
7 July 2011
Email: Andreas.Schleicher@OECD.org
PISA
International Student Assessment
OECD Programme for
… and remember:
Without data, you are just another person with an opinion
Hinweis der Redaktion
The pace of change is most clearly visible in higher education, and I want to bring two more dimensions into the picture here. Each dot on this chart represents one country. The horizontal axis shows you the college graduation rate, the proportion of an age group that comes out of the system with a college degree. The vertical axis shows you how much it costs to educate a graduate per year.
*Lets now add where the money comes from into the picture, the larger the dot, the larger the share of private spending on college education, such as tuition.The chart shows the US as the country with the highest college graduation rate, and the highest level of spending per student. The US is also among the countries with the largest share of resources generated through the private sector. That allows the US to spend roughly twice as much per student as Europe. US, FinlandThe only thing I have not highlighted so far is that this was the situation in 1995. And now watch this closely as you see how this changed between 1995 and 2005.
You see that in 2000, five years, later, the picture looked very different. While in 1995 the US was well ahead of any other country – you see that marked by the dotted circle, in 2000 several other countries had reached out to this frontier. Look at Australia, in pink.
What do we see from this?- Every country has seen improvements in terms of output The example of the UK shows that you can set ambitious national targets and actually get close to them within a decade What you cannot do is prevent others from surpassing themAnd in a global economy, it is no longer simply improvement by national standards, but the best prepared individuals, companies and countries that are the benchmarks for success. What international comparisons can do is to show how the goal post keeps changing.
We started to develop PISA in 1998 with 28 OECD countries, but since then country participation has grown and our latest PISA assessment covers 74 education systems that make up 86% of the world economy. Coverage in China and India is still patchy though, in China we have now covered 12 provinces and in India we are working in two states only.One aspect that makes PISA stand apart from traditional school tests is that PISA puts less emphasis on whether students can reproduce what they were taught, but focuses on their capacity to extrapolate from what they know and creatively apply what they know in novel situations. Some people complain that PISA is unfair, because it confronts students with tasks they have not dealt with before, but if you take that line, then you should consider life unfair, because in this fast-changing world, that is precisely what will expect students later in life. You will see that in the callout box.Students also provided data on their socio-economic context, their schools and their attitudes and engagement with school and learning.In addition, PISA collected data from parents, principals and system leaders to yield insights on school policies, practices, resources and institutional factors that help explain performance differences.
The idea of PISA is to support governments in preparing students for life. In a sense, PISA provides schools and nations with a mirror in which they can judge their performance in light of what other systems show is possible to achieve.
Let me conclude this introduction with a couple of factors that were key to the success of PISAAt the heart of PISA is not a bureaucracy but the largest international network of educators and researchers in which Japan is an active player. These experts develop and validate the assessment material and methodologies, guided by governments on the basis of shared policy interests.Whenever you engage in cross-national collaboration, you will run into the question of whether measures and policy lessons travels well across cultural and national contexts. That is an area where PISA has made unprecedented progress.Third, in the field of education, nobody really knows how learning occurs in the classroom, but everybody has a view on this. So in PISA, we approached the issues from many perspectives, collecting data from students, parents, school principals, experts and system leaders and then through triangulation tried to get to the bottom to the issues. Finally, PISA employs a range of methods to ensure adequate measurement at different grain size to serve different decision-making needs.
With that introduction, let us turn to the results. The first thing you can do is to see how countries line up with regard to the competencies of their 15-year-olds.
The red dot indicates classroom spending per student, relative to the spending capacity of countries, the higher the dot, the more of its GDP a country invests. High salaries are an obvious cost driver. You see Korea paying their teachers very well, the green bar goes up a lot. Korea also has long school days, another cost driver, marked here by the white bar going up. Last but not least, Korea provides their teachers with lots of time for other things than teaching such as teacher collaboration and professional development, which costs money as well. So how does Korea finances all of this? They do this with large classes, the blue bar pulls costs down. If you go to the next country on the list, Luxembourg, you see that the red dot is about where it is for Korea, so Luxembourg spends roughly the same per student as Korea. But parents and teachers in Luxembourg mainly care about small classes, so policy makers have invested mainly into reducing class size, you see the blue bar as the main cost driver. But even Luxembourg can only spend its money once, and the result is that school days are short, teacher salaries are average at best and teachers have little time for anything else than teaching. Finland and the US are a similar contrast.Countries make quite different spending choices. But when you look at this these data long enough, you see that many of the high performing education systems tend to prioritise the quality of teachers over the size of classes.
Let me briefly summarise the influences that we have measured in PISA.
You have seen very large performance differences among schools and countries, but how predictive are these for the success of students and nations?
To what extent is performance in school predictive of success in later life?The best way to find out whether what students have learned at school matters for their life is to actually watch what happens to them after they leave school. This is exactly what we have done that with around 30,000 students in Canada. We tested them in the year 2000 when they were 15 years old in reading, math and science, and since then we are following up with them each year on what choices they make and how successful they are in their transition from school to higher education and work.The horizontal axis shows you the PISA level which 15-year-old Canadians had scored in 2000. Level 2 is the baseline level on the PISA reading test and Level 5 the top level in reading.The red bar shows you how many times more successful someone who scored Level 2 at age 15 was at age 19 to have made a successful transition to university, as compared to someone who did not make it to the baseline PISA level 1. And to ensure that what you see here is not simply a reflection of social background, gender, immigration or school engagement, we have already statistically accounted for all of these factors. The orange bar. …How would you expect the picture to be like at age 21? We are talking about test scores here, but for a moment, lets go back to the judgements schools make on young people, for example through school marks. You can do the same thing here, you can see how well school marks at age 15 predict the subsequent success of youths. You see that there is some relationship as well, but that it is much less pronounced than when we use the direct measure of skills. What this tells you how important reliable measures of student performance are, an area where the UK is leading the field since some years.
I am going to present evidence on separate issues in turn, but it is their interdependence that is key to understanding the nature of the policy and implementation challenges. If you simply raise entrance standards for teachers, you will choke off supply unless compensation and working conditions are aligned. Raising pay and changing working conditions alone won’t automatically translate into improvements in teacher quality unless standards are raised. Teacher evaluation systems have limited impact where they only relate to compensation but not professional development and career advancement. Giving teachers more autonomy can be counterproductive if the quality and education of the teachers are inadequate.Education is ultimately about student learning outcomes……and these Learning outcomes are the result of what happens in the classroom.Instructional policies and practices, in turn, are shaped by people - teachers, principles and families. And that’s why the quality of an education system cannot exceed the quality of its teachers.But it works the other way round too: The quality of teachers cannot exceed the quality of work organization, the quality of teacher selection and education, teacher careers and teacher evaluation.And it is those processes that we can shape with policy tools. And success depends on the design and implementation of effective policies.
First, there is no question that most nations declare that education is important. But the test comes when these commitments are weighed against others. How do countries pay teachers, compared to other highly-skilled workers? How are education credentials weighed against other qualifications when people are being considered for jobs? Would you want your child to be a teacher? How much attention do the media pay to schools and schooling? What we have learned from PISA is that in high performing systems political and social leaders have persuaded citizens to make choices that show they value education more than other things. But placing a high value on education is only part of the equation. Another part is belief in the possibilities for all children to achieve success. In some countries, students are separated into different tracks at an early age, reflecting a notion shared by teachers, parents and citizens that only a subset of the nation’s children can or need to achieve world class standards. Our analysis shows that systems that track students in this way, based differing expectations for different destinations, tend to be fraught with large social disparities. By contrast, the best performing systems deliver strong and equitable learning outcomes across very different cultural and economic contexts. In Finland, Japan, Singapore, Shanghai-China and Hong Kong-China, parents, teachers and the public at large share the belief that all students are capable of achieving high standards and need to do so, and they provide great examples for how public policy can support the achievement of universal high standards.
High-performing education systems also share clear and ambitious standards across the board. Everyone knows what is required to get a given qualification, both in terms of the content studied and the level of performance needed to earn it. Students cannot go on to the next stage—be it in work or in further education—unless they show that they are qualified to do so. They know what they have to do to realise their dream, and they put in the work that is needed to do it.As discussed in the 2009 edition of OECD’s Education at a Glance¸ over the past decade, assessments of student performance have become common in many OECD countries – and the results are often widely reported and used in both public and more specialised debate. However, the rationale for assessments and the nature of the instruments used vary greatly within and across countries. Methods employed in OECD countries include different forms of external assessment, external evaluation or inspection, and schools’ own quality assurance and self-evaluation efforts. One aspect relating to accountability systems concerns the existence of standards-based external examinations. These are examinations that focus on a specific school subject and assess a major portion of what students who are studying this subject are expected to know or be able to do (Bishop, 1998, 2001). Essentially, they define performance relative to an external standard, not relative to other students in the classroom or school. These examinations usually have a direct impact on students’ education – and even on their futures – and may thus motivate students to work harder. Other standardised tests, which may be voluntary and implemented by schools, often have only indirect consequences for students. For teachers, standardised assessments can provide information on students’ learning needs and can be used to tailor their instruction accordingly. In some countries, such as Brazil, Hungary, Italy, Malaysia, Mexico, Poland and the Slovak Republic, such tests are also used to determine teachers’ salaries or to guide professional development (for data, see the 2009 edition of Education at a Glance ). At the school level, information from standardised tests can be used to determine the allocation of additional resources, and what interventions are required to establish performance targets and monitor progress.Across OECD countries, students in school systems that require standards-based external examinations perform, on average, over 16 points higher than those in school systems that do not use such examinations (Figure IV.2.6a). Among OECD countries, there are standards-based external examinations for secondary school students in the Czech Republic, Denmark, Estonia, Finland, France, Hungary, Iceland, Ireland, Israel, Italy, Japan, Korea, Luxembourg, the Netherlands, New Zealand, Norway, Poland, the Slovak Republic, Slovenia, Turkey and the United Kingdom. In Australia, these examinations cover 81% of secondary students, in Canada 51% and in Germany 35%. In Austria, Belgium, Chile, Greece, Mexico, Portugal, Spain, Sweden and Switzerland, such examinations do not exist or only in some parts of the system (Table IV.3.11).In PISA 2009, school principals were asked to report on the types and frequency of assessment used: standardised tests, teacher-developed tests, teachers’ judgemental ratings, student portfolios or student assignments. Some 76% of students in OECD countries are enrolled in schools that use standardised tests. Standardised tests are relatively uncommon in Slovenia, Belgium, Spain, Austria and Germany, where less than half the 15-year-olds attend schools that assess students through standardised tests. In contrast, the use of standardised tests is practically universal in Luxembourg, Finland, Korea, the United States, Poland, Denmark, Sweden and Norway, where over 95% of students attend schools that use this assessment at least once a year (Table IV.3.10). In Japan, 65% of students are in schools that use standardised tests.
Third, the quality of an education system cannot exceed the quality of its teachers and principals. Just like companies, high quality school systems pay attention to how they select and train their staff. They watch how they improve the performance of those who are struggling; how structure teachers’ pay packets; and how they reward their best teachers. They provide an environment in which teachers work together to frame good practice. That is where teachers conduct field-based research to confirm or disprove the approaches they develop, and they judge their colleagues by the degree to which they use these practices in their classrooms. Listen to what the Finnish Minister had to say about that.
Now, so far so good, and we all get very exited when we talk about teacher recruitment and initial training. And the reason why we get exited is because public policy can easily shape these. Some claim that the best performing systems all recruit their teachers from the top third of school graduates. That’s true for Finland and Korea. But if I look around the table, how many more countries can really claim that teaching is the first choice of graduates? Singapore perhaps? But it simply does not hold for the majority of countries around the table here. If we wait for this marvellous spaceship that will arrive one day and bring us new teachers – and maybe send those who are not effective to the moon - then we will find ourselves in the same situation years from now. And you see that other sectors have acted in much smarter ways. Imagine Nokia, the Finnish technology leader. In the 1960s, Nokia produced car tyres and rubber products. Imagine where they would be now if they had said then: We would really like to produce something more advanced than car tyres, but our engineers are not up to the task. Lets wait until they are retired, then we will train new engineers, and when the new graduates trickle into the labour-market, we will start to do something great. And there is more to this. If you put great teachers into a poor system, the system will win every time. There are plenty of examples for this. No matter how good the pre-service education for teachers is, it cannot be expected to prepare teachers for all the challenges they will face throughout their careers. So let us move on to the second summit topic. [Slide – How teachers are developed] So as important as recruitment and selection of promising graduates is, it can only be one component of human resource management in education. Successful reform requires investment in quality professional development to continuously: Simply look at how significantly the profile of teacher requirements has changed in the last decade alone: They must now place much greater emphasis on integrating students with special learning needs. They need to make more effective use of information technologies. They are required to engage more in planning within evaluative and accountability frameworks. And they are asked to do more to involve parents in schools. I think we all recognize that education is still far from being a knowledge industry, in the sense that its own practices are being continuously transformed by greater understanding of their efficacy. While in many other fields, people enter their professional lives expecting that what they do and how they do it will be transformed by evidence and research, this is still not generally the case in education. Ongoing professional development is a crucial instrument to:update teachers subject knowledge; update teachers skills and approaches in light of new teaching techniques, new circumstances, and new research; enable teachers to master changes made to curricula or teaching practice; enable schools to develop and apply new strategies; and facilitate exchange of experience; In some countries, ongoing professional development already plays an important role. In Shanghai, each teacher is expected to engage in 240 hours of professional development within five years. Singapore provides teachers with 100 hours of professional development per year. But there is a lot of variation in the incidence and intensity of teacher participation in professional development both across and within countries.
There are some lessons we have learned about teacher development: Well-structured and resourced induction programs can support new teachers in their transitionMany countries carefully induct their teachers before they confer on them all the rights and responsibilities of full-time teachers. During that period, they are supervised by master teachers and they get additional instruction and coaching from their supervisory teachers, and observe other teachers. Typically, teachers in this induction period can be coached out of the profession, if, in the opinion of their supervisory teachers, they have not demonstrated that they have the knowledge and skills needed to be a competent professional. [Slide: percentage of teachers without mentoring or induction] But our data show a substantial share of teachers is left without induction - or mentoring. Effective professional development is on-going, include training, practice and feedback, and provide adequate time and follow-up supportSecond, we have learned that effective professional development needs to be on-going, include training, practice and feedback, and provide adequate follow-up. Successful programs involve teachers in learning activities that are similar to those they will use with their students, and encourage the development of teachers’ learning communities. Teacher development needs to be linked with appraisal and feedback practices and school evaluationTeacher development also needs to be linked with appraisal and feedback. And it’s important to provide sufficient room for teachers to employ inquiry- and group-based approaches, especially in the core areas of curriculum and assessment. Our data show that teachers’ participation in professional development goes hand-in-hand with their mastery of a wider repertoire of pedagogical practices. We also see a close relationship between professional development and a positive school climate, co-operation between teachers and teacher job satisfaction. But we also see that that schools and systems need to better match the costs and benefits of, and supply and demand for, professional development.
Here you see the percentage of teachers that participate in various types of professional development across the countries that took part in our TALIS survey.
When you now contrast this with the impact of such professional development, you see that relatively few teachers participate in the kinds of professional development that they believe has the largest impact on their work, namely qualification programs and individual and collaborative research. Teacher demand for professional development is often not met, sometimes for lack of time, sometimes for lack of opportunity…More than half the teachers surveyed also said that they wanted more professional development than they received. Of course, people never feel well enough prepared, but the extent of unsatisfied demand appears large. We have no data on the extent to which this undermines the effectiveness of these teachers. But the cost of providing additional professional development needs to be seen in relation to the cost of not providing it.
Fourth, as you have seen, success has to do with incentives and accountability, and how these are aligned in the system. It has also to do with how vertical accountability to superiors is balanced with horizontal or professional accountability towards peers, how knowledge is shared and spread. For students this affects: How gateways affect the strength, direction, clarity and nature of the incentives operating on students at each stage of their education; as well as the degree to which students have incentives to take tough courses and study hard and the opportunity costs for staying in school and performing well.It also means providing incentives for teachers to make innovations in pedagogy and/or organisation, improve their own performance and the performance of their colleagues, and to pursue professional development opportunities that lead to stronger pedagogical practices.High performing systems tend to provide a balance between vertical and lateral accountability and have effective instruments to manage and share knowledge and spread innovation – and that means both communication within the system and with stakeholders around it.
The most impressive outcome of world class education systems is perhaps that they deliver high quality learning consistently across the entire education system so that every student benefits from excellent learning opportunity. To achieve this, they invest educational resources where they can make most of a difference, they attract the most talented teachers into the most challenging classroom, and they establish effective spending choices that prioritise the quality of teachers. Let me come back to the example of Shanghai once more here. Let us have a look at the struggling schools six years later.Research usually shows a weak relationship between educational resources and student performance, with more variation explained by the quality of human resources (i.e. teachers and school principals) than by material and financial resources, particularly among industrialised nations. The generally weak relationship between resources and performance observed in past research is also seen in PISA. At the level of the education system, and net of the level of national income, the only type of resource that PISA shows to be correlated with student performance is the level of teachers’ salaries relative to national income (Figure IV.2.8). Teachers’ salaries are related to class size in that if spending levels are similar, school systems often make trade-offs between smaller classes and higher salaries for teachers. The findings from PISA suggest that systems prioritising higher teachers’ salaries over smaller classes, such as those in Japan and Korea, tend to perform better. The lack of correlation between the level of resources and performance among school systems does not mean that resource levels do not affect performance at all. Rather, it implies that, given the variation in resources observed in PISA, they are unrelated to performance or equity. A school system that lacks teachers, infrastructure and textbooks will almost certainly perform at lower levels; but given that most school systems in PISA appear to satisfy the minimum resource requirements for teaching and learning, the lack of a relationship between many of the resource aspects and both equity and performance may result simply from a lack of sufficient variation among OECD countries.
Some of the most successful systems are also actively looking outward, realising that the benchmark for success is no longer simply improvement by national standards, but the best performing systems internationally. Whether Singapore is interested in designing a better sewer system, retirement system or school system, it sends key people in the relevant sector to visit those countries that are the world’s best performers in those areas with instructions to find out how they do it, and to put together a design for Singapore that is superior to anything that they have seen anywhere.
This chart shows you that a fair proportion of teachers still remain without any form of appraisal or feedback. What is interesting is to see how the role of teacher appraisal has changed in recent years. In the past, it was mainly about compliance, about monitoring adherence to centrally established procedures, policies and practices. Almost everywhere, the focus has now shifted to teaching effectiveness. Effective teacher appraisal can help to improve teachers’ practices by identifying strengths and weaknesses for further professional development – the improvement function. That involves helping teachers learn about, reflect on, and adjust their practice. Teacher appraisal can also help to hold teachers accountable for their performance in enhancing student learning – the accountability function. That's often linked with performance-based career advancement or salaries. But when you look around the table here, you see that countries typically either focus on improvement, or on accountability. And the reason is that combining improvement and accountability functions into a single teacher-appraisal process is tough. When evaluation focuses on improving practice, teachers tend to be willing to reveal their weaknesses, in the expectation that conveying that information will lead to more effective decisions on developmental needs. That's what you see in Finland. But when teachers are confronted with potential consequences on their career and salary, they tend to be less inclined to reveal weaknesses in their performance, and the improvement function, which builds on trust in the relationship between appraiser and the appraised, can be compromised. But, again, there are good examples for how this works well. And teachers generally do see appraisal and feedback in positive terms. 80% of teachers in our TALIS survey said appraisal was helpful for developing their work as teachers; and almost half of teachers reported that it led to a teacher-development or training plan to improve their teaching. One way of ensuring that teachers see evaluation in positive terms is to involve them in school evaluations, in particular by organizing school self-evaluations as a collective process in which teachers take real responsibility. Effective appraisal requires the development of considerable expertise in the system, including training evaluators, establishing evaluation processes and aligning broader school reforms, such as professional development opportunities, with evaluation and assessment strategies. All of these require considerable resources, including time.The criteria used to evaluate teachers center on learning outcomes, although they also assess significant inputs, such as teacher qualifications and the learning environment created in classrooms… Also, our data show that where teachers receive feedback on their work, they are more likely to find it fair than threatening. On average, eight in ten teachers surveyed in TALIS who received feedback thought it was fair. More than three-quarters of teachers also considered it helpful for their work, and the majority said it improved their job satisfaction and development as teachers, without reducing job security. Most importantly, they reported that appraisal leads to changes in the specific aspects of their teaching on which it focuses.
Last but not least, in high performing systems these policies and practices are aligned across all aspects of the system, they are coherent over sustained periods of time, and they are consistently implemented. And PISA shows, success is within the reach for nations that have the capacity to creating and executing policies with maximum coherence in the system. Of course, the path to reform is not easy and it can be fraught with political controversy. Moving away from administrative and bureaucratic control toward professional norms of control can be counterproductive if a nation does not yet have teachers and schools with the capacity to implement these policies and practices. Pushing authority down to lower levels can be as problematic if there is not agreement on what the students need to know and should be able to do. Recruiting high quality teachers is not of much use if those who are recruited are so frustrated by what they perceive to be a mindless system of initial teacher education that they will not participate in it and turn to another profession. Or if they become school teachers, but are so turned off by the bureaucratic forms of work organisation they find there that they leave teaching for some other occupation. So this is all about alignment.
Whether and how long students are enrolled in pre-primary education is also an important resource consideration. Many of the inequalities that exist within school systems are already present once students enter formal schooling and persist as students’ progress through school. Earlier entrance into the school system may reduce these inequities. On average across OECD countries, 72% of today’s 15-year-old students reported that they had attended pre-primary education for more than one year. Attendance in more than one year of pre-primary education was practically universal in Japan (97%), and in the Netherlands, Hungary, Belgium, Iceland and France, over 90% of 15-year-old students reported that they had attended pre-primary school for more than one year. More than 90% of students in 27 OECD countries had attended pre-primary school for at least some time, and 98% or more of students in Japan (99%), Hungary, France and the United States reported having done so. Pre-primary education is rare in Turkey, where less than 30% of 15-year-olds had attended pre-primary school for at least a year. More than one year of pre-primary education is uncommon in Chile, Ireland, Canada and Poland, where less than 50% of students attended pre-primary school for that length of time (Table IV.3.18).PISA 2009 results show that, in general, students who had attended pre-primary education perform better in reading at the age of 15 than students who had not (Figure II.5.9 and Table II.5.5). In 32 OECD countries, students who had attended pre-primary education for more than one year outperformed students who had not attended pre-primary education at all – in many countries by the equivalent of well over a school year. This finding holds in most countries even after accounting for students’ socio-economic backgrounds. However, across countries, there is considerable variation in the impact of attendance in pre-primary education and reading performance when students are 15 years old. Among OECD countries, in Israel, Belgium, Italy and France, students who attended pre-primary education for more than one year perform at least 64 score points higher in reading than those who did not, which corresponds to the equivalent of roughly one-and-a-half school years. This was the case even after accounting for students’ socio-economic background. On the other hand, in Estonia, Finland, the United States and Korea, there is no marked difference in reading scores between those who attended pre-primary school for more than one year and those who did not attend at all, after accounting for students’ socio-economic background. In Japan, the students who had attended pre-primary education for one year or more scored an average of 39 points higher on the PISA reading scale than those who did not – roughly the equivalent of one school year – and after accounting for students’ socio-economic background, the performance advantage is 24 score points. These results underline the importance of pre-primary education, and international comparisons of primary-school children show high pre-primary enrolment rates among both advantaged and disadvantaged Japanese children. The next challenge will be to increase the positive impact of pre-primary education on performance later on in students’ school careers.One factor that may explain the variations in the impact of pre-primary education on later school performance is the quality of pre-primary education. This hypothesis is supported by the fact that the impact tends to be greater in education systemswhere pre-primary education is of longer duration, has smaller pupil-to-teacher ratios or benefits from higher public expenditure per pupil (Table II.5.6). When this impact is compared according to socio-economic background, in most OECD countries, there is no significant difference in the impact between students from socio-economically disadvantaged and advantaged backgrounds (Table II.5.8). Students benefit equally from attending pre-primary school in 31 OECD countries including Japan and 25 partner countries and economies. The United States is the only OECD country where PISA shows that disadvantaged students benefit more from pre-primary education. Part of the difference in the impact of attendance in pre-primary education on the performance of students from different socio-economic backgrounds may be due to the fact that many factors other than pre-primary education (e.g. education in and out of school that students received between the ages of 6 and 15) may influence 15-year-olds’ performance.
I want to conclude with what we have learned about successful reform trajectories In the past when you only needed a small slice of well-educated people it was efficient for governments to invest a large sum in a small elite to lead the country. But the social and economic cost of low educational performance has risen substantially and all young people now need to leave school with strong foundation skills.When you could still assume that what you learn in school will last for a lifetime, teaching content and routine cognitive skills was at the centre of education. Today, where you can access content on Google, where routine cognitive skills are being digitised or outsourced, and where jobs are changing rapidly, the focus is on enabling people to become lifelong learners, to manage complex ways of thinking and complex ways of working that computers cannot take over easily.In the past, teachers had sometimes only a few years more education than the students they taught. When teacher quality is so low, governments tend to tell their teachers exactly what to do and exactly how they want it done and they tend to use Tayloristic methods of administrative control and accountability to get the results they want. Today the challenge is to make teaching a profession of high-level knowledge workers. But such people will not work in schools organised as Tayloristic workplaces using administrative forms of accountability and bureaucratic command and control systems to direct their work. To attract the people they need, successful education systems have transformed the form of work organisation in their schools to a professional form of work organisation in which professional norms of control complement bureaucratic and administrative forms of control.
I am delighted to share our analysis of the latest PISA findings with you, and I am particularly pleased to do this in Japan, a country which has maintained its high levels of student performance, and which has seen important improvements in student engagement since 2000, an area that traditionally was one of Japan’s weaknesses.I want to start with a brief overview of the objectives and origins of PISA, then analyse where Japan stands on measures ranging from student performance up to student attitudes to learning and engagement with school, and then conclude with what we have learned about effective policies and practices that may help Japan to further raise its already impressive educational performance.