This document discusses an OECD review of school evaluation frameworks in the Flemish Community of Belgium. It provides an overview of the review, including the analytical approach used and assessment of strengths and challenges. The review team found that while schools have responsibility for quality, evaluation could be better embedded in a vision linking different approaches. It also found variation in school self-evaluation capacity and made recommendations like clarifying evaluation goals and increasing the use of objective information and competency development to strengthen evaluation.

1. OECD Review on Evaluation and
Assessment Frameworks for
Improving School Outcomes
School Evaluation in the
Flemish Community of Belgium
Claire Shewbridge
Analyst, Education and Training Policy Division
claire.shewbridge@oecd.org
Brussels, 7 December 2011

2. Structure of the presentation
• The OECD review in general and the OECD review of
school evaluation in the Flemish Community
• Policy trends in the Flemish Community and the analytical
approach used by the OECD review team
• The OECD review team’s assessment of strengths and
challenges in the current approach to school evaluation
• The OECD review team’s policy recommendations to build
on the current approach to school evaluation in the
Flemish Community
• Further information and OECD review outputs

3. The OECD review of School Evaluation in the
Flemish Community of Belgium
THE REVIEW IN GENERAL AND IN
THE FLEMISH COMMUNITY

4. OECD reviews of evaluation and assessment:
School evaluation in the Flemish Community of Belgium (1)
• The Flemish Community is one of 12 systems being reviewed
– 12 other systems providing analytical reports, i.e. total 24 systems
– Presenting evidence and including stakeholder views
Country review OECD review visit Country OECD Analytical Country
strand report Report strand Report
Australia June 2010 Y Y Austria
Belgium (Fl.) January 2011 Y Y Belgium (Fr.) Y
Chile November 2011 Canada
Czech Republic March 2011 Y Finland
Denmark October 2010 Y Y France
Luxembourg June 2010 Hungary
Mexico First quarter 2012 Iceland
New Zealand August 2010 Y Ireland
Norway December 2010 Y Y Korea
Portugal February 2011 Netherlands
Slovak Republic First quarter 2012 Poland
Sweden May 2010 Y Y Slovenia Y

5. OECD reviews of evaluation and assessment:
School evaluation in the Flemish Community of Belgium (2)
• Focus of international review is on how to best use evaluation and
assessment to improve school outcomes
– Designing a coherent framework for student assessment, teacher
appraisal, school evaluation and system evaluation
– Paying attention to use of results, evaluation capacity, procedures and
implementation
• Background report developed by University of Antwerp Edubron Research
Group and the Ministry of Education and Training
• 6 days of review, including interviews of all major stakeholders and
visits to Brussels, Antwerp, Vilvoorde and Sint-Niklaas
– Seeking a broad cross-section of information and opinions on school
evaluation policies and how to improve these
• OECD review team = Claire Shewbridge and Deborah Nusche (OECD),
Marian Hulshof (Netherlands) and Louise Stoll (UK/Canada)

6. The OECD review of School Evaluation in the
Flemish Community of Belgium
POLICY TRENDS IN THE
FLEMISH COMMUNITY AND
OECD’S ANALYTICAL APPROACH

7. Major policy developments influencing
school evaluation in the Flemish Community
Increased • Inspection focus on “output”
focus on • Differentiated inspection to target schools/areas with
greater quality concerns (2009)
school • Publication of school inspection reports (2004, 2007)
quality and • Schools legally responsible for their quality (2009)
reducing • Consultation platforms within the Policy on Equal
Educational Opportunities (GOK) (2002)
inequities
• GOK funding evaluation requirements (2002)
Stimulating • School innovation projects
• Feedback to schools from National assessment
school self- • Participation Decree (2004) ensures stakeholder voice
evaluation • Promotion of “school communities”
• Inspection judges school “policy-making capacity”

8. Analytical framework for the review of
school evaluation in the Flemish Community
Effective
evaluation
procedures
Governance: a framework
for school evaluation
Competencies Use of
for evaluation / evaluation
feedback results

9. Effective
evaluation
procedures
Governance: a framework
for school evaluation
Competencies Use of
for evaluation/ evaluation
feedback results
The OECD’s assessment of the current approach
to school evaluation
STRENGTHS AND
CHALLENGES

10. Governance: a framework
for school evaluation
• Schools have the major responsibility for the school improvement process
• The Ministry fosters a degree of common understanding of basic quality
• An increasingly information rich environment for school evaluation
• Availability of robust student assessment tools for primary schools
• Increased focus on the importance of engaging all stakeholders in school
evaluation
• The Ministry does not mandate or steer school self-evaluation
• School evaluation is not well embedded in a larger vision for evaluation
and assessment
• Miss potential synergies between different evaluation approaches
• Insufficient emphasis on improvement/excellence in attainment targets
• High degree of variation in school policy-making capacity
• Lack of information flow impedes school evaluation efforts

11. Effective evaluation
procedures
• Considerable use of technology to support self-evaluation
• The Context-Input-Process-Output (CIPO) inspection framework is
empirically grounded and comprehensive
• Collection of evidence during the inspection can stimulate school self-
evaluation activities
• There is a legal basis for inspection, including the examination of
school policy-making capacity
• Lack of clarity around purpose of school self-evaluation and minimum
quality
• Inadequate information base for risk assessment prior to inspections
• Communicating school understanding and uptake of the new
inspection methodology
• Judging schools’ implementation of attainment targets

12. Competencies for
evaluation / feedback
• Recognition of the importance of a school’s policy-making capacity
• Network support to promote and develop school self-evaluation
capacity
• Identified teacher competencies to support school evaluation
• Emerging collegial relations within and between schools to support
competency development
• Efforts to improve the capacity of the Inspectorate to conduct coherent
inspections
• Variation among school leaders in policy-making capacity
• A need to strengthen educators’ evaluation literacy
• Teachers’ orientation to research and enquiry
• Clarifying and making more uniform inspectors’ judgements on quality

13. Use of evaluation
results
• Students have a voice in school policy, including growing involvement
of students in self-evaluation
• Good balance of school responsibility and external pressure to use
inspection results for improvement
• Publication of inspection reports
• Examples of schools using feedback from national and international
assessments in their self-evaluation activities
• Examples of primary schools using network test results as part of
whole-school evaluation
• Lack of strategic and consistent use of self-evaluation
• There is room for improvement in the public use of inspection results
• Schools only receive Inspectorate profile information when being
inspected
• School self-evaluation results not necessarily shared with inspectors

14. Some challenges to implementing school
evaluation for continuous improvement
Flemish Inspection Schools
learning focus on responsible Continuous
objectives output / for quality
differentiation improvement
Focus on Lack of objective Varying levels of
minimum only. information to self-evaluation capacity
Hard to judge assess “output” among schools.
how schools prior to inspection Lack of strategic use of
implement them. evaluation results for
school improvement.

15. Effective
evaluation
procedures
Governance: a framework
for school evaluation
Competencies Use of
for evaluation/ evaluation
feedback results
The OECD’s assessment of how to build on
the current approach to school evaluation
POLICY RECOMMENDATIONS

16. Suggested priorities in going forward
Clarify the
Continue to invest in
goals of school
school leader and
evaluation and
teacher capacity to
how different
Governance: a
Competencies conduct evaluation
types of framework for
school
for evaluation/ and use its results
evaluation fit feedback
evaluation for improvement
together
Effective Use of
Increase the evaluation evaluation Increase the use of
procedures results
objectivity of available information
evaluation (collected at school
procedures and and central level) in
ensure they promote both internal
improvement and and external
excellence school evaluation

17. Governance: a framework
for school evaluation
• Further clarify common goals and expectations with a view to encouraging
excellence and continuous improvement
• Strengthen consistency and coherence of different elements of school
evaluation
– Articulate appropriately with teacher appraisal
– Reinforce links with school leadership appraisal
– Better integration between self-evaluation and inspection
• Promote the use of evaluation and assessment tools by schools for
improvement
– Comparative evaluation of externally developed self-evaluation tools for
schools
– Promote examples of effective practice
• Continue to embed self-evaluation requirements in new policies/programs
– Schools should link this to school quality assurance/improvement plan
– Provide guidelines on how to map this to CIPO inspection framework

18. Effective evaluation
procedures
• Evaluate potential to develop set of criteria for learning progressions
– Reference to assess student progress in different subjects
• Clarify the criteria for inspection judgements on the quality of
education
– Define clear criteria for all components used to determine quality
– School self-evaluation capacity is a core component of its quality
• Go further in improving the inter-rater reliability of inspection reports
– Concrete rating scale; build on inspection of school groups
• Strengthen the commitment of inspection and schools to the
implementation of Flemish attainment targets
• Extend collegial practice both within and among schools
– Further develop and coordinate emerging critical friendship
• Promote further involvement of students in self-evaluation activities
– Critical feedback for learning, teaching and school improvement

19. Competencies for
evaluation / feedback
• Further strengthen professional development for effective school self-
evaluation
– Prioritise support to building school policy-making capacity
– Conduct external review of PBD tools and services
• Recognise and strengthen key role of school leaders in self-evaluation
– Flemish framework for leadership competencies
– Develop new leadership roles
– Refine leadership training
– Ministry project on school evaluation competency development
• Increase teacher understanding of policy-making capacity
– Strengthen links between teaching quality and self-evaluation
– Build competencies in data use, research and innovation
– Fund collaborative teacher research projects with cross-school peer
review

20. Use of evaluation
results
• Strengthen information flow on key indicators from and to schools
– Provide schools with access to information in Data Warehouse
– Schools to provide performance information to the Inspectorate
– Improve availability and use of objective output measures
• Ensure regular feedback to schools on key CIPO inspection
framework indicators
– Help schools with clear goals and measurable objectives
– Work with schools to build shared language of quality/ evaluation
• Promote the availability and use of appropriate self-evaluation
resources
– Comparative overview of tools available to schools
– Assess school demand for other self-evaluation tools
• Devise ways to improve public use of inspection results
– Simplified language; dedicated website; reports for school groups

21. Ensure the quality of teaching and learning
is at the heart of school evaluation
• Clarify the goals of • Continue to invest
school evaluation in school leader
and how different Support to and teacher
types of evaluation Improvement schools (PBD) capacity to
fit together conduct
Teacher appraisal Leadership
evaluation and
/development /evaluation use its results for
Self-evaluation / competencies improvement
inspection Collaboration
Quality of
teaching
and
learning
Learning School access to
• Increase the progressions central data • Increase the use
objectivity of Inspection criteria School performance of available
evaluation Tools for data to information in
procedures and improvement Inspectorate both internal
ensure they and external
promote school
improvement evaluation
and excellence

22. Outputs available on the Review’s
website:
www.oecd.org/edu/evaluationpolicy
OECD Reviews of Evaluation and
Assessment in Education:
Australia
Denmark
Norway
Sweden
Country Background Reports:
9 out of 24 available
Literature reviews /conceptual
papers including:
formative and summative
assessment; teacher evaluation;
school evaluation; education
standards; use of results...