M&E Report SEAL 2012

VVOB Cambodia
SEAL Programme

M&E Report 2012

Preamble

The Science, Environmental and Agricultural Life Skills (SEAL) programme has been implemented by
VVOB Cambodia since mid-2008 and will be phased out by the end of 2013.

This document presents an analysis of the Monitoring and Evaluation (M&E) findings based on data
collected by the SEAL programme throughout 2012. The M&E process involved observations,
logbooks, interviews and focus group discussions with direct and indirect target groups as well as a
national M&E workshop held on 6 November 2012 together with strategic and operational partners.

This M&E report also serves as input for the peer evaluation of the SEAL programme to be conducted
from 17-21 December 2012. The findings of the peer evaluation will in its turn help to identify lessons
learned and recommendations for the new programme of VVOB to be started as of 2014.

We hope that this document provides adequate information for the above purposes.

The VVOB Cambodia team

SEAL Programme: M&E report 2012 2/79

Table of Contents

Preamble _________________________________________________________________________ 1
Table of Contents ___________________________________________________________________ 2
List of tables and figures _____________________________________________________________ 5
Tables .............................................................................................................................................. 5
Figures ............................................................................................................................................. 5
List with abbreviations _______________________________________________________________ 7
1. Introduction ___________________________________________________________________ 8
2. Overview Indicators MYP2 ______________________________________________________ 10
3. Overall Objective: Learning outcomes of pupils in basic education improve as a result of more
relevant and effective teaching and learning ____________________________________________ 12
3.1 Indicators ......................................................................................................................... 12
.
3.2 Means of Verification ....................................................................................................... 12
3.3 Results .............................................................................................................................. 13
3.4 Conclusion ........................................................................................................................ 16
4. Specific Objective 1: The percentage of graduate teachers with a sufficient level of
understanding on how to integrate technological, pedagogical and content knowledge in science and
life skills teaching __________________________________________________________________ 17
4.1 Indicators ......................................................................................................................... 17
.
4.2 TPACK Concept ................................................................................................................. 17
4.4 Results RTTC student teachers ......................................................................................... 20
4.5 Results PTTC student teachers ......................................................................................... 23
4.6 Conclusion ........................................................................................................................ 31
5. Intermediate Result 1 __________________________________________________________ 32
5.1 Indicators ......................................................................................................................... 32
.
5.2 Objectives ......................................................................................................................... 32
5.4 Results .............................................................................................................................. 37
5.5 Discussion on effectiveness with partner ........................................................................ 56
6. Intermediate Result 2 __________________________________________________________ 60
6.1 Indicators ......................................................................................................................... 60
.
6.2 Objectives ......................................................................................................................... 60
6.4 Results .............................................................................................................................. 65
6.5 Discussion on effectiveness ............................................................................................. 69
7 Intermediate Result 3 __________________________________________________________ 73
7.1 Introduction ..................................................................................................................... 73
7.3 Results .............................................................................................................................. 73
7.4 Discussion on effectiveness ............................................................................................. 75
Conclusions _______________________________________________________________________ 77
Planning 2013 _____________________________________________________________________ 77


Annexes _________________________________________________________________________ 79
Annex 1a: Lesson observation form – version 1 ........................................................................... 79
Annex 1b: Lesson observation form – version 2 ........................................................................... 79
Annex 2: M&E Protocol for field trips ........................................................................................... 79
Annex 3: Evaluation Rubric for Lesson Observations .................................................................... 79
Annex 4: Survey for teacher trainers on Agricultural and Environmental Life Skills Teaching ..... 79
Annex 5: Survey teacher trainers RTTCs ....................................................................................... 79
Annex 6: Logbook page outline ..................................................................................................... 79
Annex 7a: Guide Focus Group with Teacher Trainers ................................................................... 79
Annex 7b: Guide focus Group with life skills teacher trainers ...................................................... 79
Annex 8: Interview protocol for interviews with graduated teachers .......................................... 79
Annex 9: Individual lesson observation scores teacher trainers .................................................. 79
.
Annex 10: Overview Teaching Resources ..................................................................................... 79
Annex 11a: Report Monitoring Visit RTTCs (March 2012) ............................................................ 79
Annex 11b: Report Monitoring Visit RTTCs (June 2012) ............................................................... 79
Annex 11c: Report Monitoring Visit RTTCs (November 2012) ...................................................... 79
References _______________________________________________________________________ 79


List of tables and figures

Tables

Table 1 M&E in MYP2 .............................................................................................................................. 9
Table 2 M&E toolkit in MYP2 (2012) ........................................................................................................ 9
Table 3 Planned and Achieved Indicator Values .................................................................................... 11
Table 4 Overview lesson plans collected per subject  (data 2012) ........................................................ 18
Table 5 Overview lesson plans per RTTC ............................................................................................... 18
Table 6  Summary interviews with PTTC student teachers ................................................................... 24
Table 7 Lesson Observations in 2012 per subject .................................................................................. 34
Table 8 Lesson Observations in 2012 per RTTC ..................................................................................... 34
Table 9: Overview of target population, method and timing of surveys ............................................... 35
Table 10: Composition of teacher trainer population per subject per survey ...................................... 36
Table 11: Percentage of curriculum topics teacher trainers have problems with understanding the
content ................................................................................................................................................... 37
Table 12: Percentage of curriculum topics teachers from practice schools in Kandal have problems
with in understanding ............................................................................................................................ 38
Table 13: Percentage of curriculum topics teacher trainers from RTTC Kandal have methodological
problems with ........................................................................................................................................ 39
Table 14: Percentage of curriculum topics practice school teachers (Kandal) have methodological
problems with ........................................................................................................................................ 39
Table 15 Student‐Centred Approaches included in training teacher trainers ....................................... 40
Table 16 Share of science curriculum topics with approved learning resource developed by VVOB ... 43
Table 17 Overview approved learning resources per subject and type ................................................ 43
Table 18  Average scores for lesson observations per RTTC in March and June 2012 .......................... 51
Table 19  Challenges for integrating TPACK by teacher trainers, per RTTC in 2011 (n = 75) ................. 54
Table 20  Challenges for integrating TPACK by teacher trainers, per RTTC in 2012 (n = 62) ................. 54
Table 21  Assumptions and Mitigation Strategies for IR3 (MYP2, p.59) ................................................ 76

Figures

Figure 1  Student‐centred approaches applied by student teachers during their teaching practice
during school year 2011 – 2012 (n = 400) ............................................................................................. 21
Figure 2  Technologies applied by student teachers during their teaching practice (n = 400).............. 21
Figure 3 Use of extra didactical materials by student teachers at RTTC Kandal during their practicum
in 2008 and 2010 ................................................................................................................................... 23
Figure 4  Frequency of applying SCA by science teacher trainers at RTTCs (n = 61) ............................. 41
Figure 5 Reported frequency of use of various student‐centred approaches by science teacher
trainers (n=62) ....................................................................................................................................... 41
Figure 6  Frequency of applying teaching aids for SCA by science teacher trainers at RTTCs (n = 61) . 43 .
Figure 7 Reported use of teaching aids by RTTC science teacher trainers (n = 62) ............................... 44
Figure 8 Use of school garden (left) and waste management system (right) by RTTC earth science &
biology teacher trainers (n = 25) ............................................................................................................ 45


Figure 9: Number of users of projectors per college ............................................................................. 46
Figure 10: Average projector use per teacher trainer ........................................................................... 47
Figure 11 Support provided by teacher trainers to student teachers during practicum (data baseline
study 2008) ............................................................................................................................................ 48
Figure 12  Need for help by teacher trainers in providing methodological support to student teachers
during practicum (data baseline study 2008) ........................................................................................ 48
Figure 13  Need for help by teachers of practice schools in providing methodological support to
student teachers during practicum (data baseline study 2008) ............................................................ 49
Figure 14 Support provided by science teacher trainers during practicum (2011 – 2012) (n = 57) ...... 49
Figure 15 Comparison lesson observation scores for RTTC teacher trainers in March and June 2012. 50
Figure 16 Reported use of SCA by science teacher trainers per RTTC .................................................. 51
.
Figure 17  Challenges reported by science teacher trainers for integrating TPACK in 2011 (n = 75) and
2012 (n=62) ............................................................................................................................................ 53


List with abbreviations

ADB Asian Development Bank
BIO Biology
CHE Chemistry
DCD Department of Curriculum Development (of MoEYS)
DP Development Partner
EEQP Enhancing Education Quality Project
ES Earth Science
ESDP3 Third Education Sector Development Program
GSED General Secondary Education Department (of MoEYS)
IBL Inquiry‐based lessons/learning
IR Intermediate Result
LF Logical Framework
LS Life Skills
JICA Japan Overseas Cooperation Agency
MoEYS Ministry of Education, Youth and Sport
PED Primary Education Department (of MoEYS)
PHY Physics
POE Provincial Office of Education
PTTC Provincial Teacher Training Centre
RTTC Regional Teacher Training Centre
SCA Student‐Centred Approach
SEAL Science, Environmental and Agricultural Life skills (programme)
SO Specific Objective
STEPSAM2 2nd Strengthening Teacher Education Project for Science and Mathematics (of JICA)
TDAP Teacher Development Action Plan
TPACK Technological, Pedagogical and Content Knowledge
TIMMS Trends in International Mathematics and Science Study
ToT Trainer of Trainers
TTC Teacher Training Centre
TTD Teacher Training Department (of MoEYS)


1. Introduction

As in 2011 this M&E report follows the structure of the logical framework.  Per objective (overall and
specific) and result (IR1, IR2 and IR3), indicators, means of verification, results and conclusions are
presented.

The table below gives an overview of the various M&E instruments deployed in the MYP2 per level
in the logical framework (LF) and specifies what outputs, outcomes and impact are intended.

Level capacity LF Level Activity Output Outcome Impact
building
Institutional level
MoEYS Intermediate Meetings Revised Application of Behaviour and
Result curriculum curriculum, attitude
Workshops teaching aids change w.r.t.
Approved and manuals student‐
Conferences teaching aids centred
& manuals RTTCs report approaches,
Working
Updated on science & life
groups
templates for management skills education
reporting science & life
skills
infrastructure

Organisational level
RTTCs & PTTCs Intermediate Identification Lab, Resource Usage Commitment
Result and selection Room, to
Procurement equipped maintenance
library and
Distribution improvement
Training

Structures in
place for CPD
for teacher
trainers

Individual level
  Teacher trainers   Intermediate Workshops Lesson plans More Behaviour and
Result knowledge and attitude
  Teachers Lesson skills change
observations
Usage of
Material teaching
development materials and
manuals
  Student teachers Specific / /


Objective

  Pupils Overall / / Experience Higher
Objective more relevant motivation,
and effective involvement
teaching and and
learning satisfaction.

Improved
learning
outcomes.

Table 1 M&E in MYP2

The table below gives an overview of the M&E tools applied in 2012.

LF Level Outcome Impact

Knowledge and skills Behaviour and attitude change
Usage of materials Higher motivation and satisfaction.
Improved learning outcomes (pupils)
Institutional level Interviews with MoEYS Interviews with MoEYS officials
officials
(MoEYS) Plans (TDAP, ICT Master plan)

Organisational level Logbooks Interviews with RTTC and PTTC
(TTCs) Management
Reports to MoEYS

Checklist Life Skills Activities

Individual level

  Teacher trainers   Intermediate Logbooks Focus Group discussion
Result
Lesson Observations Interviews with teacher trainers

Survey

  Teachers Intermediate Lesson Observations In‐depth interviews young teachers
Result

  Student teachers Specific Lesson Observations during Interviews with student teachers
Objective practicum
Interviews with young teachers
Lesson plan analysis

  Pupils Overall Lesson observations in schools
Objective

Table 2 M&E toolkit in MYP2 (2012)


Compared with 2011 following changes were made to the toolkit:

1. The TPACK survey was replaced by smaller survey.
2. Form for lesson observations was revised
3. A rubric for scoring lesson observations was introduced.
4. A representative sample of lesson plans from student teachers, made during their
practicum, was collected.

2. Overview Indicators MYP2

LF level Indicator description Baseline Value end Value end
2010 2012 2013

The average percentage of pupils showing
active involvement in learning science and life
skills in practice schools NA NA 80%

Overall objective The percentage of pupils passing tests on
science subjects including problem solving and
life skills in practice schools NA NA 80%

The percentage of graduating student teachers
with a sufficient level of understanding on how
to integrate technological, pedagogical and 0% 50% 70%
content knowledge in science and life skills
Specific Objective
teaching.

The percentage of student teachers that
integrate technological, pedagogical and
content knowledge in science and life skills 0% 50% 70%
during their teaching practice.

Teacher trainers’ understanding on how to
content knowledge in science and life skills 15% 70% 85%
teaching.
Intermediate
Result 1 Teacher trainers’ application in their lessons of
how to integrate technological, pedagogical
and content knowledge in science and life skills 15% 60% 70%
teaching.

Teacher trainers’ coaching of student teachers
in how to integrate technological, pedagogical
and content knowledge in science and life skills 10% 50% 60%
teaching during the teaching practice.


Intermediate The percentage of PTTC teacher trainers with a
Result 2 sufficient level of understanding on how to
integrate technological, pedagogical and 0% 65% 85%
content knowledge in life skills teaching.

The percentage of PTTC teacher trainers that
content knowledge in their life skills teaching. 0% 65% 70%

The percentage of PTTC teacher trainers that
systematically provide guidance during
students' teaching practice on how to integrate 0 % 65% > 50%
technological, pedagogical and content
knowledge in science and life skills teaching

Educational materials (including manuals, RTTC: 0 RTTC:6 RTTC: 6
posters, a DVD box and digital learning objects) PTTC: 4
to support 4 science subjects and life skills PTTC: 0 PTTC:4
teaching are approved and disseminated (in
Intermediate
print or on‐line) by TTD
Result 3
80 % of developed educational material Printed: 0 Printed:
(printed and audio‐visual) integrated into TTD's >50%
professional development programs and the Audio‐
national teacher training curricula Visual: 0 Audio‐
Visual:
>50%

The integration of technological, pedagogical Not Partly Integrated
and content knowledge framework into the integrated integrated
M&E processes of TTD

Table 3 Planned and Achieved Indicator Values


3. Overall Objective: Learning outcomes of pupils in basic
education improve as a result of more relevant and effective
teaching and learning

3.1 Indicators

The target group of the overall objective are pupils in basic education (grades 1 – 9).  The way we
aim to make classes more effective is by making sure that new teachers who graduate are better
equipped to teach science, environmental and agricultural life skills at primary and secondary
schools.

A success rate of 80% by 2013 is considered appropriate and feasible (MYP2, p.60). Success can be
described as:
‐ The lesson is positively evaluated by the observers.
‐ Pupils indicate a positive change in teaching style.
‐ Teachers indicate satisfaction and motivation to apply SCA.

Following indicators were identified in MYP2
‐ The average percentage of pupils showing active involvement in learning science and life skills in
practice schools
‐ The percentage of pupils passing tests on science subjects including problem solving and life
skills in practice schools

Note that the population are the pupils from the 36 practice schools, affiliated to 6 (out of 6) RTTCs
and the 24 practice schools, affiliated to 4 (out of 18) PTTCs.

3.2 Means of Verification

3.2.1. Lesson Observations & Interviews with pupils (planned in 2013)

Pupils from primary schools and pupils from grades 7 – 9 will be interviewed before and after lesson
observations.  Lessons will be observed at 6 schools, one per RTTC in 2013.  Schools will be selected
from the 36 practice schools. The impact on pupils from non‐practice schools is assumed from
observations at the practice schools.

For RTTC: Four lessons per school will be observed and 4 pupils per lesson invited for a short
interview, resulting in 96 interviews.

For PTTC: Lessons will be observed at 4 schools, 1 practice school of each of the 4 target PTTCs.
Schools will be selected from 24 practice schools. Also lessons will be observed at 2 practice schools
of 2 non‐target PTTCs as there are no baseline data for the PTTC level.

The objectives of the interviews are to learn:
‐ What are pupils’ descriptions of a “normal” lesson?
‐ How do pupils perceive the lesson compared with a “normal” lesson?


‐ What are pupils’ emotions when confronted with a student‐centred teaching approach?
‐ Did pupils feel that they learned more with this approach?
‐ What would pupils change to the teaching?

3.2.2. In‐depth interviews with recently graduated students (aka young teachers)

During interviews with young teachers, we asked some questions about pupils’ reactions when
applying student‐centred techniques and technologies, and about any evolution in pupils’ learning
outcomes. (See a more detailed description of the tool under 3.2.2)

3.3 Results

3.3.1 RTTC

There is a time lag between programme activities and a noticeable impact with the target group.
Training activities with teacher trainers in Kandal started in earnest in 2009 and application of SCA
and ICT by teacher trainers at the pilot institution (RTTC Kandal) wasn’t in full swing until 2010.  This
implies that the student cohort graduating in July 2011 was the first cohort that could have
experienced a more student‐centred teaching approach by teacher trainers. This means they are
the first group of graduated teachers who can reasonably be expected to teach in a more student‐
centred way.   For the other RTTCs trainings started at the end of 2011 and continued throughout
2012. For RTTCs outside Kandal students graduating in 2014 will be the first group to have
experienced student‐centred approaches. The impact in the RTTCs outside Kandal will therefore
largely take place beyond the lifetime of the programme.

Some indications of the impact of the programme can be obtained from anecdotal evidence and
informal dynamics. For example, draft versions of manuals, posters and experiment descriptions
have been repeatedly requested and copied by students. POE Kandal has requested copies of
posters to distribute to schools and many student teachers have been using our YouTube videos of
experiments to prepare their practice lessons (23.041 views on VVOBCambodia YouTube Channel on
November 12, 2012).

Recently graduated teachers indicate that they are quickly made head of subject, often after a few
months of teaching.  Lesson observations convince school directors of the stronger capacity of newly
graduated teachers.  This could be because directors are aware of recent policies to promote the use
of SCA in schools.  As head of subject they can lead monthly technical subject meetings and is in a
position to introduce SCA to their colleagues.

“The first monthly technical meeting takes place tomorrow, in total, 8 teachers from physics
and chemistry teaching G7 to G12 attend the meeting. Last year, our agenda discussion
focused on lesson content for example electrostatic, the use of chemical periodic table etc.
The discussion on teaching methodology was also in our agenda. In the meeting I mentioned
the number of techniques of SCA I learned during my training program at RTTC Kandal, I
indicated the name of techniques, but not really show them the procedure and how to apply
those techniques yet. I plan to introduce them in the next meeting because the agenda for
tomorrow is fully covered on other issues. When I applied SCA in my lesson, most teachers


showed their interested and requested me to share with them someday. My director also
satisfied my lesson when applying SCA, because he can compare my good lesson to the
lesson without SCA integration, students are more active and have a good discussion and
interaction”.  (Young Teacher A)

In a student‐centred classroom also students need to come along. Stories from young teachers
show that some pupils find it difficult to be active and think for themselves. Some pupils reportedly
have low literacy skills, making it difficult to apply student‐centred approaches.

“SCA is seen as challenges to apply in every lesson. Most students in my class have low ability
to read and write, they are familiar with the approach to just copy the text from the board
into their notebook, and not able write independently or with their own words. I can
conclude that one reason is the quality of their primary education is not high. Another reason
is that teachers are afraid to not finish the curriculum by the end of the school year, so the
lesson went fast and students find it difficult to follow. In my lesson last year, I spent more
time lecturing than involving students in the discussion, I hope in this school year, the student
prior knowledge is better and stimulate to apply SCA more frequently”. (Young Teacher

The main barrier for young teachers to apply SCA in their schools is time. Many teachers have a
second job to complement their income. In school they teach more than one subject or teach in
upper secondary classes as well. More teaching hours leaves them little time to prepare their
lessons.

“When I became a teacher at this school I am appointed to teach more than one subject, this
had leaded me to have little time preparing my lesson plan, also if I integrate and apply SCA
in every lesson, I will not able to finish all topics in the curriculum at the end of the school
year. Besides the teaching, I also have to support my family in farming to make up for the
income”.

Time also relates to the pressure to finish the curriculum in time.  All young teachers found that the
curriculum is too extended.  In order to cover all the topics they tend to resort to lecturing and other
teacher‐centred approaches.

“Applying SCA more frequently can slow down my lesson. Sometimes lecturing is a good
choice rather than encouraging student in the discussion”.

3.3.2 PTTC

As mentioned above also for the life skills programme for PTTCs there is a time lag between
programme activities and a noticeable impact with the target group. Teacher trainers from PTTC
Siem Reap have been doing try‐out lessons at PTTC Siem Reap during the school year 2010‐2011.
The first group of students that could have benefitted from the new approaches in the subjects
Agriculture and Environment graduated in June 2011. From this group 8 students have been
interviewed then, and again in October 2012. This means they are the first group of graduated
teachers who can reasonably be expected to teach in a more student‐centred way. The interviews of


October 2012 show that they have been using SCA and are experimenting and trying to improve
themselves.

Training activities for teacher trainers of all 18 PTTCs started in early 2011 and improvements in the
environment and agriculture lessons and application of SCA by teacher trainers there only fully
started with the school year 2011‐2012. The student cohort graduating in 2012 was the first cohort
that could have experienced a more student‐centred teaching approach in the life skills lessons by
teacher trainers.  The impact in the PTTCs outside Siem Reap will therefore largely take place beyond
the lifetime of the programme.

Some indications of the impact of the programme can be obtained from anecdotal evidence and
informal dynamics. Students mention the change in their attitude towards keeping the school
grounds at PTTC Siem Reap clean and seeing the benefits of segregating waste. When they came
back from their teaching practice they notice there was no waste segregation system in their
practice schools and they are planning to implement this in their schools. They also mention that
they do get support from their school directors to implement SCA in the lessons.

“My director completely supports applying SCA and helps with finding or producing teaching aids.”

The main barrier for young teachers to apply SCA is lack of teaching aids they mention. The school
has some budget but it is not enough. They do try and make teaching aids themselves and use easy
to find real life objects in their lessons or ask pupils to bring materials from home.

Pupils often are not used to different student centred techniques.

“Students have not seen some methods before, like the questioning game, they are scared to play it
or afraid to answer questions.”

Pupils’ reactions

During the follow up visit to PTTC Banteay Meanchey and PTTC Battambang in May 2012 pupils were
invited to answer some questions. A group of 3 pupils from grade 6 at the practice school in
Battambang said:

“Yes, my teacher gives a nice lesson. He leads us to play educational games and teaches
about reading. I know that environment is everything around us such as school premises,
toilets, etc. I have learned the meaning of 3 colours bins. We have learned how to collect the
waste, clean the classroom and school ground, plant and water the flowers. Sometimes we
have a lesson outside and the teacher asks us to collect the waste and put it in the bins. We
have learnt how to grow vegetables such as cabbage and morning glory, but we did not learn
how to raise both chicken and fish yet. We have practice at outside very often.”

Quotes from a group of 4 pupils from grade 6 at the practice school Battambang:

“Yes, I like going to school. The teacher gives nice lessons such as sports, agriculture, telling
tales and joke stories. Environment means everything around us such as tree, chair, swing,
teachers, students, water, rain, wind, soil and so on. We have learnt about waste disposal.
The teacher explains me about the role of each colour waste bin. We now collect the waste


around the school ground, classroom and put them in the right waste bins.  I learn how to
grow vegetables at school. I learn how to roof the chicken cage, and we have learned about
doing vaccination and making chicken food. Also we learn the theory of digging the hole for a
fish pond. What I like best is growing chili, digging the soil, water the vegetables.”

3.4 Conclusion

Measuring impact is challenging due to the time lag and the difficulty to link changes in behaviour
causally to programme activities. There is anecdotal evidence that graduated teachers are using
student centred approaches in their schools, but this needs to be confirmed by lesson observations
and more interviews with pupils in 2013. The graduated teachers who were interviewed did not
report resistance by elder teachers and the management as an important barrier.  Conversely, some
have been made head of subject already.  Instead, time often presented the main challenge as they
have multiple jobs, teach various subjects and get classes in upper secondary school assigned.


4. Specific Objective 1: The percentage of graduate teachers with a
sufficient level of understanding on how to integrate
technological, pedagogical and content knowledge in science
and life skills teaching

4.1 Indicators

Two indicators were defined for SO1:
‐ The percentage of graduate teachers (graduated student teachers) with a sufficient level of
understanding on how to integrate technological, pedagogical and content knowledge in science
and life skills teaching.
‐ The percentage of student teachers that integrate technological, pedagogical and content
knowledge in science and life skills during their teaching practice.

Target values for 2013 were determined.
‐ The target value for 2013 for both indicators is an impact on 70% of all student teachers in an
effective way. The baseline value in 2011 is 15% as only student teachers in Kandal can be
expected to be impacted by the programme (Table 3).

4.2 TPACK Concept

Technological Pedagogical Content Knowledge (TPACK) attempts to capture some of the essential
qualities of knowledge required by teachers for technology integration in their teaching, while
addressing the complex, multifaceted and situated nature of teacher knowledge. At the heart of the
TPACK framework, is the complex interplay of three primary forms of knowledge: Content (CK),
Pedagogy (PK), and Technology (TK).

Figure 1 TPACK Concept (Source: http://tpack.org/)


A central element in the TPACK concept is that applying a student‐centred approach or technology in
itself is not sufficient. There is only an improvement in teaching if there is a negotiated balance
between content knowledge, pedagogy and technology, whereby technology is not limited to digital
technologies.  Indeed, in the Cambodian context, non‐digital technologies such as posters, cartoons
and games arguably play a more important role than digital technologies.

4.3 Means of Verification

4.3.1 Analysis lesson plans from teaching practice

Lesson plans developed during teaching practice were collected from a random sample of student
teachers from the 6 RTTCs.  Table 2 and

Subject Number of student teachers Number of lesson plans
Kandal 9 82
Phnom Penh 9 51
Takeo 9 32
Prey Veng 9 113
Kampong Cham 9 65
Battambang 9 80
Total 54 423
Table 5 provide an overview of the sample characteristics.

Subject Number of student Number of lesson
teachers plans
Math‐Physics 15 60

Physics‐Chemistry 18 178

Biology‐Earth Science 17 185

Table 4 Overview lesson plans collected per subject (data 2012)

Subject Number of student teachers Number of lesson plans
Kandal 9 82
Phnom Penh 9 51
Takeo 9 32
Prey Veng 9 113
Kampong Cham 9 65
Battambang 9 80
Total 54 423
Table 5 Overview lesson plans per RTTC

Misinterpretation of instructions caused the low number of lesson plans from RTTC Takeo.


Student teachers prepare lesson plans for the 2 subjects of their specialisation (mathematics‐
physics, physics‐chemistry or biology‐earth science).  The subject listed first is the major subject and
student teachers are required to prepare more lesson plans for their major subject. Chemistry and
earth science can only be studied as the second subject, which explains the lower number of lesson
plans for these subjects.

4.3.2 Interviews with student teachers

Interviews with small groups of student teachers were systematically done during follow‐up visits to
the 5 RTTCs in March, June and November.  Per RTTC 3 or 4 student teachers were interviewed per
visit, resulting in a total of 60 interviews.

In March and May 2012 interviews were done with student teachers from the 3 target PTTCs
(Banteay Meanchey, Battambang and Kandal). In total 12 student teachers took part in the
interviews.

4.3.3 Lesson observations (during teaching practice)

20 Lesson observations were carried out at the practice schools of RTTC Kandal during the practicum
of 2nd year students in the period February– April 2012.

As for practice schools of PTTC Siem Reap the teacher trainer responsible for collecting lesson
observation forms during teaching practice is at the moment processing the collected data. The
results will be integrated in the next version of this report.

Objectives of the lesson observations are:
‐ To asses student teachers’ awareness of SCA and their capability to successfully integrate SCA
into their lessons.
‐ To provide feedback to student teachers about the integration of SCA into their lessons.
‐ To assess the suitability of learning materials developed by the SEAL programme in a local
classroom context.

4.3.4 In‐depth interviews with recently graduated students (aka young teachers)

We selected in‐depth interviews as a research methodology as we wanted to obtain insights in
deeper motives of participants.  Since we would like to discuss barriers and challenges encountered
during their teacher training, we considered individual interviews more appropriate in this context
than focus group interviews.

In‐depth interviewing is a qualitative research technique that involves conducting intensive
individual interviews with a small number of respondents to explore their perspectives on a
particular idea, program, or situation (Boyce and Neale, 2006).  We selected the subjects for the in‐
depth interviews with purposive sampling. This means that we deliberately selected participants
according to preselected criteria relevant to our research question.  When the same stories, themes,
issues, and topics emerge from the interviewees, a sufficient sample size has been reached (Boyce
and Neale, 2006).


In May 2011 we seelected 9 RT and 8 PTTC student for a sem
TTC ts mi‐structured in‐depth in
d nterview.
Interview were con
ws nducted in J
June 2011 a follow‐up interviews in October 2012. In 2
and s r 2013 the
same paarticipants w be interv
will viewed for a last time. The 17 stu
a udents were selected fro RTTC
om
Kandal and PTTC Siem Reap, s since only th
hese studen were fam
nts miliar at the time with student‐
e
centred teaching approaches.

For RTTC
C they were selected acc
cording to following crite
eria:
‐ Bala ance between 3 speciializations (earth science/ biology physics/ chemistry, physics/
y,
matthematics);
‐ Gen nder balance; ;
‐ Bala ance between province oof origin (Kan
ndal, Kampon ng Speu, Kam
mpong Chnanng)

For PTTC
C they were randomly seelected ensu uring gender balance.

For the s
start intervie
ew the objecctives were to:
‐ Obtain insights i into their understanding, applicationn and appreciation of SCA A;
‐ Lear rn about their plans and expectations about their r teaching;

For the f
follow‐up intterviews thee objectives a
are to:
‐ Com mpare the tea achers’ expeeriences withh their plans and expecta ations the year before.
‐ Desc cribe their ch
hanges in SCCA appreciatiion and unde erstanding.
‐ Dete ermine lessoons learned rregarding strrategies to en
nhance adop ption of SCA in school pra
actice.
‐ Disc cuss possible changes in teacher education in order to better prepar and equi future
e n re ip
teac
chers.

4.4 Results R
RTTC stude
ent teacher
rs

During t
their practic
cum in 2012 about half of the student teachers used IBL o group wo One
s or ork.
student in five applied SCA, introduced by VVOB. Ne
y early 60% used a poste in their practicum
er
(Figure 2
2).

Figure 2 P
Percentage of st
tudent teacher
rs applying SCA
A (left) and usin
ng teaching aids (right) (n=53)
)

Student teachers ap pplied a varie
ety of student‐centred a approaches ( Group work (in more
(Figure 3).  G
than ¼ of lesson plans)and inquiry‐based learning (in about 1/6 of lesson plans) prov
p n 6 ved most

SEAL Prog
gramme: M&E r
report 2012 20/79

popular, the forme being a c
, er catch‐all term covering a wide var
riety of active (and less active)
instructi
ional techniq
ques and the
e latter has b ted by Stepsam2 since 20
been stimulat 008.

Techniques introducced by VVOB
B do show up in student ts’ lesson pla
ans, albeit not in large n
numbers.
Howeve trainings for teacher trainers st
er, r tarted only in October 2011 with the bulk o
organized
between or having an impact on th
n February and May 2012, too late fo he practicumm.

Figure 3  S
Student‐centre
ed approaches applied by stud
dent teachers d
during their tea
aching practice
e during school year 2011
– 2012 (n = 400)

Technologies applied by stude
Figure 4  T ent teachers du
uring their teac
ching practice (
(n = 400)

Posters are the mosst popular technology use ed by studen
nt teachers d
during their practicum (FFigure 4).
More thhan 1/3 of th
he lesson plaans featured the use of a
a poster and
d nearly 60%% of student teachers
used one.  About 1 iin 8 lesson p
plans include
ed a low‐cost
t experimentt and 1 studeent in 4 used
d at least

SEAL Prog
gramme: M&E r
report 2012 21/79

one experiment during their teaching practice (Figure 2).  Posters include mainly copies of textbook
pictures, self‐made posters and (in Kandal) posters developed by VVOB, which are available for
borrowing by the students. Distribution of posters to other RTTCs took place after the practicum
(February – April), so that usage figures are expected to rise in 2013.

Most SCA were applied by student teachers from RTTC Kandal, except for IBL. Nevertheless, in some
RTTCs, notably Takeo, lesson plans contain SCA and experiments introduced by VVOB in 2012.  These
data are confirmed by interviews with student teachers. For example, students physics and
chemistry at RTTC Kampong Cham indicate to do at least one experiment per week.  Earth science
and biology students report that posters are frequently used by their teacher trainers, and plan to
use them in their practicum.

Interviews revealed that student teachers appreciate that their teacher trainers apply SCA.
Following reasons were mentioned:

‐ Lessons are more enjoyable, increasing their motivation
“SCAs lead to a better learning environment.  I feel more motivated to come to class.  Later in
my school I want to create the same feeling with my pupils.” (Student Teacher RTTC
Kampong Cham, March 2011).

‐ They get a better understanding of the content
“I am not from science subject, but when received cascade trainings from colleagues and
have applied with STs, I found that student teachers like the lessons and learn better.”
(Pedagogy teacher trainer RTTC Kampong Cham)

‐ Materials are relevant for them, saving them the cost to buy materials themselves.
“For science posters, student teachers like them. With the A3 format of periodic table, some
student teachers do not need to buy personal pocket periodic table anymore.” (Teacher
trainer chemistry, RTTC Prey Veng)

‐ They get inspiration to apply SCA in their teaching, thus becoming better teachers.  Student
teachers discuss with each other which techniques and materials could be useful for their
own teaching.
“I liked the lesson on the Sun and the Moon.  First we watched a video, and then the teacher
used a poster to explain about the solar eclipse.  Then we used [concept] cartoons to discuss
about the reason of solar eclipse. In my school I cannot use the video, but I can use the
cartoon to let my students discuss about the solar eclipse.” (Student teacher RTTC Prey
Veng, March 2011)

Data are difficult to compare with baseline values, collected in 2008 and 2010, as these were based
on self‐assessment data, and not on analysis of lesson plans and were limited to RTTC Kandal.  Data
in 2010 showed a strong rise of the use of didactical materials in RTTC Kandal by student teachers
during their practicum (Figure 5).


Figure 5 Use of extra didactical materials by student teachers at RTTC Kandal during their practicum in 2008 and 2010

Up to now, only model teachers from practice schools from RTTC Kandal are acquainted with SCA
and the supply of teaching aids.  Training activities for model teachers from other RTTCs are planned
in early 2013.  This is important as lesson observations by teacher trainers are very limited during
the practicum, and usually only during the final lesson.

Data don’t give information about the quality of the applied technique.  Experiments, for example,
are often used to ‘prove’ the theory taught before, rather than as a tool to encourage student
thinking.

4.5 Results PTTC student teachers

In‐depth interviews with student teachers in 2011

In July 2011 a number of 8 graduate student teachers were asked to participate in an in‐depth
interview. The student teachers were chosen randomly, 4 female and 4 male student teachers. The
questions of the interview (Annex 8) are follow up questions related to the TPACK questionnaire to
find out more about student teachers understanding of aspects of teaching and learning, especially
the Student Centred Approach.

Below is a summary of the interviews:

M/F Scale Can explain Nr.of SCA How often did Important? TTs showed Expected
SCA approaches you use it/when? SCA? difficulties in
concept mentioned future school
and its during
usefulness interview
1 M 7 ++ 8 During teaching Strongly believe Sometimes, also Lack of teaching
practice, maths it’s important learn by myself aids
example


2 M 8 + 8 Example of Strongly believe Yes a lot Lack of teaching
maths. aids, absence of
students
3 M 7 ++ 12 Often, maths Yes Encouraged me Low capacity of
example teachers, lack of
teaching aids
4 M 7 ++ 9 Maths example Yes vital Often and lots Low capacity of
teachers, lack of
teaching aids, bad
management
5 F 7.5 +++ 10 During teaching Yes, experienced Showed good Lack of teaching
pract. Maths that it helps examples and aids, lack of
example. encouraged participation of
during teaching students but SCA
practice can help with that
6 F 8 + 7 In the Nat.educ. Yes, it saves time, Yes they helped Making teaching
program so I have st.help each aids takes a lot of
to, used in other, share ideas time, students are
teaching pract. noisy and lazy in
group discussion
7 F 8 ++ 12 During teaching I experienced it 70% of them yes I have to do more
pract., soc.study helps so yes, and self-study on SCA,
example. poor research prepare good
skills in students lesson plans,
need to be
improved
8 F 8 + 11 Math.example. Vital and good Yes they give Too many students
and during expl.why ideas in one class,
teaching pract producing teaching
aids is time
consuming,
Av 7.56
Table 6  Summary interviews with PTTC student teachers

On understanding the concept of SCA female students rated themselves a little bit higher than the
male students (7.9 against 7.3 for the male students) on a scale from 1 to 10. When asked to explain
what SCA is and how useful it is, 1 girl gave a very good explanation, 2 gave a sufficient explanation
and the 4th female student teacher’s explanation was a bit under average but still showed some
understanding.
Out of the 4 male students 3 were able to give a complete and thorough explanation. All 8 students
were able to give a good variety of examples of teaching approaches (average nearly 10 examples
per student). 2 of the 8 student teachers mention spontaneously they are familiar with the list of 36
SCA activities.

All student teachers mention that they used SCA activities during their teaching practice and they
were encouraged to do so by the teacher trainers.  When asked to give an example of using SCA in
their own teaching most students (6 out of 8) gave an example related to Mathematics.

The biggest problem they expect to encounter when they start teaching in their future schools is a
lack of teaching aids. Some mention that they can address that challenge by making teaching aids
themselves or use real life teaching aids that do not take much time to prepare.


In‐depth interviews 2012 with 4 primary school teachers (graduated and interviewed in 2011)

In October 2012 an interview was held with 4 of the 8 above mentioned student teachers, now
primary school teachers. Most questions are the same as last year. Some questions were added
about their experience with SCA after 1 year of teaching and some questions about their school. The
4 primary school teachers were chosen randomly from the total of the 8 above mentioned student
teachers that were interviewed in July 2011. This is a summary of their answers.

M/ Scal Explain SCA Examples Why do Challenges Would you Characteristics of a good
F e concept and its given you use consider using teacher
usefulness it? more or other
SCA?
1 F 9 More activities for Educational Students Students are Yes I am still Motivation is the most
students, student games, are able not used to improving and important weather they
are confident and experiments to figure group could do other are in a rural area or in a
learn , group things out discussion, they educational town
independently work, and it hesitate to games, painting,
and clearly practice, makes show ideas, but thinking‐
understand, they questioning, student now they get entertain‐
work together brainstorm, responsibl used to it rethinking
and learn observation e. because the technique.
cooperatively. s outside group
the discussion can
classroom be arranged
much faster no.
Lack of teaching
aids, but use
low cost
materials.

2 F 8.5 It makes students It serves Is still use it and Strongly respects the
able to find teaching try other school discipline and make
solutions, Group work, and Lack of methodologies students understand and
collaborate and questioning, learning. resources, which makes it follow, solving problems
help each other, experiments If during example giving: easier for students for the students
and they are lesson a lesson on to they want to
active teacher electricity learn
talks a lot,
students
get bored
and lose
attention,
with SCA
they stay
interested
, they can
share
ideas

3 F 5 Students are Learning Because Students have I use SCA once in a Trying to find new
more active than games, students not seen some while depends on knowledge
the teacher, it guided can methods the lesson, I still
makes students questions, express before, like the use it and am


more interested brainstormi their ideas questioning trying to find out
ng, analyse, and it is game, they are more methods to
discuss in quick for scared to play it use.
pairs them to or to answer
remembe questions
r and they
will
remembe
r more.

4 M 7 Students are Group Students Difficult to Pupils’ ability to Try to provide pupils with
more active, discussion,   learn handle the SCA evolves. In lessons that they can
students can search for collaborat variety of levels group discussion, understand, teach
grasp the content information ively and in the class. they sometimes regularly, make a lesson
of the lesson as it easier don’t understand plan and prepare teaching
immediately, and homework, for them Some pupils are the content. That aids.
students are able pair to grasp not so literate is why the lesson
to show their discussion, the and then group should also use
ideas. small content. discussion is other techniques.
groups awkward.
solving Through class
maths discussions or pair
assignment, discussion pupils
pupils at different skill
present   levels can learn.
findings

demonstrati
on with real
life objects,
let students
draw
numbers on
the board

M/F Meetings in school Do colleagues Support from director Budget to buy Suggestions how to
apply SCA? materials for life skills improve the teacher
in applying SCA and other resources? education PTTC to be better
prepared?

1 F Once a month, if it is Yes, some do I have to demonstrate a lesson 2 or 3 Yes, but it is not To provide even more new
not too busy. Used to times a year. POE and my director enough, students techniques and more ideas
show a model lesson and visitors from Japan and VVOB. shared by bringing to make low costs
and the team gives My director completely supports brooms and collect a materials/teaching aids.
feedback and share applying SCA and helps with finding bit of money to buy If all student teachers apply
materials used. or producing teaching aids. small bins or bring SCA in their schools the
some wood to make primary schools will surely
them.  I can make be good in the future.
copies but the budget
is not enough.

2 F Every month, Yes Yes, for example when some lessons Yes the school has a
discussions about require additional resources the budget to support
good methodology, director helps to facilitate this. During school expenses, but it


M/F Meetings in school Do colleagues Support from director Budget to buy Suggestions how to
apply SCA? materials for life skills improve the teacher
in applying SCA and other resources? education PTTC to be better
prepared?

cascading methods, technical meetings a model lesson is mainly supports the
sharing ideas on SCA demonstrated and discussed fence, but we can also
afterwards. make copies.  I
suggested more
experimental and
teaching resources.

3 F Every month, Yes Yes, for example when some lessons No budget. Request the teachers to
discussions and require additional resources the teach regularly. Teacher
trying new teaching director helps to facilitate this. School I would like more shows up during agriculture
methods,   director observed my lesson. resources but the lesson and asks students to
school cannot provide try and understand by
During technical meetings a model it. themselves. Request the
lesson is demonstrated and discussed students to bring the
afterwards. resources by themselves for
the agriculture lesson.

4 M Every month. Yes, for School director helps with materials PB Budget, but not so If VVOB has new good
example group and methodology to teachers. Giving much. I will share some lesson, manuals or other
Discuss about discussion, feedback after a lesson. money to buy things background information
geometry, search for but not so much. about agriculture and
measurement unit, information, Teachers from clustered schools, in‐ environment, it can help
dictation, essay assigning tasks school teachers have done a lesson share with student teachers
writing. If one and exercises to observation. so that they can apply when
teacher does not test students’ they are at the future
understand the ability. schools. VVOB can
methods well, one distribute some manuals to
another will provide keep the old teachers that
knowledge of this. have never been involved
with VVOB posted.

Compared to last year 3 out of 4 teachers rate their understanding of SCA higher. The 4th teacher
rated herself lower (5) but when asked to explain the concept and its usefulness and to provide
examples she was able to do so. She does mention that she is using it once in a while and is trying to
find out more methods to use. All 4 teachers can explain why they feel SCA is useful.
The 4 teachers can name a total of 13 different SCA techniques that they use. Challenges they
mention is that pupils are not used to the new methodology and are sometimes shy to answer
questions or participate in a game. They also mention lack of teaching aids. There is a budget to buy
materials but it is not enough. All 4 teachers have a monthly meeting at their school with the team
where methodologies are shared and model lessons observed. They all feel that their school director
is supportive of SCA and tries to help in finding or making teaching aids. After 1 year of teaching the
teachers are still motivated to keep teaching. When asked do you plan to keep working as a teacher
all 4 primary school teachers answer in a positive way.

“I will keep working as a teacher especially it is useful for young pupils. If the pupils are well‐
educated at the basic level, they will be strong at higher levels. It is useful for myself, my
family. Due to low salary I sometimes feel demotivated. Anyway I keep doing it
enthusiastically. It provides me high honour in society.”


“Yes I will keep working as a teacher. Working as a teacher provides me with high honour
and helps people, especially children to escape from illiteracy. Benefits: It provides students
with knowledge, as teacher I can provide knowledge and skills. I have a range of techniques
to apply to students. And I can development myself. Disadvantages are: Low living standards
and the high workload.

In‐depth interviews with 8 student teachers graduated in 2012

In June 2012 an interview was held with a new group of 8 graduate student teachers. The student
teachers were chosen randomly, 4 female and 4 male student teachers. The questions of the
interview are nearly the same as the questions that were asked to the 8 graduate student teachers
last year. The interview contains questions to find out more about student teachers understanding
aspects of teaching and learning especially the Student Centred Approach and Life Skills education.
This is a summary of their answers.

M/ Scale Can explain Example How often did Important? TTs showed SCA TTs showed SCA in
F SCA given you in Environment Agriculture lessons
experience lesson
concept and SCA at PTTC
its lessons
usefulness

1 F 7.5 ++ Group Very often Yes, it The Environmental The use of fertilizers and
discussion encourages lessons include the insecticides, soil preparation
involvement relationship between techniques for crops growing.
lives on earth and
the natural protected
area.

2 F 6.5 ++ Group All lessons Yes, I stimulates Outdoor activities, Building beds for growing
discussion pupils to learn by learning games, and crops, making composed
themselves songs. fertilizer, and using insecticide.

3 F 8 + Group Not very often Yes, pupils do lots (Outdoor activities) Outdoor activities. Producing
discussion of activities Biodiversity lesson: composed fertilizer and dry
Each student is has fertilizer, and building beds for
to find plant’s name. crops growing.

4 F 7 + Group Very often Yes, pupils can (Outdoor activities) Outdoor activities: Example:
discussion check their won Example: Growing vegetable
answers Biodiversity

5 M 7 ++ Group Yes, Students Yes. It Group discussion Activities inside and outside
discussion learn more encourages pupils was also the classroom (e.g. compost
to share opinions accompanied by production)
which enable a pictures presentation
better (e.g. deforestation)
understanding

6 M 8 ++ Group Yes and I Yes. Pupils like Group work (e.g. Activities outside classroom
discussion appreciate it learning this way Ecosystem) (e.g. chicken raising)

7 M 6 ++ Learning Sometimes Yes, it make Individual work, Outdoor activities: Example:
game pupils self- example: Recycling. Fish raising and crops growing


M&E Report SEAL 2012

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (18)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie M&E Report SEAL 2012

Ähnlich wie M&E Report SEAL 2012 (20)

Mehr von Stefaan Vande Walle

Mehr von Stefaan Vande Walle (20)

M&E Report SEAL 2012