Project Monitoring and Evaluation Techniques

PROJECT
MONITORING AND EVALUTION
Christine Joyce B. Mendoza
Assisstant Professor
DCERP, College of Human Ecology
University of the Philippines Los Baños
20 January 2017
Calapan City
Short Course on Environmental Planning
DCERP & HUMEIN Phils. Inc.
1

LECTURE OBJECTIVES
By the end of the lecture, participants are
expected to:
 Discuss the importance of monitoring and
evaluation of projects
 Understand monitoring and evaluation tools
and techniques
 Analyze good indicators and evaluation
questions
2

MONITORING DEFINITIONS
• United Nations: Process of periodic routine
measurement of development indicators for the
purpose of identifying developmental trends
• World Bank: The timely gathering of information on
project inputs, activities and outputs used as mechanism
to warn program/project management about potential
implementation problems requiring corrective actions
• NEDA: Is the timely gathering of information of
project development inputs, activities and outputs
with the use of development indicators as a
management tool
3Short Course on Environmental Planning

DEFINITIONS OF MONITORING
• Routine process of collecting and analyzing data on
activities and results
• Keeping a track of the implementation process and
basically answers if the project:
Is on schedule?
Is on planned budget?
Delivers expected outputs?

IMPORTANCE OF MONITORING
• Ensures projects are implemented efficiently and
effectively
• Improves performance and achieve results
• Identifies actual or potential problems to facilitate
timely adjustments

DEFINITIONS OF EVALUATION
• Systematic and objective assessment of an on-
going or completed project, program or policy
including its design implementation and results
(OECD)
• An assessment, as systematic and impartial as
possible of an activity, project, programme, strategy,
policy, topic, theme, sector, operational area,
institutional performance, etc. (UN Evaluation Group)

IMPORTANCE OF EVALUATION
• As an analysis, it enables one to understand and
appreciate the relative merits or deficiencies of
program; how far program has progressed and how
much further it should be carried to accomplish
objectives
• Progress is examined against goals, purposes and
output targets which can be measured by the use of
indicators

FUNCTIONS OF M&E
Assessing results
– outcomes or
impacts
Identifying project
sustainability and
efficiency
Assessing
implementation
and process
On-going learning
and improvement
of program/project
design

WHAT’S THE DIFFERENCE?
Monitoring Evaluation
Assess changes from
planned vs. actual
Validates what, how, and
why results were
achieved (or not)
Performance tracking
and assessment through
analysis and comparison
of indicators
Compares before and
after intervention
Relies on monitoring
data and information
Continuous and
systematic
Time bound

WHAT’S THE DIFFERENCE?
Monitoring Evaluation
Done by project
managers and partners
Can be done by internal
or external evaluators
Alert mechanism for
managers on
performance problems,
alternatives and actions
Provides information to
donors, stakeholders,
policy makers/planners

M or E
Typical priority
function
Function M E
Routine tracking and reporting ✔
Support accountability ✔ ✔
Identify results (interim, long-term) ✔ ✔
Identify what works (what does not) ✔ ✔
To replicate or scale up ✔ ✔
Continuous learning and improvement ✔ ✔

When should
project monitoring and
evaluation activities
be conducted?
12

13
Conception
/Identificatio
n
Formulation/
Preparation
Analysis and
Appraisal
Implementation
and
Supervision
Operations
Completion
PROJECT
DEVELOPMENT CYCLE
Starts with
Project Ideas
Monitoring: shortly after
project begins
implementation
Midterm Evaluation:
During project
implementation/operation
Before project completion
Terminal evaluation:
after project
completion

M&E TIMING ALTERNATIVES
Upon project
Implementation
Sometime during
Project Operation
After completion of
project operation
Actual construction
costs have been
incurred and
hopefully
documented
Comparisons of
actual and estimated
operating and
maintenance cost
will not be feasible
Actual benefits are
already generated
and operating and
maintenance costs
are being incurred
Actual vs target
benefits and
operating and
maintenance cost
may be assessed
Actual benefits and
operating and maintenance
costs have been generated
and hopefully documented
All actual vs. target benefits
and costs will be assessed
Identification and
measurement of overall
economic and social impact
can be obtained

ON-GOING EVALUATION
An analysis during the project implementation
of project outputs and effects
Provides project management information on
a continuing basis to enable to assess, and if
necessary, adjust policies, objectives,
institutional arrangements and resources
affecting the project during implementation

EX-POST EVALUATION
Executed at the last stage of the project
development cycle to investigate
(a) project’s performance and
(b) direct and indirect impact on the development
goals
In terms of project performance, it seeks to
determine projects at preparation stage are made,
if so to what extent. If there was divergence, it
seeks to identify and analyze the cause

IMPACT EVALUATION
Attempts to identify, measure and validate all the
effects of the project, whether anticipated or not.
It aims to:
• Determine whether stated goals were achieved
• Attribute the identified effects in relation to the
program or project
• Determine conditions under which project is most
effective
• Identify any unanticipated consequences or side
effects of the implemented project

Its goal is beyond improving project development
processes but also improving planning process
• Done at the end or sometime after the end of
the project
• Comprehensive and addresses question of
what has happened and why
• Provides information to policy and program
managers, not to project management
IMPACT EVALUATION

TYPES OF EVALUATION
When
Formative/
Process
Summative
/ Outcome
Who
Internal
External
Stage
On-going
Terminal
Ex-post

TYPES OF EVALUATION
Level
Program
Portfolio
Organization
Theme
Goal/object
ive
Sector
Function
Develop-
mental
Formative

GOOD AND EFFECTIVE M&E
The components of a good M&E system include:
• Basic M&E requirements
• Type of M&E information
• Learning from good practices

Basic M&E requirements should answer:
• What function will the M&E serve?
• Who are the key audiences and stakeholders? What
information do they need? (donors, program
managers, stakeholders, LGUs)
• What M&E products will be produced? How will they
be used? (Reports, dashboards, frequency)
• What are the data collection and reporting
requirements?
• What are the information management system
requirements? Who have access to it? Who will
manage?

Type of M&E information needed:
• Required data for routine reporting including
project status, budget allocations, progress or
planned activities
• Short-, medium- and long-term goals
• Additional information for specific purposes
• Indicators
• Baselines
• Targets

Learning from good practices on M&E
• Use available information
• Ensure stakeholder communication and
engagement throughout
• Pilot/prototyping systems
• Provide trainings and capacity building
• Budgeting on different resources
• Learning from others
• Adapting the system over time

M&E TOOLS &TECHNIQUES
Formal Informal Others
• Project profiles
• Project
progress/status
report
• Project
schedule chart
• Project financial
report
• Scheduled
Meetings
• First hand
information/
direct
observation
• Press reports
• Complaints
and petitions
•Graphic
representati
ons
•Photos
• Stakeholders
participatory
reviews

Includes basic project data, objectives, description and
status of performance. It can be generated quarterly,
semi-annual or annual.
26
PROJECT REPORTS
Activity
report
Progress
report
Project
expenditure
statements
Project
financial
status report
Audited
financial
report
Final Report

WORK AND FINANCIALPLAN
• Contains the chronological major activities of a project
with corresponding cost allotment, unit or person
responsible, and the duration of implementation, and the
expected outputs (indicator).
• Serves as basis for financial programming of the project. It
can be classified as:
• Global WFP which is prepared before project approval
Annual WFP which is prepared by PMO in advance for
a particular year
• Reference of project management office and oversight
agencies for planning budgeting, M&E and audit functions
• Incorporated and integrated in a well-prepared logframe

PROJECT MONITORING PLAN
• Schedule of projects to be visited and inspected by the
monitoring team
• Projects in the monitoring plan includes:
• Projects incurring slippage of negative 15%
• Reported by concerned entities as problematic
• Major PPAs in the development plan
• Projects need to be inspected as instructed by
national/sub-national bodies (N/RPMC, RDC)
• National and ODA-funded projects

MONITORING WORK
PROGRAM
• An operational checklist on the sequential activities
required to achieve the objectives of the M&E
• Includes detail of activities, responsibilities, timetables,
targets and schedules of implementation

GANTTCHART
A Gantt Chart provides feedback information and
variance analysis of project activities to control
and manage the project
Can provide calculation of the percentage
completion for each week, or month, quarter
semester and or year (scheduled and actual
accomplishment)

BARCHART
30
20
10
0
40
90
80
70
60
50
US$ inMillion US$ inMillion US$ in Million US$ inMillion
2010 2011 2012 2013
Actual Availment
ScheduledAvailment

S-CURVE
• "S-curve" or schedule's cumulative progress is a curve
that represents the cumulative progressing separate (1)
financial and (2) physical targets and the actual
performance of project implementation
• The name derives from the “S” like shape of the curve
(sometimes practitioners call it a lazy curve because it is
flatter at the beginning and end and relatively steep after
the mid schedule of the cycle or towards the end of the
cycle)

•
1 2 3 4 5 6 7 8 9 10
Target 1 8.15 16.8 26.15 44.95 67.05 81.75 93.85 95.9 100
Actual 0 0 2.1 6.7 14.85 27.04 58.25 83.23 95.9 100
0
20
40
60
80
100
0
20
40
60
80
100
PERCENT
YEAR
Target
Actual
S-Curves of Target and Actual Accoplishments of
Construction of an Express Highway
120
120
34
SAMPLE S-CURVE

PROJECT PERFORMANCE
• Averaging in Computing Performance
 Dangerous – can pull down to Unsatisfactory level
due to low accomplishment, or pull up to Very
Satisfactory level due to high accomplishment, on a
low priority activity.
 Erroneous – average of % accomplishment of all
activities
 Misleading – all activities treated equal some are
heavier/more important when given weights
• Weights are values assigned to project components
and activities indicative of their relative importance
based on costs and other considerations

FINANCIAL PERFORMANCE
Financial Support
(%)
=
Total Releases
Total Programmed Amount
X 100
Expenditure Rate
(%)
=
Total expenditures
Total Programmed Amount
X 100
Total expenditures = Unpaid Obligations + Cash Disbursement

PHYSICAL PERFORMANCE
For Single Project
1. Compute the project’s accomplishment
2. Compute the project performance and
slippage
Overall Slippage (%) = Actual (%) – Target (%)
Performance (%) =
Actual Accomplishment (%)
Target Accomplishment (%)
X 100

Project with several components
1. Identify project components and their costs
2. Compute the relative weight of each
component
Relative weight component 1
= cost of component /total project cost
3. Compute the percent accomplishment
(target and actual) of each component

4. Compute for the weighted accomplishment
of each component
Weighted accomplishment component 1
= percent accomplishment * relative weight of the
project component
5. Compute the overall weighted
accomplishment

6. Compute the overall weighted performance
and overall project slippage.
Overall Weighted Performance
= Actual Weighted Accomplishment / Target
Weighted Accomplishment x 100
Overall Project Slippage (%) = Actual (B) – Target
(A)

SUB-ACTIVITY OVERALL WEIGHTED
PHYSICAL PERFORMANCEMATRIX
SUB-ACTIVITY
COST
[PhP M] WEIGHT
INDICA
TOR
TAR
GET
ACT
UAL
PERFOR
MANCE [%] OWPP [%]
[1] [2]
[3]
=SAC/TSAC
[4] [5] [6] [7]=6/5*100 [8]=3*7
Sub-Activity 1
20.25 0.1149
No.
meetings
6 5 83 10
Sub-Activity 2
45.50 0.2582
Survey
Instrume
nt 1 1 100 26
sub-Activity 3
30.00 0.1702
Report
1 1 100 17
Sub-Activity 4
80.50 0.4567
Fieldwork
days
20 18 90 41
Total
176.25 1.0000 94
SAC - Sub-Activity Cost
TSAC - Total of Sub-Activities Costs
OWPP - Overall Weighted Physical Performance
41

ACTIVITY OVERALL WEIGHTED
PHYSICAL PERFORMANCEMATRIX
AC - Activity Cost
TAC - Total of Activities Costs
ACTIVITY
COST
[PhP M] WEIGHT
INDICA-
TOR
TAR
GET
ACT
UAL
PERFOR
MANCE [%]
OWPP
[%]
[1] [2] [3]=AC/TAC [4] [5] [6] [7]=6/5*100 [8]=3*7
Activity 1 176.25 0.2266 % 94 21
Activity 2 200.55 0.2579 % 66 17
Activity 3 100.20 0.1288 % 77 10
Activity 4 300.65 0.3866 % 88 34
Total 777.65 1.0000 % 82
42

COMPONENT
COST
[PhP] WEIGHT
INDI
CATOR
TAR
GET
ACT
UAL
PERFOR
MANCE [%]
OWPP
[%]
[1] [2] [3]=CC/TCC [4] [5] [6] [7]=6/5*100 [8]=3*7
Component 1
777.65 0.3414 % 82 28
Component 2
500.00 0.2195 % 67 15
Component 3
400.00 0.1756 % 88 15
Component 4 600.00 0.2634 % 99 26
Total 2,277.6
5
1.0000 84
CC - Component Cost
TCC - Total of Components Costs
43
COMPONENT OVERALL WEIGHTED
PHYSICAL PERFORMANCE MATRIX

PROJECT
COST
[PhP M] WEIGHT
INDICA
TOR
TAR
GET
ACT
UAL
PERFOR
MANCE [%]
OWPP
[%]
[1] [2]
[3]
=AC/TAC [4] [5] [6]
[7]
=6/5*100
[8]
=3*7
Project 1 2277.65 0.2138 % 84 18
Project 2 1789.00 0.1679 % 81 14
Project 3 2588.00 0.2429 % 79 19
Project 4 4000.00 0.3754 % 40 15
Total 10654.65 1.0000 % 66
TPC - Total Project Cost
TOPC - Total of Department ODA Projects Costs
OWPP - Overall Weighted Physical Performance 44
DEPARTMENT OVERALL WEIGHTED
PHYSICAL PERFORMANCE MATRIX

THANK YOU!

EVALUATION INDICATORS
Christine Joyce B. Mendoza
20 January 2017
Calapan City

INDICATORS
47
• Indicators demonstrate progress and result by
providing a reference point for monitoring, decision-
making, stakeholder consultations and evaluation
Measure
progress and
achievements
Clarify
consistency
Ensure
legitimacy and
accountability to
all stakeholders
Assess project
and staff
performance

INDICATORS
48
• Supports effectiveness throughout the process of
planning, implementation, monitoring, reporting and
evaluation
• May be used at any point along the results chain of
inputs, activities, outputs, outcomes and impacts
Actual vs. Target
Baseline vs Current
Annual targets
Intermediary benchmarks

INDICATORS
49
• Indicators indicate, but do not explain
• Credibility of indicator is key
• Approximate intended changes

TYPES OF INDICATORS
50
Situational Outcome Output
Provides broad
picture
Do development
changes occur?
Assesses
progress vs
specified
outcomes
Assesses
progress against
specific
operational
activities

Assessing performance along results, UNDP

TYPES OF INDICATORS
52
Qualitative/ Categorical Indicators
Signal Scale Outcome
Indicator
Output Indicators
Existence (yes/no) Local governance
act passed/not
passed
Policy
recommendation
submitted/not
submitted
Category (e.g. x or y or z) Level of social
housing
development policy
focus “high”,
“medium” or “low”
Poverty analyzed in
“region east”, “west”
or “nationally”

TYPES OF INDICATORS
53
Quantitative/ Numerical Indicators
Scale Outcome
Indicator
Output Indicators
Number (e.g. 1, 20 or
5,000)
Number of new
jobs created in
small enterprise
sector
Number of
entrepreneurs
trained
Percentage (e.g. 12% or
95%)
Percentage share
of rural population
with access to
basic health care
Percentage share of
government budget
devoted to social
sectors
Ratio (e.g. 1/3 or 125
per 100,000)
Ratio of female to
male school
enrolment
Ratio of trained
female to male
members of
parliament

SELECTING INDICATORS
54
• Indicators should meaningfully capture key changes
that are substantively relevant and reflection of
desired results
• Is the indicator SMART?
• Specific
• Measurable
• Attainable
• Relevant
• Trackable/ Time-bound
• Should be accompanied by specific methodology for
recording and reporting

BASELINE, TARGET, AND
TIMEFRAME
55
• Indicators require a baseline, target and timeframe
in order to be useful in verifying the results of a
development intervention over tome
• Baseline is the situation before a programme or
activity. It is the starting point for results monitoring.
• Target is the situation expected at the end of a
programme or activity. There maybe several
milestones to expect between the baseline and
target.
• Timeframe refers to observations taken at specified
points in time or within a given time period.

BASELINE
56
Time
Solidwastereduction
Static
Dynamic
With project

ESTABLISHING BASELINE
57
• Baseline should be gathered and agreed upon by
stakeholders when a programme is being
formulated
• It may be possible to retroactively ascertain
approximately where one was when the programme
started, perhaps from data included in past annual
review exercises

TARGETS
58
Time
Solidwastereduction
Project Target
Baseline
Actual
Latest Report

ESTABLISHING TARGETS
59
• Targets should be based on a thorough review of
the factors that influence the development problem
being addressed, what partners are doing, and what
degree of change can realistically be associated
with the project’s contribution.
• Factors to consider in establishing targets include:
• Past trends
• How well others have done
• Limits to progress
• Existence of objective international, sectoral or
other quality standards

INDICATOR DATA COLLECTION
60
• Indicators are selected because of their relevance
and not because they are easy to track
Data can be obtained through: questionnaire, public
records, common knowledge, a legislation or policies,
development of instruments and/or capacities to
capture information (surveys, administrative records,
expert pan

Relevance Efficiency
Effectiveness Impact
Sustainability
Cross-cutting
Issues

RELEVANCE
64
• Appropriateness of project’s objectives to the real
problems, needs and priorities of its target groups
(relevance at beneficiary level) that the project
was supposed to address, and to the policy
environment (strategic relevance) within which is
is operated

RELEVANCE
65
• Did the planned target groups benefit from the
project intervention?
• Were the project purpose and overall objective
consistent with, and supportive of partner policies
and relevant sector programs?
• To what extent have the interventions corresponds
to the local development priorities?
• Did the intervention contribute to the overall
organization or the Sustainable Development
Goals?

EFFICIENCY
66
• How well means/inputs and activities were
converted into results (as in outputs) and their
quality

EFFICIENCY
67
• Cost/benefit efficiency: to what extent do overall
project costs relate to project results and benefits?
• Partners’ efficiency which can be based on
monitoring reports and stakeholder’s observations
• Have all the planner results delivered to date? What
is the quality of the results?
• To what extent have key observations and
recommendations, if any, from previous M&E visits
been taken account for improving achievement and
quality of results?

EFFECTIVENESS
68
• The contribution made by the project’s results (as in
“outcomes”) to the achievement of the project
purpose up to the end of the implementation period

EFFECTIVITY
69
• To what extent have the planned results been
achieved? To what extent have the intervention
contribute to delivering equitable training skills and
better employability to its target groups?
• Did all planned beneficiaries have access to project
results/services?
• To what extent have the target groups had the
opportunity to utilize their newly acquired knowledge
and competence?

IMPACT
70
• The effect of the project on its wider environment
and its contribution to the wider (sector) objectives
summarized in the project’s overall objective
• Direct impacts are those directly arising from the
project’s results as defined in the project purpose
• Indirect impacts are attributable to the project only
via a series of intermediary steps with interplay of
other factors

IMPACT
71
• What impact did the intervention have in the
national, intermediate and local levels?
• How many beneficiaries did the project directly and
indirectly reach? How does the intervention
influence non-beneficiaries?
• How far are lessons learned/good practice properly
documented in order to facilitate replication?

SUSTAINABILITY
72
• The continuation in the stream of benefits produced
by the project after the period of external support
has ended
• One of the key criteria in ex-post M&E together with
impact and quality of the project
• Should include sustainability criteria which emerged
from ex-post sustainability findings

SUSTAINABILITY
73
• How far was, and is, the intervention embedded in
local/national structures?
• To what degree are interventions continued by
authorities/institutions and fully ‘owned’ by them?
• To what extent does the intervention strengthened
ownership and leadership of intended users and
public/private enterprises beyond the project
duration?

CROSS-CUTTING ISSUES
74
• Looks the relationship between cross-cutting issues
and project’s performance
• Were practical and strategic gender interests
adequately considered in the project strategy?
• Did the project respect environmental needs?
• Was (good) governance mainstreamed in the
project/ program?
• Did the project actively contribute to the promotion
of Human Rights?
• To what extent does the intervention improve the
fragile situation of women and vulnerable sectors to
the society?

LESSONS LEARNED
75
• Include those identified and used by the project
• These are transferrable conclusions
• Should be formulated as a generalized principle that
can be applied in other interventions
• Cannot be too general of too specific
• Should capture the context from which it is derived
and serve as guide for potential replication

LESSONS LEARNED
76
• Thematic lessons learned (specific to the sector)
• Examples: Microfinance in urban areas, gender-
oriented employment initiatives,
• Intervention strategies
• Examples: direct assistance, capacity building at
community, institutional and/or policy level
• Organization and implementation modalities
• Examples: Multi-stakeholder co-management
structures, internal M&E systems, application of
learning tools, inter-institutional arrangement for
project management, coordination platforms with
beneficiaries

REFERENCES
Environmental Project Planning and Administration Lecture Materials
NEDA (1984). Project Development Manual.
NEDA. Regional Project Monitoring and Evaluation System Manual.
NEDA (2014). Project Development Training
UNEP. Monitoring and Reporting Manual.
World Bank Institute. Open Learning Campus.
77

Project Monitoring and Evaluation Techniques

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Project Monitoring and Evaluation Techniques

Ähnlich wie Project Monitoring and Evaluation Techniques (20)

Mehr von Primum Nocere

Mehr von Primum Nocere (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Project Monitoring and Evaluation Techniques