Ideas global assembly presentation (april 14, 2011) maram barqawi
Theory of change
1. Monitoring & Evaluation
“Watch your school succeed”
Sep 28 2012
Maram Barqawi,
M&E Manager, mbarqawi@hotmail.com
Jordan Education Initiative
2. Monitoring & Evaluation
Watch your school succeed
1 Introduction M&E
2 What is monitoring?
3 What is Evaluation?
4 Theory of Change
4 Agreeing on Outcomes
5 Selecting Key Indicators to
Monitor Outcomes
6 Set Baseline and Target
7 Outcome Matrix
8 Indicators Matrix
3. Monitoring & Evaluation
6
Management tool 4
0
0 8
0
2 1
0 0
0
Measures how well 0 1
2
0
your school is
performing
4. Monitoring & Evaluation
M&E shifted from
auditing and inspections
to supporting the
performance of program’s
work to achieve the
demanded goals
5. The Power of Measuring Results
If you do not measure results, you can not tell success from failure
If you can not see success, you can not reward it
If you can not reward success, you are probably rewarding failure
If you can not see success, you can not learn from it
If you can not recognize failure, you can not correct it
If you can demonstrate results, you can win public support
6. Monitoring & Evaluation
What is Monitoring?
Day-to-day follow up of
activities during
implementation to measure
progress and identify
deviations
Answers the question, “what
are we doing?”
7. Monitoring Tools
Examples of monitoring tools in the school:
• Personal diaries
• Check lists,
• Attendance sheets
• Field visits reports
• Students scores
8. Monitoring & Evaluation
What is Evaluation?
assessment of overall
achievement and impacts
Answers the question, “what
have we achieved and what
impact have we made”?
9. Theory of Change
You want to make CHANGE
in your school?
You have to have a THEORY about how
to make that CHANGE.
10. Theory of Change
It is an explanation of how the use of (inputs) by
(activities) produce a group of (outputs and
outcomes) that sets the stage for producing (impact).
11. Theory of Change is how
to use the resources you
have to win the change you
want
12. Theory of Change (Logical Model)
Goal • Long-term, widespread
(Impacts)
Results
improvement in the school
Outcomes
• Changes in behaviors or skills
as a result of the implemented
project.
• Products and services
Implementation
Outputs
produced
• Tasks personnel undertake to
Activities transform inputs to outputs
Inputs
• Financial, human, and material
resources
13. Theory of Change - Adult Literacy program
Goal • Higher income levels; increase
(Impacts)
Results
access to higher skill jobs
Outcomes
• Increased literacy skill; more
employment opportunities
• Number of adults completing
Implementation
Outputs
literacy courses
Activities • Literacy training courses
Inputs
• Facilities, trainers, materials
14. Why Is It Important to
Choose Outcomes?
“If you don’t know where you’re
going, any road will get you there.”
Paraphrased from
Lewis Carroll’s
Alice in Wonderland
15. Why Is It Important to
Choose Outcomes? Participative approach
Note: When Choosing Outcomes,
Remember – “Do Not Go It Alone!”
key stakeholder
16. How to Developing Outcome
Statements?
From To
School is not Clean environment
clean is created in the
school
Children are
Encouraged families
dropping out of
To Keep Kids In
School
School
ICT skills are low Improved ICT literacy
in the school among students and
teachers
17. How to Developing Outcome
Statements?
Consider this Outcome Statement:
“Students in rural areas improve learning
and gain better quality of life”
What are the measurement issues??
19. Theory of Change - Simple
Example (Training)
Input Activity Output Outcome Impact
•Materials • Conduct • # of • Capable • Better
•Trainers training Participants participant decisions
•Participants • Followup trained s to apply
M&E
• # of
techniques
workshops
20. Theory of Change (Logical Model)
Group work 1 (15min):
• Suggest Theory of Change related to
school’s intervention, discuss the
component and draft the flow chart.
21. To know your progress you need to
measure outcomes?
Outcomes must be translated to a
set of key indicators
22. Indicator
An outcome indicator identifies
a specific numerical measurement
that tracks progress (or not)
toward achieving an outcome
23. Selecting Outcome Indicators
A good performance indicator must be CREAM:
Clear (Precise and unambiguous)
Relevant (Appropriate to subject at hand)
Economic (Available at reasonable cost)
Adequate (Must provide a sufficient basis to assess
performance)
Monitorable (Must be amenable to independent validation)
24. Outcome:
Reduction in dropping out of school.
Indicators ?
• % of students finished the Elementary level
• % of students finished the Secondary level
• % of children in child labor
• More teachers hired
25. Indicator
• You will need to develop your own
indicators to meet your own needs.
• Developing good indicators often takes
more than one try!
• Arriving at the final indicators you will
use will take time!
• Pilot, Pilot, Pilot!
27. Outcome Matrix
Group work 2 (10min):
• go back to your intervention ToC and
suggest indicators for each outcome
28. Indicator Baseline
“If you do not know where you are,
you will have difficulty determining
where you need to go.”
Harry Hatry
Urban Institute, 1999
29. Indicator Baseline
• Is the starting point from where implementation
begins, or the situation prior to a development
intervention, against which progress can be assessed
or comparisons made.
30. Indicators Matrix
Indicators are usually summarized in a matrix which will guide the
process of monitoring those indictors.
Indicators matrix include information about the data sources
,collection methods, frequency a, it also assign the responsibilities
for data collection ,analysis and reporting.
31. Indicators Matrix
Outcome1 (objective1)
Who will
Data Who will Frequency analyze and
Data
Indicator Baseline collection collect of data report data Target
source
method data collection
1.
2.
3.
4.
32. Source of Indicators
• Written records (paper and electronic)
• Individuals involved with the program
• General public
• Trained observers
• Mechanical measurements and tests