Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center; 200 Panlong Road-Shanghai, October 10, 2008

1. Understanding the Evaluation Context and Program Theory of Change 理解评价背景和项目变革理论 Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center 200 Panlong Road-Shanghai, October 10, 2008 Linda Morra-Imas

15. Checklist of Stakeholder Roles 利益相关者职能列表 Individuals, groups, or agencies 个人、团体或机构 To make policy 制定政策 To make operational decisions 制定经营决策 To provide input to evaluation 为评价提供资金 To react 作出反馈 For interest only 只为获取利益 Developer of program 计划开发者 Funder of program 计划资助者 Boards/agencies 委员会 / 机构 etc. 其他

18. Stakeholder Analysis “How To” 利益相关者分析方法 Stakeholder 利益相关者 Stakeholder Interest(s) in the Project 利益相关者在该项目中的利益 Assessment of Impact 影响评价 a) evaluation on stakeholder b) evaluation on stakeholder Potential Strategies for Obtaining Support or Reducing Obstacles 赢得支持、减少障碍的可能策略

19. Program Stages and the Broad Evaluation Question 计划各阶段与主要评价问题 Stage of program development 计划发展阶段 Evaluation question to be asked 提出的评价问题 Assessment of social problem and needs 对社会问题和需求的评价 To what extent are community needs and standards met? 社区的需求和生活水准能够得到多大程度的满足？ Determination of goals 目标的确定 What must be done to meet those needs and standards? 必须采取什么措施来满足这些需求和标准？ Design of program alternatives 计划备选方案的设计 What services could be used to produce the desired changes? 提供哪些服务可以产生预期的变化？ Selection of alternative 备选方案的选择 Which of the possible program approaches is best? 最佳的计划实施途径是什么？ Program implementation 计划实施 How should the program be put into operation? 如何将计划付诸实施？ Program operation 计划运行 Is the program operating as planned? 计划是否按照既定方案运行？ Program outcomes/effects/impact 计划成果 / 效果 / 影响 Is the program having the desired effects? 计划是否产生了预期的效果？ Program efficiency 计划的效率 Are the program effects attainted at a reasonable cost? 为取得计划效果所花费的成本是否合理？

25. Potential Influences on Program Results 对于计划结果的可能影响 www.dadangsolihin.com Inputs 投入  Activities 活动  Outputs 产出 Outcomes 成果  Impacts 影响 Black Box 黑箱 Political Environment (Governance, etc.) 政治环境（治理，等） Public Attitudes 公众态度 Environment 环境 Aid Players 援助参与者 Macro-economic Picture 宏观经济形势 Policy Context 政策背景

29. Process for Constructing a Theory of Change 建立变革理论的流程 www.dadangsolihin.com Research 研究

35. Theory of Change Template 变革理论模版 IPDET www.dadangsolihin.com Strategies 战略 Assumptions 假设 Influential Factors 有影响力的因素 Problem or Issue 问题 Community Needs/Assets 社区需求 / 资产 Desired Results (outputs, outcomes, and impact) 期望结果（产出，成果，影响） 1 2 3 4 5 6 Source: WK Kellogg Foundation, 2004

36. Simple Theory of Change Model 简单逻辑模型 By training 通过培训 www.dadangsolihin.com Training 培训 High Quality Evaluations 高质量的评价 Useful Information 有用的信息 Better Decisions 更好的决策 We should get better decisions 我们将获得更好的结果

37. Theory of Change with Assumptions 变革理论及其假设 www.dadangsolihin.com Resources given to follow what they learned 足够的资源 Training 培训 High Quality Evaluations 高质的培新 Useful Information 无用的信息 Better Decisions 更好的决定 Needs of trainees met 培训者的需求得到满足 Report writing skills adequate to communicate with government 报告写作技巧足够同政府交流 Enough time to learn 足够的时间来学习

38. Example of Program Theory Model 计划理论模型示例 Visits by teachers to students’ homes 老师对学生进行家访 www.dadangsolihin.com Sharing of views by parent and teacher 家长和老师交流各自的观点 Teachers’ understanding of the home culture 老师了解学生家庭文化 Teachers’ sympathy with children and their view of the world 老师体谅学生及其世界观 Teaching in terms comfortable and understandable to students 以轻松、易接受的方式教育学生 Student moral 学生道德 Achievement in reading 阅读方面的成绩 Parents’ knowledge of school’s expectations for students 家长了解学校对学生的期望 Parental support and encouragement with child’s homework and school assignments 家长支持并鼓励学生完成家庭作业及学校分配的任务 Conscientiousness of work by students 学生的工作责任心 Identification of special problems that retard student’s achievement ( health, emotional, etc.) 发现影响学生表现（健康、情绪等）的特殊问题 Parents’ knowledge of school’s expectations for students 家长了解学校对学生的期望 Student’s receipt of special help 学生接受特殊帮助 Improvement of condition (health, emotional) 状态（健康、情绪）的改善 Parental support for better attendance at school 家长支持学生保持高出勤率 Student attendance 学生出勤

39. Example Basic Model Theory of Change (Research Grant Proposal) 变革理论的基本模型举例 www.dadangsolihin.com Selection Criteria 选择标准 Selecting Applicants 选择申请者 Enhanced Selection Process 提升的选择过程 Improved Research Quality 提升了的研究质量 Increased Use of Research Findings 研究发现的利用 Activities 活动 Outputs 产出 Immediate Results 立即的结果 Intermediate Results 暂时的结果 Final Results 最终的结果

40. Flow Chart of Classic Logic Model 经典逻辑模型流程图 活动 产出 近期结果 中期结果 最终结果 www.dadangsolihin.com

42. Results Chain 结果链 www.dadangsolihin.com Inputs (Resources) 投入（资源） Activities 活动 Outputs 产出 Short-Term Results (Direct) 短期结果（直接） Intermediate Results (Indirect) 中间结果（间接） Long Term Result 长远结果 Area of Control Internal to the Organization 对于组织的内部控制领域 Outputs Reach Direct Beneficiaries 产出直接联系到受益人 Area of Influence External to the Organization 外部对于组织的影响 External Factors 外部因素

43. www.dadangsolihin.com

44. www.dadangsolihin.com Impact nutrition, health, and clothing 对营养、健康和着装有影响 Women have limited access to economic opportunities due to lack of access to credit and other productive resources and to social control 由于缺少获得信贷和其它生产资源的机会，以及社会控制的原因，妇女很难获得经济机会 Project offers credit, technical assistance and group information 计划能够提供信贷、技术援助和大量信息 Women create business 妇女创办企业 Generate profits 盈利 Profits re-invested 利润再投资 Short-term improvement in household welfare 家庭福利的短期改善 Permanent improvement in household welfare 家庭福利的永久改善 Improved housing 居住条件得以改善 Improved education for girls 女孩的教育条件得到改善 Economic improvements 经济状况改善 Business sustained 企业持续发展

45. Example of Program Outcome Model 计划成效模型示例 www.dadangsolihin.com  Resources 资源 ▪ Money 资金 ▪ Staff 人员 ▪ Volunteers 志愿者 ▪ Supplies 物料供应 • Eligible participants 合格的参与者 Services 服务 ▪ Training 培训 ▪ Education 教育 ▪ Counselling 咨询服务 Products 产品 ▪ Total # of classes 班级总数 ▪ Hours of service 服务时间（小时） ▪ Number of participants completing course 完成课程的参加者人数 Benefits 效益 ▪ New knowledge 新知识 ▪ Increased skills 技能提升 ▪ Changed attitudes 态度改变 ▪ New employment opportunities 新的就业机会 Changes 改变 ▪ Trainers earn more over five years than those not receiving training 培训者在 5 年内比那些没有接受培训的人挣得更多 ▪ Trainees have higher standard of living than the control group 接受培训的人的生活水平高于控制组 Inputs 投入 Activities 活动 Outputs 产出 Impacts 影响 Outcomes 成效

47. Logframe Model 逻辑框架 Narrative Summary 描述性总结 Performance Indicators 绩效指标 M&E/Supervision/ Verification 监测与评价 / 监督 / 核实 Key Assumptions 关键假设 Program Goal: 计划目标： Project Development Objective: 计划发展目标： Outputs: 产出： Components: 组成：

48. IPDET www.dadangsolihin.com GOAL (general objective) Improve the economic and social welfare of women and their families INDICATORS Improvements in family income in x% of participating families Improvements in measures of health status, nutritional status, and educational participation VERIFICATION Household surveys of the economic, social, and health condition of all family members PURPOSE (specific objective) Provide women with opportunities to earn and learn while their children are cared for in home day care centers INDICATORS  day care homes functioning, providing accessible, affordable care of adequate quality during working hours and thus allowing shifts in women's employment and education activities VERIFICATION From surveys: changes in women's employment and education and their evaluations of the care provided Evaluations of quality of care provided based on observation ASSUMPTIONS Other family members maintain or improve their employment and earnings Economic conditions remain stable or improve OUTPUTS Trained caregivers, supervisors, and directors Day care homes upgraded and operating Materials developed Administrative system in place MIS in place INDICATORS  caregivers trained  homes upgraded and operating  materials created and distributed  a functioning MIS VERIFICATION Data from MIS on trainees, homes, and materials Evaluations of trainees after initial training and during course of continuous training ASSUMPTIONS Family conditions allow home day care mothers to carry through on their agreements to provide care ACTIVITIES Select caregivers and supervisors and provide initial training Upgrade homes Develop materials Develop administrative system Deliver home day care Provide continuous training and supervision Develop monitoring and evaluation system RESOURCES Budget Technology Human resources VERIFICATION Plan of action, budgets, and accounting records Studies showing that the chosen model and curriculum work Evaluations to see that the activities were not only carried out but done well Survey o

51. Thank you! 谢谢！