SlideShare ist ein Scribd-Unternehmen logo
1 von 35
Sharing information to improve evaluation
How to navigate the maze of
  evaluation methods and approaches
                       A BetterEvaluation Podcast




BetterEvaluation.org   Simon Hearn & Ron Mackay
The Challenge:
Overwhelming number and
diversity of PME tools
Wikipedia page for Evaluation Methods
Steps Taken:
Tools are just too many, focus on
Log Frame and Outcome Mapping
Image: Michael Heilemann. http://www.flickr.com/photos/heilemann
Image: 8lettersuk. http://www.flickr.com/photos/8lettersuk/
Image: Lottery Monkey. http://www.flickr.com/photos/lotterymonkey/
Image: Simon Kneebone. http://simonkneebone.com/
Image: Susan Kistler. http://aea365.org
Image: Reven. http://www.flickr.com/photos/reven/
Image: russelldavies. http://www.flickr.com/photos/russelldavies//
Image: nickherber. http://www.flickr.com/photos/nickherber/
Simple organisational cycle


                  Plan
   Assess


            Act
CIPP Model

         Context


Inputs   Process   Product
Image: meg_riordan. http://www.flickr.com/photos/meg_riordan/
Evaluation is the activity of
systematically collecting, analyzing
and reporting information so that it
can be used by appropriate people to
answer their questions so that they
can make decisions about the future of
a project or program.
•   Who carries out the ‘activity’?
•   Who or what is an evaluator?
•   How does an activity become ‘systematic’?
•   What activites and skills do ‘collecting, analysing and reporting’
    involve?
•   How do you go about ‘collecting, analysing and reporting’?
•   Is that an exhaustive list of ‘the activity’ that is called ‘evaluation’
•   What kind of ‘information’ is being referred to?
•   What is the thing or things you collect information about?
•   Who should be involved in an evaluation?
•   Who are the ‘appropriate people’ who ‘use’ the ‘information’
    supplied by an evaluation?
•   And for what purposes can they ‘use’ this information?
•   What kinds of questions can they ask?
Rainbow Framework



     Over 200
 methods/options
related to 35 tasks
   in 7 clusters
Rainbow Framework



     Over 200
 methods/options
related to 35 tasks
   in 7 clusters
BetterEvaluation.org
See IOCE.net for a list of
national evaluation societies
Evaluation Competencies
1. Reflective Practice
2. Technical Practice
3. Situational Practice
4. Management Practice
5. Interpersonal Practice



 Competencies for Canadian Evaluation Practice, Canadian Evaluation Society
Evaluation Standards
1. Utility Standard
2. Feasibility Standards
3. Propriety Standards
4. Accuracy Standards
5. Evaluation Accountability
   Standards


          The Program Evaluation Standards developed by the JCSEE
How to navigate the maze of
  evaluation methods and approaches
                       A BetterEvaluation Podcast




BetterEvaluation.org   Simon Hearn & Ron Mackay
How to navigate the maze of evaluation methods and approaches

Weitere ähnliche Inhalte

Was ist angesagt?

Creating Your Theory of Change
Creating Your Theory of ChangeCreating Your Theory of Change
Creating Your Theory of Changepkebel
 
Prt 595 week 2 lecture 2
Prt 595 week 2 lecture 2Prt 595 week 2 lecture 2
Prt 595 week 2 lecture 2ejadair
 
Do-It-Yourself Logic Models: Examples, Templates, and Checklists
Do-It-Yourself Logic Models: Examples, Templates, and ChecklistsDo-It-Yourself Logic Models: Examples, Templates, and Checklists
Do-It-Yourself Logic Models: Examples, Templates, and ChecklistsInnovation Network
 
Public Relations and Research
Public Relations and ResearchPublic Relations and Research
Public Relations and ResearchChris Huebner
 
2. grantseeking creating a program logic model
2. grantseeking   creating a program logic model2. grantseeking   creating a program logic model
2. grantseeking creating a program logic modelRebecca White
 
Program logic models: design and use
Program logic models: design and useProgram logic models: design and use
Program logic models: design and useJana Garay
 
Program Planning: Logic Model
Program Planning: Logic Model Program Planning: Logic Model
Program Planning: Logic Model Alaa Qari
 
Program Logic Models
Program Logic ModelsProgram Logic Models
Program Logic ModelsRebecca White
 
Evaluating Communication Programmes, Products and Campaigns: Training workshop
Evaluating Communication Programmes, Products and Campaigns: Training workshopEvaluating Communication Programmes, Products and Campaigns: Training workshop
Evaluating Communication Programmes, Products and Campaigns: Training workshopGlenn O'Neil
 
CSU Extension, Engagement and the Logic model
CSU Extension, Engagement and the Logic modelCSU Extension, Engagement and the Logic model
CSU Extension, Engagement and the Logic modelSteven Newman
 
Logic Model and Education
Logic Model and EducationLogic Model and Education
Logic Model and Educationdledingham
 
Needs Assessment Presentation
Needs Assessment Presentation Needs Assessment Presentation
Needs Assessment Presentation Eric Schauster
 
Integrating communications in evaluation
Integrating communications in evaluationIntegrating communications in evaluation
Integrating communications in evaluationGlenn O'Neil
 
1. Para qué la evaluación sobre educación en museos
1. Para qué la evaluación sobre educación en museos1. Para qué la evaluación sobre educación en museos
1. Para qué la evaluación sobre educación en museosMuseo del Oro Colombia
 
Introduction to Logic Models
Introduction to Logic ModelsIntroduction to Logic Models
Introduction to Logic ModelsDiscoveryCenterMU
 
Types & Uses of PR Research
Types & Uses of PR ResearchTypes & Uses of PR Research
Types & Uses of PR Researchmargyparker
 
Tracking Use of Campaign Evaluation Findings of Two International Organisations
Tracking Use of Campaign Evaluation Findings of Two International OrganisationsTracking Use of Campaign Evaluation Findings of Two International Organisations
Tracking Use of Campaign Evaluation Findings of Two International OrganisationsGlenn O'Neil
 

Was ist angesagt? (20)

PR Research
PR ResearchPR Research
PR Research
 
Creating Your Theory of Change
Creating Your Theory of ChangeCreating Your Theory of Change
Creating Your Theory of Change
 
Prt 595 week 2 lecture 2
Prt 595 week 2 lecture 2Prt 595 week 2 lecture 2
Prt 595 week 2 lecture 2
 
Do-It-Yourself Logic Models: Examples, Templates, and Checklists
Do-It-Yourself Logic Models: Examples, Templates, and ChecklistsDo-It-Yourself Logic Models: Examples, Templates, and Checklists
Do-It-Yourself Logic Models: Examples, Templates, and Checklists
 
The PR Campaign: Research
The PR Campaign: ResearchThe PR Campaign: Research
The PR Campaign: Research
 
Public Relations and Research
Public Relations and ResearchPublic Relations and Research
Public Relations and Research
 
2. grantseeking creating a program logic model
2. grantseeking   creating a program logic model2. grantseeking   creating a program logic model
2. grantseeking creating a program logic model
 
Program logic models: design and use
Program logic models: design and useProgram logic models: design and use
Program logic models: design and use
 
Program Planning: Logic Model
Program Planning: Logic Model Program Planning: Logic Model
Program Planning: Logic Model
 
Program Logic Models
Program Logic ModelsProgram Logic Models
Program Logic Models
 
Evaluating Communication Programmes, Products and Campaigns: Training workshop
Evaluating Communication Programmes, Products and Campaigns: Training workshopEvaluating Communication Programmes, Products and Campaigns: Training workshop
Evaluating Communication Programmes, Products and Campaigns: Training workshop
 
CSU Extension, Engagement and the Logic model
CSU Extension, Engagement and the Logic modelCSU Extension, Engagement and the Logic model
CSU Extension, Engagement and the Logic model
 
P process plus sa (1)
P process plus sa (1)P process plus sa (1)
P process plus sa (1)
 
Logic Model and Education
Logic Model and EducationLogic Model and Education
Logic Model and Education
 
Needs Assessment Presentation
Needs Assessment Presentation Needs Assessment Presentation
Needs Assessment Presentation
 
Integrating communications in evaluation
Integrating communications in evaluationIntegrating communications in evaluation
Integrating communications in evaluation
 
1. Para qué la evaluación sobre educación en museos
1. Para qué la evaluación sobre educación en museos1. Para qué la evaluación sobre educación en museos
1. Para qué la evaluación sobre educación en museos
 
Introduction to Logic Models
Introduction to Logic ModelsIntroduction to Logic Models
Introduction to Logic Models
 
Types & Uses of PR Research
Types & Uses of PR ResearchTypes & Uses of PR Research
Types & Uses of PR Research
 
Tracking Use of Campaign Evaluation Findings of Two International Organisations
Tracking Use of Campaign Evaluation Findings of Two International OrganisationsTracking Use of Campaign Evaluation Findings of Two International Organisations
Tracking Use of Campaign Evaluation Findings of Two International Organisations
 

Ähnlich wie How to navigate the maze of evaluation methods and approaches

Advanced Social Media Techniques CBMI 2013
Advanced Social Media Techniques CBMI 2013Advanced Social Media Techniques CBMI 2013
Advanced Social Media Techniques CBMI 2013Christopher Rice
 
Content Strategy: A Framework for Marketing Success
Content Strategy: A Framework for Marketing SuccessContent Strategy: A Framework for Marketing Success
Content Strategy: A Framework for Marketing SuccessLaura Creekmore
 
UCISA Learning Anaytics Pre-Conference Workshop
UCISA Learning Anaytics Pre-Conference WorkshopUCISA Learning Anaytics Pre-Conference Workshop
UCISA Learning Anaytics Pre-Conference WorkshopMike Moore
 
BetterEvaluation: A framework for planning evaluations
BetterEvaluation: A framework for planning evaluationsBetterEvaluation: A framework for planning evaluations
BetterEvaluation: A framework for planning evaluationsSimon Hearn
 
Implementing Ethics in the Workplace: Creating the Process ppt
Implementing Ethics in the Workplace: Creating the Process pptImplementing Ethics in the Workplace: Creating the Process ppt
Implementing Ethics in the Workplace: Creating the Process pptOsama Yousaf
 
Psychometric instrument development
Psychometric instrument developmentPsychometric instrument development
Psychometric instrument developmentJames Neill
 
Foolproof Assessments
Foolproof AssessmentsFoolproof Assessments
Foolproof Assessmentsdaniel.pahlow
 
Content Strategy: A Framework for Marketing Success
Content Strategy: A Framework for Marketing SuccessContent Strategy: A Framework for Marketing Success
Content Strategy: A Framework for Marketing SuccessLaura Creekmore
 
Inter raterpresent
Inter raterpresentInter raterpresent
Inter raterpresentmattwalter
 
Learning Analytics in Higher Education
Learning Analytics in Higher EducationLearning Analytics in Higher Education
Learning Analytics in Higher EducationJose Antonio Omedes
 
What are we learning from learning analytics: Rhetoric to reality escalate 2014
What are we learning from learning analytics: Rhetoric to reality escalate 2014What are we learning from learning analytics: Rhetoric to reality escalate 2014
What are we learning from learning analytics: Rhetoric to reality escalate 2014Shane Dawson
 
Applied Testing Heuristics in the Context of eBay
Applied Testing Heuristics in the Context of eBayApplied Testing Heuristics in the Context of eBay
Applied Testing Heuristics in the Context of eBayDominik Dary
 
Evaluating conferences and events: new approaches and initiatives
Evaluating conferences and events: new approaches and initiativesEvaluating conferences and events: new approaches and initiatives
Evaluating conferences and events: new approaches and initiativesGlenn O'Neil
 
Use of Qualitative Approaches for Impact Assessments of Integrated Systems Re...
Use of Qualitative Approaches for Impact Assessments of Integrated Systems Re...Use of Qualitative Approaches for Impact Assessments of Integrated Systems Re...
Use of Qualitative Approaches for Impact Assessments of Integrated Systems Re...Water, Land and Ecosystems (WLE)
 
Flexible Evaluation
Flexible EvaluationFlexible Evaluation
Flexible EvaluationpasicUganda
 
Helping Social Work Students develop a Professional Online Presence for Inter...
Helping Social Work Students develop a Professional Online Presence for Inter...Helping Social Work Students develop a Professional Online Presence for Inter...
Helping Social Work Students develop a Professional Online Presence for Inter...Laurel Hitchcock
 
There is a method to it: Making meaning in information research through a mix...
There is a method to it: Making meaning in information research through a mix...There is a method to it: Making meaning in information research through a mix...
There is a method to it: Making meaning in information research through a mix...Lynn Connaway
 

Ähnlich wie How to navigate the maze of evaluation methods and approaches (20)

Advanced Social Media Techniques CBMI 2013
Advanced Social Media Techniques CBMI 2013Advanced Social Media Techniques CBMI 2013
Advanced Social Media Techniques CBMI 2013
 
Content Strategy: A Framework for Marketing Success
Content Strategy: A Framework for Marketing SuccessContent Strategy: A Framework for Marketing Success
Content Strategy: A Framework for Marketing Success
 
UCISA Learning Anaytics Pre-Conference Workshop
UCISA Learning Anaytics Pre-Conference WorkshopUCISA Learning Anaytics Pre-Conference Workshop
UCISA Learning Anaytics Pre-Conference Workshop
 
Dlf 2012
Dlf 2012Dlf 2012
Dlf 2012
 
BetterEvaluation: A framework for planning evaluations
BetterEvaluation: A framework for planning evaluationsBetterEvaluation: A framework for planning evaluations
BetterEvaluation: A framework for planning evaluations
 
Implementing Ethics in the Workplace: Creating the Process ppt
Implementing Ethics in the Workplace: Creating the Process pptImplementing Ethics in the Workplace: Creating the Process ppt
Implementing Ethics in the Workplace: Creating the Process ppt
 
Psychometric instrument development
Psychometric instrument developmentPsychometric instrument development
Psychometric instrument development
 
Foolproof Assessments
Foolproof AssessmentsFoolproof Assessments
Foolproof Assessments
 
Content Strategy: A Framework for Marketing Success
Content Strategy: A Framework for Marketing SuccessContent Strategy: A Framework for Marketing Success
Content Strategy: A Framework for Marketing Success
 
Inter raterpresent
Inter raterpresentInter raterpresent
Inter raterpresent
 
Learning Analytics in Higher Education
Learning Analytics in Higher EducationLearning Analytics in Higher Education
Learning Analytics in Higher Education
 
What are we learning from learning analytics: Rhetoric to reality escalate 2014
What are we learning from learning analytics: Rhetoric to reality escalate 2014What are we learning from learning analytics: Rhetoric to reality escalate 2014
What are we learning from learning analytics: Rhetoric to reality escalate 2014
 
Qualitative Research
Qualitative ResearchQualitative Research
Qualitative Research
 
Applied Testing Heuristics in the Context of eBay
Applied Testing Heuristics in the Context of eBayApplied Testing Heuristics in the Context of eBay
Applied Testing Heuristics in the Context of eBay
 
Evaluating conferences and events: new approaches and initiatives
Evaluating conferences and events: new approaches and initiativesEvaluating conferences and events: new approaches and initiatives
Evaluating conferences and events: new approaches and initiatives
 
Review of eAssessment Quality
Review of eAssessment QualityReview of eAssessment Quality
Review of eAssessment Quality
 
Use of Qualitative Approaches for Impact Assessments of Integrated Systems Re...
Use of Qualitative Approaches for Impact Assessments of Integrated Systems Re...Use of Qualitative Approaches for Impact Assessments of Integrated Systems Re...
Use of Qualitative Approaches for Impact Assessments of Integrated Systems Re...
 
Flexible Evaluation
Flexible EvaluationFlexible Evaluation
Flexible Evaluation
 
Helping Social Work Students develop a Professional Online Presence for Inter...
Helping Social Work Students develop a Professional Online Presence for Inter...Helping Social Work Students develop a Professional Online Presence for Inter...
Helping Social Work Students develop a Professional Online Presence for Inter...
 
There is a method to it: Making meaning in information research through a mix...
There is a method to it: Making meaning in information research through a mix...There is a method to it: Making meaning in information research through a mix...
There is a method to it: Making meaning in information research through a mix...
 

Mehr von Simon Hearn

Monitoring stakeholder engagement
Monitoring stakeholder engagementMonitoring stakeholder engagement
Monitoring stakeholder engagementSimon Hearn
 
BetterEvaluation - Webinar number 7: Report and support use
BetterEvaluation - Webinar number 7: Report and support useBetterEvaluation - Webinar number 7: Report and support use
BetterEvaluation - Webinar number 7: Report and support useSimon Hearn
 
BetterEvaluation - Webinar number 2: Define
BetterEvaluation - Webinar number 2: DefineBetterEvaluation - Webinar number 2: Define
BetterEvaluation - Webinar number 2: DefineSimon Hearn
 
RAPID perspective on Research into Use
RAPID perspective on Research into UseRAPID perspective on Research into Use
RAPID perspective on Research into UseSimon Hearn
 
Online faciliation
Online faciliationOnline faciliation
Online faciliationSimon Hearn
 
Introduction to Outcome Mapping
Introduction to Outcome MappingIntroduction to Outcome Mapping
Introduction to Outcome MappingSimon Hearn
 
OM for policy influencing
OM for policy influencingOM for policy influencing
OM for policy influencingSimon Hearn
 
OM and Complexity
OM and ComplexityOM and Complexity
OM and ComplexitySimon Hearn
 

Mehr von Simon Hearn (11)

What is impact?
What is impact?What is impact?
What is impact?
 
Monitoring stakeholder engagement
Monitoring stakeholder engagementMonitoring stakeholder engagement
Monitoring stakeholder engagement
 
BetterEvaluation - Webinar number 7: Report and support use
BetterEvaluation - Webinar number 7: Report and support useBetterEvaluation - Webinar number 7: Report and support use
BetterEvaluation - Webinar number 7: Report and support use
 
BetterEvaluation - Webinar number 2: Define
BetterEvaluation - Webinar number 2: DefineBetterEvaluation - Webinar number 2: Define
BetterEvaluation - Webinar number 2: Define
 
RAPID perspective on Research into Use
RAPID perspective on Research into UseRAPID perspective on Research into Use
RAPID perspective on Research into Use
 
Online faciliation
Online faciliationOnline faciliation
Online faciliation
 
Roma for seval
Roma for sevalRoma for seval
Roma for seval
 
Introduction to Outcome Mapping
Introduction to Outcome MappingIntroduction to Outcome Mapping
Introduction to Outcome Mapping
 
OM for policy influencing
OM for policy influencingOM for policy influencing
OM for policy influencing
 
OM and Complexity
OM and ComplexityOM and Complexity
OM and Complexity
 
OM Introduction
OM IntroductionOM Introduction
OM Introduction
 

How to navigate the maze of evaluation methods and approaches

Hinweis der Redaktion

  1. There is increasing recognition of the importance of evaluation. But when NGOs or government agencies start to plan an evaluation, or to engage an evaluator, it is difficult to find accessible, comprehensive and credible information about how to choose the right combination of evaluation methods, and how to implement them well – or how to manage these issues with a potential evaluator. Many guides to evaluation focus on only a few types of data collection methods and a few types of research designs. Few guides cover newer types of data collection (such as GIS logging on mobile phones, or tracking social media mentions), designs and strategies that can be used when experimental or quasi-experimental research designs are not feasible or appropriate, and methods to address issues such as clarifying and negotiating the values that should underpin the evaluation. There is information about some of these available on various web sites, but they can be difficult to locate, of varying quality, and not co-ordinated. And much of the information is partisan for particular methods, or does not include examples from practice.
  2. There is increasing recognition of the importance of evaluation. But when NGOs or government agencies start to plan an evaluation, or to engage an evaluator, it is difficult to find accessible, comprehensive and credible information about how to choose the right combination of evaluation methods, and how to implement them well – or how to manage these issues with a potential evaluator. Many guides to evaluation focus on only a few types of data collection methods and a few types of research designs. Few guides cover newer types of data collection (such as GIS logging on mobile phones, or tracking social media mentions), designs and strategies that can be used when experimental or quasi-experimental research designs are not feasible or appropriate, and methods to address issues such as clarifying and negotiating the values that should underpin the evaluation. There is information about some of these available on various web sites, but they can be difficult to locate, of varying quality, and not co-ordinated. And much of the information is partisan for particular methods, or does not include examples from practice.
  3. There is increasing recognition of the importance of evaluation. But when NGOs or government agencies start to plan an evaluation, or to engage an evaluator, it is difficult to find accessible, comprehensive and credible information about how to choose the right combination of evaluation methods, and how to implement them well – or how to manage these issues with a potential evaluator. Many guides to evaluation focus on only a few types of data collection methods and a few types of research designs. Few guides cover newer types of data collection (such as GIS logging on mobile phones, or tracking social media mentions), designs and strategies that can be used when experimental or quasi-experimental research designs are not feasible or appropriate, and methods to address issues such as clarifying and negotiating the values that should underpin the evaluation. There is information about some of these available on various web sites, but they can be difficult to locate, of varying quality, and not co-ordinated. And much of the information is partisan for particular methods, or does not include examples from practice.