The document describes using a Theory of Change (ToC) model throughout a mixed-methods systematic review on handwashing and sanitation behavior change programs. The ToC was developed based on behavioral models and refined with stakeholders. It mapped outcomes, influencing factors, and the relationships between program activities and goals. The ToC helped with question development, study selection criteria, data extraction, synthesis, and identifying evidence gaps. After the review, the ToC was adapted based on findings and further stakeholder input.
The use of a ToC model in a mixed-methods systematic review
1. The use of a Theory of Change model in a mixed-
methods systematic review: an example from the
Development Aid sector
Hans Van Remoortel, Emmy De Buck, Axel Vande Veegaete, Thashlin Govender,
Karin Hannes, Philippe Vandekerckhove, Taryn Young
2. I have no actual or potential conflict
of interest in relation to this
presentation.
Project is (co-)funded by
4. QUANTITATIVE
RESEARCH STUDIES
QUANTITATIVE
RESEARCH STUDIES
QUALITATIVE
RESEARCH STUDIES
QUALITATIVE
RESEARCH STUDIES
Q1: What is the
effectiveness of different
approaches for promoting
handwashing and sanitation
behaviour change in low-
and middle-income
countries?
Q2: What factors influence the
implementation of
approaches to promote
handwashing and sanitation
behaviour change in low-and
middle-income countries?
Background: our Mixed methods systematic
review (MMSR)
5. Objective
✚ To demonstrate the added value of a Theory of Change
(ToC) model throughout the conduct of a MMSR about
the effectiveness (quantitative arm) and implementation
(qualitative arm) of sanitation and handwashing
promotion programs on behavior change
Phase 1: Scoping phase - ToC
development (October 2015-
February 2016)
Phase 2: MMSR (March-
November 2016)
Phase 3: Adaptation ToC based
on evidence MMSR +
stakeholder input (December
2016 – February 2017)
6. Theory of change (ToC)?
✚ An ongoing process of reflection to explore
change and how it happens – and what that
means in a particular context, sector, and/or group
of people
• Contains hypothesized
causal links
• Short versus
intermediate versus
longer-term outcomes
• Possible effect of
positive/negative
mediators or
moderators
7. Objective
✚ To demonstrate the added value of a ToC throughout the
conduct of a MMSR about the effectiveness (quantitative
arm) and implementation (qualitative arm) of sanitation
and handwashing promotion programs on behavior
change
Phase 1: Scoping phase - ToC
development (October 2015-
February 2016)
Phase 2: MMSR (March-
November 2016)
Phase 3: Adaptation ToC based
on evidence MMSR +
stakeholder input (December
2016 – February 2017)
8. ToC development: methodology (1)
1) WASH behavioural models
RANAS model as a basis: intermediate outcomes,
contextual factors (Mosler 2012, IJEHR)
IBM-WASH model: some additional contextual
factors (Dreibelbis 2013, BMC Public Health)
2) PROGRESS framework: additional contextual factors
(O’Neill 2014, JCE)
3) Checklist for implementation (Ch-IMP) & SURE
framework: selection of factors playing a role in
implementation (Cargo 2015, BMC Med Res Method & SURE collaboration
2011)
9. Intermediate outcomes
Longer-term
outcomes
EDUCATIONEDUCATION
Intention to
practice
handwashing
and
sanitation
interventions
(readiness,
willingness)
Use: uptake,
adherence,
and longer-
term use
of/to
handwashing
and
sanitation
activities
(latrine use,
faeces
disposal,…)
Habit to
practice
handwashing
and
sanitation
interventions
(routinized
behaviour)
Knowledge
concerning
sanitation
and
handwashing
program
Skills
concerning
sanitation
and
handwashing
program
Attitude
concerning
sanitation and
handwashing
program
Norms
concerning
sanitation
and
handwashing
program
Self-
regulation
concerning
sanitation
and
handwashing
program
BEHAVIOUR
CHANGE:
Short-term outcomes
Psychosocial
theories
Psychosocial
theories
Community-
based
participatory
approaches
Community-
based
participatory
approaches
Marketing
approaches
Marketing
approaches
IncentivesIncentives
AdvocacyAdvocacy
Target population
Process
evaluation
factors:
-recruitment
-attrition
-reach
-dose
-fidelity
-adaptation
-engagement
-satisfaction
-acceptability
Process
evaluation
factors:
-recruitment
-attrition
-reach
-dose
-fidelity
-adaptation
-engagement
-satisfaction
-acceptability
(Elements of)
promotional
approach
Outputs
Potential
Influencing factors
Program environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-(adapted) training materials
-partnership, coordination between
providers of same intervention or
other health interventions
-funding
-contextual factors implementers (e.g.
gender)
-intent of program to change a
specific outcome
Program environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-(adapted) training materials
-partnership, coordination between
providers of same intervention or
other health interventions
-funding
-contextual factors implementers (e.g.
gender)
-intent of program to change a
specific outcome
Recipient-related factors:
-awareness of personal risk
-self-efficacy
-awareness about costs and benefits
-public commitment
-others showing behaviour
-planning skills
Recipient-related factors:
-awareness of personal risk
-self-efficacy
-awareness about costs and benefits
-public commitment
-others showing behaviour
-planning skills
Recipients socio-cultural context: dignity/respect, culture, religion, ethnicity, law/regulation, socioeconomic status/authority/role model, minorities, social capital, information environment, division of labour
Recipients physical context: place of residence (rural vs urban), low vs middle income countries, natural and built environment ((quality/maintenance of) infrastructure, geophysical), safety, remote areas, available space
Recipients personal context: demographic variables (age, gender, race, cast, language, education, occupation), physical health, mental health
Other promotional
elements such as
the use of pride and
disgust, behaviour
change
techniques,…
Other promotional
elements such as
the use of pride and
disgust, behaviour
change
techniques,…
Morbidity,Mortality
10. Intermediate outcomes
Longer-term
outcomes
EDUCATIONEDUCATION
INTENTION
USE
HABIT
KNOWLEDGE
SKILLS
ATTITUDE
NORMS
SELF-
REGULATION
BEHAVIOUR
CHANGE:
Short-term outcomes
Psychosocial
theories
Psychosocial
theories
Community-
based
participatory
approaches
Community-
based
participatory
approaches
Marketing
approaches
Marketing
approaches
IncentivesIncentives
AdvocacyAdvocacy
Target population
Process
evaluation
factors:
-recruitment
-attrition
-reach
-dose
-fidelity
-adaptation
-engagement
-satisfaction
-acceptability
Process
evaluation
factors:
-recruitment
-attrition
-reach
-dose
-fidelity
-adaptation
-engagement
-satisfaction
-acceptability
(Elements of)
promotional
approach
Outputs
Potential
Influencing factors
Program environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-(adapted) training materials
-partnership, coordination between
providers of same intervention or
other health interventions
-funding
-contextual factors implementers (e.g.
gender)
-intent of program to change a
specific outcome
Program environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-(adapted) training materials
-partnership, coordination between
providers of same intervention or
other health interventions
-funding
-contextual factors implementers (e.g.
gender)
-intent of program to change a
specific outcome
Recipient-related factors:
-awareness of personal risk
-self-efficacy
-awareness about costs and benefits
-public commitment
-others showing behaviour
-planning skills
Recipient-related factors:
-awareness of personal risk
-self-efficacy
-awareness about costs and benefits
-public commitment
-others showing behaviour
-planning skills
Recipients socio-cultural context: dignity/respect, culture, religion, ethnicity, law/regulation, socioeconomic status/authority/role model, minorities, social capital, information environment, division of labour
Recipients physical context: place of residence (rural vs urban), low vs middle income countries, natural and built environment ((quality/maintenance of) infrastructure, geophysical), safety, remote areas, available space
Recipients personal context: demographic variables (age, gender, race, cast, language, education, occupation), physical health, mental health
Other promotional
elements such as
the use of pride and
disgust, behaviour
change
techniques,…
Other promotional
elements such as
the use of pride and
disgust, behaviour
change
techniques,…
Morbidity,Mortality
11. Intermediate outcomes
Longer-term
outcomes
EDUCATIONEDUCATION
INTENTION
USE
HABIT
KNOWLEDGE
SKILLS
ATTITUDE
NORMS
SELF-
REGULATION
BEHAVIOUR
CHANGE:
Short-term outcomes
Psychosocial
theories
Psychosocial
theories
Community-
based
participatory
approaches
Community-
based
participatory
approaches
Marketing
approaches
Marketing
approaches
IncentivesIncentives
AdvocacyAdvocacy
Target population
Process
evaluation
factors:
-recruitment
-attrition
-reach
-dose
-fidelity
-adaptation
-engagement
-satisfaction
-acceptability
Process
evaluation
factors:
-recruitment
-attrition
-reach
-dose
-fidelity
-adaptation
-engagement
-satisfaction
-acceptability
(Elements of)
promotional
approach
Outputs
Potential
Influencing factors
Program
environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-(adapted) training materials
-partnership, coordination between
providers of same intervention or
other health interventions
-funding
-contextual factors implementers (e.g.
gender)
-intent of program to change a
specific outcome
Program
environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-(adapted) training materials
-partnership, coordination between
providers of same intervention or
other health interventions
-funding
-contextual factors implementers (e.g.
gender)
-intent of program to change a
specific outcome
Recipient-related
factors:
-awareness of personal risk
-self-efficacy
-awareness about costs and benefits
-public commitment
-others showing behaviour
-planning skills
Recipient-related
factors:
-awareness of personal risk
-self-efficacy
-awareness about costs and benefits
-public commitment
-others showing behaviour
-planning skills
Recipients socio-cultural context (e.g. SES), Recipients physical context (e.g.: place of residence (rural vs urban),) and Recipients
:personal context (e.g. demographic variables (gender))
Other promotional
elements such as
the use of pride and
disgust, behaviour
change
techniques,…
Other promotional
elements such as
the use of pride and
disgust, behaviour
change
techniques,…
Morbidity,Mortality
12. Intermediate outcomes
Longer-term
outcomes
EDUCATIONEDUCATION
INTENTION
USE
HABIT
KNOWLEDGE
SKILLS
ATTITUDE
NORMS
SELF-
REGULATION
BEHAVIOUR
CHANGE:
Short-term outcomes
Psychosocial
theories
Psychosocial
theories
Community-
based
participatory
approaches
Community-
based
participatory
approaches
Marketing
approaches
Marketing
approaches
IncentivesIncentives
AdvocacyAdvocacy
Target population
Process
evaluation
factors:
-recruitment
-attrition
-reach
-dose
-fidelity
-adaptation
-engagement
-satisfaction
-acceptability
Process
evaluation
factors:
-recruitment
-attrition
-reach
-dose
-fidelity
-adaptation
-engagement
-satisfaction
-acceptability
(Elements of)
promotional
approach
Outputs
Potential
Influencing factors
Program
environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-(adapted) training materials
-partnership, coordination between
providers of same intervention or
other health interventions
-funding
-contextual factors implementers (e.g.
gender)
-intent of program to change a
specific outcome
Program
environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-(adapted) training materials
-partnership, coordination between
providers of same intervention or
other health interventions
-funding
-contextual factors implementers (e.g.
gender)
-intent of program to change a
specific outcome
Recipient-related
factors:
-awareness of personal risk
-self-efficacy
-awareness about costs and benefits
-public commitment
-others showing behaviour
-planning skills
Recipient-related
factors:
-awareness of personal risk
-self-efficacy
-awareness about costs and benefits
-public commitment
-others showing behaviour
-planning skills
Recipients socio-cultural context (e.g. SES), Recipients physical context (e.g.: place of residence (rural vs urban),) and Recipients
:personal context (e.g. demographic variables (gender))
Other promotional
elements such as
the use of pride and
disgust, behaviour
change
techniques,…
Other promotional
elements such as
the use of pride and
disgust, behaviour
change
techniques,…
Morbidity,Mortality
13. ToC development: methodology (2)
Stakeholder meeting 1 (February 2016, Cape Town, South Africa)
4 development practitioners
2 donors
1 topic expert
2 qualitative research experts
1) Refining definitions
promotional approaches
2) Clear distinction between
output vs short-term vs
intermediate vs longer-
term outcomes
3) Factors implementation:
adding layer “recipient-
related factors” / “program
environment factors” and
completion list contextual
factors
14. Objective
✚ To demonstrate the added value of a ToC throughout the
conduct of a MMSR about the effectiveness (quantitative
arm) and implementation (qualitative arm) of sanitation
and handwashing promotion programs on behavior
change
Phase 1: Scoping phase - ToC
development (October 2015-
February 2016)
Phase 2: MMSR (March-
November 2016)
Phase 3: Adaptation ToC based
on evidence MMSR +
stakeholder input (December
2016 – February 2017)
15. ToC as a useful tool in MMSR?
SR step How a ToC helped in our case
1. Research question Refined our 2 research questions;
clear distinction between
quantitative vs qualitative research
2. Study selection (selection criteria) Clear description of the selection
criteria (e.g. primary vs secondary
outcomes); helpful tool to involve
stakeholders in formulating criteria
3. Data extraction Coding sheet for data extraction
based on ToC
4. Data synthesis Explaining heterogeneity; ToC as a-
priori model for “Best fit framework
synthesis”
5. Formulating implications Overview evidence gaps;
implications for practice and policy
based on boxes in ToC
16. ToC adaptation based on evidence MMSR +
stakeholder input
Stakeholder meeting 2 (December 2016, Geneva, Switzerland)
13 development practitioners -
consultants
2 topic experts
2 qualitative research experts
4 donors
Scoping phase - ToC
development (October 2015-
February 2016)
MMSR (March-November
2016)
Adaptation ToC based on
evidence MMSR +
stakeholder input (December
2016 – February 2017)
17. ToC adaptation based on evidence MMSR + stakeholder input
EducationEducation
Psychosocial
theories
Psychosocial
theories
Community-
based
participatory
approaches
Community-
based
participatory
approaches
Marketing
approaches
Marketing
approaches
IncentivesIncentives
AdvocacyAdvocacy
(Elements of)
promotional
approach
Other promotional
elements such as
the use of pride and
disgust, behaviour
change
techniques,…
Other promotional
elements such as
the use of pride and
disgust, behaviour
change
techniques,…
Community-
based
promotional
elements
Community-
based
promotional
elements
Assessment
Formative
research,
elements of
psychosocial
theory
Decide about
choice of
promotional
elements
Assessment
Formative
research,
elements of
psychosocial
theory
Decide about
choice of
promotional
elements
Promotional approach
Social
marketing
promotional
elements
Social
marketing
promotional
elements
Sanitation
and hygiene
messaging
Sanitation
and hygiene
messaging
Before MMSR After MMSR
18. Before MMSR After MMSR
Potential
Influencing factors
Program
environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-…
Program
environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-…
Recipient-related
factors:
-awareness of personal risk
-self-efficacy
-…
Recipient-related
factors:
-awareness of personal risk
-self-efficacy
-…
Recipients socio-cultural context (e.g. SES),
Recipients physical context (e.g.: place of residence
(rural vs urban),) and Recipients :personal context
(e.g. demographic variables (gender))
Factors influencing
implementation
Program
environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-…
Program
environment factors:
-training/qualifications of
implementers
-leadership of implementing
organization
-…
Recipient-related
factors:
-awareness of personal risk
-self-efficacy
-…
Recipient-related
factors:
-awareness of personal risk
-self-efficacy
-…
Implementer-related
factors:
-awareness of personal risk
-self-efficacy
-…
Implementer-related
factors:
-awareness of personal risk
-self-efficacy
-…
Recipients socio-cultural context (e.g. SES),
Recipients physical context (e.g.: place of residence
(rural vs urban),) and Recipients :personal context
(e.g. demographic variables (gender))
Implementer-related
contextual factors:
-socio-cultural
-physical
-personal
Implementer-related
contextual factors:
-socio-cultural
-physical
-personal
ToC adaptation based on evidence MMSR + stakeholder input
19. After MMSR
Implementer-related contextual factors:
•Socio-cultural context: dignity/respect, culture,
religion, ethnicity, law/regulation, socioeconomic
status/authority/role model, minorities, social
capital, information environment, division of
labour, social-political environment
•Physical context: place of residence (rural vs urban),
low- vs middle-income countries, natural and built
environment ((quality/maintenance of)
infrastructure, geophysical), safety, remote areas,
available space
•Personal context: demographic variables (age,
gender, race, cast, language, education,
occupation), part of the community, physical
health, mental health
ToC adaptation based on evidence MMSR + stakeholder input
New factors
Factors without evidence
20.
21. Conclusions
✚ Discussion with the different stakeholders resulted in
a ToC that is more relevant to our target group
✚ Stakeholder involvement in the ToC development
created a sense of ownership and stakeholder buy-in
✚ ToC was used as a central thread during the
development of a systematic review (SR) + was
further refined with findings from the SR
✚ Theory-based approach will help policy makers to
understand the important role of implementation,
and the processes determining behaviour change in
handwashing and sanitation
