A study on what works more for TSC- shame or subsidy.

1. 3ie Delhi Seminar Series
Shame or Subsidy
What explains the impact of Total Sanitation
Campaign
Sumeet Patil 1
S. K. Pattanayak 2, K Dickinson 3, J-C Yang 3, C. Poulos 3
1 NEERMAN, Mumbai (formerly RTI International)
2 Duke University, USA (formerly RTI International)
3 RTI International, USA

2. Study Team Partners
 Rajiv Gandhi Drinking Water Mission, Govt of India
 Orissa State Water and Sanitation Mission and Department
of Rural Development –
 World Bank – Funders
 WHO, USAID, UNICEF, ICMR - Multidisciplinary technical
advisory group
 RTI International – Principal investigating agency
 Duke University – Analysis and publication phase
 NEERMAN – Analysis and publication phase
 TNS Mode – Survey agency
2

3. Overview of Presentation
 Policy Context for Study (4 slides)
 Study Objectives (1 slide)
 Intervention (3 slides)
 Methodology and Implementation (9 slides)
 Results (12 slides)
Approximately, 45 minutes
3

4. Race against Time
Source: WSP 2009 calendar
4

5. Policy Context for Study
 Child Diarrhea - key underlying link for India’s MDG
targets
 Sanitation is expected to break fecal-oral transmission
and thus improve health
 Universal access to toilets (no open defecation) by
2022 is a goal of Nirmal Bharat Abiyaan (NBA)
 Heated comparison between supply “pushed” subsidy
based TSC and CLTS based demand driven “no
subsidy” based approaches
 Limited evidence to guide implementers and policy
makers
5

6. Govt M&E/MIS data highly unreliable
Source: Chambers and Von Medeazza (2013): working paper
6

7. Policy Context for Study
 Evidence to make determination is very thin
 Impact evaluation in sanitation sectors (are (were) few
 Cross-sectional assessments (lacking baseline, control,
statistical power)
 Need for rigorous impact evaluation
 2005 RCT. Hammer and Spear (2013). Working Paper
 2006 RCT. Pattanayak et al. (2009). This paper
 2011 RCT. Patil et al. (2013). Working Paper
 2011 4-arm QE. We hope that endline happens
7

8. Overview of Presentation
 Policy Context for Study
Study Objectives
 Intervention
 Methodology and Implementation
 Results
8

9. Study Objectives
 Whether CLTS based behavior change coupled with
subsidy based intervention (TSC) impacts latrine use
and child health?
 Track the logic chain from inputs to intermediate outputs
to outcomes to health impacts
 Generate operational knowledge to guide policy
 Ability to study the effect of Shame only and shame +
subsidy because of the TSC program design feature
9

10. Overview of Presentation
 Policy Context for Study
 Study Objectives
Intervention
 Methodology and Implementation
 Results
10

11. Community-Led Total Sanitation in
Bhadrak
Knowledge
Links
11

12. Intervention: Community-Led Total Sanitation
(Kar, IDS)
 Knowledge alone does not change behavior; need
to create “triggering events” and intensive Behavior
Change campaign
 “walk of shame”
 “defecation mapping”
 “fecal calculation”
 TSC related Incentives for BPL for latrine
construction (Rs 1500)
 Supply side: masons, rural sanitation mart, know
how, motivation, monitoring
 Immediate outputs: Out of 20 villages, 9 resolved
to end OD, 2 agreed in principle, 5 decided to meet,
and 4 were unable to reach a consensus
12

13. CLTS Program – Logic Model
13

14. Overview of Presentation
 Policy Context for Study
 Study Objectives
 Intervention
Methodology and
Implementation
 Results
14

15. Study Design
 Randomized Control Experiment
 Well controlled. Random and blind assignment of
treatment
 Sample Size: 20 CLTS villages + 20 control villages and 25
HHs per village (with u5 children)
 Baseline (2005) and Endline (2006)
 Panel Surveys
2 rounds, same season, same households
 Difference in Difference (DID) estimation of impacts
 Difference: Before and After and With and Without
15

16. Sample Selection
Selected a district (Bhadrak) with adequate water
Selected blocks (Tihidi & Chandbali) without prior TSC
Restricted villages to have >70 HHs and < 500 HHs
Restricted to 1 village per GP to reduce spill over
Selected 40 villages & randomly assigned 20 to
treatment
Listed and mapped all households in 40 villages
Randomly selected and surveyed 25 households with
child < 5 yrs in each village
16

17. Study Villages
20.- Controlled Villages of
Nayananda
RTI - WB Study
10 0 10 Km.
Birabarpur
N
Aigiria Budhapur
Mangrajpur Haripur W E
Tentulida
Barikpur Arjunbindha S
TIHIDI
Sasankhas
BLOCK
Nuasahi
Badapimpali
Amarpur Padisahi
Sansamukabedhi
Balipada
Satuti
Bhimpur
Rajnagar
Orali Baincha
Hengupati Hatapur
Madhupur Talabandha Gouriprasad
Jashipur
Taladumka Sanasingpur Baliarpur
Bahu
CHANDBALI BLOCK
Guanal
Begunia Balisahi
Jaladharpur
Ambolo
Bhuinbruti Deuligaan
Tentulida Jaydurgapatna
Dhurbapahalipur
LEGEND:
Control village
Villages
17

18. Data: Measurements
 Outputs, Outcomes, Impacts:
 Household pit latrines (IHL): constructed, operational and in-use
 Diarrhea frequency & severity (> 3 episodes in 24-hr, 2-week
recall)
 Child growth (anthropometrics – MUAC, weight, height)
 Additional parameters:
 Individual - sex, age, class, caste, religion
 Household - family size and composition, education, housing
conditions, asset holdings, occupation and expenditures,
services
 Community – roads, electricity, environmental sanitation,
employment, clinics, schools, credits, markets
 Institutional - main governmental and NGO programs, local
government size and composition
18  Water quality (E. coli & total coliform) – community sources (all),

19. Data: Household Survey
 Respondent - Primary Care Giver
 Water samples collected from approx 50% of surveyed households
 Modular questionnaire
 Knowledge, Attitudes
 Household SEC
 Sanitation Behaviors – outputs and outcomes
 Hygiene Behaviors
 Water Sources and their use
 Water Treatment/safety behaviors
 Food safety behaviors
 Environmental conditions – HH and community
 Budget constraints
 Community Participation
19

20. Data: Community Survey
 Respondent – sarpanch, GP member, Informal leader, Doctor, etc
 Water samples collected from up to 10 in-use drinking water
sources
 Modular questionnaire design
 Background: population, households, area, arable land, major crop
grown
 Public infrastructure: roads, water supply, sanitation, hygiene,
electricity, clinics, schools, STD booths, telegraph offices, post offices,
credits and markets
 Environmental sanitation: general cleanliness, drainage, animal and
household waste, use of water sources, open defecation practices
 WSS scheme: Swajaldhara, piped water, hand pumps, etc
 Development Programs: Health, education, women support etc
 Economy: employment opportunities, major governmental and NGO
programs, prices
 Local government: structure, composition, activities
20

21. Survey Implementation - I
 Schedule & Resources
1 month of data collection to catch the monsoons!!
 Field Teams – RTI (3 + 1 consultant) and TNS (30 field
people + 2 researchers)
 Focus groups
 Pre-testing (2 rounds of 50 household surveys)
 Training (8 days. Mix of in-class and field practice)
(manuals prepared)
 Supervision: Supervisors  executives 
Managers  Researchers. Back checks, spot
checks.
21

22. Fieldwork
22

23. Survey Implementation - II
 Data Processing
 On field editing, 100% scrutiny before data entry
 CSPro based data entry
 Cross-tabulation based cleaning
 WQ Samples
 50% HHs and up to 10 in use sources.
 Sterilized bottles
 Cold chain transport to lab within 24 hours
23

24. Overview of Presentation
 Policy Context for Study
 Study Objectives
 Intervention
 Methodology and Implementation
Results
24

25. Baseline Balance - I
T C p-value
SEC
From scheduled caste 28 26 0.858
From other backward classes 29 24 0.449
Below poverty line 60 61 0.91
WASH
Used improved water sourced 37 42 0.602
Boiled or treated drinking water 9 13 0.192
Adults washed hands at 5 critical instances 11 9 0.564
Dumped garbage outside of house 68 69 0.794
Threw wastewater in the backyard 46 48 0.705
With individual household latrine 6 12.7 0.03
25

26. Baseline Balance - II
T C P-value
Attitudes
Completely dissatisfied with current sanitation 72 61 0.011
Water supply is most important improvement 7 12 0.149
Sanitation is most important improvement 5 8 0.264
Women lack privacy during defecation 32 30 0.82
Women are not safe defecating in the open 29 29 0.463
Government should bear the cost of sanitation 53 50 0.561
Health
U5 diarrhea in past 2 weeks 28 23 0.218
(MUAC)-for-age z-score for U5 -1.3 -1.3 0.677
height-for-age z-score for U5 -2 -1.9 0.687
weight-for-age z-score for U5 -2.2 -2.3 0.341
26

27. Estimation
27

28. % Households owning & using Toilets
(by intervention and year)
DID= 26%-0%= 26%***
40% 32%
30%
20%
26% 13% 0% 13%
6%
10%
0%
2005 2006 2005 2006
CLTS Villages Control Villages
I indicates the 95% confidence interval.
28

29. E Coli Levels in HH Drinking Water
25
20
15
10
5
0
2005 2006 2005 2006
CLTS Control
29

30. Elusive Health Impacts
BL/EL T C T-C DID
0 27% 23% 4.30%
U5 diarrhea Prevalence -4.90%
1 14% 15% -0.60%
0 -1.34 -1.33 -0.011
MUAC-for-age z-score 0.133
1 -1.2 -1.32 0.123
0 -1.95 -1.94 -0.007
Height-for-age z-score 0.281
1 -2.01 -2.29 .273*
0 -2.16 -2.25 0.088
Weight-for-age z-score -0.192
1 -2.22 -2.3 0.069
30

31. Shame or Subsidy?
Full sample BPL APL
Impact (mean test with EL) 19 23.7 12
Impact DID 28.7 34.2 20.7
 Triple Difference to get the relative effect of
shame and subsidy
 BPL = Subsidy + Shame and APL = Shame
alone
 DID for BPL – DID for APL = 34.2 – 20.7 = 13.5%
31  13 % effect (about 1/3rd) by the “subsidies”

32. Is this result replicable elsewhere?
 Another RCT in Madhya Pradesh
 A scaled up and more “realistic” program
 50% to shame + “less subsidy” and 50% to
shame and “more subsidy (by Rs 2700)
Control Treatment
N Mean N Difference
Overall 1514 0.22 1525 0.19 (0.035) ***
Poor 375 0.17 300 0.32 (0.046) ***
Non-poor 1139 0.24 1225 0.15 (0.037) ***
32

33. 33
A
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0%
Ba mb
da ol
pi a
m
pa
Ba la
lia
rp
Ba ur
lip
ad
Ba a
rik
pu
Be r
gu
n
Bh ia
i
Bi mp
ra u
ba r
ra
D Bo pur
hr
ub dha
ap pu
ah r
G
Treatment Villages
al
ou ip
ra ur
pr
as
ad
H
at
a
M pur
ad
Percent of Households Owning a Latrine,
hu
pu
N
ua r
s
Pa ahi
dh
is
Sa R ahi
na ajn
sa ag
m ar
uk
av
Are effects sustainable?
Sa ed
sa i
nk
Ta ha
la
ba s
Ta nd
la h
2007 GoO Data
2004 GoO Data
2006 HH Survey
2005 HH Survey
du a
m
uk
a
2006 Community Survey

34. Findings from Mixed Methods - I
 Some factors indicate “possibility of sustainability”
 Increased satisfaction with sanitation situation
 Increased belief that improving sanitation is the family’s
responsibility
 Lack of knowledge of the “germ theory” is not the
most important BUT privacy and dignity are key
 Households prioritize. Toilets may be “our” priority,
not theirs: 80% want health dispensary, 59% roads.
Compare to 7-9% for water supply and sanitation
34

35. Findings from Mixed Methods - II
 Support structure – NGOs, district officials,
involvement of triggering team, village institutional
capacity are important success factors
 Subsidies are tricky business
 may have created an incentive for NGOs to “cut
corners” and produce lower quality latrines
 Concern that subsidies in general defeat the sense of
self-reliance
 Will subsidy be counteractive in long term?
 How and when you give subsidies will matter
 Community based incentives (e.g. NGP) instead of
individuals?
 Is “post” incentives practical for poor population?
35

36. 7 years later…
 Credible evidence that “shaming” works
 BUT, so do subsidies
 BUT, does the relative contributions depend on
“intensity” of CLTS or amount of subsidy?
 Seems to be continued increase in toilet
coverage
 BUT, what about use? And toilet maintenance?
 BUT, will we reach 100% open defecation free status?
 BUT, what about health impacts?
 7 year later, we still stare at above critical
36 questions without credible answers

37. Thank You
 Sumeet Patil: srpatil@neerman.org
 Other papers
 Pattanayak et al. (2010), “ “How valuable are
environmental health interventions?...” Bull WHO,
88:535-542.
 Pattanayaket al. (2009), “Shame or subsidy revisited:
…” Bull WHO, 87:1-19.
 Pattanayak et al. (2009), “Of taps and toilets….”, J of
Water and Health, 7(3): 434–451.
 World Bank (2011). “Of Taps an Toilets”. WB report on
Evaluation of CDD program in RWSS.
37