1. Surveys as a tool to
improve aid in fragile states
The Asia Foundation’s Afghanistan
experience

2. A fragile state environment
 Highly dynamic environment; uncertain &unstable
 Strained state-society relations; distrust & suspicion
 Multiple interests and actors at play; high costs of navigation
 Insufficient and unreliable information
 Weak local capabilities
 Weak physical and transportation infrastructure

3. Some ways that surveys can help
 Capture trends in a dynamic environment, if done over time
 Monitor state-society relations over time
 What matters? Public opinion vs. expert opinion
 Help capture key issues that are normally sensitive and deeply political
 Specific issues unique to the state can be studied in depth
 Can be a public good and used by multiple actors, where
coordination risks and fund flows are high
 Better inform program design
 Better inform spending (targeting and resourcing)
 Test assumptions on key issues
 As a common baseline to track progress
 Build local capacity to provide reliable information

4. Varying objectives of TAF surveys
Objective Survey
1) Inform host government policy- Survey of Public Perceptions of the Mindanao Peace Process (2008)
makers and local policy debates Local Economic Governance Index (EGI) in 5 countries
Voter surveys in pre-election periods (Bangladesh, Cambodia,
Indonesia, Pakistan, among others)
2) Articulate/reflect key concerns Democracy & Conflict in Southern Thailand
of conflict-affected populations Survey of the Afghan People (2006-11)
3) Inform program design Public perceptions of police (Timor-Leste, Bangladesh, Sri Lanka,
Indonesia)
Leaders of Influence surveys (Maldives, Bangladesh)
4) Inform donor strategies or Survey of the Afghan People (2006-11)
international policy Survey of the Sri Lankan People (2010)
5) Measure impact of programs Community policing pilot program in Sri Lanka, Timor-Leste (2009-11)

5. Case in point: Survey of the Afghan People
 Nationwide quantitative survey conducted annually using a
structured questionnaire among more than 6200 Afghans
 6 surveys conducted since baseline in 2006
 Findings at http://www.asiafoundation.org/publications/pdf/989

7. Survey decision environment
 Pre survey
 During survey
 Post survey

8. Some Pre-survey Challenges
 Objective (s) of the survey
 Partnerships
 For what and with whom
 ‘Localisation’ (Afghanisation) of the survey
 Ownership and accountability
 Objectivity
 Capacity

9. Some Challenges during Survey
 In general, minimal secondary information available for design
 In Afghanistan, last census in 1979, partial only
 No electoral rolls
 Questionnaire development
 Compatibility to other surveys within & outside the context
 Collaborative inputs
 Curiosity questions vs. actionable findings
 Scale of the fieldwork
 Weak to nonexistent local capacity
 Challenges with female interviewers

10. …Some Challenges during Survey
 Weak infrastructure and harsh weather conditions
 Mountainous terrain & lack of (good) roads
 Long and harsh winters
 Ethno-linguistic sensitivities/closed society
 Conservatism
 Insularity
 Respondent selection and gender balance
 Balanced gender coverage at each sample point
 Travel restrictions for female interviewers

11. …Some Challenges during Survey
 Quality control
 Added oversight and supervision of field team and data cleaning
 Backchecks and accompanied interviews
 Managing sampling and error
 Increasing insecurity and inaccessibility
 Greater replacement of sampling points
 Error margin can increase

12. Some Post-Survey and Other Challenges
 Analysis and reporting considerations
 Overstating the findings
 Interpreting every finding
 Emphasizing magnitude vs. direction
 Looking beyond topline findings to in-depth analysis, when to move
along?
 When to integrate with comparisons to other worldwide surveys
 Dissemination plans and the ethics of research
 Selective data dissemination, suppression of findings
 Duplicative efforts, sharing of information

13. …Some Post-Survey and Other Challenges
 Risk management
 Antigovernment elements, local strongmen, competitors
 Host Government
 Donors
 Way forward: one-off exercise vs. longitudinal – time series
 Changing objective over time of conducting large scale surveys of
Afghans
 Disciplined to team, timing, and, quality control costs
 Choice between parachute survey firm and in-house management
in early years

14. Pictures from the field

15. Team training…

21. The terrain…

29. Interviews…

36. Thank You!

37. Objectives of the Afghan Survey
 Use strong social science research to provide credible, policy-
relevant information about public opinion on recent changes and
trends in government performance, public policy, politics and
political processes, and development progress
 Build social research capacity in the country

38. Dealing with outdated population data
 Rural sampling -- Combination of PPS and SRS
 Urban sampling -- use of maps/locality lists in the absence of
database such as electoral rolls
 Innovative field movements for selection of starting points and
households

39. Building the field team
 Extensive training of interviewers & supervisors
 Questionnaire
 Sampling procedure- field movements, selection of households
 Kish grid
 ‘Train the trainer’ sessions
 Mock interviews
 Female interviewer recruitment in each district of selected sample
point

40. Innovations
 Team of women interviewers recruited locally
 ‘Mahrams’ for women interviewers
 Men and women interviewers in each sampling area
 Modifications in Kish selections
 Women interviewers obtained details of female HH members; men
obtained details of males