Trend Analyses of Nationally-representative Survey Data: What story can be told and what is missing
•
0 gefällt mir•977 views
Presented by Lia Florey, MEASURE DHS/ICF International, as part of a symposium organized by MEASURE Evaluation and MEASURE DHS at the 6th MIM Pan-African Malaria Conference.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (12)
Ähnlich wie Trend Analyses of Nationally-representative Survey Data: What story can be told and what is missing
Ähnlich wie Trend Analyses of Nationally-representative Survey Data: What story can be told and what is missing (20)
Mehr von MEASURE Evaluation
Mehr von MEASURE Evaluation (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Trend Analyses of Nationally-representative Survey Data: What story can be told and what is missing
- 1. Trend analyses of nationallyrepresentative survey data: What story can be told and what is missing. Lia Florey, MEASURE DHS - ICF International Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa Symposium 38
- 2. Acknowledgements The Multiagency Malaria Control Impact Evaluations are a joint effort of many partners. The core team includes members from the following organizations: •PMI/USAID – Erin Eckert, Christine Hershey, Rene Salgado •PMI/CDC – Achuyt Bhattarai, Carrie Nielsen, Steven Yoon •MEASURE DHS – Fred Arnold, Lia Florey, Cameron Taylor •MEASURE Evaluation – Ana Claudia Franca-Koh, Samantha Herrera, Jui Shah, Yazoume Ye Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 3. Outline • • • • • Introduction to Impact Evaluation Project General plausibility approach Challenges to the plausibility approach Examples What else can be done Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 4. Introduction to Impact Evaluation Goal: Determine if scale-up of malaria control interventions has had an impact on malaria outcomes 1. What impact have malaria control interventions had on malaria-related morbidity and mortality? 2. Can we demonstrate and quantify plausible association between intervention and impact? 3. What else could have contributed? Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 5. Evaluation Framework Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 6. Core Analytic Questions Input Process Output Outcome Impact Question 1: Has the availability of services for malaria prevention and treatment increased and are services equitably distributed? a. Funding/spending for malaria programs (from X partner organizations and/or domestic funding) b. Vector management (ITNs, IRS) X X c. Case management X X d. IPTp X X Question 2: Has mortality decreased? a. All-cause under-five mortality (ACCM) X b. Malaria-specific under-five mortality X Question 3: Have the malaria incidence and prevalence decreased? a. Morbidity (anemia prevalence, parasite prevalence, malaria cases) b. Is there anecdotal evidence suggesting additional potential impacts of malaria control (burden placed on health facilities, etc.) Question 4: Have other health programs (non-malaria) been scaled up in recent years? a. Vitamin A, immunizations, etc. X X X X X
- 7. Plausibility approach • Show trends in scale-up of malaria control interventions (ITNs, IRS, IPTp, Effective Case Management) • Show trends in malaria outcomes (Morbidity, Mortality) • Show trends in other factors that could have influenced trends in outcomes (Contextual Factors) • Conclude whether it is plausible that malaria control interventions reduced malaria-related deaths Malaria related interventions Morbidity Mortality Contextual factors Contextual factors Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 8. Why use a Plausibility Approach? • Data on malaria-specific outcomes poor or lacking • Difficult to measure cause-specific mortality in most of Africa – Weak vital registration system – Cause of death difficult to verify • We do not have individual-level data needed for directly measuring causal relationships – ITN use questions ask about previous night – Mortality is measured over a five year period – Exposures to interventions do not always precede outcomes Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 9. Impact Model Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 10. Plausibility Scenario Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 11. Challenges to determining plausibility using survey data Low levels of coverage throughout evaluation period – Maybe insufficient to expect impact on mortality Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 12. Challenges to determining plausibility using survey data Mortality trend Malaria intervention coverage 2000 2010 Lack of baseline data for interventions – Started measuring half way through evaluation period – New/improved interventions introduced during period Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 13. Example from Angola Impact Evaluation Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 14. Challenges to determining plausibility using survey data Mortality decline began before intervention scale-up Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 15. Challenges to determining plausibility using survey data Mortality trend scenario 1 Malaria intervention coverage 2000 2010 Intervention coverage plateaued but mortality trends continued Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 16. Example from Malawi Impact Evaluation Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 17. Challenges to determining plausibility using survey data Seasonal variation in data collection, Malawi – DHS low transmission season, MIS high transmission season – Affects use of interventions as well as outcomes
- 18. Challenges to determining plausibility using survey data Ecological fallacy Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 19. Challenges to determining plausibility using survey data Ecological fallacy Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 20. Examples Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 21. Rwanda Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 22. Ethiopia Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 23. What else can be done? Tell the story with more detail – Stratifications – by malaria risk, wealth, urban/rural, age Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 24. What else can be done? Tell the story with more detail – Accessibility, health systems, specific intervention campaigns
- 25. What else can be done? Use other methodological approaches – District-level ecological analyses - Malawi • Requires large number of sampled districts • Allows inclusion of contextual factors • Few national surveys representative at district level Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 26. What else can be done? Use other methodological approaches – Decomposition analyses - Rwanda • • • • Survival models Individual level Allows inclusion of contextual factors Timing issue with exposure data Decomposition models show that the observed increase in household bed net ownership, from 8% to 94% could have explained as much as 45% of the observed decline in ACCM between 2000 and 2010, equivalent to a reduction of 37 deaths per 1,000 live births. Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 27. What else can be done? • Use other sources of data – Subnational Anemia & Parasitemia surveys Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 28. What else can be done? • Use other sources of data – Demographic Surveillance Systems (DSS) Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
- 29. Questions? Sixth MIM Pan-African Malaria Conference October 9, 2013, Durban, South Africa
Hinweis der Redaktion
- Timing of the change Space Dose-response Age-pattern Correspondence malaria morbidity – mortality change Other factors LiST deaths averted (magnitude expected)
- Reference Rowe paper
- Conceptual Framework
- These are both plausible patterns for impact. Scenario 1 shows a negative linear relationship between intervention coverage and mortality. Scenario 2 shows a threshold effect.
- Timing of collection of intervention data compared to measurement of outcomes Changing drug policies make it difficult to link trends in treatment with trends in morbidity/mortality Early surveys did not contain standard questions necessary for calculating some of these indicators (i.e. ITN use). Difficult to assess a trend with few data points, especially with five-year intervals Data not always available for the required period Plausibility versus causality Time series data on interventions not available Accounting for possible contextual factors
- Alternative scenario are possible with very different implications for the plausibility argument. Treated nets Few ITNs in 2000 LLINs distributed en masse ~2007/8 IPTp in 39 countries by 2007 Malawi is an exception (1993) ACTs 2006/2007 IRS Usually in targeted areas and not useful as a national measure
- ACCM not specific to malaria and therefore cannot be attributed fully to malaria interventions
- We see this especially with IPTp, and with ITNs in some countries.
- Steep declines during the period before intense intervention began.
- Timing of collection of intervention data compared to measurement of outcomes Changing drug policies make it difficult to link trends in treatment with trends in morbidity/mortality Early surveys did not contain standard questions necessary for calculating some of these indicators (i.e. ITN use). Difficult to assess a trend with few data points, especially with five-year intervals Data not always available for the required period Plausibility versus causality Time series data on interventions not available Accounting for possible contextual factors
- This is hypothetical. Can’t test it without disaggregated data.
- Significant mortality decline Significant and rapid intervention coverage Timing IPTp not implemented throughout Data on % of households with ITNs not available from 2000 So successful (look at such high coverage) what next? Will need another approach as coverage gets so high and morbidity so low.
- Rapid declines in mortality before uptake of interventions Rapid uptake of interventions followed by stagnation yet mortality declines continue. How to interpret this? MIS vs. DHS potential effects Potential effects of varied epidemiologic conditions (elevation etc.)
- Timing of the change Space, dose-response, age pattern Correspondence malaria morbidity and mortality change Other factors LiST deaths averted (magnitude expected)
- Timing of the change Space, dose-response, age pattern Correspondence malaria morbidity and mortality change Other factors LiST deaths averted (magnitude expected)
- Timing of the change Space, dose-response, age pattern Correspondence malaria morbidity and mortality change Other factors LiST deaths averted (magnitude expected)
- Timing of the change Space, dose-response, age pattern Correspondence malaria morbidity and mortality change Other factors LiST deaths averted (magnitude expected)
- Timing of the change Space, dose-response, age pattern Correspondence malaria morbidity and mortality change Other factors LiST deaths averted (magnitude expected)