Accountability, transparency and corruption in global health: the critical role of health metrics and evaluation<br />Step...
2<br />Outline<br />Context<br />Two examples<br />Tracking immunization coverage<br />Conditional cash transfers to women...
3<br />Outline<br />Context<br />Two examples<br />Tracking immunization coverage<br />Conditional cash transfers to women...
Development Assistance (Billions US$) for Health by Institution, 1990, 2007<br />4<br />
5<br />Global goals, funders and initiatives<br />Goals<br />1978: Alma-Ata: Primary Health Care and Health for All<br />1...
6<br />National health sector reforms and programs<br />2001: Thailand Universal Coverage<br />30 Baht Scheme<br />2003/04...
Critical role of health metrics and evaluation<br />High-quality measurement of health indicators and evaluation of progra...
8<br />Outline<br />Context<br />Two examples<br />Tracking immunization coverage<br />Conditional cash transfers to women...
9<br />Tracking childhood immunization coverage<br />Substantial resources have been invested in delivering immunization s...
10<br />Tracking childhood immunization coverage<br />GAVI’s Immunization Services Support (ISS) is the funding that aims ...
11<br />Tracking childhood immunization coverage<br />Two main questions:<br />What is the trend in the fraction of childr...
12<br />Data sources<br />Micro-data from standardized multi-country surveys <br />DHS, MICS, CDC<br />Crude coverage: thr...
13<br />Quick review of Immunization Services Support (ISS)<br />Performance-based payment<br />Number of additional child...
14<br />
15<br />
16<br />
17<br />
18<br />Estimating missing survey coverage<br />Two purposes:<br />Generate plausible estimates over time to allow monitor...
19<br />
20<br />Global trends in DTP3 coverage<br />Survey-based global coverage of DTP3 (black) with 95% uncertainty estimates co...
21<br />Regional trends<br />
22<br />Does ISS lead to over-reporting?<br />Statistical analysis of over-reporting (officially reported coverage minus s...
23<br />GAVI Immunization Services Support (ISS)<br />Number of additional children vaccinated in 51 countries receiving I...
24<br />Implications<br />At the global level, survey-based immunization coverage has increased continuously and gradually...
25<br />Outline<br />Context<br />Two examples<br />Tracking immunization coverage<br />Conditional cash transfers to wome...
JananiSurakshaYojana – “Safe motherhood scheme”<br />Launched in 2005; 100% centrally funded<br />Goal: reduce maternal an...
National guidelines, Eligibility<br />In 10 high-focus states <br />All pregnant women delivering in government facility o...
National guidelines, Cash payments<br />28<br />1 U.S. dollar ~ 45 Indian Rupees<br />
Questions<br />What is the level of implementation of JSY at district and state-levels?<br />Is JSY reaching its intended ...
Data<br />India District-level Household Surveys (DLHS)<br />DLHS-2: ~1,000 households from 593 districts, 2002 to 2004<br...
31<br />Births receiving JSY, 2007/08<br />In-facility birth coverage, 2001 to 2003<br />
JSY uptake by socioeconomic indicators, national-level<br />32<br />
33<br />In-facility birth coverage, 2007/08<br />In-facility birth coverage, 2001-2003<br />
Evaluating impact of JSY on coverage and outcomes<br />Exact matching<br />Match births receiving JSY to those not receivi...
Outcomes<br />Intervention coverage<br />Antenatal care with at least three visits<br />In-facility birth<br />Skilled bir...
Potential confounders<br />Controlled for:<br />maternal age;<br />number of live births;<br />birth interval;<br />single...
JSY and intervention coverage, national level<br />Change in probability of receiving intervention: JSY vs no JSY<br />
Impact on mortality, national level<br />Change in probability of death: JSY vs no JSY<br />
Variation by State: Intervention coverage<br />
Variation by State: Mortality<br />
Implications<br />Varied uptake of JSY across states; not reaching the very poor<br />Increases in ANC coverage and intra-...
42<br />Summary<br />Substantial resources are being directed towards improving population health<br />Need to track in a ...
Nächste SlideShare
Wird geladen in …5
×

Understanding the Effect of the GAVI Initiative on Reported Vaccination Coverage Levels

976 Aufrufe

Veröffentlicht am

Steve Lim
05/07/10

  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Understanding the Effect of the GAVI Initiative on Reported Vaccination Coverage Levels

  1. 1. Accountability, transparency and corruption in global health: the critical role of health metrics and evaluation<br />Stephen S Lim<br />Assistant Professor of Global Health<br />
  2. 2. 2<br />Outline<br />Context<br />Two examples<br />Tracking immunization coverage<br />Conditional cash transfers to women for delivering in a health facility<br />
  3. 3. 3<br />Outline<br />Context<br />Two examples<br />Tracking immunization coverage<br />Conditional cash transfers to women for delivering in a health facility<br />
  4. 4. Development Assistance (Billions US$) for Health by Institution, 1990, 2007<br />4<br />
  5. 5. 5<br />Global goals, funders and initiatives<br />Goals<br />1978: Alma-Ata: Primary Health Care and Health for All<br />1984: Universal childhood immunization by 1990<br />2000: Millennium Development Goals<br />2003: 3 by 5 – 3 million people on antiretroviral drugs by 2005<br />2008: Malaria Elimination<br />Private philanthropic organizations<br />2000: Bill and Melinda Gates Foundation<br />2006: Warren Buffet pledged additional US$30 to BMGF<br />Global Health Initiatives<br />2000: Global Alliance for Vaccines and Immunizations (GAVI)<br />2002: Global Fund for Aids, Tuberculosis and Malaria (GFATM)<br />2003: US President’s Emergency Plan for AIDS Relief (PEPFAR)<br />2005: US President’s Malaria Initiative (PMI)<br />2007: International Health Partnership (IHP+)<br />
  6. 6. 6<br />National health sector reforms and programs<br />2001: Thailand Universal Coverage<br />30 Baht Scheme<br />2003/04: Mexico Health Sector Reforms<br />System of Social Protection in Health including Seguro Popular<br />2005: Indian National Rural Health Mission<br />Conditional cash transfers for women to give birth in health facilities <br />
  7. 7. Critical role of health metrics and evaluation<br />High-quality measurement of health indicators and evaluation of programs is central to issues of transparency and accountability<br />Are resources being used effectively? <br />Have initiatives and reforms led to improvements in health system delivery and population health?<br />Mounting pressure from funders, civil society, etc to document this <br />Economic crisis has led to rising fears that development assistance budgets will be cut <br />7<br />
  8. 8. 8<br />Outline<br />Context<br />Two examples<br />Tracking immunization coverage<br />Conditional cash transfers to women for delivering in a health facility<br />
  9. 9. 9<br />Tracking childhood immunization coverage<br />Substantial resources have been invested in delivering immunization services<br />Basic vaccines, e.g. three-dose diptheria, pertussis and tetanus (DTP3) as well as new vaccines, e.g. HiB, rotavirus<br />Global initiatives<br />1984: Universal Childhood Immunization (UCI) by 1990 initiative, defined as 80% immunization coverage<br />1999: Global Alliance on Vaccines and Immunizations (GAVI)<br />Monitoring the extent of immunization delivery is critical for evaluating how effective these investments and initiatives have been<br />
  10. 10. 10<br />Tracking childhood immunization coverage<br />GAVI’s Immunization Services Support (ISS) is the funding that aims to increase coverage of basic vaccines such as three-dose diptheria, tetanus and pertussis (DTP3) vaccination. <br />ISS payments are performance-based with funds disbursed in proportion to the number of additional children targeted or reported to receive DTP3.<br />Number of additional children receiving DTP3 is based on official reports from countries to WHO and UNICEF. <br />
  11. 11. 11<br />Tracking childhood immunization coverage<br />Two main questions:<br />What is the trend in the fraction of children receiving three-dose diptheria, tetanus and pertussis vaccination (DTP3 coverage) over the period 1986 to 2006?<br />Do target-oriented initiatives such as universal childhood immunization (UCI) and results-based financing initiatives such as GAVI’s Immunization Services Support (ISS) lead to over-reporting of DTP3 immunization coverage? <br />
  12. 12. 12<br />Data sources<br />Micro-data from standardized multi-country surveys <br />DHS, MICS, CDC<br />Crude coverage: three DTP vaccinations by maternal recall or card documented, irrespective of vaccine schedule<br />Estimated for each birth cohort (up to 5 years prior to the survey)<br />225 surveys<br />Survey reports and WHO/UNICEF database:<br />78 multi-country surveys<br />142 country-specific surveys with sample size reported<br />145 country-specific surveys without sample size reported<br />Administrative data estimates based on health service provider registries<br />Reported routinely to WHO and UNICEF since 1990<br />Officially reported estimates since 1980<br />
  13. 13. 13<br />Quick review of Immunization Services Support (ISS)<br />Performance-based payment<br />Number of additional children reported by countries to have received DTP3<br />Reports largely based on administrative data<br />Baseline is the year prior to approval of the proposal<br />US$20 is paid once per additional child<br />Data quality audit (DQA) of administrative data system before reward payments commence (from Year 3)<br />
  14. 14. 14<br />
  15. 15. 15<br />
  16. 16. 16<br />
  17. 17. 17<br />
  18. 18. 18<br />Estimating missing survey coverage<br />Two purposes:<br />Generate plausible estimates over time to allow monitoring of indicators<br />Reduce compositional bias in in causal inference that can stem from missing data<br />We use validated statistical approaches that are<br />Objective<br />Replicable<br />Characterizes uncertainty<br />
  19. 19. 19<br />
  20. 20. 20<br />Global trends in DTP3 coverage<br />Survey-based global coverage of DTP3 (black) with 95% uncertainty estimates compared to countries’ officially reported (red) and WHO and UNICEF estimates (blue), 1986 to 2006.<br />
  21. 21. 21<br />Regional trends<br />
  22. 22. 22<br />Does ISS lead to over-reporting?<br />Statistical analysis of over-reporting (officially reported coverage minus survey coverage) by years since the GAVI ISS baseline<br />
  23. 23. 23<br />GAVI Immunization Services Support (ISS)<br />Number of additional children vaccinated in 51 countries receiving ISS funding up to the year 2006 :<br />Based on official reports: 13.9 million<br />Survey-based: 7.3 (5.5 to 9.2) million<br />ISS payments<br />Based on official reports: US$289 million<br />Survey-based: US$148 million<br />Over-reporting is not uniform<br />4 countries that reported increases, number of additional children did not increase<br />6 overestimated by > 4x<br />10 overestimated by > 2x but ≤ 4x<br />23 overestimated by > 1x but ≤ 2x<br />8 countries underestimated<br />
  24. 24. 24<br />Implications<br />At the global level, survey-based immunization coverage has increased continuously and gradually over the last 20 years<br />Reflects time and investment needed to expand health services<br />Improvements more pronounced in some regions (e.g. Central, West sub-Saharan Africa) and countries (e.g. Cameroon) during recent time period<br />Targets and payments for performance such as GAVI’s ISS can incentivize improvements but also lead to over-reporting<br />Over-reporting likely reflects pressures throughout the reporting system to meet targets<br />Monitoring and evaluation systems need to be based on independent, rigorous, empirical measurements that are robust to these effects<br />
  25. 25. 25<br />Outline<br />Context<br />Two examples<br />Tracking immunization coverage<br />Conditional cash transfers to women for delivering in a health facility<br />
  26. 26. JananiSurakshaYojana – “Safe motherhood scheme”<br />Launched in 2005; 100% centrally funded<br />Goal: reduce maternal and neonatal mortality<br />Works by incentivizing women to deliver in a health facility<br />Implemented through Accredited Social Health Activists (ASHAs)<br />ASHAs also receive a cash benefit<br />Budget allocation of US$342 million in 2009-10<br />26<br />Accredited Social Health Activist (ASHA)<br />Madhya Pradesh, India<br />Photo: Department for International Development, UKAID<br />
  27. 27. National guidelines, Eligibility<br />In 10 high-focus states <br />All pregnant women delivering in government facility or accredited private institutions<br />Other states & home deliveries<br />Below the Poverty Line <br />>19 years of age<br />First two live births<br />Targeted to women from scheduled caste or tribe <br />27<br />
  28. 28. National guidelines, Cash payments<br />28<br />1 U.S. dollar ~ 45 Indian Rupees<br />
  29. 29. Questions<br />What is the level of implementation of JSY at district and state-levels?<br />Is JSY reaching its intended beneficiaries?<br />Does receipt of financial assistance under JSY lead to increased antenatal care and in-facility delivery and reduced perinatal, neonatal and maternal mortality? <br />29<br />
  30. 30. Data<br />India District-level Household Surveys (DLHS)<br />DLHS-2: ~1,000 households from 593 districts, 2002 to 2004<br />DLHS-3: 1,000 to 1,500 households from 611 districts, late 2007 to early 2009<br />Ever-married women aged 15 to 44, for most recent pregnancy<br />Antenatal care (no. of visits)<br />Delivery care (type of provider, location)<br />Outcome (live birth, still birth, spontaneous or induced abortion)<br />Survival of the child in the case of a live birth<br />Receipt of financial assistance under JSY (DLHS-3)<br />Individual and household characteristics, e.g. asset-based wealth, caste, education, location of residence and distance to facility<br />30<br />
  31. 31. 31<br />Births receiving JSY, 2007/08<br />In-facility birth coverage, 2001 to 2003<br />
  32. 32. JSY uptake by socioeconomic indicators, national-level<br />32<br />
  33. 33. 33<br />In-facility birth coverage, 2007/08<br />In-facility birth coverage, 2001-2003<br />
  34. 34. Evaluating impact of JSY on coverage and outcomes<br />Exact matching<br />Match births receiving JSY to those not receiving JSY in DLHS-3<br />Matching covariates: urban/rural residence, BPL card ownership, wealth quintile, caste, education, parity, and maternal age <br />Logistic regression on matched data allows more precise control for confounders<br />With-vs-without<br />Logistic regression, comparing births receiving JSY to births that did not receive JSY in DLHS-3 and all births in DLHS-2<br />District-level differences-in-differences<br />Compare districts by level of JSY uptake, controlling for baseline differences (DLHS-2)<br />580 district aggregates from DLHS-2 to DLHS-3.<br />34<br />
  35. 35. Outcomes<br />Intervention coverage<br />Antenatal care with at least three visits<br />In-facility birth<br />Skilled birth attendance (in-facility birth or birth outside of a facility with a skilled attendant)<br />Mortality<br />Perinatal death (stillbirth or death up to and including 7 days after a live birth)<br />Neonatal death (death up to and including 1 month after a live birth)<br />Maternal mortality* (death of women aged 15 to 49 during pregnancy or up to 6 weeks after birth or termination)<br />* In district-level analysis only<br />35<br />
  36. 36. Potential confounders<br />Controlled for:<br />maternal age;<br />number of live births;<br />birth interval;<br />single or multiple birth;<br />maternal education;<br />household wealth based on asset ownership; <br />caste/tribe; <br />religion; and <br />location of residence with respect to distance to the nearest health facility<br />Varied using district, state-level fixed and random effects<br />36<br />
  37. 37. JSY and intervention coverage, national level<br />Change in probability of receiving intervention: JSY vs no JSY<br />
  38. 38. Impact on mortality, national level<br />Change in probability of death: JSY vs no JSY<br />
  39. 39. Variation by State: Intervention coverage<br />
  40. 40. Variation by State: Mortality<br />
  41. 41. Implications<br />Varied uptake of JSY across states; not reaching the very poor<br />Increases in ANC coverage and intra-partum care coverage<br />Likely reductions in perinatal and stillbirth/neonatal mortality<br />But potential quality of care issues in high-focus states indicates<br />Alternative monitoring approach needed for maternal mortality<br />Continued monitoring and evaluating the program is critical<br />41<br />
  42. 42. 42<br />Summary<br />Substantial resources are being directed towards improving population health<br />Need to track in a valid, reliable and comparable way health indicators and evaluate the impact of programs<br />Ensure that increased resources for health are being utilized intended purpose and are making a difference to the health of populations<br />Increasing relevance during a time of global financial crisis<br />Independent and empirically-based monitoring of health indicators and evaluation of programs<br />

×