Presented by Heidi Reynolds and Elizabeth Sutherland for the USAID Brown Bag Series for GHI Principles.

1. A Systematic Approach to Monitoring and
Evaluating Integrated Health Interventions in
the Era of the Global Health Initiative
Heidi Reynolds, MPH, Ph.D. and
Elizabeth Sutherland, Ph.D.

2. Rationale

3. Definition of integration
Linkages
Primary
Comprehensive care One stop shop
Coordination
Holistic Interoperable
Not vertical Synergies
Case management

4. Defining Integration
 The effort, within any building block of the health
system, to improve the continuum of care for
clients over the life course.
• Integration is from the
client’s perspective
Client
• Goal is to improve health
outcomes

5. Health System Building Blocks
and Integration
 Health financing
 Direct funding by external donors -> General health care
budget
 Leadership and governance
 Disease policies -> integrated health policies
 Decisions made without -> with consideration of general
health care activities
 Health services
 Single purpose ->multi-purpose service
Atun et al 2009;
Mitchell et al 2004

6. Health System Building Blocks
and Integration (con’t)
 Work-force
 Providers and supervisors with specialized -> generalized
knowledge
 Medical products, vaccines and technologies
 Vertical -> general systems
 Health information systems
 Single purpose reporting ->patient centered system
Atun et al 2009; Mitchell 2004

7. Health Systems and Integration
 Health systems do not need to be integrated to
result in integrated care
 Health systems do need to be strong
 Whether and how health systems will be
integrated is context specific

8. So now what?
 How do we operationalize integration in a
systematic fashion?
 How do we monitor and evaluate integrated
interventions?
 How do we use the data to adapt program
response and inform the global evidence base
for integration?

9. No Need to Panic!

10. Existing M&E Best Practices
Apply
Are we doing 8. Are collective efforts
them on a large impacting the epidemic?
enough scale?
7. Are Interventions making a
difference?
6. Are we implementing the program as
Are we doing
planned?
them right?
5. What are we doing? Are we doing it correctly?
4. What interventions and resources are needed?
Are we doing
the right things?
3. What interventions can work (efficacy & effectiveness)?
2. What are the contributing factors?
1. What is the problem?
Adapted from: Organizing a framework for a functional national HIV
monitoring and evaluation system. A report. UNAIDS. April, 2008.

11. Key M&E Steps for Integration
1. Begin with end in mind
2. Define essential packages of services
3. Develop logic model
4. Improve health information systems
5. Use the data

12. 1. Begin with the end in mind
 Key health outcomes and impacts
 MDGs 4, 5, and 6
 National priorities and targets
 Proximate health outcome indicators where
appropriate
 E.g. delivery with skilled birth attendants

13. 2. Define essential packages of
services
 Built around specific health care entry points
 ANC/maternity
 HIV testing and treatment
 Curative or ambulatory services
 Child wellness

14. 2. Essential packages of services
(con’t)
 Precedent setting examples of such
packages exist
 ANC
 Women presenting in pregnancy
 WHO package includes range of services
 HIV testing and screening for other STIs
 Blood pressure and anemia screens
 Tetanus toxoid injection and malaria prophylaxis

15. 2. Essential packages of services
(con’t)
 Need international guidance on service
packages
 Choice of package determined by health
needs
 Tailored for country-specific priorities and
epidemiology
 Service delivery guidelines for different
service delivery levels

16. 3. Develop logic models
 Defines how and where integration occurs at each
level of intervention
 Inputs, processes, outcomes, impacts
 National, district hospital, health facility, community
 Health system building blocks
 Promotes stakeholder buy in at national-level

17. IHP+ Common M&E Framework
Inputs and Processes Outputs Outcomes Impacts
• Service
• Improved
Governance
• Infrastructure Readiness • Coverage
Financing
Health
• Workforce
Outcomes
• Commodities • Access • Prevalence
• Information of risk
• Efficiency
Systems • Quality of factors
Care
Adapted from: Monitoring the building blocks of the
health system. WHO Report. October 2010.

18. 4. Improve health information
systems
 Support provider access to client health
information
 Electronic medical records
 3 interlinked patient monitoring systems
 ANC client cards
 Strong linked/interoperable routine health
information systems still needed
 Track progress in service delivery

19. 4. Health information systems (con’t)
 Indicators
 Derived from logic model
 Access, readiness, coverage, health outcomes
 New indicators are needed
 Quality
 Met needs and prevention
 Referral

20. 4. Health information systems (con’t)
 Consistent with efforts to strengthen the
broader M&E system
 IHP+, CHeSS, evaluation platform, etc.
 Map data needs (from indicators in logic model)
to existing survey data, surveillance, RHIS, etc.
 Determine what new data collection efforts are
necessary

21. 5. Use the data
 Inform program decision making
 Refine logic model inputs, processes, indicators
 Strengthen the evidence base of what works

22. Research and Evaluation Agenda

23. Current state of the evidence
 Some improved client-level outcomes noted
 increased uptake of services
 increased client satisfaction
 Pilot tests
 Little info on how to implement/effectiveness at
scale
 Value added
 Little/no info on relative value of integration

24. Research agenda
 Conduct outcome and impact evaluations
 At scale/going to scale
 Across several models and countries
 Evaluate essential packages of services
 What should they contain?
 What is the effectiveness of package in improving key health
outcomes?
 Evaluate effectiveness of improved patient monitoring tools on
 Continuum of care
 Provider access to client health info

25. Research agenda
 Evaluate effective capacity building approaches to
intervention
 For building human resources to provide
integration
 Evaluate effectiveness of referral mechanisms
 Conduct case studies of integrated interventions to
inform impact evaluations

26. Conclusions
 Integration is fundamentally client-centered
 Interventions should improve continuum of care
 Approach assumes integration will be country led
 Context specific and tailored to epidemiology
 Experience needed to understand how to adapt
approach to reality of country setting
 Role for international community to guide and
help build evidence base

27. Conclusions
 Health system needs to be strong but not
necessarily integrated
 Integrated interventions take place within health
system building blocks
 Intersects with innovative National Evaluation
Platform design, IHP+ initiative, HSS questions,
and the CHeSS initiative
 Leadership is needed from GHI on expectations
for implementing the integration principle

28. MEASURE Evaluation is a MEASURE project funded by the
U.S. Agency for International Development and implemented by
the Carolina Population Center at the University of North Carolina
at Chapel Hill in partnership with Futures Group International,
ICF Macro, John Snow, Inc., Management Sciences for Health,
and Tulane University. Views expressed in this presentation do not
necessarily reflect the views of USAID or the U.S. Government.
MEASURE Evaluation is the USAID Global Health Bureau's
primary vehicle for supporting improvements in monitoring and
evaluation in population, health and nutrition worldwide.