A presentation from the 2014 Annual Results and Impact Evaluation Workshop for RBF, held in Buenos Aires, Argentina.

1. 1
Opportunities for Strengthening Quality of Health Care
in Results-based Financing Programs
March 8th 2014
Kathleen Hill, M.D.
Consultant to the World Bank

2. 2
Table of Contents
Executive Summary............................................................................................................................3
Introduction ......................................................................................................................................4
Quality Dimensions and Improvement Principles…………………………………………………………………………………..5
Measuring Quality of Care…………………………………………………………………………………………………………………..7
Incentivizing Quality in RBF Programs……………………………………………………………………………………………….12
Aligning RBF programs with Improvement and Health System Strengthening Efforts………………………..14
Promising Directions for Strengthening Quality of Care in RBF Programs…………………………………………..15
References………….…………………………………………………………………………………………………………………………….17
Appendices……………………………………………………………………………………………………………………………………….18

3. 3
Executive Summary
This background paper examines current directions and opportunities to improve quality of care in the
context of World Bank supported results-based financing (RBF) programs. Common quality of care gaps
in low and middle-income countries (LMICs) are reviewed ranging from a lack of essential commodities
to poor adherence with evidence-based standards to lack of improvement capacity in most health
systems. Six dimensions of quality are reviewed including care that is safe (does no harm), care that is
effective (adherent with guidelines), efficient (not wasteful), timely, patient-centered and equitable.
Core principles of quality improvement are described including effective teamwork, understanding how
processes of care function within a health system, use of data to track results, and an understanding and
focus on patient needs. While there is no single best method to improve quality, several approaches for
improving quality of care are discussed including the model for improvement and audit and feedback.
Opportunities and challenges for measuring quality of care in LMICs are explored in depth due to the
importance of quality measures for RBF programs that incentivize quality performance measures. The
paper explores advantages and disadvantages for a range of specific measurement methods including
observation, simulation, record audit, provider questionnaire and patient interviews. Feasibility of
specific measurement methods is explored within the context of common health information
constraints encountered in LMICs (e.g. non-standardized patient charts, missing data, etc.) The paper
explores differences between measuring quality for the purpose of routine monitoring (e.g. quarterly
performance results for payment) versus periodic measurement of quality for the purpose of external
verification versus measurement for the purpose of an external assessment (e.g. evaluation). The paper
notes the wider range of measurement methods that are typically feasible in a one-time assessment of
quality (versus routine monitoring) and notes the common challenge of overcoming discrepancies
between results generated through routine supervision and results generated through external
verification by an independent auditor.
The section on measurement reviews the construction of reliable quality measures in which a
numerator, a denominator, and a measurement method (data source, sample size, frequency of
measurement) are clearly defined at the outset. The structure of common quality of care process
measures is reviewed (e.g. % adherence with minimum best practices; % cases meeting minimum
standard). The paper briefly reviews integration of quality of care measures into established
structurally-oriented checklists. Examples of checklists from World Bank supported RBF programs
incorporating maternal and newborn care quality performance measures for hospitals and clinics are
provided in an appendix.
There are a number of ways in which RBF programs may seek to incentivize better quality of care,
including: (1) incentives for quality performance measures; 2) incentives for quality improvement
activities; and 3) alignment of RBF programs with broader health system strengthening and quality
improvement efforts. The distinct incentive approaches are discussed including the process of
prioritizing specific conditions and services to incentivize (including potential phasing of priority
conditions/services) and the construction of quality performance measures related to prioritized health

4. 4
conditions and services (e.g. incentivizing high-impact intervention “bundles”). Illustrative quality
improvement activities that may be incentivized are discussed including maintenance of provider
certification, continuous quality improvement by facility QI teams (setting aims, testing changes,
tracking results), peer to peer evaluation and structured death audits. Approaches to aligning or
designing RBF programs linked to broader quality improvement and health system strengthening efforts
such as national QI strategies, accreditation and workforce efforts are discussed including an example of
a World Bank supported RBF program in Liberia being implemented in hospitals in parallel with a
provider residency training program and a World Bank supported RBF program in Zimbabwe being
implemented in hospitals and clinics in parallel with the roll out of a national QI strategy (Zimbabwe).
Finally, promising directions for continuing to strengthen quality of care in RBF programs are explored at
the level of country-specific directions and cross-country directions linked to an explicit learning and
knowledge management agenda.
Introduction
Poor quality of health care is widely recognized as a major barrier to achieving optimal health outcomes
and strong economies in low, middle and high-income settings. Despite a plethora of guidelines based
on proven best practices, health care services world-wide often fail to deliver evidence-based care to
patients when and where they need such care. Studies of care delivery in LMIC demonstrate
widespread deficiencies in the care provided to patients. Many factors contribute to poor quality of care
and weak health systems including:
Governance and policy: Inconsistent national policies, standards, leadership and accountability
mechanisms
Financing: lack of financing for priority services, workforce and essential inputs
Essential commodities: lack of functional supplies at the point of service delivery
Provider competence: weak knowledge and skills and poor maintenance of provider
competence after training
Organization of care processes: poor organization of care processes and poor adherence with
evidence-based standards.
Health information systems: lack of routine quality of care data collection and analysis of data
to improve services
Improvement capacity within health system: lack of improvement capacity throughout health
system.
Results based financing programs seek to address many of the above deficits through combinations of
incentives that may target commodities and basic infrastructure, adherence with best practices (clinical
quality measures), timeliness and coordination of care (organization of care processes, referrals), and
sometimes improvement activities. Due to strong emphasis on data and continuous monitoring and
verification of performance measures, RBF programs contribute to the strengthening of health
information systems.

5. 5
Quality Dimensions and Improvement Principles
In 2001 the Institute of Medicine published a seminal report, Crossing the Quality Chasm, highlighting
the chasm between the “the health care we could have and the care that we have” in the U.S. (Institute
of Medicine, 2001.) The report emphasized the fact that “problems come from poor systems—not bad
people” and that efforts to improve care must be anchored in a consideration of how systems of care
operate in real-life complex delivery systems.
The Institute of Medicine report highlighted six primary aims of quality care including:
1. Safe: Care should be as safe for patients (do no harm)
2. Effective: The science and evidence behind health care should be applied and serve as the standard in the
delivery of care (adherence with evidenced-based standards)
3. Efficient: Care and service should be cost effective, and waste should be removed from the system
4. Timely: Patients should experience no waits or delays in receiving care and service
5. Patient centered: The system of care should revolve around the patient, respect patient preferences, and
put the patient in control
6. Equitable: Unequal treatment should be a fact of the past; disparities in care should be eradicated.
There are several useful conceptual models of quality of care. One model developed by Donabedian
that continues to be widely used proposes three main categories from which information about
quality of care can be drawn (see Figure 1): 1) Structure, 2) Process and 3) Outcomes. Structure
describes the context in which care is delivered, including infrastructure, staff, financing and
equipment. Process denotes the actions that make up health care as reflected in the transactions
between patients and providers and staff throughout the delivery of health care. Processes can be
further classified as technical processes (how care is delivered) or interpersonal processes (the
manner in which care is delivered). Outcomes refer to the effects of health care on the status of
patients and populations and are considered to be a result of inputs and processes of care.
Figure 1: Conceptualizing Quality of Health Care: Inputs, processes and Outcomes
(Source: Donabedian 2005)
1. What is done
2. How it is done
Health services
delivered
Change in
health behavior
Change in
health status
Patient Satisfaction
People
Infrastructure
Materials
(i.e. vaccine)
Information
Technology
Resources
(Inputs)
Activities
(Processes)
Results
(Outputs or Outcomes)

6. 6
One approach to improving health care widely used in high-resource settings is the model for
improvement (Figure 3). The model for Improvement is a change management strategy that stems
from the work of William Edwards Deming. The model includes three basic questions to help structure
improvement:
1. What are we trying to accomplish?
2. How will we know that a change is an improvement?
3. What changes can we make that will result in improvement?
Figure 3: Model for Improvement
A key tenet of improvement is that making care better always requires change, although not all change
necessarily leads to improvement. Without “change” every system will continue to produce the same
results it has always produced. Or, in other words, “every system is perfectly designed to get the results it
gets” (Paul Batalden.) Managing change is central to improvement efforts whether or not such efforts
are prospective (e.g. defining aims and proactively testing changes to processes of care to try to reach
the aim) or retrospective (e.g. auditing and examining adverse events to identify and correct root
problems contributing to poor quality).
The Plan-Do-Study-Act (PDSA) cycle demonstrated in Figure 3 is one approach to managing change; the
PDSA cycle guides tests of change by health care teams to determine if a change leads to improvement.
Teams new to improvement benefit from supportive supervision (coaching) to identify and test changes

7. 7
to processes of care to improve adherence with best practices. Ideally, supportive supervision of QI
teams includes integrated clinical, QI and data-management capacity-building over time. Improvement
teams are typically made up of managers, front-line health care workers and staff who possess the
necessary deep knowledge of their local systems to be able to identify and test feasible and sustainable
changes to “usual processes” to improve care in their local setting.
While context has a strong influence on which changes may be most feasible and effective for
overcoming gaps in a specific setting, categories of quality and system gaps and effective changes
(solutions) are often common across settings. Diverse settings can learn from each other to overcome
common quality and system gaps. Increasingly, many improvement approaches mobilize teams to work
together across health system levels and geographic sites to identify, test and share successful changes
for overcoming important quality and system gaps. Promoting regular shared learning among teams
helps to accelerate and scale up best practices for overcoming common barriers to delivery of high
quality of care.
Audit and feedback is a systematic review of care provided in relation to standards and guidelines of
care. Audit and feedback alone, without action to correct problems may not improve care but can
provide valuable insights into critical quality gaps in order to support change and solutions for
improvement. Audit of adverse events and near-miss audits allow teams to reflect upon, understand
and learn from rare, catastrophic (or near-catastrophic) events through peer review of cases that caused
concern, affected patient safety, or resulted in an unfortunate outcome. Clinical audits are systematic
reviews of patient charts to determine the care given in relation to the standard of care; they are done
by sites for monthly monitoring and conducted externally for data validation. Team-based and individual
self- assessments allow teams to assess themselves or each other through systematic chart reviews to
determine the quality of care. It allows for a sustainable internal system and is potentially cost-effective.
Measuring Quality of Care: Opportunities and Challenges
Regular measurement and analysis of quality measures is a core principle of all improvement work and
is a central component of RBF programs that incentivize quality performance measures. However,
measuring quality is not simple in any setting as highlighted by a quote from a recent issue of the
Journal of the American Medical Association:
“Quality measurement is in rapid flux….despite the challenges of a rapidly expanding number of quality
measures, much of health care remains poorly measured or unmeasured.” Journal of the American
Medical Association Nov 13. 2013
Measures of quality can encompass any of the quality dimensions discussed above, including timeliness
of care, coordination of care (e.g. referral/counter-referral), clinical effectiveness of care (adherence
with best practices), safety of care (adverse events), equity of care (same treatment for everyone),

8. 8
efficiency (not wasteful) and others. Often stakeholders think of clinical effectiveness (adherence with
best practices) when they hear the word “quality”.
It is useful to consider which stakeholders need which quality of care information and for what purpose.
For example facility staff may benefit from tracking quality of care process measures related to the
specific services they are providing. District and regional managers may benefit from tracking
performance of essential system functions at district level such as distribution of commodities and
workforce, functionality of referral systems, etc, in addition to tracking a few sentinel process of care
quality measures in the facilities they supervise. It may or may not make sense for a district health
supervisor to track all quality of care indicators being monitored by facility health care team. In some
cases, it may be preferable to incorporate a few sentinel measures of quality into routine information
systems so as not to burden HMIS with too many indicators. Clients may benefit from having access to
measures of client-centeredness in addition to other quality measures.
National policy makers may find it most useful to track primarily health outcome measures and a
few sentinel quality indicators. Global stakeholders who use data in large part for advocacy and
accountability purposes may need to track an even higher level of disease burden and health
outcome measures.
Measurement Methods
There are a many methods that can be used to measure quality of care. Table 1 illustrates common
methods of assessing quality, including advantages and disadvantages for specific methods. Individual
methods have unique strengths and weaknesses depending on the purpose and context of the
measurement exercise. It is important to tailor the method to the specific need, including the specific
quality dimension being measured. For example a patient interview may be the best method to assess
client-centeredness of care but may not be a reliable method to measure adherence with treatment
standards for a complex disease due to the knowledge asymmetry between provider and patient. Often
a combination of methods can yield a fuller picture of quality than any one single method.
Measurement methods feasible for use as part of a one-time assessment of quality of care (e.g. to
evaluate a program intervention) may be impractical for use for routine measurement of care in an RBF
program or health care improvement intervention.
Table 1: Common Methods of Measuring Quality of Health Care
Measurement Method Advantages Disadvantages
Observation -Considered “gold standard”
-Only method that measures performance of
health service (as opposed to provider
knowledge and competence which may not
-Hawthorne effect
-Resource intensive
-Difficult to sustain in
routine practice

9. 9
correlate with provider performance)
-May be best method for assessing highly
procedural health care tasks (e.g. surgery)
Patient Interview
(e.g. exit interview;
household interview)
-Client centeredness of care
-May be reliable for simple clinical measures
-Recall problems
-Knowledge asymmetry between
provider and patients
-Patient reluctance to give honest
feedback for fear of negative
consequences (e.g. facility exit
interviews)
Death & Near-miss Audit -Targets adverse outcomes
-May identify common quality deficits
-Accountability
-Retrospective (after the fact)
-Limited evidence for association
between routine audit and improved
outcomes
Simulation -Next best method after observation for
complex procedural tasks and processes (e.g.
emergency resuscitation)
-Resource intensive
-Unclear relationship between
simulated competence and actual
performance
Provider Questionnaire -Assesses provider knowledge, self-reported
practice and attitude
-Does not assess provider competence
or performance
( knowledge and problem solving (e.g.
vignettes)
Facility & Patient Records
Individual patient
record
Registers
Other facility
documents
-Relatively sustainable and low-cost
-May encourage better documentation and
point-of-care use of data for decision-making
-Records in LMICs often inadequate or
absent altogether (e.g. no
standardized individual patient record)
-Providers/supervisors may falsely
document data (intentionally or
unintentionally)
Routine information system -Efficient extraction of data -Most HMIS track few (if any) quality
of care measures.
Standardized individual patient records that capture patient-specific and clinical care data serve two
important functions: 1) support of real-time clinical decision-making at the point of care; 2) permit data
extraction for calculation, aggregation and analysis of quality measures across different units of the
system (e.g. provider-specific, facility-specific, district, national). Although the ideal, many health
systems in low resource settings are still far from having individual medical records, and instead use

10. 10
registers to track patient-specific information. Such registers are often no more than columns drawn
into a local notebook and may contain varying amounts of patient-specific clinical data depending on
the register. Never the less, such registers can be manually adapted to capture simple routine best
practices (e.g. addition of column for immediate post-partum oxytocin) while stronger patient records
and more robust information systems are being developed.
Defining Indicators of Quality
Clinical quality of measures can be constructed in varying ways depending on the specific technical
content, data source and measurement method and feasibility in an individual context. In most cases,
clinical quality of care process indicators measure the adherence of care with proven best practices (e.g.
evidence-based standards):
% cases adherent with standards – “all or nothing adherence” (e.g. % PPH cases managed per
minimum standard; % cases pediatric pneumonia treated per standard)
Average % adherence with minimum standards (e.g. average % adherence with newborn sepsis
case-management standards; N=30 cases reviewed)
It is very important to standardize operational definitions of quality of care indicators (performance
measures) that include at a minimum: 1) clear numerator; 2) denominator; 3) source of data; and 4)
frequency of data collection (see Table 2). The specific measurement method that is best suited to a
particular indicator depends on a range of factors, including feasible data sources, and must be
considered when constructing an indicator. For example, it is impossible to measure the quality of
the highly procedural resuscitation of a newborn using a chart audit method. Instead, periodic
observation of real care or simulated care using a structured checklist may be the most appropriate
measurement method.
Table 2: Illustrative Quality of Care Measures: Clinical Effectiveness of Care
Quality of Care Measure
Operational Definition
Data Source/Sample
(measurement method)
Frequency of
Data
CollectionNumerator Denominator
% births in last month
benefitting from Active
Management of the Third
Stage of labor for PPH
prevention
# births in last
month with
administration of
10 units of
oxytocin within
one minute of
delivery of fetus
and controlled
cord traction and
uterine massage.
Total number
births in last
month
Birth register OR
Partogram
(specify one)
Monthly
Average adherence # eclampsia case Total All eclampsia cases Monthly

11. 11
with eclampsia case-
management standards
management
standards met
pneumonia case
management
standards
in hospital perinatal
record in last month
Measuring Quality of Care in Real-life Settings: The Challenge
Measuring quality of care is difficult in any setting. However, routine measurement of quality is
especially challenging in low resource settings due to a range of factors including:
Relative absence of quality of care (content) measures in many routine HMIS in low-resource
settings
Absence of standardized individual patient records in many facilities
Lack of primary data to permit calculation of quality indicators (e.g. registers and individual
partograms/records lack essential data; records may not be standardized or if standardized
records may not include essential information.)
Multiple competing vertical registers in facilities, often containing duplicative data (e.g. TB
register; hypertension register)
Few routine indicators of performance of essential system functions (e.g. % maternities in
district with functional neonatal bag & mask at bedside.)
Inadequate data management skills among providers and managers.
Although measurement methods in addition to chart audit such as observation and client/provider
questionnaires may be useful for one-off periodic assessments, such methods are not typically
sustainable for routine measurement of quality in low resource settings. Routine measurement of
clinical quality may require a combination of measurement approaches, including adaptation of local
records and/or registers, periodic patient and provider interviews, and periodic observation of care.
Even when primary data is available in local records data is often inconsistent and of poor quality.
Building staff capacity to capture and extract data to calculate quality measures is central to building
capacity for continuous improvement in low resource settings and is a central component of all quality
improvement and RBF programs.
In general it is far easier to measure adherence with routine best practices relevant for every patient
than it is to measure adherence with more complex processes of care such as case management of
complications. For example, a routine best practice such as administration of Tetanus Toxoid during
antenatal care can be tracked fairly easily by checking a “box” in a standard record or register column. It
is considerably more difficult, however, to construct a simple and feasible measure of adherence with
eclampsia case-management standards across a necessary continuum of care including: 1) timely
accurate diagnosis; 2) stabilization and successful timely referral (primary facility); 3) prompt and
ongoing treatment/monitoring (hospital); 4) discharge planning and follow up (hospital).
The emergence of electronic health records and electronic health information systems in many
countries is increasing the efficiency of data management and holds great promise for standardizing and
improving measurement of quality of care. However, as is being learned in high-resource settings,

12. 12
electronic health records and registries are not without significant challenges. Unless EHRs are pro-
actively designed to be user-friendly and to capture and automatically aggregate quality of care data
and indicators, they will continue to face the same challenges as traditional paper records.
Incentivizing Quality in RBF Projects
Health systems in high-resource settings have incentivized quality of care performance measures for
many years. Many RBF schemes in high-resource settings incentivize a combination of clinical process
measures (e.g. adherence with diabetes care standards) and outcome measures (e.g. blood glucose
control in diabetes, hospital re-admission rates, post-operative death rates, etc.)
Increasingly, RBF programs in LMICs are beginning to incentivize quality-related performance measures
in addition to volume of services (e.g. number of deliveries) and structural measures (e.g. availability of
essential drugs). There are a number of ways in which RBF programs may seek to improve quality of
care, including: (1) incentives for quality performance measures; 2) incentives for quality improvement
activities; and 3) alignment of RBF programs with broader health system strengthening and quality
improvement efforts.
Incentivizing quality of care performance measures
As is the case for any program that seeks to improve quality of care, RBF programs must define which
quality of care measures to incentivize. In LMICs where quality of care deficits are significant across a
range of clinical conditions and service delivery types it can sometimes be challenging to know where to
start since it is clearly impossible to incentivize and measure everything at once.
In general, it is useful to consider the following factors, at a minimum, when deciding which clinical
conditions and service delivery types to prioritize:
Target high-burden conditions in country and local context responsible for the greatest burden
of disease in local context (leading causes of mortality and morbidity)
Target high-burden conditions for which there is strong evidence of effective health care
interventions (preventive and curative)
Align selection of priority conditions and services with government priorities; involve national
and local decision makers and experts in the decision about which areas to incentivize
When possible, consider phasing improvement priorities during RBF program implementation
since it is not possible to “improve everything at once”
Once an RBF program has defined a set of priority health conditions and services for which it seeks to
improve quality, then program managers need to engage local and international experts to examine
country standards and guidelines against global evidence since evidence is constantly changing. In
general, review of World Health Organization guidelines is a good starting point since such guidelines
are usually based on a systematic review of the evidence and are likely to be respected by country

13. 13
government experts. When possible, it is preferable to distill standards for priority conditions and
services into the minimum highest-impact “intervention bundles” most likely to yield the best outcomes,
to be understood by local providers, and to be measurable with relatively simple measurement
methods. When possible, it is best to avoid long lists of standards that are difficult to measure and
verify and difficult to understand among providers working in high-volume messy systems. For example,
an established RBF program seeking to incorporate a greater focus on quality can add a set of quality
indicators to the structural checklist.
A challenge faced by many structurally oriented RBF programs as they transition to include a greater
focus on quality is the need to revise the qualifications of supervisors and independent verifiers who
assess the quality performance measures. Regardless of the measurement method used (chart audit,
provider knowledge/case study, observation, etc) supervisors who assess performance on quality
measures and independent assessors who verify results must possess a minimum level of clinical
qualifications. As RBF programs incentivize more complex clinical processes such as hospital case
management of severe pneumonia the clinical qualifications of the supervisor/assessor becomes
increasingly important. Because many supervisors are primarily administrative managers who no longer
(or never did) practice clinical care it may sometimes be necessary to involve clinician experts in
assessing and verifying quality measures in RBF programs. Another challenge faced by many RBF
programs is a discrepancy between quality scores calculated by supervisors versus scores calculated by
independent assessors. This challenge is may be particularly difficult when complex process of care
performance measures is involved.
Incentivizing Quality Improvement and Health System Strengthening Activities
In addition to incentivizing quality of care performance measures, RBF programs can incentivize specific
improvement activities at the level of the individual provider, the facility, the regional or district health
management team, or other unit. Table 3 provides examples of improvement activities that may be
incentivized as part of RBF programs.
Table 3: Illustrative quality improvement activities that can be incentivized in RBF programs
Activity to Improve Care Description
Regular facility QI team meetings Regular facility QI team meeting (review of meeting notes and action plans)
Routine collection & analysis of
quality of care measures
Routine collection and analysis of priority quality measures by facility teams
and /or district health management teams
Continuous quality improvement Continuous improvement led by facility teams: clear measurable
improvement aims and indicators; regular tests of change and tracking of
results to determine if care is (or is not) improving

14. 14
Regular peer to peer evaluation Regular structured peer to peer review of medical records with feedback;
regular peer to peer structured observation of procedural tasks (e.g.
newborn resuscitation)
Provider professional development
and or certification
Incentives tied to minimal standards of professional development and/or
provider maintenance of certification
Facility accreditation Incentives for maintenance of external accreditation by facilities
Laboratory and pharmacy quality
assurance processes
Death and near-miss audits Structured retrospective review and analysis of adverse outcomes, linked to
an action plan
Aligning RBF with Improvement & Health System Strengthening Efforts
Many governments and donors worldwide are increasingly investing in strengthening health systems
and improving quality of care. Such efforts are likely to increase in the era of universal “effective”
health coverage and there are many opportunities for RBF programs to align with such initiatives,
including policy, regulation and standards-setting/protocol initiatives.
Policy related to improvement falls into two distinct areas: 1) policy outlining a country’s approach to
improvement (e.g. national QI policy and strategy); and 2) policy on specific clinical or non-clinical health
related areas. National QI strategies typically cut across all technical areas, defining strategic objectives
and implementation plans to address quality and system gaps in disease-specific programs (e.g. T.B.,
MNCH) and cross-cutting system functions (e.g. health information systems, supply chain and
workforce). RBF programs may be designed to reinforce the implementation of a national QI strategy.
For example, in Zimbabwe the second phase of a World Bank supported RBF program is being
introduced in parallel with the roll-out of a national quality improvement strategy that includes pre- and
in-service training and the phased introduction of an electronic medical record and DHIS2 to strengthen
the country health management information system (HMIS). RBF program supervision checklists with
quality indicators in prioritized clinical areas are building capacity of supervisors and front-line facility
health care teams to pay attention to and continually assess quality of care.
Standards are explicit statements of expected quality in the performance of a health care activity. They
may take the form of procedures, clinical practice guidelines, treatment protocols, critical paths,
algorithms, standard operating procedures, or statements of expected health care outcomes, among
other formats. For example the WHO Integrated Management of Childhood Illness (IMCI) treatment
protocol spells out specific standards of care for care of the sick child. Such clinical protocols may be
incentivized in RBF programs; indeed several World Bank supported RBF programs incentivize
adherence with clinical protocols.

15. 15
Regulation approaches help maintain and improve quality, ensure patient safety, provide legal
recognition to qualified health professionals, and verify that design or maintenance specifications are
met. Regulation approaches include accreditation (facility), professional licensure and renewal, and
certification and re-certification of professionals and facilities. Many countries are using regulatory
approaches to try to improve or ensure a minimum level of quality and RBF programs may incentivize
specific regulation activities (e.g. incentives for hospitals that maintain accreditation).
In many cases, RBF programs are being proactively designed and implemented in tandem with
complementary health system strengthening activities For example, in Liberia a World Bank supported
Hospital RBF program is being implemented in parallel with a hospital-based residency training program
to address critical workforce gaps. Programs that incentivize availability of essential commodities such
as medications and equipment can be implemented in parallel with supply chain strengthening efforts.
Global efforts in many technical areas are increasingly identifying quality of care as one essential
element for achieving targeted health outcomes (e.g. MDGs). For example, the World Health
Organization (WHO) Department of Maternal, Child and Adolescent health convened a three-day
meeting on quality of maternal, newborn and child health in December 2013 and is planning several
follow-up initiatives including an Every Mother and Every Newborn (EMEN) quality initiative to
complement the soon-to-released Every Newborn Action Plan (ENAP).
Promising Directions for Strengthening Quality as Part of RBF programs
There are many promising directions for strengthening quality of care in RBF programs in low-resource
settings. First and foremost is the recognition of the importance of quality and the huge quality of care
deficits that undermine the performance of health systems worldwide.
Promising directions for strengthening quality in RBF programs are categorized with respect to
country/local directions and cross-cutting multi-country directions:
Illustrative Directions for Strengthening quality of Care in Country RBF programs
Phased introduction of clinical quality performance measures focused on high-burden diseases
with demonstrated quality of care gaps and for which there are evidence-based interventions
Incentives for performance measures tailored to distinct system levels (e.g. primary facilities,
hospitals, district management teams)
Inclusion of incentives for improvement activities at facility level and for improvement capacity-
building by supervisors/managers (see Table 3)
Design of complementary RBF and health system strengthening interventions (HMIS, workforce,
supply chain, etc)
Alignment of RBF programs with established health system strengthening and improvement
efforts (e.g. provider pre-service training; electronic HMIS, etc)

16. 16
Consideration of inclusion of incentives for quality measures related to safety (adverse events),
coordination of care (referral/counter-referral), and client-centeredness of care
Consideration of inclusion of incentives for measures of equity of health care (accessible, high-
quality care for the poorest and most vulnerable)
Regular participation of RBF program managers in country inter-agency technical working
groups relevant to program technical focus
Improvement capacity-building of Government and private sector stakeholders, including
strengthening data management skills
Illustrative Directions for Strengthening Quality Across Country RBF programs
Participation (or regular interface with technical experts) in global technical working groups and
technical conferences (e.g. Inter-agency Newborn Indicators Working Group; NCD Alliance;
International Forum on Quality and Safety; etc.)
Regular tracking and alignment of quality component of RBF programs with global initiatives in
priority technical areas (e.g. WHO Department of Service Delivery and Strengthening (SDS)
initiatives; UN NCD Global Action Plan and monitoring and evaluation framework; No Child Left
Behind; Every Newborn Action Plan)
Articulation of a priority learning agenda linked to an operational research and knowledge
management strategy
Regular synthesis and analysis of learning across RBF programs (knowledge management)

17. 17
References:
Berwick DM. Lessons from developing nations on improving health care, BMJ 2004;328:1124-8.
Donabedian, A. (1997). The quality of care. How can it be assessed? Archives of pathology &
laboratory medicine, 121(11), 1145-1150.
Donabedian, A (2005). "Evaluating the quality of medical care. 1966.". The Milbank quarterly 83
(4): 691–729
Donabedian, A. (2003). An introduction to quality assurance in health care. (1st ed., Vol. 1).
New York, NY: Oxford University Press.Institute of Medicine.
Haynes et al. A surgical safety checklist to reduce morbidity and mortality in a global
population. New England Journal of Medicine. 2009; 360:491-499.
Institute of Medicine. Crossing the Quality Chasm: A new health system for the 21st
century.
Washington D.C. National Academy Press; 2001.
Institute of Medicine. To Err is Human: Building a Safer Health System. 1999.
Institute for Healthcare Improvement. The Breakthrough Series: IHI’s Collaborative Model for
Achieving Improvement Innovation Series. Cambridge, MA: Institute for Healthcare
Improvement, 2003.
Nolan T, Angos P, Cunha AJLA et al. Quality of Hospital Care for seriously ill children in less
developed countries. Lancet 2001; 357:106-10.
Rowe AK et al. How can we achieve and maintain high-quality performance of health workers in
low resource settings? Lancet 2005;366:1026-35.

18. 18
Appendices: Hospital Eclampsia Chart Audit Tool Liberia RBF program
Eclampsia Chart Review TABLE
See Eclampsia Chart Review Guide
(based on quarterly review of X randomly selected charts)
Site ………. ……… Month ……………… Year……….
Chart review elements (see chart review guide for specific
criteria) ; each element if recorded = 1 point
Charts
1. Evaluation
1 2 3 4 5
1. Blood pressure (BP) recorded
2. Gestational age (GA) recorded (per one of criteria
indicated in GUIDE)
3. Urine protein quantified (dipstick +, ++, +++)
4. Danger signs assessed (see chart review guide)
Evaluation Score (x/4)
2. Diagnosis pre-eclampsia or eclampsia recorded (if
applicable criteria met)
1. DBP > 90 and at least 2+ proteinuria pre-eclampsia (+
seizure if eclampsia)
Documentation of Diagnosis (x/1)
3Treatment
(a) Mild pre-eclampsia
1. BP check and surveillance for danger signs at least every
4 hours in labor and post-partum
(b) Severe pre-eclampsia
1. 4 gm loading dose of MgSO4 IV ; monitor for toxicity
(reflexes, urine output, respirations)
2. 5 mg MgSO4 IM every 4 hours until minimum 24 hrs after
delivery or convulsion (whichever comes later

19. 19
May consider anti-hypertensive for DBP > 90 (e.g.
hydralazine 5 mg or any safe in pregnancy)
3.
4. Stabilize and deliver immediately if GA > 34 weeks
5. If GA < 34 weeks and patient stable administer
corticosteroids to promote fetal lung maturity
(c) Eclampsia
1. 4 gm loading dose of MgSO4 IV ; monitor for toxicity
(reflexes, urine output, respiration)
2. 5 mg IM every 4 hours (national guideline) until minimum
24 hours after delivery or convulsion (whichever comes
later)
3. Deliver within 12 hours
4. Consider anti-hypertensive (e.g. hydralazine 5 mg or any
safe in pregnancy)
Treatment score (x/3)
4. In-hospital monitoring (labor & post partum)
1. Vital signs monitored- (including BP) every 4 hours
2. Danger signs assessed at least twice daily (HA, visual
disturbance, epigastric pain
3. Provider note in chart at least once daily
In-hospital monitoring score (x/3)
5. Discharge
1. Vital signs documented for at least 24-48 hours
(continued risk of eclampsia)
2. Pre-discharge physical exam documented with pulmonary
and cardiac exam
3. Pre-discharge counseling noted : danger signs ; one
week follow up ; counseling for recurrence risk future
pregnancies ; FP method
Discharge score (x/4)
Numerator = Total of 1, 2, 3, 4, 5
Denominator = Total possible elements (14)
% adherence with pre/eclampsia standards per chart
(numerator/denominator x 100)
Average % adherence with pre/eclampsia standards (all charts)

20. 20
% Charts at least 80% adherence w/ pre-eclampsia standards
Guidelines for a Section of a Rural Health Center Supervision Checklist
Zimbabwe RBF Program
Maternity Services Instructions for Completion of Checklist Quality Item
Routine MNH best practices; PPH & sepsis
prevention/management (mother &
newborn); post-partum FP
% partograms in last month completed per
guideline (random review minimum 10
partograms)
-FHR, cervical dilatation, descent of presenting part,
maternal BP, pulse, and temperature documented
at admission and at least every 4 hours from
admission until delivery
Randomly review (every 3rd
partogram) at least 10
partograms completed in last month.
Calculate: % partograms completed per minimum
standard.
Numerator: Total # partograms reviewed that document
FHR, cervical dilation, descent of presenting part, maternal
BP, pulse, temperature at admission and at least every 4
hours until delivery
Denominator: Total # partograms reviewed
% women with prolonged labour referred to
higher level facility
-Review all charts meeting obstructed labour criteria
in past month: active labour > 12 hours (from
admission at minimum 4 cm dilation or per patient-
reported labour onset if admitted > 4 cm
--Review all partograms completed in past month and
select out partograms for analysis in which birth occurred
> 12 hrs after onset active labor (from time of admission
or time of 4 cm dilation or > 12 hrs from patient-reported
onset labor if admitted > 4 cm cervical dilation) (note total
# in HMIS maternity section below)
-Calculate: % of partograms with labor > 12 hours in
which women referred to higher level facility
Numerator: Total # partograms documenting active labor
> 12 hours referred to higher level facility
Denominator: Total # partograms documenting active

21. 21
labor > 12 hours
% women delivered in last quarter
administered uterotonic within one minute of
delivery of foetus
(Active Management Third Stage of Labor
(AMTSL) for PPH prevention).
-administration oxytocin 10 units IM within
one minute of delivery of fetus (or misoprostol
or ergomertrine, if BP normal, and oxytocin
unavailable)
-Review birth register (or partograms) for all births in past
month and calculate:
% total births in last month documenting administration
of oxytocin 10 units IM within one minute of delivery of
fetus (or misoprostol or ergometrine if BP normal and
oxytocin unavailable)
Numerator: # of births in past month in which oxytocin
(or another uteronic) administered within one minute
(immediately after) delivery of fetus
Denominator: Total # births within past month
% newborns BF within one hour of birth Review birth register, post-partum register (or other
appropriate register) and calculate:
% total births in last month documenting breastfeeding
within one hour of birth
Numerator: Total # births documenting initiation of
breastfeeding (BF) within one hour of birth
Denominator: Total # births in past month
% women delivered in last quarter monitored
per standard in early post-partum period for
early identification of danger signs
(review at least 10 charts)
- vaginal bleeding, BP, pulse, respiratory rate,
temperature at least every 30 minutes 1st
2 hrs
after birth and then at least twice per day until
discharge
-Randomly review (every 3rd
case) records post-partum
register entries for at least 10 women delivered in last
month
-Calculate: % post-partum women monitored per
standard (birth to discharge)
Numerator: Total # women delivered in past month with
documentation of following items at least every 30
minutes first 2 hours after birth and then at least twice per
day until discharge: vaginal bleeding (present or not), BP
value, pulse value, respiratory rate, temperature value

22. 22
Denominator: Total cases reviewed of women delivered
in past month