Presented at the 2014 CUGH Conference.

1. 2
METHODOLOGY
The core objectives of the collaboration between the MoH and MEASURE Evaluation were to:
1. describe the process for ensuring data quality at each level of the health system, including level
specific responsibilities, and
2. provide guidelines for data quality monitoring procedures.
KEY DELIVERABLES IN THE PROCESS INCLUDED:
1. Data Quality Standard Operating Procedure (SOP)—General, high level protocol for ensuring
data quality at the service delivery, district, and national levels
2. Routine Data Quality Assessment (RDQA) SOP—Protocol for the implementation of RDQAs as a
monitoring tool to routinely review the quality of data at the service delivery, district, and national
levels
3. Customized RDQA Tool for Botswana (B-RDQA Tool)—Excel worksheet consisting of data entry
forms and graphical output
4. B-RDQA Tool User Manual—Detailed guidance on implementing and disseminating results using
the B-RDQA Tool for conducting an RDQA of any health program
5. Data Quality Curriculum—Curriculum for use in MoH trainings on data quality, including presen-
tations, exercises, and a full participant’s guide. Curriculum covers the content of the two SOPs
and the collection and use of data from RDQAs.
The development of the protocols and curriculum occurred over the course of a year (2012),
followed by training workshops (2012/2013).
January Established scope of work & objectives
February Developed B-RDQA Tool
March Drafted SOPs
April Pre-tested B-RDQA Tool in the field
Reviewed SOPs
May Finalized B-RDQA Tool
Revised SOPs
June Finalized SOPs
July-Oct. Developed the data quality curriculum
November Conducted 1st data quality training workshop
April Conducteddataqualitytrainingoftrainers(ToT)workshop
Supported 2nd data quality workshop
The development of this poster was supported by funds from the USAID MEASURE Evaluation project.
RESOURCE REQUIREMENTS
Time & cost: The process of developing the SOPs, customized tool, training materials, and
conducting the trainings took about 16 months. The total cost to MEASURE Evaluation, primarily
in staff time and travel for in-country consultative workshops and trainings, was $300,000, funded
by the United States Agency for International Development. The MoH was responsible for funding
the in-country costs, including workshop venues, per diems, and transportation costs.
Staff: The newly formed Department of Health Policy Development, Monitoring and Evaluation
(DHPDME) at the MoH initiated the activities with MEASURE Evaluation. The Principal Health
Officer was a key champion for the process, supported by the Chief Health Officer. The MEASURE
Evaluation team that worked with the MoH included three Senior M&E Advisors and two M&E
Associates.
Travel: Five trips were made to work in-country with the MoH and other stakeholders including:
1. January 2012—Planning visit to develop the scope of work
2. April 2012—B-RDQA Tool customization workshop and pilot testing
3. June 2012—Consultative workshops to finalize SOPs and user manual
4. November 2012—Training of M&E officers
5. April 2013—Training of trainers and training of M&E officers
7
MEASURE Evaluation is funded by USAID through cooperative agreement GHA-A-00-08-00003-00 and implemented by the Carolina
Population Center at the University of North Carolina at Chapel Hill, with Futures Group, ICF International, John Snow, Inc., Management
Sciences for Health, and Tulane University.
The views expressed in this publication do not necessarily reflect the views of USAID or the United States government.
Botswana’s Integration of Health Data Quality Assurance Into Standard Operating Procedures
STANDARD OPERATING PROCEDURES
The SOPs for data quality and RDQA, as well as the B-RDQA Tool User Manual were drafted for
review while the B-RDQA Tool was being customized.
• The data quality SOP was written as a high level document on the
various dimensions and considerations of data quality, intended for
senior MoH officials, other policymakers, and M&E Officers.
• The RDQA SOP was written as a general protocol for conducting
a RDQA, including responsibilities by level, intended for any MoH
or district official responsible for initiating, managing or conducting
routine assessments.
• The B-RDQA Tool User Manual was written specifically for those
staff using the B-RDQA Tool to conduct RDQAs in the field.
Draft SOPs and a draft user manual were reviewed and discussed
with stakeholders at consultative workshops. Both Ministry and
external stakeholders participated in the consultations, and
documents were finalized based on the recommendations from the
workshops. Final documents were printed in country for distribution
by the MoH.
5
BACKGROUND
To support improved data quality throughout the health system, the Botswana Ministry of Health
(MoH) collaborated with experts from MEASURE Evaluation to develop a national procedure for
routine monitoring of data quality, as well as provide specific guidance on developing action
plans to address challenges using a bottom-up approach.
Because information is a key building block of a health system, efforts to improve data quality
directly support improvements in a country’s information system across program areas.
1
The quality of health data impacts a government’s ability to make strategic health-related
decisions. At the national level, data ultimately inform budget and policy decisions. In the
District Health Management Teams (DHMTs) and Service Delivery Sites, data enable providers
and Monitoring and Evaluation (M&E) Officers to understand the broader health activities and
priorities in their respective areas.
SERVICE DELIVERY
HEALTH WORKFORCE
INFORMATION
MEDICAL PRODUCTS, VACCINES & TECHNOLOGIES
FINANCING
LEADERSHIP/GOVERNANCE
IMPROVED HEALTH (LEVEL AND EQUITY)
RESPONSIVENESS
SOCIAL AND FINANCIAL RISK PROTECTION
IMPROVED EFFICIENCY
THE WHO HEALTH SYSTEM FRAMEWORK
SYSTEM BUILDING BLOCKS
OVERALL GOALS / OUTCOMES
ACCESS
COVERAGE
QUALITY
SAFETY
FIGURE 1: INFORMATION IN THE WHO HEALTH SYSTEM BUILDING BLOCKS
Health data are
collected at
service delivery
sites
Health data are
aggregated at
district & national
levels
Assessment
of data impacts
policy &
budgets
Policy & budget
decisions impact
health outcomes
Data collection Aggregation & Analysis Impact on health
FIGURE 2: DATA & HEALTH IMPACT
ADAPTATION OF THE B-RDQA TOOL
The B-RDQA Tool is an Excel dataset with multiple worksheets for a user to complete to verify
data on up to four indicators at various levels of the health system, as well as conduct a system
assessment to evaluate the key functional components of the M&E system. The tool was
customized for the Botswana country context, with changes to the language used to describe the
various levels of the health system to reflect the Botswana data flow from service delivery sites to
health districts to National M&E.
One of the significant changes to the tool was the addition of the “use of data for decision making”
functional area in the system assessment component of the tool. The importance of this component
was reinforced in subsequent consultative workshops and during the first training, where district
M&E officers identified the use of data for decision making as a key challenge.
4
M&E structures,
functions &
capabilities
Use of data for
decision making
Indicator
definitions
and reporting
guidelines
Training
Data management
processes
Data collection
and reporting forms
and tools
Six functional
areas of an M&E
system
FIGURE 5: FUNCTIONAL AREAS OF THE M&E SYSTEM
DATA QUALITY CURRICULUM
& TRAINING
A comprehensive curriculum was developed by MEASURE Evaluation to train national and district
M&E officers on how to implement and use the SOPs and the B-RDQA Tool. The curriculum
supports a two and a half day training with a balance of presentations and hands-on exercises
that give attendees first-hand experience using the tool, interpreting outputs, and developing
action plans.
The 1st training of 22 M&E Officers was
conducted in November 2012, followed by
a ToT workshop in April 2013 with select
participants from the first data quality
training. The trainers then conducted the
2nd training with support from MEASURE
Evaluation. Overall feedback on the
trainings was very positive as indicated
by the quotes to the left.
6
“This [process] will really
reduce work burden…
very exciting, can’t wait to
implement. This was one of
the best trainings which will
really address our district
data quality problems.”
– District Health Officer
“A very good
training that came at the right time,
providing skills that are sustainable and
very easy to use…Bringing out very
valuable results to improving health
information systems, important to system
improvement and decisions making.”
- Trainee
KEYS TO SUCCESS
Country ownership: This activity was initiated by the MoH, who requested technical assistance
to develop and implement protocols for improving data quality and adapt global tools to the
Botswana context. The country-led foundation of this process was essential in connecting with
the correct stakeholders to give input and insight. Furthermore, responsibility for the in-country
expenses proved beneficial with respect to the MoH taking ownership of the process.
Going forward, the MoH will conduct an annual national M&E forum where data quality issues will
be addressed by M&E Officers based on findings from their RDQAs. The MoH also has identified
a consultant to develop the National Health M&E Plan for the ministry, which will include RDQAs
as a routine activity for districts and at national level.
Champions: Also key to the entire process was having a strong champion for data quality
activities at the MoH. Without a strong technical voice, it would have been challenging to find
the momentum to support the development and implementation of the protocols.
Decentralization: Finally, the protocols decentralize the process of
planning targeted activities to improve data quality, allowing service
delivery sites and district-level officials to take ownership of data
quality in a systematic and structured way. Service delivery sites and
districts develop their own recommendations and action items, putting
the power in local hands.
Evidence of the success of the RDQA process was found at a district
outside the capitol. A District M&E Officer implemented a log book
(see left) to track data quality indicators after attending the training
workshop in April 2013.
8
Authors: Suzanne Cloutier, Sergio Lins, Amanda Makulec,
David Boone, Ernest Fetogang, Rosinah T. Dialwa,
Segametsi Duge and Sophia Magalona
With growing interest and investment in health system
strengthening measures, the Botswana adaptation
of global data quality tools operationalizes a system
for health information system improvements that
could be adopted by other countries facing data
quality challenges. Having conducted this adaptation
in Botswana, the customization approach has been
tested and streamlined, such that the deliverables
could be adapted to other country contexts.
CONCLUSIONS:
3
BOTSWANA’S APPROACH
The data quality protocols in Botswana were developed to ensure the accuracy, timeliness and
completeness of health data being transmitted throughout the health information system. The
Botswana adaptation of the RDQA is unique as it provides a national protocol for routine data
quality assessment across program areas.
The ideal data flow for Botswana health data is illustrated below. Botswana’s health data
currently flow through more than 39 different information systems, including both electronic and
paper-based systems that feed into various data management systems. With the creation of a
national M&E unit, the MoH is working to streamline processes and move towards this ideal flow.
FIGURE 3: IDEAL BOTSWANA HEALTH DATA FLOW
FIGURE 4: BOTSWANA CONCEPTUAL FRAMEWORK FOR DATA QUALITY
JOHN SNOW, INC.
A global conceptual framework for data quality was adapted to reflect the Botswana data flow
and priorities within the country’s M&E system.