Data Demand and Use Workshop

Data Demand and Use
Workshop
Isabel Brodsky and Verne Kemerer
MEASURE Evaluation
October 24–27, 2017
Freetown, Sierra Leone

Knowledge learning objectives
 Describe the role of data demand and use (DDU)
in a functional M&E system
 Define DDU
 Understand the context of data use
 Identify barriers to data use and develop
recommendations to overcome them
 Identify locally relevant programmatic questions
and link those questions to available data
 Understand types of descriptive data analysis
Skills learning objectives
 Use tools and strategies to facilitate data use during
decision making
Workshop Objectives

 Framework for linking data
with action
 Action plan for overcoming barriers to
data use and improving information
flow
 Three commitments to improve data
use in your job
Workshop Outputs

Session 1: Monitoring and Evaluation
Concepts

Session 1: Learning Objectives
Knowledge learning objective:
 Describe the role of data demand and use
in a functional monitoring and evaluation
(M&E) system

DDU’s Role in a Functional M&E System

Strategic Information
 Are we making progress towards or are we
achieving our goals, objectives, and targets?
 How does service delivery vary across facilities?
 What are we doing well and how are we
under-performing?
 Can we improve service coverage with
existing resources?
 How should we allocate resources?
 Do we need to advocate for additional resources or
for changes to policies
and programmes?

Review of Basic M&E Concepts
 Goals
 Objectives need to be smart
• Specific
• Measureable
• Achievable
• Realistic
• Time bound

Results Chain
Inputs Activities Outputs Outcomes Impacts
Processes
Results
Objectives Goal(s)

Source: UNAIDS RST-ESA, 2010
Results Chain

Results Framework
Result
The outputs, outcomes, impacts identified in the results chain, that is, what the
programme plans to achieve at different result levels.
Result
level
 Outputs are the products, like the number of people enrolled on
antiretroviral therapy (ART) during the past 12 months.
 Outcomes reflect the programme’s objectives to change knowledge,
attitudes, behaviour, and biomedical markers like viral load and service
coverage.
 Impacts align with the goal(s) of the programme and describe changes in
morbidity and mortality, typically at the national level.
Indicator
An indicator is a quantitative or qualitative variable that provides a valid and
reliable way to measure achievement, assess performance,
or reflect changes connected to an intervention. Single indicators are limited in
their utility for understanding programme effects and must be collected and
interpreted as part of a set of indicators.
Baseline
The first measurement of an indicator that measures the current
condition against which we can track future changes.
Target
Targets state the desired level of performance the program wants to achieve
within a specified period. Each indicator needs a target. Setting targets
understanding what can be realistically achieved with available resources.

Indicator
 Measures an aspect of a program’s
performance
 Measures changes over a period of time
• # of new family planning users
• # of clients currently on ART
 Expressed as a number or percentage

Baselines
Result
Result
level
 Outputs are the products, like the number of people enrolled
on ART during the past 12 months.
attitudes, behaviour, and biomedical markers like viral load
and service coverage.
Indicator
An indicator is a quantitative or qualitative variable that provides a
valid and reliable way to measure achievement, assess performance,
Baseline
The first measurement of an indicator that measures the current condition
against which we can track future changes.
Target

 What are the sources of data?
 What are the data collection
methods?
 Who collects the data?
 How often are data collected?
 Who analyses these data?
 Who reports these data?
 Who uses these data?
Establishing a Baseline

Targets
Result
Result
level
 Outputs are the products, like the number of people enrolled on
ART during the past 12 months.
attitudes, behaviour, and biomedical markers like viral load
and service coverage.
Indicator
An indicator is a quantitative or qualitative variable that provides a
valid and reliable way to measure achievement, assess performance,
Baseline
The first measurement of an indicator that measures the current
condition against which we can track future changes.
Target

Calculating Targets
Things to consider
 Existing capacity of staff and facilities
 Staff turn-over
 Budget allocations through the
Ministry of Health (MOH)
 Other funding streams like bilateral
and multilateral donors

Learning Activity #1
Setting targets

Learning Activity #2
Session 1 review

 Define what data demand and use is

Data use
 Create or revise
a program or
strategic plan
 Develop or revise
a policy
 Advocate for a policy
or program
 Allocate resources
 Monitor a program
Data Demand and Use Defined
Data demand
 Decision-makers
specify what kind of
information they
need to inform the
decision-making
process and then
seek it out

Data-Informed Decision-Making Cycle

Learning Activity 3:
Session 2 review

Session 3: Context of Decision Making

 Understand the context of data use
Session 3: Learning Objective

Data-Informed Decision Making
Not an event, but a process
Data
x
Stakeholders
x
Decisions

“We are always giving patient forms and
data to our M&E Unit, who then gives
data to donors and the government. I am
the head doctor and I never have the
chance to look through the data before
they go up. We just keep giving data up
and up, and we never hear back about
it…”
Head of ART facility, Nigeria

 Census
 Vital events data
 Surveillance data
 Household surveys
 Facility-level service statistics
 Financial and management information
 Modeling, estimates, and projections
 Health research
Data and Information

 Paves path between data collectors
and users
 Leads to greater appreciation
of data
 Important element of management
and supervision
Working toward a Culture of Data Use

Data
x
Stakeholders
x
Decisions
Data-Informed Decision Making
Not an event, but a process

 View activities from different perspectives
 Have different degrees of understanding
 Need/want different information
 Need information at different levels
of complexity
 Have different intensities of interest
 Have different roles in the decision-making
process
Knowing Your Stakeholders
Why is it important?

Learning Activity 4
Stakeholder analysis matrix
Stakeholder
Description
Role
Knowledgelevel
Commitment
Resources
Constraints
Government Sector
Political Sector
Commercial Sector
Source: MEASURE Evaluation, 2011

Involving Stakeholders in the
Data Use Process
Relevance of data
Ownership of data
Appropriate dissemination
of data
Use of data

 Program design and evaluation
 Program management
and improvement
 Strategic planning
 Advocacy and policy development
Decision Areas

Learning Activity 5
Session 3 review

Session 4: 7 Steps for Using Information

Knowledge learning objectives:
 Define data demand and use
 Identify barriers to data use and develop
recommendations to overcome them
 Identify locally relevant programmatic
questions and link those questions to
available data
Session 4 Learning Objective

No Need to Fish
Photo by Bread for the World

1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential sources
4. Transform data into information
5. Interpret information and assess findings
6. Craft solutions and take action
7. Continue to monitor key indicators
7 Steps to Use Information

 Should we expand care and treatment
programs in Njombe province?
 Am I meeting the HIV prevention needs of
my key populations?
 Are men and women seeking HIV
care equally?
 What is the quality of my HIV prevention
service?
Step 1: Identify Questions
Examples

Should we expand care and treatment
programs in Njombe province?
 What is the burden of HIV in the province over
time?
 How many facilities are offering counseling
and testing (C&T) in the region?
 How many people are being seen
at each facility?
 Are there key facilities that have capacity to
expand C&T to reach gaps?
Step 2: Prioritize Questions

 Demographic and health surveys (DHS),
AIDS indicator surveys (AIS), etc.
 Routine health information system (RHIS)
 External data sources: behavioral surveillance systems,
mapping exercises, surveys
 Projections
Step 3: Identify Data Sources

Step 4: Transform Data into Information
Source: Key Findings of the 2013 Sierra Leone Demographic & Health Survey, page 9

Step 5: Interpret Information
and Draw Conclusions
Relevance
of finding
Reasons
for finding
Consider
other data
Conduct
further
research

 Gather additional information
 Prioritize Njombe in national expansion strategy
 Expand voluntary counseling and testing (VCT) and
C&T in high priority facilities
 Current VCT facilities do outreach to hotspots
 Create referral protocol to ensure those tested are
entering into C&T
Step 6: Craft Solutions and Take Action

 Monitor implementation of action plan
 Consider frequency and duration of monitoring
 Develop tool for monitoring
Step 7: Continue to Monitor Key Indicators

Learning Activity 6
Session 4 review

Session 5: Linking Data to Action

Knowledge learning objective
 Identify locally relevant programmatic
questions and link those questions to
available data
Skills learning objective
 Use tools and strategies to facilitate data
use during decision making

Questions
 Are we implementing the
program as planned?
 Are we fully meeting the
needs of our clients?
 What additional resources are
needed to meet
the needs of our clients?
 Are our interventions making
a difference?
Questions Lead to Decisions
Decisions
 Allocation of resources across
districts/community-based
organizations (CBOs)
 Revising program approaches
to emphasize results
 Addition of outreach
education programs to most-
at-risk populations
 Advocacy for new program
areas

 Brainstorming about what different staff
are interested in knowing
 Participatory discussion of indicators/data
— desire to know more
 Gathering feedback from clients
 Preparing for upcoming decisions
that have to be made or
planning exercises
 External factors — donors questions
How Do I Identify Questions of Interest?

3. Identify data needs and potential
sources
5. Interpret information and
assess findings

 Relevance
 Answerable
 Actionable
 Timeliness
Criteria for Prioritizing Questions

 Creates a time-bound plan for
data-informed decision making
 Encourages greater use of existing
information/generated demand
for data
 Monitors the use of information
in decision making
Framework for Linking Data with Action

Framework for linking data with action
Question Data
source/
indicator
Person
responsible
(analysis/
research)
Timeline Communication
channel
Possible
decision/
action

Session 5 review

Session 6: Transforming Data into Information

sources
5. Interpret information and assess
findings

 Understand types of descriptive
data analysis
Session 6: Learning Objective

Descriptive Analysis
 Turns raw data into useful information
 Provides answers to questions
 Describes the sample/target population
(demographic and clinic characteristics)
 Does not define causality – tells you what, not
why

 Question: Is my program meeting its
objectives?
 Analysis: Compare program targets and
actual program performance
to learn how far you are from target
 Interpretation: Why you have, or have
not, achieved the target and what
this means for your program
Descriptive Analysis Answers
Programmatic questions

 Service delivery descriptive analysis
 Service coverage analysis
 Unit cost analysis
 Trend analysis
Types of Descriptive Analyses
Using Routine Data

Service Delivery Descriptive Analysis
Selection of ARV Indicators
Private
sector
Public
health
system
Civil
society
# of clients who have refilled their prescriptions on-time at a
fee-for-service pharmacy
√
# of clients on ARVs who have been admitted into a
care facility for end-of-life support in the past six months √ √ √
# of clients on ARVs who have been counselled on ARV
adherence in the past three months
√ √ √
# of clients who have had CD4 and VL tested in the past six
months
√ √ √
# of clients enrolled in ARV programme √ √
# of client eligible for ARVs √ √ √
# of clients who have refilled their prescriptions on time at a
subsidized pharmacy
√ √
# of CD4 test kits procured through bilateral development
partners and distributed by NGOs
and INGOs to public-sector ARV clinics
√ √

Service Coverage Analysis
 Supply of services data
 Need for HIV services
 Calculate:
Service coverage =
Supply of services
Demand for services

Service Coverage Analysis
Service
coverage
=
# of individuals who have
received the services
# of individuals in need
of the service
Service coverage =
467 individuals have received
HIV testing and counselling (HTC) services
1,000 individuals in need of HTC services
= 47%

Unit Cost Analysis
What is the unit cost of delivering antenatal care (ANC) services
through an existing mobile health unit that provides maternal and
child health (MCH) services?
Item Amount
Fixed costs per month
Overhead SLL 7.8M
Personnel SLL 30.5M
Variable costs per month
Commodities SLL 7.8M
Other SLL 15M
# MCH clients/month 100
# MCH clients/month receiving ANC services 23
Cost data adapted from 10.1371/journal.pone.0119236

 Observe the overall pattern of change
 Reveal patterns due to seasonal variations
 Identify outliers and high/low performance in
certain time periods
 Compare the effect before or after
an event
 Produce projections to help with programme
planning and target setting
Descriptive Statistics
Trend analysis

Trend Analysis
Comparing time periods and sites
Source: 2016 Sierra Leone Malaria Indicator Survey
Full Report, page 51

 Display trends, relationships, and comparisons
 Simple and self-explanatory
Charts and Graphs

Column Charts
Source: Key Findings of the 2013 Sierra Leone Demographic & Health Survey, page
8

Stacked Bar Chart
Source: Key findings of the 2013 Sierra Leone Demographic & Health Survey,
page 10

Line Graph
Source: 2016 Sierra Leone Malaria Indicator Survey Full Report, page 33

Pie Chart
Source: Key Findings of the 2013 Sierra Leone Demographic & Health Survey, page 11

Trend analysis

Session 6 Review: Interpreting Data
Relevance of
finding
Reasons for
finding
Consider
other data
Conduct
research
 Have we met our target?
 How does performance compare
to other time periods? Facilities?
 What could explain the observation?
 Talk to beneficiaries and community
leaders
 Ask program managers
 Analyze other data sources
• Do they confirm or reject
your finding?
o Why?
 If there are no data, need for research
 Methodology depends on question
and available resources

Session 6 review

Session 7: Craft Solutions, Take Action, and
Monitor Progress

sources
5. Interpret information and assess findings

 Identify barriers to data use
and develop recommendations
to overcome them
Skills learning objective
 Use tools and strategies to facilitate
data use during decision making
Learning Objectives

What Determines DDU?
* Based on PRISM analytical framework (LaFond, Fields et al. (2005). The PRISM: An analytical framework for
understanding performance of health information systems in developing countries. MEASURE Evaluation).
Organizational Technical
Behavioral

Organizational Technical
Behavioral
Politics
Culture
Society
What Determines DDU?

Technical
 Have you ever had an experience while making a policy or
program-related decision when you were concerned about the
quality of the information being used?
Behavioral
 What specific challenges have you experienced among your
staff when it comes to using data?
Organizational
 How does your organization support having the necessary
information to make decisions?
Identifying barriers to data use: discussion questions

Developing an action plan to address
barriers to data use
Barrier Action Steps
Responsible
person
Timeline Indicators
1. 1a.
1b.
1c
1a.
1b.
1c.
1a.
1b.
1c.
1a.
1b.
1c.
2. 2a.
2b.
2c.
2a.
2b.
2c.
2a.
2b.
2c.
2a.
2b.
2c.

 Plan, plan, plan!!!
 Engage key stakeholders to fully
understand:
• Decisions they make
• Information they need
• Appropriate dissemination
Building Data Use into Your Work

 Institute regular meetings
 Document and
communicate data
use successes
Practical Steps to Integrate DDU
 Institutionalize tools (ex.,
framework for linking
data with action)

 Identify a group leader
 Identify 2–3 things your group can do
to facilitate data use within the first two
months of returning back to work, based
on your action plan
 Small group discussion: 20 mins
 Group presentations: 15 mins
Make Commitments

This presentation was produced with the support of the United
States Agency for International Development (USAID) under the
terms of MEASURE Evaluation cooperative agreement AID-OAA-L-
14-00004. MEASURE Evaluation is implemented by the Carolina
Population Center, University of North Carolina at Chapel Hill in
partnership with ICF International; John Snow, Inc.; Management
Sciences for Health; Palladium; and Tulane University. Views
expressed are not necessarily those of USAID or the United States
government.
www.measureevaluation.org

Data Demand and Use Workshop

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Data Demand and Use Workshop

Ähnlich wie Data Demand and Use Workshop (20)

Mehr von MEASURE Evaluation

Mehr von MEASURE Evaluation (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Data Demand and Use Workshop