Prospective Country Evaluation 2019 Synthesis Findings
â˘Als PPTX, PDF herunterladenâ˘
1 gefällt mirâ˘954 views
The Prospective Country Evaluation is an embedded mixed-methods evaluation platform designed to examine the Global Fund business model, investments and contribution to disease program outcomes and impact in eight countries. Findings were synthesized across the 8 countries to provide timely and actionable recommendations to support program improvements and accelerate progress towards the objectives of the Global Fund 2017-2022 Strategy.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ăhnlich wie Prospective Country Evaluation 2019 Synthesis Findings
Ăhnlich wie Prospective Country Evaluation 2019 Synthesis Findings (20)
Mehr von Institute for Health Metrics and Evaluation - University of Washington
Mehr von Institute for Health Metrics and Evaluation - University of Washington (20)
KĂźrzlich hochgeladen
KĂźrzlich hochgeladen (20)
Prospective Country Evaluation 2019 Synthesis Findings
- 1. THE GLOBAL FUND PROSPECTIVE COUNTRY EVALUATION Synthesis of Findings from 2018 Cambodia, Democratic Republic of the Congo, Guatemala, Mozambique, Myanmar, Senegal, Sudan, and Uganda
- 2. STRUCTURE OF PRESENTATION ď§ Purpose and approach ď§ Methods ď§ PCE focus in 2018 ď§ Key findings and recommendations ⢠Business model ⢠Human rights, key and vulnerable populations, gender ⢠Resilient and sustainable systems for health ⢠Sustainability, co-financing, and transition ⢠Value for money ď§ Added value of the PCE ⢠How are findings being used in PCE countries ⢠Lessons learned ď§ PCE plans for 2019
- 4. Evaluation of the Global Fund business model, investments and contribution to disease program outcomes and impact in eight countries Generation of timely evidence to support programme improvements and accelerate progress towards the objectives of the 2017-2022 Strategy PURPOSE APPROACH Disease results chains explore links between Global Fund inputs, outputs, programme outcomes, and impact. Programmatic changes to be observed through tracking key indicators. Theory-based approaches and related evaluation frameworks explored how and why Global Fund investments, policies and practices influence disease impact pathways in the results chains
- 5. METHODS
- 6. ď§ Quantitative results led to qualitative exploration and vice versa principally through the results chains analyses ď§ Evidence triangulated and strength of evidence ranked across findings MIXED METHODS Impact assessment ⢠Existing or modelled secondary data ⢠Analysed retrospective sources to provide context and trends Process evaluation ⢠Document review, meeting observations, key informant interviews, root cause analyses, process mapping
- 7. FOCUS OF PCE IN 2018
- 8. FOCUS OF PCE Building on analysis of funding request and grant making phase in 2017, 2018 focused on early implementation of 34 grants in eight countries, totalling over $2.1 billion in investments during this allocation Tracked how Global Fund investments translated into activities and programmatic outputs Identified how the business model enabled and constrained early grant implementation Examined the efficiency and effectiveness of early grant operationalisation
- 9. PCE Portfolio Characteristics and 2018-2020 Grant Budgets Source: Approved 2018-2020 budgets in $USD. * Income category shifted between 2017 and 2018 Global Fund eligibility lists Sudan (Lower LMI) ⢠Core portfolio (formerly high impact) ⢠Grants total: $128.4m ⢠COE Myanmar (Lower LMI) ⢠High impact portfolio ⢠Grants total: $321.5m ⢠Matching funds Cambodia (Lower LMI*) ⢠High impact portfolio ⢠Grants total: $98.4 m Uganda (LI) ⢠High impact portfolio ⢠Grants total: $478 m ⢠AGYW priority country ⢠Intensive support for human rights (all) ⢠Matching funds ⢠CCM evolution Mozambique (LI) ⢠High impact portfolio ⢠Grants total: $523m ⢠AGYW priority country ⢠Intensive support for human rights (HIV, TB) ⢠Matching funds ⢠CCM evolution Guatemala (Upper LMI) ⢠Core portfolio ⢠Grants total: $38.2 m ⢠CCM evolution Senegal (LI*) ⢠Core portfolio ⢠Grants total: $73.1 m ⢠Intensive support for human rights (HIV) ⢠Matching funds DRC (LI) ⢠High impact portfolio ⢠Grants total: $542.9 m ⢠COE ⢠Intensive support for human rights (HIV, TB) ⢠Matching funds ⢠CCM evolution
- 10. KEY FINDINGS & RECOMMENDATIONS Business model Human rights, key and vulnerable populations, gender Resilient and sustainable systems for health Sustainability, transition and co-financing Value for money
- 11. KEY FINDINGS & RECOMMENDATIONS Business Model
- 12. KEY FINDINGS: Business Model ď§ The Secretariat approved the majority of PCE grants on time ď§ First disbursements (Global Fund to PRs) for the majority of grants were made on time ď§ Approval processes for Matching Funds were aligned with main grants in some cases ď§ Country Teams allowed flexibilities which helped with grant transition ď§ Country Teams played important roles in resolving early bottlenecks Some grant start up processes worked well and as intended
- 13. ď§ Concurrent business model-related processes reduced time and attention from grant start up including for program continuation grants ď§ PR transition created initial implementation delays ď§ Lengthy selection and contracting of implementers, particularly Sub-Recipients by Principal Recipients delayed activity implementation ď§ Some Matching Funds approvals and disbursements were mis-aligned with main grant approvals and this impacted on activity implementation However, some processes worked less well and this affected grant implementation efficiency, contributing to delays and low early absorption rates in most PCE countries FINDINGS: Business Model
- 14. Summary of early grant implementation milestones
- 15. Budget absorption for Q1 and Q2 PUDRs 2018 highly variable but low overall: ⢠HIV: 14% ⢠TB: 47% ⢠Malaria: 30% ⢠RSSH: 7% Despite this, our qualitative data suggests that core services (e.g. treatment services provided by national programs) did not stop between grants Q3/Q4 absorption is expected to be higher FINDINGS: Business Model
- 16. Reflecting that the provision of core services did not stop between grants, the majority of countries are meeting or nearly meeting performance indicators ⢠HIV: 79% ⢠TB: 96% ⢠Malaria: 80% This is primarily due to performance indicators being focused on coverage, outcome and impact metrics that relate to the overall national program (rather than grant) performance FINDINGS: Business Model
- 17. ď§ There is limited/no correlation between financial data on short-term budget absorption and programmatic M&E data ď§ The reliance on imperfect quantitative financial and programmatic M&E data: ď§ Leads to inaccurate grant ratings; ď§ Creates perverse incentives to implementers; and ď§ Poses a risk to Secretariat grant oversight and Board governance functions ď§ While an over-emphasis on absorption for monitoring grant performance has drawbacks, it remains an essential first condition for grant impact. Low early absorption is reflective that budget execution rates should be more carefully anticipated ď§ The lack of reported data on grant-specific outputs (except in some cases) and data at the sub-national level fundamentally compromises analysis and measurement of VfM and could hamper efficient portfolio management FINDINGS: Business Model
- 18. The Global Fund Secretariat should ď§ Consider flexibilities to the three-year grant cycle to facilitate smoother transition between grants, facilitate early grant implementation and enable adequate time for grant implementation ď§ Update and strengthen guidance for CCMs and PRs on the selection and contracting of SRs to increasingly âfront loadâ PR/SR selection and contracting processes prior to grant implementation. Guidance should include ⢠Metrics which clearly define the time period within which SRs are expected to be selected and contracted by PRs ⢠PRs to work with identified SRs to ensure roles, scope of activities and budgets are agreed during grant making, ahead of the implementation period ⢠PRs should be strongly encouraged to effectively use Pre-Financing Policy flexibilities to facilitate SR preparation (e.g. staff contracting, pre-financing some activities) in advance of grant implementation ď§ Consider embedding matching funds in the timeline for the design, approval and implementation of the main grants to facilitate timely implementation of activities ď§ Consider trying to better link financial and programmatic data by collecting data on the specific outputs achieved through grant implementation, as well as collecting data at the sub-national level, at least for some indicators RECOMMENDATIONS: Business Model
- 19. Human rights, key and vulnerable populations, gender KEY FINDINGS & RECOMMENDATIONS
- 20. KEY FINDINGS: Human rights Activities to reduce human rights-related barriers to services are well represented in HIV grants, but there is less focus in TB and malaria grants Global Fund interventions to address human rights-related barriers in country grants
- 21. KEY FINDINGS: Key and vulnerable populations (KVP) ⢠Investments by disease program largely cover key and vulnerable populations as defined by the Global Fund but many country-defined KVPs do not align with the Global Fund definitions Other HIV ⢠Fisher people: Uganda, Senegal ⢠Bridge populations (truckers, security forces, traders, tourism): Senegal ⢠Pregnant women: Senegal ⢠Partners of PLWH: Cambodia ⢠Partners of MSM: Cambodia ⢠Partners of PWID: Cambodia ⢠Clients of FEW: Cambodia ⢠Partners of sex workers: Guatemala ⢠High risk individuals not identifying as KVP: Myanmar Other TB ⢠People living with diabetes: Cambodia, Guatemala, Myanmar, Senegal, Uganda ⢠People living in overcrowded housing / urban slums: Senegal, Uganda ⢠Elderly people: Cambodia, Myanmar ⢠TB contacts: Cambodia, Uganda ⢠Students: Senegal ⢠Health care workers: Myanmar ⢠Urban and rural poor: Myanmar Other Malaria ⢠Pregnant women: Sudan, Uganda ⢠Children <5: Senegal, Sudan, Uganda ⢠Forest workers, dwellers: Cambodia, Myanmar ⢠Seasonal workers: Cambodia, Myanmar ⢠Military, border, armed groups: Myanmar ⢠Camp/settlement occupants: Myanmar
- 22. KEY FINDINGS: Gender and Human Rights ď§ Gender and human rights dimensions are not well understood or discussed by stakeholders ⢠Perception that sex-specific targeting alone is sufficient for gender- responsive programming ⢠Lack of experience among Ministry of Health and other stakeholders on gender and on programs to reduce human rights-related barriers to services (including legal dimensions) ⢠Few examples of programs that are actually addressing gender-related vulnerabilities (DRC SASA! pilot project is an exception) ď§ TB and malaria activities are less gender responsive ď§ For example, despite greater TB prevalence in men, most programs lacked interventions that addressed menâs gender-related risks ď§ Overall implementation delays due to sub-contracting issues
- 23. RECOMMENDATIONS: Human Rights, key and vulnerable populations, gender The Global Fund Secretariat should ď§ Ensure that Global Fund-supported programs clearly defines key and vulnerable populations, aligned with national epidemiological context and that programs are designed to allow for tracking of progress against key intervention areas (e.g. disaggregation of male/female/trans sex workers, youth, women who inject drugs) ď§ Continue efforts to build in-country capacity and expertise on gender and human-rights related issues, e.g.: ⢠Developing clearer and more accessible guidance on human rights and gender programming and implementation (already underway by Secretariat/CRG); ⢠Ensuring TA is consistent with country needs and facilitating countries seeking technical assistance for reducing gender- and human rights-related barriers (e.g. help the CCM to know that the mechanism exists and see the value in accessing TA to enable stronger more gender responsive planning, implementation, and monitoring) Country Stakeholders should ď§ More explicitly articulate the gender-related vulnerabilities of men/boys, women/girls, transgender and gender non-conforming individuals, the impact of these on disease- specific outcomes, and specific strategies to mitigate these effects in funding requests and designing disease-specific strategies.
- 24. RECOMMENDATIONS: Human Rights, key and vulnerable populations, gender The Global Fund Secretariat and Country Stakeholders should ď§ Encourage more explicit promotion of gender and human rights integration throughout the grant lifecycle, particularly for TB and malaria, including: ⢠Determine the appropriate mechanisms for ensuring the that high quality gender assessments are conducted (or integrated into other assessment practices); e.g. further direct engagement by Global Fund technical staff in specific country gender assessments ⢠Ensure each CCM has a qualified gender expert engaged throughout the grant design and implementation process with the requirement that the gender expert is fully represented in all processes and decisions ⢠Expand the requirements for addressing gender in funding requests and reporting, using clear guidance that is understandable for both country teams and reviewers ⢠Programming and grant design (e.g. to address social norms, stigma, time use, and intra- household decision-making, not just sex-based targeting) ⢠Implementation (e.g. collection and analysis of programmatic data disaggregated by key populations)
- 25. Resilient and sustainable systems for health KEY FINDINGS & RECOMMENDATIONS
- 26. KEY FINDINGS: RSSH Absorption across RSSH modules during Q1-Q2 2018 was low, in part due to the factors hindering implementation more generally Delays in RSSH implementation are due to similar factors hindering overall grant implementation, such as SR selection and contracting, administrative / logistical hurdles, timing of disbursements, staff turnover, etc. Absorption by RSSH module, by country, across grants with RSSH investments, Q1-Q2 2018
- 27. KEY FINDINGS: RSSH RSSH coverage indicators predominantly align with HMIS/M&E module, missing an opportunity for monitoring other RSSH priorities Despite limited early absorption, most grants achieved 90+% on RSSH indicators Achievement progress on RSSH HMIS indicators, by PCE country, by grant, Q1-Q2 2018
- 28. KEY FINDINGS: RSSH Many RSSH investments are considered shorter-term gap investments rather than longer- term investments in more sustainable health system strengthening needs Cambodia: RSSH investments are mainly targeting âfixableâ and âshorter termâ issues and lacking strategic focus, rather than tackling systemic âlonger termâ challenges facing the country, such as human resource capacity Uganda: Many direct RSSH investments target the malaria program with limited integration efforts (e.g. HRH investments support national malaria control program staff; community response investments to improve malaria component of district epidemic response system) Myanmar: the HRH investment is largely spent supporting seconded staff on project coordination and logistics related to Global Fund grants, which does little to address the major health workforce shortages facing the country Sudan: the scattered approach to compiling RSSH activities to fill gaps in short-term disease-specific work plans is thought to have resulted in limited investment in community systems. Stakeholders note this as a missed opportunity for Global Fund contributions to Sudanâs intended national policy shift toward health promotion and primary care
- 29. RECOMMENDATIONS: RSSH ď§ During the funding request development, consider mechanisms to incentivize stronger alignment of crosscutting RSSH investments to longer-term national strategies for health system strengthening, rather than addressing short-term, disease-specific health system gaps ď§ Improve standardization for categorization of RSSH investments within grant budgets to ensure accurate quantification of Global Fund contributions toward RSSH, including whether simplification is feasible or increased guidance and examples are necessary ď§ Improve monitoring and measurement of the outcomes of RSSH investments, e.g.: ⢠Clear articulation of expected RSSH outcomes, which can be translated into investment guidance, the modular framework and grant performance framework where relevant; ⢠Stronger alignment of grant activities to indicators; and ⢠Consideration (and development of) community systems and responses indicator(s) in the modular framework The Global Fund Secretariat should
- 30. Sustainability, transition and co-financing KEY FINDINGS & RECOMMENDATIONS
- 31. KEY FINDINGS: Sustainability, Transition, Co-financing ď§ All governments of PCE countries have made commitments to meet or exceed Global Fund co-financing requirements ď§ However, external stakeholders (e.g. CSOs, advocates, and evaluators) have been unable to verify fulfillment in a timely manner in most countries ⢠Countries report to the Global Fund on their co-financing and realization of specific commitments periodically, with assurance on country reporting through a variety of country-specific mechanisms ⢠Reporting structures for domestic financing vary between countries, including whether they are accessible to stakeholders ⢠National Health Accounts are one of the primary mechanisms for tracking domestic financing and fulfillment of co-financing commitments, but most mechanisms are retrospective in nature
- 32. KEY FINDINGS: Sustainability, Transition, Co-financing Even when countries do meet co-financing requirements, PCE countries remain heavily reliant on donor resources to finance the disease program ⢠This may pose a threat to transition readiness, programmatic and financial sustainability ⢠Budget shortfall varies by disease, with HIV typically coming closer to total needs costing than TB ⢠Guatemala is an exception TB 2018-20 Funding Landscape
- 33. KEY FINDINGS: Sustainability, Transition, Co-financing There is evidence of countries embedding sustainability and transition considerations into program design and implementation Long-term sustainability planning ⢠Sustainability and transition considerations were integrated into grant design and national strategic plans in Cambodia, Myanmar, Guatemala, and Senegal ⢠Uganda finalized health financing strategy, included a Social Health Protection system and strategic pooling of finances to strengthen purchasing mechanisms ⢠The HIV NSP in Senegal has prioritized the mobilization of domestic resources to sustain the national HIV response Preparation for transition ⢠Guatemala has embedded a sustainability and transition plan into the TB NSP and is developing a sustainability plan annex to the HIV NSP ⢠A transition readiness assessment in Cambodia found that funding shortfalls in prevention would lead to an increase in HIV incidence and compromise the sustainability of the HIV response
- 34. RECOMMENDATIONS: Sustainability, Transition, Co-financing ď§ Consider restructuring the country co-financing requirement to more ambitiously increase domestic expenditure on health and the three diseases, with a view to ensuring that domestic financing increases to a level that more fully supports transition and sustainability objectives. Specifically, this might involve: ⢠Expanding upon the co-financing requirement to better reflect the governmentâs existing financial commitments overall and within the wider health financing landscape (e.g. by setting the co-financing requirement based on more parameters than just the current two) ⢠Increasing the minimum level of co-financing that is acceptable to the Global Fund (i.e. increasing the co-financing requirement but not necessarily the co- financing incentive) ⢠Strengthening the incentive for countries to increase domestic expenditure on health and the three diseases by making additional resources available to countries that invest above the minimum acceptable level of co-financing (via a separate mechanism than the existing incentive, which can only be taken away) The Global Fund Secretariat should
- 35. Value for money KEY FINDINGS & RECOMMENDATIONS
- 36. KEY FINDINGS: Value for money: Economy Analysis of PQR data suggests that economy has improved over time across the grants, with health commodity prices falling, often below global reference prices Some unit costs used for budgeting do not closely reflect actual costs, potentially leading to: ⢠Global Fund paying above the lowest possible cost for inputs; and or ⢠Low budget absorption
- 37. KEY FINDINGS: Value for money: Efficiency & Effectiveness Strong examples of efforts to improve efficiency of grant design and national programs, particularly in countries facing significant reductions in program budgets Program management costs vary significantly across countries and by type of PR, with substantially higher costs for UN agencies and CSOs than for governments Cost-effectiveness considerations inform program design and decision making in most settings (such as through modelling) but not systematically 25% 7% 2% 24% 15% 12% 25% 33% 0% 10% 20% 30% 40% 50% 60% 0 100 200 300 400 500 600 DRC (LIC) Mozambique (LIC) Uganda (LIC) Myanmar (Lower LMIC) Sudan (Lower LMIC) Senegal (LIC) Cambodia (Lower LMIC) Guatemala (Upper LMIC) Programmanagementas%oftotalgrantbudget Grantbudgets(USDmillions) Program management budget Total grant budget Program management as % of grant amount
- 38. KEY FINDINGS: Value for money: Equity ď§ While equity is often discussed, trade-offs between equity, cost-effectiveness and programmatic targets are dealt with differently (often informally) ď§ More could be done to ensure that Global Fund-supported activities (and their benefits) are fairly distributed amongst target recipients, e.g.: ⢠A review of the gender-related dimensions of equitable access to services and reduction of vulnerabilities for men, women, boys, girls, and transgender individuals ď§ Some evidence that over ambitious target setting vis-Ă -vis available funding has been counterproductive to the prioritization of hard-to-reach areas ď§ Despite some examples of Global Fund support being used to reduce financial barriers to accessing services, this still poses a significant issue
- 39. RECOMMENDATIONS: Value for money The Global Fund Secretariat, together with partners, should ď§ Expedite work to collect unit/service delivery costs at the country level and use this as a basis for budgeting, with clear guidance on appropriate formulae to inflate estimates to allow for inflation, price changes, currency shifts, etc. ď§ Consider ways to strengthen country-level and/or grant-specific analysis of VfM throughout the grant life-cycle (while considering the burden of reporting), such as by: ⢠Collecting and analysing grant-specific output data ⢠Extending reporting tools to collect sub-national data ⢠Creating performance targets that better address equity considerations ⢠Requesting that PRs/countries report against trends in efficiency and effectiveness
- 41. PCE analysis provides in-depth knowledge of the complexities of grant implementation unlikely to be found in thematic reviews/short country visits PCE evidence is informing and/or validating findings from TERG Thematic and Strategic Reviews (e.g. RSSH and Partnerships) and is able to use other TERG Review findings prospectively PCE is able to respond to emerging TERG or Secretariat issues (e.g. business model issues, CRG needs) and rapid requests e.g. feasibility of implementing new MDR- TB treatment guidelines (see example in next slide) PCE synthesis represents a whole that is greater than the sum of the parts with recommendations derived from and consistent with the evidence from multiple countries PCE findings on lessons learned for key processes (e.g. funding request development, SR selection, etc.) will inform Secretariat planning of the next implementation cycle Added value of the PCE: Global level
- 42. The PCE platforms in Myanmar, Sudan and Uganda were mobilized at short notice by the TERG to facilitate a preliminary review of the feasibility and bottlenecks for transitioning to and implementing the new guidelines (Dec 2018 start -Jan 2019 end) High levels of trust between PCE team and country stakeholders enabled access to key informants, documents and data and this contributed to the reviewâs success The PCE reviewâs findings have provided a baseline for the PCEs to follow up prospectively in Myanmar, Uganda and Sudan The PCE review provided country recommendations for programmatic course correction and Global Fund recommendations for the current and next grant cycle The PCE platforms are developing capacity and agility to respond to emerging issues and have the potential to be further development and utilized. Discussions are underway for further TB-related findings to be generated through PCE in this way Added Value of the PCE Conducting a rapid review on the feasibility of implementing new treatment guidelines for MDR/RR-TB and LTBI in three PCE countries
- 43. Added value of the PCE: Country level Targeting PCE findings to national program managers: The ability to disseminate emerging findings in a timely manner is a core strength of prospective evaluations and provides an opportunity for the PCE to contribute to continuous quality improvement Opportunities for subnational data collection and analysis can add value to national level perspectives Country stakeholdersâ appreciation for documenting the challenges, successes, and learnings throughout the Global Fund grant cycle â some of which are previously known, but not systematically or independently documented, nor synthesized across countries Synthesis adds value at country level, enabling stakeholders to compare their responses to those of other countries as well as understanding how the PCE is part of a larger strategic process
- 44. What have we learned from the PCE approach? Platform/Methods ⢠Results chain is helpful analytic tool ⢠PCE knowledge of Global Fund takes time to develop but now seeing capacity & agility to respond to emerging issues ⢠Balancing competing priorities and multiple stakeholders is challenging ⢠Difficulty with timely feedback when evaluating processes that happen once during the grant cycle â findings relevant in 3 years PCE Team Structure ⢠Strong linkages between global and country evaluation partners is essential ⢠Various staffing models among GEP and CEP â but tracking 3 diseases requires sufficient people for embedded evaluation model ⢠Opportunities for cross GEP/CEP learning: in- person, webinars, TERG meetings ⢠Relationship building with country stakeholders is critical PCE Reporting/Dissemination ⢠Dissemination needs to be aligned with critical data use periods ⢠Annual report deliverable may be inconsistent with stakeholder preferences; shorter, more frequent briefs likely to be better ⢠PCE teams lack knowledge translation expertise â this could help in dissemination findings and strengthening feedback loop PCE / TERG / Country Team Engagement ď§ TERG meetings and presentations require significant time and input (high transaction costs) ď§ Some inconsistencies in TERG feedback over time ď§ CT engagement early and often is critical to ensure PCE is helpful to CTâs work ď§ Stronger engagement with Global Fund Secretariat could help ensure added value and synergies, while avoiding duplication
- 45. Process ď§ Continued grant implementation monitoring and business model process tracking ď§ Greater use of root cause analysis to understand implementation barriers and facilitators ď§ âDeeper diveâ inquiries into linkages between activities and outcomes along the result chains to help explain observed trends, using thematic areas as possible analytic lenses ď§ Stronger emphasis on timely feedback to country stakeholders and use of PCE findings Plans for the PCE in 2019
- 46. Impact ď§ Differentiated approach by country and disease ⢠Extend analysis of results chains ⢠Additional indicators and paths ⢠Country-specific tailored analysis o Programs, populations or geographic regions of specific interest to the country ď§ Model-based impact analysis ⢠Statistical correlations between adjacent elements of results chains (i.e. inputs vs. outputs; outputs vs. coverage, etc.) ⢠Structural equation modeling where complete data at sub- national level are available ⢠Alternative (e.g. causal inference; epidemic) models where less complete data are available Plans for the PCE in 2019
- 47. Example results chain for malaria
Hinweis der Redaktion
- Results chain example found at slide 46
- Broad diversity of countries represented with grants of over $2.1b during the 2017-19 allocation Mix of Upper LMI, Lower LMI, and LI Mix of high impact and core portfolios 2/8 are COE (DRC, Sudan) 2/8 are AGYW priority countries (Uganda, Mozambique) 5/8 eligible for matching funds (Mozambique, Myanmar, Uganda, DRC, Senegal) 4/8 participating in CCM evolution pilot (DRC, Guatemala, Mozambique, Uganda)
- The Secretariat succeeded in approving âdisbursement readyâ grants on time in almost cases (Nov 2017 â Dec 2017). Depicted in Milestones slide as ochre coloured blocks The Secretariat was successful in the timely transfer of funds to PRs in most cases, closely aligned to the timing of grant approvals. Depicted in Milestone slide as dark red blocks. Matching funds were approval and aligned with main grant approvals in Myanmar and Mozambique (although this was not the case for DRC and Uganda). Depicted in Milestone slide as light blue blocks CTâs allowed for flexibilities: Pre-financing of commodities or some SR activities (Ug and DRC resp); rapid budget revisions (Ug); 3 month contract extensions and/or bridge funding (Guatemala, Senegal, DRC). Strong role played by CTs in helping address country specific start up issues acknowledged in at least half of the PCE countries .
- Most PCE countries spent the first 6 months starting up grants and closing previous grants â reported as time consuming in several countries (Sudan, My, Uganda). In Sudan, old grants needed closing before ânewâ continuation grants could start. Transitioning PR-ship to national entities led to implementation delays in Senegal, Cambodia, Sudan (more bureaucracy, tighter risk measures, re-contracting of staff etc) A major bottleneck in the majority of PCE countries was the selection, contracting and disbursement processes in-country (PR to SR). SR activities delayed due to relaunch selection process; lengthy PR/SR negotiations on role, scope of activities and budgets; size and complexity of grant implementation arrangements introduced e.g. in Sudan, 14 new SRs involved and subject to capacity assessments by new PR. Misalignment of M Funds approvals which had knock on effect of delaying grant activities, particularly for SRs and their contracts linked to MF approvals (eg Uganda). Note: selection and contracting of SRs is the area where GF guidance is least developed. Trade offs: PR transition: as GF increasingly shifts responsibilities to national programs/systems there may be trade offs on absorption, particularly during early implementation. PR/SR contracting: GF promotion of country ownership by assigning responsibility of SR selection and contracting to PRs (under CCMs). Trade offs between ensuring quality SRs are selected, time required for contracting and knock on effects for grant planning, implementation and performance.
- Source: Process tracking and document review in each country Clustering of ochre blocks to depict on time signing of disbursement ready grants Clustering of crimson blocks to depict timely first disbursement of funds (Secretariat to PR) Clustering of light blue blocks in Dec 2017 to depict Matching Funds approved with main grants (and funds incorporated into those grants) in two of the four PCE countries. Matching Funds approved for Uganda and DRC at least 4 months later with disbursements following 2.5 months later. SR contracting processes depicted by light grey bars indicating the duration of when multiple SR contracts were being signed. The darker grey is the duration of when 1st disbursements to multiple SRs were happening. The overlap in these durations is why thereâs a third shade of medium grey (indicating the overlap).
- Available financial data (which is only for January to June 2018) shows that budget absorption has been low HIV expenditure in Q1 and Q2 of 2018 was only 14% of the original budget across the eight PCE countries, with some modules (e.g. prevention for transgender people and PSM) not beginning at all Overall absorption of funds for malaria and TB grants was higher at 30% and 47%, respectively Absorption for RSSH was very low at just 7% [If not discussed above] The drivers of low absorption are numerous, including: Delays in SR contracting and subsequent implementation Grant architecture and grant management issues such as PR transition (Guatemala, Sudan, Cambodia, Senegal), SR selection/signing (DRC, Uganda, Senegal, Mozambique, Cambodia), variable Country Coordinating Mechanism (CCM) leadership (Guatemala, Mozambique), early budget revisions (Cambodia, Sudan, Uganda, Senegal), and burdensome administrative processes such as risk mitigation measures and grant closure processes (Mozambique, Myanmar, Uganda, Sudan). Further, a number of country-specific contextual factors played important roles in early grant implementation progress e.g. the cash and fuel crisis in Sudan. We do however understand that the absorption data looked worse than the situation in countries for two main reasons: In practice, core services (e.g. treatment services provided by national programs) did not stop in between grants Some expenditure that had been incurred was not reflected in the Global Fund collected data While some activities were slow to get going, they are now underway and absorption is increasing dramatically.
- Reflecting that the provision of core services did not stop between grants, early performance against grant coverage targets is mixed but generally high, with most countries achieving or nearly achieving targets on most performance indicators Among HIV indicators, the median percent target achievement (i.e. the âachievement ratioâ) was 79% across PCE countries For TB it was 96% For malaria it was 80% The high achievement in relation to performance indicators is primarily because the indicators are focused on coverage, outcome and impact metrics that relate to the overall national program performance (rather than output metrics that can be attributed directly to grant performance, such as continuity in their contribution to the national program) Of particular concern is the very low target achievement ratios for performance indicators focused on key populations, with a median achievement ratio of 15%. This may be partly explained by these performance indicators being some of the few that are focused on grant-specific outputs, where the implementation of activities is reliant on Global Fund funding.
- There is no/limited correlation between: financial reporting data, which shows that budget absorption is well below what was planned; and programmatic data on performance, which suggests that grants are performing relatively well in most areas. Note that this is short term budget absorption only. If low absorption sustained for longer it may begin to show a correlation As such, financial reporting data and programmatic data on performance are often misaligned with short-term grant performance and implementation progress. However, these form the basis for developing grant ratings, which influence Annual Funding Decisions and are reported to the Board. The lack of correlation between the two: creates perverse incentives to implementers, by placing an over-emphasis on absorption, which encourages implementers to budget for activities that can be spent quickly and easily, possibly at the expense of activities that might have more impact; and poses a risk to the Secretariatâs grant management function, and the Boardâs governance function, at least in the short term Work Plan Tracking Measures can be used to more accurately reflect short-term grant performance and implementation progress, but appear to be used inconsistently across countries. In our view, and while we understand the sensitivity around this issue given the Global Fundâs previous decisions not to collect this data, some of these issues could be overcome by collecting data on the actual outputs achieved through the implementation of grant activities. This would allow for better linkage of financial and programmatic data (i.e. on investments and results) and facilitate analysis of efficiency and effectiveness at the country level, which could inform programming Analysis of data along the results chain at the sub-national level would also facilitate more comprehensive and systematic analysis of equity across the Global Fund portfolio, which could also inform programming. In the context of countries increasingly targeting interventions to increasingly concentrated epidemics, this sub-national analysis will become ever more important.
- The Secretariat should consider trying to better link financial and programmatic data by collecting data on the specific outputs achieved through grant implementation, as well as collecting data at the sub-national level, at least for some indicators. This has the potential to facilitate improvements to grant performance monitoring functions, such as through more comprehensive and systematic analysis of VfM across the Global Fund portfolio, as well as through the development of more nuanced grant performance ratings which inform annual funding decisions.
- This finding raises questions around how best to promote more explicit gender and human rights integration into TB and malaria funding requests in the next cycle, where appropriate.
- Global Fund and country definitions of key and vulnerable groups broadly align, but some inconsistencies exist This raises questions around the appropriateness of the Global Fundâs KVP definitions, which have potential implications for investments for program targeting and coverage of KVPs (such as, is the Global Fund targeting the most appropriate KVPs for that country context, do we know the size of those populations, etc.?).
- Gender and HR dimensions not well understood Program managers see interventions targeting women (e.g., female drug users or malaria prevention for women and children) as gender responsive, but this kind of sex-specific targeting (while complementary) is distinct from gender-responsive programming which addresses gender-related vulnerabilities Exception is DRC (SASA! pilot) where there is a real attempt to reduce vulnerability of adolescent girls and young women to GBV by working in the community, schools, and health centers to change harmful social norms â if implemented as designed/intended, which is not always the case. In countries like Myanmar and Sudan, there was limited recognition of gender inequity in disease risk and burden Absent a concrete focus on gender-related vulnerabilities, it may be difficult for Global Fund programming to have a significant impact on dismantling gender barriers More detail on DRC SASA! if done properly, SASA! falls into the category of gender-transformative approaches. The full SASA! approach is designed that way, and the âgender communityâ looks at it that way. There is a risk that SASA! âadaptationsâ in different countries fall short of the full approach (i.e. sometimes people put up some posters and have a few community dialogues and not much more, and say weâre doing SASA!) and it therefore end up as more gender-responsive than actually gender-transformative. Depends on how theyâre doing it, and how well. Sub-recipient contracting delays hindered the launch of key and vulnerable populations, human rights, and gender-related interventions in most countries. PRs often rely on SRs with community experience to deliver KVP, HR and gender interventions In Cambodia, DRC, Mozambique, Senegal and Uganda, disbursements were delayed until Q2 or in some cases Q3 of 2018 Disagreement over implementation modalities and delayed arrival of matching funds Lack of SR engagement during grant making was commonly cited across countries as a barrier, with many countries suggesting earlier SR involvement to ensure alignment on activities, targets, and budgets
- Gender and human rights dimensions are not well-understood or discussed among stakeholders, causing delays in conceiving of, prioritizing and operationalizing initiatives. Human rights initiatives that require understanding new content and forming new partnerships (e.g. with the legal community) may fall to a lower priority because of the time associated with building new relationships and understanding the key concept. In many of the PCE countries, in part because of these delays, absorption was low in the first few quarters of 2018 for gender and human rights activities
- Gender and human rights dimensions are not well-understood or discussed among stakeholders, causing delays in conceiving of, prioritizing and operationalizing initiatives.
- Source: PUDRs Jan-Jun 2018 Myanmarâs relatively higher RSSH absorption is driven by the large proportion of the budget allocated to program management (salaries, transport, office costs). Guatemala not included here due to grants being at a different stage of start up PCEs will continue tracking implementation progress for RSSH activities in 2019, including whether absorption improves and if not, why not Linkage to TERG RSSH Thematic Review: CEPA: âEmphasis on absorption as performance metric creates perverse incentives for RSSHâ
- Source: PUDRs Jan-Jun 2018 Most grants with RSSH investments contain only one RSSH-related coverage indicator in the performance framework, usually related to either the timeliness or completeness of HMIS reporting. PCE countries only included coverage indicators for HMIS and PSM (Senegal & Guatemala only). The modular framework does not include RSSH indicators related to community systems or national health strategies. There was a suggestion from one of the Secretariat reviewers to examine the Workplan Tracking Measures for RSSH â this is certainly an area we can cross check in the PCEs during 2019 to understand the extent to which RSSH interventions are being monitored by WPTMs⌠and whether those measures are appropriate/sufficient. Finding consistent with TRP RSSH review and TERG RSSH Thematic Review: TRP: Weak RSSH performance metrics and limited RSSH indicators CEPA: Poor linkage between RSSH investments and outcome indicators in Modular Framework
- The Uganda case highlighted some difficulty in applying guidance from the RSSH information note / investment framework document which states direct investments should be crosscutting in nature (with the exception of HMIS/M&E, all of which should be captured as âdirectâ)
- Building on findings presented, we note challenges with both absorption and performance monitoring for RSSH investments. Absorption for RSSH investments was very low in most countries. In addition to the reasons noted above, concerns about using RSSH funding for activities without clear performance metrics emerged, furthering challenges around misalignment of activities and performance.
- TB Shortfall Larger than other diseases in percentage terms Mean percentage gap for TB: 45% Mean percentage gap for HIV: 30% Mean percentage gap for malaria: 26%
- The Global Fund Secretariat should consider restructuring the country co-financing requirement to more ambitiously increase domestic expenditure on health and the three diseases, with a view to ensuring that domestic financing increases to a level that more fully supports transition and sustainability objectives. Specifically, this might involve: Expanding upon the co-financing requirement to better reflect the governmentâs existing financial commitments overall and within the wider health financing landscape. For instance, more explicit criteria related to government funding as a proportion of GDP, general government expenditure, total health expenditure and total disease program budget shortfalls, with the latter two including other donor contributions and private out-of-pocket (OOP) expenditures. Increasing the minimum level of co-financing that is acceptable to the Global Fund. We understand that the Secretariat has been reluctant to do this as under the current incentive mechanism, the consequence of failing to meet the requirement is that an equivalent value would be withheld by the Global Fund from a future disbursement. This would place uncertainty over a significant proportion of the grant, which could jeopardise Strategic Objective 1. In our view, while this concern is valid, it could be avoided by restructuring the incentive mechanism to reduce the value of grant funds at stake for non-compliance. This would however serve to weaken the incentive mechanism somewhat. Strengthening the incentive for countries to increase domestic expenditure on health and the three diseases beyond the minimum acceptable level of co-financing. The most striking mechanism to do this would be provide additional funding (i.e. over and above the budget allocation) to countries that exceed the minimum co-financing requirement (for instance by a certain percentage and ideally in a graduated manner). There are a number of potential ways that the Secretariat could do this: Increasing the Global Fund budget allocation for high performing countries in the next grant cycle period. Prioritising high performing countries for additional funding through their PAARs and/or catalytic funding, in particular, matching funds (which itself could be further targeted towards activities that promote sustainability objectives). Such an approach might allow for greater dialogue between the Secretariat and country stakeholders on short-term domestic funding expectations, as well as longer-term objectives of the Global Fund related to transition and sustainability. We do however acknowledge the need for flexibility in the setting of co-financing requirements and incentives, such as in COE settings and particularly where there are strong arguments to suggest that increasing domestic funding for the three disease will come with a substantial/unacceptable opportunity cost (for instance with a proportional decrease in domestic funding for other health priorities The provision of matching funds already requires countries to commit an equal level of domestic resources to areas of strategic importance to the Global Fund.
- Analysis of PQR data suggests that economy has improved over time across the grants, with health commodity prices falling, often below global reference prices. Across the eight PCE countries, prices paid for ARVs, TB medications and antimalarial medications largely decreased over time. In many cases, prices were at or below the global reference price (mean and median) by 2018. Around 65% of procured commodities are through the PPM, meaning that substantial volumes of commodities were procured outside of the Global Fundâs procurement systems. The chart shows the prices paid for Abacavir (one of the most common ARVs purchased) have substantially declined over time to well below the global reference price in 2016 and 2017 across the 8 PCE countries, although there are some instances of countries purchasing at above the reference price. This is just one example, but similar declines were observed for other commodities across diseases. There is evidence to suggest that the unit costs used as a basis for budgeting do not closely reflect the actual cost of inputs, posing a risk to both the economy and efficiency of Global Fund support. The clear incentive for stakeholders is to ensure that sufficient resources are available to implement the planned activities, in order to avoid requesting additional resources from the Global Fund and to meet grant targets within the agreed resource envelope. In Cambodia and Myanmar, the budgeted unit cost for inputs tend to be based on the estimates used for previous grant budgeting exercises, which are known to be inflated to allow for changes in prices and currency shifts and inflation, which can be significant when budgeting up to four years in advance. This can lead to: the Global Fund potentially paying above the lowest possible cost for inputs; and/or low budget absorption, where inputs are purchased at a lower cost than budgeted, creating savings that then need to be reprogrammed. For instance, in Guatemala, commodities were over budgeted by up to 72%. Others were however under budgeted. Whereas the quantity of inputs is scrutinized during grant making and linked to output targets, there is little evidence of input unit costs being subject to the same level of scrutiny. This is partly due to a lack of data on unit costs and service delivery costs available at the country level. The Global Fundâs support to the Global Health Costing Consortium (GHCC) seeks to address this issue by providing both a repository and reference prices for unit costs for HIV and TB interventions from published and grey literature. However, our review of this database suggests that cost estimates are only available in four out of eight PCE countries, three of which contained cost estimates for only 25 or fewer interventions, and many were extracted from studies more than 10 years old.
- Examples of efforts to improve efficiency of grant design. Grant management arrangements have been changed in a number of countries, in part to reduce program management and administration costs. For instance in Cambodia where procurement arrangements were changed, and in Uganda where a standalone RSSH grant was embedded in disease grants. Examples of efforts to improve efficiency of national program design. More targeted prevention services in Cambodia, Myanmar and Sudan; In DRC, bundled commodity distribution across programs is being implemented with positive expectations for efficiency, requiring fewer vehicles and human resources per shipment; Integration efforts across countries Program management costs. The budget for program management as a proportion of the total grant budget varies from 2% in Uganda to 25% in DRC and Cambodia, with no clear link between a countryâs stage along the development continuum and the program management budget. There is also no clear link between the size of grants and the proportion of the grant required for program management. Our analysis does show that program management costs are significantly higher for UN agencies and CSOs (which are mostly international and where costs are 26-27%) than for governments (where costs are 6%). In absolute terms, if the program management costs in Cambodia and Myanmar, where there are international PRs, could be brought down to a level equivalent for government PRs in other countries, this would result in a saving of around $30m. Other points on efficiency: Efficiency of grant implementation covered under business model Initial analysis suggests that the efficiency of malaria programming is improving, with the cost per malaria case treated declining in DRC and Uganda over time. We intend to advance this sort of analysis, including explanations on what these trends represent and how and why they have occurred, in the year to come. Cost-effectiveness considerations inform program design and decision making in most settings (such as through modeling) but not systematically There is some evidence in all countries of grant interventions being targeted to correspond with burden of disease and address the specific needs of key populations, although how this allocation has been made has varied considerably, for instance with some countries not using the outputs of any analysis as the basis for programming, particularly for TB and malaria.
- While equity is often discussed, trade-offs between equity, cost-effectiveness and programmatic targets are dealt with differently (often informally) In Mozambique and Sudan, while cost-effectiveness considerations are taken into account, we understand that these can be superseded by political decisions on the allocation of resources between states and regions (such as for TB, for which resources and commodities are simply allocated equally between states), and not systematically used as the basis for decision-making. More could be done to ensure that Global Fund-supported activities (and their benefits) are fairly distributed amongst target recipients Our findings in relation to KVP, human rights and gender suggest that the Global Fund grants have not been designed to sufficiently address barriers/issues to ensure universal access to health and disease-specific services. Removing these barriers is critical to fairly distributing the benefits of Global Fund support amongst target recipients (i.e. to improve equity by ensuring that the benefits of activities are fairly distributed among target recipients.). Our analysis of the correlation between Global Fund investments and the distribution of program interventions to burden of disease and KVP size estimates also supports the finding (although this does vary by country). Some evidence that over ambitious target setting vis-Ă -vis available funding has been counterproductive to the prioritization of hard-to-reach areas In Myanmar, there is some evidence that the Global Fundâs requirement to set ambitious targets (which stakeholders feel are overly ambitious) with a modest resource envelope meant that activities had to be focused in areas where TB burden is high, but also where the cost of service delivery is low. Because the Global Fund budget is insufficient to access all hard-to-reach areas (where the service delivery cost is high), prioritizing these areas would not allow for the achievement of Global Fund targets. Relatedly, in Sudan, the reduced budget coupled with ambitious targets for HIV KVP services has led to the need to trial innovative service models that have not been trialled fully and have yet to demonstrate their effectiveness, which poses some degree of uncertainty that the approaches may not be effective enough to reach the targets. Despite some examples of Global Fund support being used to reduce financial barriers to accessing services, this still poses a significant issue There are high out-of-pocket payments for health care in all PCE countries, except for Mozambique. Although this is lower for HIV, TB and malaria services than for other health services, these barriers can still be significant. Evidence in Myanmar suggests that the Global Fundâs support for the living costs of TB patients has been a critical factor in reducing the (often significant) financial barriers associated with accessing TB services and is associated with Myanmarâs high MDR-TB treatment success rates as compared to regional and global levels. In Uganda as well, over US$1.7 million has been allocated to socioeconomic approaches in their combined HIV/TB grant, the effects of which will be explored as implementation begins.
- The Secretariat should expedite work to collect unit/service delivery costs at the country level and use this as a basis for budgeting, with clear guidance on appropriate formulae to inflate estimates to allow for inflation, price changes, currency shifts, etc. The Secretariat should consider ways to strengthen country-level and/or grant-specific analysis of VfM (while seeking not to add to the burden of reporting), such as by: Collecting and analysing grant-specific output data, at least for some indicators. This would ensure that reporting of programmatic data more closely reflects actual grant implementation performance (i.e. to strengthen the linkages between financial and programmatic data), and would enable some quantitative analysis of efficiency and effectiveness Extending reporting tools to collect sub-national data; and creating performance targets that better address equity considerations. We expect both of these to become increasingly important as epidemics become increasingly concentrated, resources reduce and prevention activities are more and more targeted. Requesting that PRs/countries report against quantitative trends for some indicators as proxies for efficiency and effectiveness, with qualitative explanations of what the trends represent, and how and why the observed trends occurred
- Country examples for bullet 1: DRC: PCE team shared timely findings from their data quality assessment with the Director and Data Manager of PNLP Uganda: PCE team identified operational challenges during subnational data collection activities that were shared with national MOH officials and thereby promptly addressed Synthesis point for bullet 4: found in Cambodia and Myanmar in 2017/18 at dissemination meeting Added value in being present and different to LFA and OIG â these seems to have been important in Myanmar.