Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
WHAT ARE INDICATORS OF
THE STRENGTH OF PUBLIC
MANAGEMENT SYSTEMS?
ISPMS BACKGROUND,
PURPOSE AND APPROACH
Initial motivation behind ISPMS
2

Lots of $ spent without
much results

Ambitious/costly
initiatives not sustained

Gaps ...
Main purposes of ISPMS
3

Provide a comprehensive and comprehensible set of indicators that:


Strengthen government owne...
ISPMS Approach
4

Overarching principle: Draw on existing efforts and avoid duplication
Define a set of
systems

Identify ...
Setting the scope: Defining a set of public
management systems and cross-cutting themes
5

The focus is on upstream instit...
ISPMS Criteria
6

ISPMS should meet 4 “utility” criteria…
Criterion

Definition

1. Action-worthy

We know (or strongly be...
The process: how the criteria were applied
7

Over 750 indicators
1. Identify potential
datasets
(see datasets)

2. Check ...
Existing indicators that meet criteria
8

PFM

Shortlist of most “action-worthy”

Public Adminsitration and Civil Service
...
Building consensus: ISPMS/AGI Governance
Global Steering
Group
(Donors, researchers,
international
organizations,
internat...
Steering Group Members
10

Organization
German Ministry for Economic Cooperation and Development (BMZ)
UK Department for I...
Status and Next steps
11

Phase 1: 2012-2013



Launched Secretariat
Developed dataset of existing
indicators that met t...
12

Additional details
•
•

Examples of the uses of ISPMS
Details on the process used to identify
existing indicators that...
Purpose of ISPMS
Strengthen government ownership of reform: example of PEFA
13

Mackie and
Caprio
(2011)
surveyed 11
count...
Purpose of ISPMS
Support donors in meeting Busan commitments
14






Ongoing efforts to revise indicators monitoring t...
Purpose of ISPMS
Improve project effectiveness: setting time-bound targets
What are reasonable
expectations for how
much a...
Purpose of ISPMS
Improve project effectiveness: hold donors accountable
16

Corporate Scorecard Tier II Indicators : Count...
Purpose of ISPMS
Improve targeting of reforms: providing empirical evidence to test
assumptions
17

Do countries with
high...
The Process
Step 1-2: Identify potential datasets and check replicability,
feasibility and actionability
18

Afro/Arab/Asi...
The Process
Step 3: Apply “action-worthy” criterion
19

1.

At the first stage the ISPMS Secretariat mapped to
CPIA standa...
The Process
Step 4: Apply “actionable” criterion
20



An indicator is “actionable” if it meets any of the
following stan...
The Process
Step 5: Apply “behavioral” criterion
21



Two steps:
Does the indicator measure de facto changes?

1.


Doe...
The Process
Step 6. Technical Expert Groups (TEGS) review and refine
22

PFM TEG

• Action-worthiness reviewed based on ev...
Nächste SlideShare
Wird geladen in …5
×

ISPMS Background, Purpose and Approach

3.412 Aufrufe

Veröffentlicht am

What are Indicators of the Strength of Public Management Systems? ISPMS Background, Purpose and Approach

Veröffentlicht in: Technologie, Business
  • Als Erste(r) kommentieren

ISPMS Background, Purpose and Approach

  1. 1. WHAT ARE INDICATORS OF THE STRENGTH OF PUBLIC MANAGEMENT SYSTEMS? ISPMS BACKGROUND, PURPOSE AND APPROACH
  2. 2. Initial motivation behind ISPMS 2 Lots of $ spent without much results Ambitious/costly initiatives not sustained Gaps exist in coverage of current indicators across themes, countries and time Failure of prior indicator development efforts Desire to provide overall view of what government looks like in a country New thinking shifted debate from form to function Better set of metrics would improve project effectiveness Many current indicators focus on de jure aspects World Bank PSM Approach 2011 World Bank IEG 2008 report
  3. 3. Main purposes of ISPMS 3 Provide a comprehensive and comprehensible set of indicators that:  Strengthen government ownership of reforms   Support donors in meeting Busan commitments   Help governments monitor and evaluate the results of reforms (see an example) Enable donor decisions to use country systems by providing information about their strengths and weaknesses (see an example) Improve project effectiveness    Help set time-bound targets for progress, which is particularly important for results-based lending approaches (see an example) Strengthen donor accountability by helping to track impact (see an example) Improve targeting of reforms   Enable research to build the evidence base on which institutions matter and in which contexts they are feasible (see an example) Track regional differences and changes over time  Help identify outliers (countries and systems that deserve a closer look)
  4. 4. ISPMS Approach 4 Overarching principle: Draw on existing efforts and avoid duplication Define a set of systems Identify strategy to measure conceptually difficult areas Define a set of criteria Apply criteria to available indicators Build consensus Are there gaps? What stories can we tell?
  5. 5. Setting the scope: Defining a set of public management systems and cross-cutting themes 5 The focus is on upstream institutions at the center of government. Public Financial Management Procurement Tax Admin. Public Admin. and Civil Service Public Information Transparency, Accountability & Participation *The scope of the project may be expanded to cover more public management systems.
  6. 6. ISPMS Criteria 6 ISPMS should meet 4 “utility” criteria… Criterion Definition 1. Action-worthy We know (or strongly believe) that they contribute to results 2. Actionable They are amenable to government action and project interventions 3. Behavioral Focus on function, not form, and performance, not design 4. Replicable Generated transparently and can be reproduced by others …and a “feasibility” criterion. Criterion Definition Sustainability Should be amenable to cost-effective and regular collection
  7. 7. The process: how the criteria were applied 7 Over 750 indicators 1. Identify potential datasets (see datasets) 2. Check if datasets are feasible, replicable and actionable 3. Apply “actionworthy” criteria to indicators (see details) 6. TEGs review results and refine (see details) 5. Apply “behavioral” criteria to indicators (see details) 4. Apply “actionable” criteria to indicators (see details) 50% aren’t action-worthy 5% aren’t actionable 55% aren’t behavioral About 100 indicators meet all criteria
  8. 8. Existing indicators that meet criteria 8 PFM Shortlist of most “action-worthy” Public Adminsitration and Civil Service Don’t yet meet feasibility criteria Tax Procurement Public Information Systems* 0 5 10 15 20 Number of indicators that meet ISPMS criteria * Public Information Systems are a subset of Public Accountability Mechanisms. 25
  9. 9. Building consensus: ISPMS/AGI Governance Global Steering Group (Donors, researchers, international organizations, international NGOs) Technical Expert Groups (World Bank staff, donors, data collectors) Procurement Tax Administration Public Financial Management Secretariat (World Bank) Public Administration and Civil Services Systems AGI/ Public Accountability Mechanism
  10. 10. Steering Group Members 10 Organization German Ministry for Economic Cooperation and Development (BMZ) UK Department for International Development (DFID) Australian Government Overseas Aid Programme (AUSAID) Swedish International Development Cooperation Agency (SIDA) Canadian International Development Agency Asian Development Bank European Commission Global Integrity International Budget Partnership Transparency International Organization for Economic Cooperation and Development Government of Bangladesh World Bank
  11. 11. Status and Next steps 11 Phase 1: 2012-2013   Launched Secretariat Developed dataset of existing indicators that met the criteria   Download the full list of indicators and data at: http://go.worldbank.org/99F3L CSFR0 Financed by the German Ministry for Economic Cooperation and Development (BMZ) Phase 2: 2014-2015  Proposal with the following elements is under preparation:  sourcing new indicators and improving coverage through a “marketplace”  incubating indicators in areas that are conceptually difficult to measure  exploring the feasibility of administrative data sources
  12. 12. 12 Additional details • • Examples of the uses of ISPMS Details on the process used to identify existing indicators that meet the criteria
  13. 13. Purpose of ISPMS Strengthen government ownership of reform: example of PEFA 13 Mackie and Caprio (2011) surveyed 11 countries during 2009 and 2010.  9 out of 11 countries surveyed use PEFA to take control of their PFM reform agendas LICs use to benchmark status of PFM systems, becoming a centerpiece of dialogue with Budget Support donors  MICs use to inform national-level PFM reform processes 
  14. 14. Purpose of ISPMS Support donors in meeting Busan commitments 14    Ongoing efforts to revise indicators monitoring the strength and use of country financial and procurement systems. ISPMS work on PFM and procurement will feed into this process Research on action-worthiness can help deepen learning on determinants of success
  15. 15. Purpose of ISPMS Improve project effectiveness: setting time-bound targets What are reasonable expectations for how much a wage bill can be decreased over a 5 year period? -In most countries, the wage bill didn’t change much or even increased slightly. The top 20% of countries experienced a decrease of about 10%. -Donors and partner countries can use this information to set realistic targets for project outcomes. Upper quintile 15
  16. 16. Purpose of ISPMS Improve project effectiveness: hold donors accountable 16 Corporate Scorecard Tier II Indicators : Country Results Supported by the World Bank Success rate is derived by looking at ISPMS in projects Institutions and Governance Civil Service No of Countries with WB Projects Success Rate No of Countries with Tax Administration WB Projects Success Rate Public Financial Management No of Countries with WB Projects Success Rate 44 55% 12 67% 26 73%
  17. 17. Purpose of ISPMS Improve targeting of reforms: providing empirical evidence to test assumptions 17 Do countries with higher transparency of taxpayer obligations collect more taxes [as % of GDP]?
  18. 18. The Process Step 1-2: Identify potential datasets and check replicability, feasibility and actionability 18 Afro/Arab/Asian barometer Medium-Term Expenditure Framework Dataset Bertelsman Transformation Index Public Investment Management Country Policy and Institutional Assessments (CPIA) BEEPS Institutional Profiles Database Regional tax administration datasets Wage and Bill Pay Compression Doing Business International Budget Practices and Procedures Database Transparency International Global Corruption Barometer Quality of Government Public Accountability Mechanism Enterprise Surveys Methodology for Assessing Procurement Systems (MAPS) OECD Comparative Information Series (Tax) Public Expenditures and Financial Accountability Human Resource Management AGI World Bank GAC Diagnostic Surveys Global Integrity Indicators Open Budget Survey Evans and Rauch IMF Government Financial Statistics/ fiscal decentralization IAMTAX ILO Public Sector Employment Data Replicability: Does the dataset have an institutionalized review/cross-checking mechanisms? (100%) Feasibility: Have data been collected twice for at least 20 countries? (orange datasets don’t meet feasibility criterion) Actionable: Are disaggregated data available by country (i.e. not just composites)? (yellow don’t meet criterion)
  19. 19. The Process Step 3: Apply “action-worthy” criterion 19 1. At the first stage the ISPMS Secretariat mapped to CPIA standards  2. About 50% of indicators didn’t map to CPIA standards Technical expert group leaders reviewed list and compared to widely accepted conceptual framework (e.g. PEFA for PFM, MAPS for procurement), theory and empirical evidence
  20. 20. The Process Step 4: Apply “actionable” criterion 20  An indicator is “actionable” if it meets any of the following standards:  It has been used in World Bank or other donor projects before (and is therefore amenable to reform efforts)  It captures a well specified concept/is specifically worded  About 5% of indicators didn’t meet actionability criterion
  21. 21. The Process Step 5: Apply “behavioral” criterion 21  Two steps: Does the indicator measure de facto changes? 1.  Does the indicator measure performance rather than design features? 2.    De jure indicators measure the contents of government documents (laws, regulations, etc.), such as the Public Financial Management Law or Recruitment Regulations (e.g. policy requires an e-procurement portal) Design = a system is in place (e.g. IT systems and interface for eprocurement portal established) Performance = to what extent is the system being used (e.g. % of transactions conducted through portal OR (further down the results chain) cost savings resulting from use of e-portal) 55% of indicators didn’t meet the behavioral criterion
  22. 22. The Process Step 6. Technical Expert Groups (TEGS) review and refine 22 PFM TEG • Action-worthiness reviewed based on evidence, theory, PEFA framework, used to develop shortlist • List of existing indicators proposed with new indicators to fill gaps Tax TEG • Action-worthiness reviewed based on theory; consensus being built via TADAT (work in progress) • List of existing indicators proposed, new indicators likely to be developed by TADAT PACS TEG • Framework proposed based on intrinsic value, currently soliciting comments • Limited existing indicators meet criteria, expanded set to those that don’t yet meet feasibility criterion Procurement TEG • Currently drafting a discussion paper proposing a framework to determine action-worthiness and the application of criteria to develop a list of existing indicators AGI/PAM TEG •Suggested expanding indicator set to focus on: Transparency, Rule of Law, Non-executive accountability institutions, and costs of corruption •April 2013 in-person workshop to discuss methodologies for measuring difficult concepts •Analytical paper drafted with discussion of concepts, trade-offs, and action steps for measuring/assessing governance

×