1. 1
Nick Manning
Jurgen Blum
May 3nd, 2012

2. • The numbers suggest quantity more
than quality
• But we have some rebuttals
• But, between us, we have some other
problems
• Where to from here?

3. 3

6. Percent of IDA and IBRD countries with CPIA improvements
80
70
60
50
40 Any relevant lending
no relevant lending
30
20
10
0
CPIA 13 - Quality of budget CPIAfinancial managementadministration revneue mobilization
and 15 - Quality of CPIA 14 - Efficiency of - Corruption, transparency and accountability
public CPIA 16

7. 7

8. Tax projects are the only projects that consistently perform above average
(compared to non-PSM projects) at statistically significant levels (5 percent
confidence level)
20.00%
15.00%
10.00%
15.40%**
5.00%
4.00%
2.00%
0.00% -1.00%
-7.00%*
-5.00% -9.00%*
-10.00%
-15.00%

9. Tax is the only contents characteristic that remains a statistically significant
predictor of project performance (IEG outcome ratings), when controlling for
other contextual and management variables
Percentage point change in project success rates associated with a 10 percent increase in PS thematic share
12
10
8
6
9.74**
4
2
0.883 0.42 1.34
0
-0.963 -1.99
-2
-4

10. Compared to non-PSM projects, (upstream) PSM projects are particularly
targeted to :
(i) the Africa (AFR, 30.5 percent) and Latin America and the Caribbean
(LAC, 30.5 percent) region (see Table 12) and
(ii) towards countries with significantly lower GDP per capita growth (1.6
percent for PSM versus 3.4 percent for non-PSM projects);
(iii) with significantly higher aid flows than non-PSM projects (11.4 versus
6.3 percent); (iv) with a slightly higher degree of political and civil
liberties (Freedom House ratings of political and civil liberties are about
0.25 to 0.3 points lower for countries with PSM projects, on a 7-point
scale, see Table 10).
(iv) By contrast, there is no conclusive evidence of a “needs-based”
upstream PSM project targeting towards countries with lower initial
administrative capability.

11. 11

12. We know what We are
formal uncertain
changes we about what
can make needs to
happen in
between!
We know the sorts of
results that we would
like to see

13. OECD has a set of propositions around these
behaviors/upstream performance competencies:
1. Workforce planning and management
2. Core values
3. Staff performance and capacity
4. Cooperation between levels of government
The HRM AGI work also had propositions:
1. Attract and retain required staff
2. Depoliticized, meritocratic HRM practices
3. Performance-focusing HRM practices
4. Fiscally sustainable wage bill
5. Ethical behavior
6. Effective working relationships with other cadres

14. 1. Management of operations within the core administration
a. HRM practices within the core administration are structured around education background and merit criteria
clearly and publicly defined. [Q15a1]
b. There are clear mechanisms within the public administration to ensure accountability of staff and avoid conflict
of interest and abuse of power. [15a2, Q16c1, Q16c2]
c. There is an effective arbitration system for violations of the HRM regime and its decisions are made publicly
available in a timely manner. [Q15a3, Q16b1]
d. The employment regime for the core administration is effective in attracting, retaining and motivating
competent staff. [Q15a4, Q15a5, Q15a6]
2. Quality management in policy and regulatory management
a. The policy process is transparent and ensures credibility in social and sectoral policy pronouncements.
[Q15b1]
b. There is quality and credibility in the actions of sector regulators. [Q15b2, Q15b4]
c. Sector regulators are held accountable for their decisions or lack of. [Q15b5]
d. Decisions and regulations are made public available in a timely manner. [Q16b1]
3. Coordination of the public sector HRM regime outside the core administration
a. In the public sector outside of the core administration, HRM practices are structured around merit criteria that
are clearly and publicly defined and the employment regime is effective in attracting, retaining and motivating
competent staff. [Q15c1]
b. There are clear mechanisms within the public administration to ensure accountability of staff and avoid conflict
of interest and abuse of power. [Q15a2, Q16c1, Q16c2]
c. The aggregate public sector wage bill is not high by comparison with other similar countries. [Q15c2]

15. We don’t know that they are correct!
(although they are not unreasonable!)

16. 16

17. Maybe replicate PEFA – balancing use with research
1. A rapid 50
expansion 40 7
18
since 2005: 19
30 14 13
135 17 5
20 9
countries, 32 13
5
10
5 10 3 15
23
12
23
11
assessments 0
8
in March 2005 2006 2007 2008 2009 2010
2012
Led by EC Led by Other Led by WB
2. Growing Nr. 135 first 52 second 8 third generation
of repeat
assessments
Note: PEFA assessments are shown by lead agency. The year refers to the main mission when most of the interviews for the PEFA assessment were conducted. The
number of assessments reported includes both "finalized" and "draft assessments". While every effort was made to take stock of PEFA Assessments
comprehensively, some assessments may have been omitted in the dataset.

18. 1. IEs provide great value-added for evidence-based reform design, by
rigorously identifying the effect of an intervention
2. The PSM IE research agenda is lagging compared to other sectors.
3. There is significant opportunity for expanding the IE research agenda on
Public Sector Governance – and the Bank should play a leading role in
this.
4. This is true despite a typically small “n” in PSM reform.
• By contrast to health or education reforms, that can be applied
selectively to schools or health centers to construct a
counterfactual, PSM reforms often focus on central agencies, such that
n is small or a randomized roll-out difficult.
• However, recent research demonstrates that IE methods can
successfully be applied to learn about the effects of upstream reform
elements – e.g. tax inspector incentives or the impact of performance-
related pay schemes.

19. Move towards more standardized clusters of project indicators to encourage peer learning
NOT: a straight-jacket! (not standardizing indicators)
BUT: A way of better peer-learning from projects through
standardized “menus” of indicators, e.g. organized by:
1. What is the specific reform area? (pay
reform, regrading, agency creation etc.)
ISPMS
2. Where in the results-chain does
the indicator measure?
19

20. 1. Focus on the functional problem, rather than the solution
2. Engage stakeholders in identifying functional problems
and binding constraints
3. Use political economy analysis prospectively
4. Use available evidence and accepted theory on whether20a
reform will fix the problem

21. Benefit from the results-based-lending instrument (PforR) for
PSM reform
The Sierra Leone Experience:
• A different, problem-focused engagement
process
• Results dialogue as a vehicle for
encouraging MoF and line-ministries to
engage in joint problem-solving
• Balancing quantity and quality in indicator
design is not easy
• Planning to build in Impact Evaluation
design
21

22. • The numbers suggest quantity more than quality
• But we have some rebuttals
• But, between us, we have some other problems
• Where to from here?
o Let’s find out more about what tends to work
in general – new metrics, impact
evaluation, standardized project indicator
buckets
o Let’s focus also on what is likely to work here
– better diagnostics and more flexible
instruments