Métricas que Marcan la Diferencia: Uso de los Indicadores de Bienestar y del Desarrollo Sostenible en América Latina y el Caribe/Metrics that Make a Difference: Policy Uses of Well-being and Sustainable Development Indicators in Latin America and the Caribbean, 23-24 October 2019, Bogotá, Colombia. More information at: www.oecd.org/statistics/lac-well-being-metrics.htm

1. Sustainable Development Indicators—Too
Few or Too Many?
Ravi Kanbur
Bogota, Colombia
24 October 2019

2. Outline
• Introduction
• Some History
• Too Few? From 1 to 3 to 8 to 17 (and 232)
• Too Many? Policy Bandwidth and Statistical
Capacity
• What to Do? The Atkinson Principles
• Conclusion

3. Introduction
• We all know that there are 17 SDGs, (169
Targets), and 232 indicators.
• Is this too few? Or too many?
• In answering these questions I want to first
look at how we got here, then at how best we
might proceed now that we are where we are.

4. • Let me, however, put my cards on the table.
• I have spent much of my career arguing first for
going beyond per capita income to income
poverty and inequality, and then for going
beyond income to non-income dimensions of
wellbeing.
• But I am now beginning to think about whether
and in what precise sense we may have too many
dimensions, and too many indicators, in the
policy discourse.

5. • In this presentation I use as a platform:
• Ravi Kanbur, Ebrahim Patel and Joseph Stiglitz,
“Sustainable Development Goals and the
measurement of economic and social
progress,” in J.E. Stiglitz, J-P Fitoussi and M.
Durand (Eds.), For Good Measure: Advancing
Research On Well-Being Metrics Beyond
GDP.” OECD, 2018.

6. • A second publication on which this
presentation is based is:
• A.B. Atkinson, Monitoring Global Poverty,
2017, Report of the Atkinson Commission.

7. Some History
• A simple narrative, which one hears quite
often, is that in the bad old days before the
1990s the policy, and analytical, world was
ruled by GDP per capita.
• But the history of analytics, and politics, is not
quite so straightforward.
• Richard Jolly (1976) spoke of the
“enthronement of basic needs”. Morris (1980)
proposed a Physical Quality of Life Index.

8. • Actually, going further back from the 1970s
and to the period before the war, direct non-
income measures of wellbeing were more
prominent in policy discourse.
• This is because they were more easily
available through administrative statistics eg
birth and death registration) than national
income, which is pretty much a post-second
world war phenomenon.

9. • Winston Churchill famously said
• “…..when I call for statistics about the rate of
infant mortality, what I want is proof that
fewer babies died when I was Prime Minister
than when anyone else was Prime Minister.
That is a political statistic.”

10. • The quote can be used to illustrate many things,
including the deeply political nature of seemingly
technical aspects of wellbeing measurement.
• But here I highlight the fact that the statistic
referred to is not income, but infant mortality.
• It was more readily available through the birth
and death registration system, while national
income calculations were still in their nascency.

11. • The use of administrative statistics on
education and health was in fact quite
prevalent before the second world war,
including in the colonies of the European
powers.
• From 1866 onwards, the British Parliament
mandated a report entitled “Statement
Exhibiting the Moral and Material Progress
and Condition of India.”

12. • The 1906-1907 edition of the Report was
written by none other than John Maynard
Keynes, then a young civil servant in London
working in the India Office.
• The Keynes Papers in the Archives of Kings
College Cambridge have an exchange between
Keynes and his boss, with the boss
commenting on a part of Keynes’s draft as
follows:

13. • “The only part about which I am exercised is
that which deals with plague. The facts are
stated quite correctly, but the impression they
produce is that the country has been ravaged
by plague, and that the Govt has done
nothing….As we know, the only people who
read the….Report are persons who want to
find fault with the Govt…I should like to see
the paragraph marked A on pages 4-5
revised…”

14. • Note:
• (i) political use of statistics is nothing new;
• (ii) focus not on income, but on a health
indicator.

15. Too Few?
From 1 to 3 to 8 to 17 (and 232)
• Let us then move back to the 1990s and the
movement towards broadening out from
income.
• Despite non-income and multidimensional
antecedents, I believe the Human
Development index (HDI) in 1990 was a major
marker in the shift away from income as the
sole indicator of wellbeing.

16. • In an early exchange between the late
Mahbub Ul Haq, the founder of the HDI, and
Amartya Sen, Sen raised various technical
concerns about the index.
• In his response, Mahbub said that his object
was not to produce the perfect index, but to
set in train a movement towards consideration
of education and health as dimensions of
wellbeing on equal footing with income.

17. • I too had technical issues with the HDI
(Kanbur, 1990), but I have to say that although
I might have had a point technically, Mahboob
was right in substance in bringing the
multidimensional perspective center stage.
• By the time we got to the MDGs in 2000, 3
had gone to 8. And then with the SDGs in
2015, 8 had gone to 17, with 232 indicators.

18. • Apart from substantive arguments for
broadening, which were present in the earlier era
as well, the politics and pressures that led to this
massive expansion of scope with the SDGs are
clear.
• Each constituency argued for its own particular
goal to be represented in the overall list.
• Thus, for example, Doyle and Stiglitz (2014)
argued, with success, for inequality reduction to
be an explicit goal.

19. • The urban constituency got their goal, to
“make cities and human settlements inclusive,
safe, resilient and sustainable.”
• For constituencies that did not get “their own
goal”, the extent to which the SDG
document(s) mention their specific issue is a
key point of focus.

20. • Worries about being left out of the SDG
documents are illustrated by Jonathan Glennie’s
2014 article while the SDGs were being drafted:
• “The great danger in compiling a list of priorities
for international development….is the dreaded
“shopping list” or “Christmas tree”. This is where
everyone’s pet problem is included and we don’t
have a list of priorities at all, but a list of almost
everything wrong with the world….

21. • Nevertheless, there is a gaping hole.
Indigenous people are conspicuous only in
the fleeting nature of references to them.
In the draft of the SDGs released last
month by the open working group, they get
only two quick mentions….”
• Same story with many other groups eg
International Disability Alliance etc.

22. • And where they couldn’t get their priorities
enshrined in the “top level” goals, the focus
shifted to the indicators—to ensure that at
least their particular interests were in the
global indicators framework.
• Advocates believed that inclusion increased
the chance of progress in their area of
concern.

23. • In this sense therefore, there were, there are,
and there always will be, too few goals and
indicators.
• But are there, in another sense, too many?

24. Too many?
(I) Policy Bandwidth
• The implicit model, and also I think a correct
model, is that getting their issue at as high a level
as possible, and as often as possible, in the SDGs
would give advocacy groups a bigger slice of
“policy bandwidth” with which to advance their
agenda.
• An assessment of the overall outcome depends
crucially on whether we believe that policy
bandwidth is ever expandable or essentially
limited.

25. • If policy bandwidth is indeed limited, there is
a commons problem. More bandwidth for me
means less bandwidth for you.
• If on the other hand policy bandwidth is
expandable and has indeed expanded to fill
the needs of going from 1 to 3 to 8 to 17 (and
232), then we should not worry, or at least not
worry quite so much.

26. • Which assessment do we think is right?
• I do not know for sure, and I welcome
discussion on it.
• But my instinct is that policy bandwidth is
indeed limited, and if the injunction is really to
take all of these many dimensions and
indicators seriously, then we may have tipped
over the “too many” line.

27. Too Many?
(II) Statistical Capacity
• Of course one response to the 1 to 3 to 8 to 17
(and 232) process is to say that the last stage
merely provides a platform, a smorgersboard,
from which countries can pick and choose
whatever is suitable for them.
• Indeed this is how it is presented in the
documents and in high level statements.

28. • But the ground level reality is that there is a
danger of these 232 indicators being used as
the basis for a new conditionality by the
international community.
• This is particularly the case in low income
countries, especially in Africa, where statistical
capacity is limited and funds for statistical
services are also limited, making them
susceptible to “indicator conditionality”.

29. • The constraints and tradeoffs faced by statistical services
are present everywhere, but especially so in Africa. Here is
how a former Head of National Accounts for South Africa
puts it:
• “… getting GDP measures and its components right is not
trivial and there are many challenges that a middle-income
country like South Africa, let alone developing countries,
face in getting a set of conventional economic indicators
right… [S]hould we gear up our statistical infrastructure to
track, as accurately as we can, the business cycle or
sacrifice this for something else – like putting more
resources into estimating the value added of the informal
sector, conduct area sampling to better understand small
enterprises?”

30. • Despite all the statements to the contrary, and
despite all protestations that the international
community is really only trying to help the
national statistical services do the best job
they can, the broadening of the mandate
across many dimensions of wellbeing, and
across many indicators within each dimension,
is bound to bring additional pressure to bear
on already overburdened statistical services,
especially in low income countries.

31. • Finally, consider Statistical Capacity in a different
sense.
• “The cost of the assembly of the proposed
statistics... is not limited to the state budget. The
cost arises…in the diversion of the time of
farmers in a busy harvest season….The survey
process may have unseen costs via the generation
of fear and suspicion about the use to which the
information could be put.” (Atkinson
Commission).

32. • Are populations being “over surveyed”? If so,
they answering strategically, or simply to get rid
of the surveyer (think political polling).
• And what are the ethics of using poor people’s
uncompensated time to generate information to
satisfy the insatiable needs of an ever expanding
statistical agenda.
• “New sources of data, such as via the suppliers
of Internet services, may initially mask the
collection of data but later give rise to
widespread concern.” (Atkinson Commission).

33. What To Do?
The Atkinson Principles
• We have seen that the broadening of the concept
of wellbeing, and expansion of the number of
associated indicators, has a certain substantive
logic to it.
• At the same time the politics of international goal
setting has led to a massive expansion of
indicators, and a consequent pressure from the
international community, particularly on low
income countries, to deliver on what is in effect
an unfunded mandate.

34. • What can be done to reign in what often seems like an
indicator and statistical juggernaut?
• The Atkinson Commission faced a similar dilemma
when proposing a dashboard of indicators to
complement the standard income poverty measure:
• “…. a long list would be counterproductive….In order to
be effective, the list of indicators included in
the….portfolio has to be sufficiently short that the new
indicators get attention from the outside public and
from policy makers.”

35. • The Commission then proposed a set of
principles for selection of indicators:
• 1. The coverage of the indicator should be truly
global, covering the whole of the world
population.
• 2. The indicator should be transparent and
identify the essence of the problem.
• 3. The definition of the indicator should be
generally accepted as valid and have a clear
normative interpretation.

36. • 4. The indicator should be sufficiently robust
and statistically validated; there should be a
clear structure of accountability for tis
definition and construction.
• 5. Indicators constructed with global coverage
of countries should be cross-checked against
information available at the level of individual
countries.

37. • 6. Where indicators are either combined as in a
multi-dimensional measure, or presented in
conjunction as in a dashboard, the portfolio of
indicators should be balanced across different
dimensions.
• 7. The design of social indicators should,
wherever possible, make use of information
already available. Where new information is
needed, then I should be obtained, as far as
feasible, using existing instruments or by making
use of administrative data.

38. • These seven principles are not of course new
or novel.
• And they are put forward in the particular
context of the charge to the Commission.
• What is new perhaps is that such principles
“are not, however, typically made explicit.”

39. • One question the international community might
ask itself is:
• How many of the 232 indicators would survive
the scrutiny of these principles, or an equivalent
set of principles derived from the context of the
SDGs?
• Might a review of the global indicators
framework, with a view to reducing not
increasing their number, be appropriate as we
approach the half stage mark of the SDG time
horizon?

40. Conclusion
• The use of non-income wellbeing indicators
predates the SDGs, going back to colonial
times.
• In the modern era, the broadening of
perspective from income alone is obviously
welcome. 1 is clearly too few.
• But 232 indicators may be too many because
of policy bandwidth and statistical capacity
constraints.

41. • Despite myriad statements to the contrary,
international level pressures and ground level
realities raise real dangers of indicator
conditionality and statistical overload,
especially in low income countries.
• As the mid-term review of the SDGs
approaches, this perspective calls for a
substantive and process assessment of
whether in fact 232 is too many.

42. Thank You!