1.
Performance Information Use:
Cognition & Design
Donald Moynihan
Washington DC, May 15, 2015

2.
Research & Practice
• We are surrounded by data – what do we do
with it?
• What’s the role of research?
– Practice and reforms outpace research
– We “do” before we “know”
– Research is better at explaining what is, than what
can be

3.
Two factors that matter
• Cognition: how individuals process performance
information
– Strong evidentiary basis on biases & preferences
– Cannot control, but can be aware of, compensate for
• Design: how organizations structure how
information is received and used
– Weak evidentiary basis – case studies & surveys
– Can more directly control
– Some cognitive insights can inform design

4.
COGNITION: BIASES, HEURISTICS
AND PREFERENCES
It aint what you don’t know that gets you into trouble, its
what you know that just ain’t so - Mark Twain

5.
Negativity bias
• Loss aversion
– People are more motivated by loss than
equivalent gain
– We see this in how media chooses to cover public
sector

6.
Negativity bias
• Citizens evaluate services more negatively if
the same performance data is presented in a
negative rather than a positive terms
– if individuals are told that 10% of patients are
dissatisfied, they are more likely to be critical of a
program than if told that 90% are satisfied (Olsen
2015)

7.
Negativity bias
• Public more critical of UK political incumbents
when performance data is labeled as poor, but
do not provide equivalent credit when
performance is good (James 2011).
• Less votes for incumbent local govt officials
with low performance scores, but positive
performance scores are not equivalently
rewarded (Boyne et al 2009).

8.
Negativity bias in resource decisions
• Limited effect of Bush-era program
performance scores concentrated on poor
performers (Gilmour and Lewis 2006; Rhee
2014)
• Punishing poor performers might be
counterproductive:
– Low NCLB scores lead to lower success in
referenda on school levies, which in turn lower
performance (Kogan, Lavertu and Peskowitz 2015)

9.
Partisanship
• Conservative elected officials more likely to
believe in logic of personalized responsibility –
that leaders control outcomes (Nielsen and
Moynihan 2015A)
• Ideological distance affects how performance
data is used:
– Agencies defined as liberal tended to receive
lower performance scores, more scrutiny, more
improvement plans than conservative peers
(Lavertu, Lewis & Moynihan 2013)

10.
Advocates can mute negativity bias
• Advocacy can be successful…
– The County Sheriffs Office is responsible for a wide variety of law
enforcement activities. The Narcotics Task Force focuses on illegal drug
use. A number of media reports in the last year have focused on an
increase in drug-related crime, and have been critical of local law
enforcement. Treatment: The County Sheriff acknowledges that drug
use and related crime may be up, but says “We usually see this sort of
thing spike when the economy is in recession. This means that the
Narcotics Task Force needs more money to tackle this problem.”
(Moynihan 2015)
• …but might depend on audience
– Comments by teachers unions criticizing test scores reduced tendency
to hold school principals personally accountable, but only for liberal
elected officials (Nielsen and Moynihan 2015b)

11.
Expectancy disconfirmation
• Expectations about appropriate level of
performance affect how actual performance
interpreted
• Targets anchor expectations about what
constitutes good or bad performance
• Experiment:
– Control: job training program showing good
performance
– Treatment: Addition of performance targets, which
are higher than actual performance

12.
Result: Missed targets (even unrealistic ones)
weakens support for programs (Moynihan 2015)

13.
Leadership attribution bias
• Organizations are complex systems, outsiders
need simple explanations
• Leader a personal heuristic
– We tend to find personalized explanations more
salient than situational ones
– Leaders are easily accessible explanation
– The organization becomes understood through a
person
• Need for explanation increases in context of
unusual outcomes

14.
Elected officials more likely to believe school principals
cause performance when performance high or low
(Nielsen and Moynihan 2015a)5.8
6
6.26.46.6
-.5 0 .5-.4 -.3 -.2 -.1 .1 .2 .3 .4
Performance Level
Predicted values of leadership attribution
L
e
a
d
e
r
s
h
i
p
a
t
t
r
i
b
u
t
i
o
n

15.
Data sources may affect credibility
• Public find data showing very good
performance more credible when it comes
from independent source, rather than agency
itself (James and Van Ryzin 2015)

16.
DESIGN
Any jackass can kick down a barn, but it takes a good
carpenter to build one - Sam Rayburn

17.
From case & survey-based studies
• Performance information use higher:
– Culture that values mission or innovation
– Leader is committed to performance, articulates
compelling vision
– Goal clarity
(Moynihan & Lavertu 2012)

18.
• Problem with governmentwide
reforms: can’t use experimental
methods
• Can we find reasonable
“treatment” for reform?
Effect of governmentwide
performance reforms

19.
Findings
• Involvement in Government Performance and
Results Act, or Program Assessment Rating
Tool not associated with purposeful use
• Involvement is associated with passive use

20.
Learning forums
• Governments built performance systems on
routines of measuring and disseminating data
• Routines of performance information use an
afterthought
• Case studies: Organizations that use performance
data a lot tend to have learning forums
– Routine discussions of data focused on goal
achievement, mixes data and experiential knowledge
(Moynihan 2008;Hatry and Davies 2011; Behn 2014)

21.
Learning routines: quarterly reviews
• Is exposure to quarterly reviews mandated by
Modernization Act associated with learning?
• Evidence from GAO (2012-2013) survey
• Exposure to quarterly reviews associated with
higher performance information use
(Moynihan and Kroll 2014)

22.
A well-designed learning forum
• Meetings take place on a routine basis
• Focus on important goals
• Agency leaders are involved and seen as committed
• Multiple level of employees facilitate learning and problem solving
• Need appropriate and timely information
• Need staff and technological capacity to analyze data
• Quality data (reliable, accurate, valid, disaggregated to the right
level, comparative)
• facilitates analysis
• Follow-up on issues raised in prior meetings
• Positive reinforcement
• Constructive feedback
• Reviews establish process of analysis

23.
Learning forums: quarterly reviews
• When quarterly reviews are well run,
association with performance information use
even stronger (Moynihan and Kroll 2014)

24.
Managerial autonomy and
performance information use
• Basic claim: autonomy increases incentive to
use performance data
• Experimental test from Denmark: some school
principals given more autonomy on hiring
than others
• Those granted more autonomy more likely to
download performance data seven months
later

25.
Motivating effects of comparison
• Provision of comparative data (how you
ranked compared to other schools) made
principals more likely to download data

26.
PRACTICAL LESSONS?

27.
Guard against inherent tendency to…
• Use measures punitively
• Overvalue importance of leaders
• Be more critical in use of data for programs
we disagree with

28.
Design Principles
• Present performance in terms of achievement
not failure
• No unrealistic targets
• Look to work with independent sources to
communicate performance
• Advocate for your program, explain outcomes
• Construct learning routines & run them well
• Build in autonomy
• Compare with others

29.
Conclusion
Welcome your feedback and questions
Google: Performance Information Project
dmoynihan@lafollette.wisc.edu
@donmoyn

30.
References
Boyne, George, Oliver James, Peter John, Nicolai Petrovsky. 2009. “Democracy and
Government Performance: Holding Incumbents Accountable in English Local
Governments.” Journal of Politics 71: 1273–1284. http://bit.ly/1tOHsFj
Calmar Andersen, Simon and Donald Moynihan. 2015. Bureaucratic Investments in
Expertise: Evidence from a Randomized Controlled http://bit.ly/1L45Eax
Gilmour, John B., and David E. Lewis. "Assessing performance budgeting at OMB: The
influence of politics, performance, and program size." Journal of Public Administration
Research and Theory 16.2 (2006): 169-186. http://bit.ly/1HjkziQ
Lavertu, Stephane, David Lewis and Donald P. Moynihan. 2013. “Administrative
Reform, Ideology, and Bureaucratic Effort: Performance Management in the Bush Era.”
Public Administration Review 73(6): 845-856. http://bit.ly/1AFfgVU
James, Oliver. 2011. “Managing Citizens’ Expectations of Public Service Performance:
Evidence from Observation and Experimentation in Local Government.” Public
Administration 89(4): 1419–1435. http://onlinelibrary.wiley.com/doi/10.1111/j.1467-
9299.2011.01962.x/abstract
Kogan, Vladimir, Stéphane Lavertu, and Zachary Peskowitz. 2015. Performance
Federalism and Local Democracy: Theory and Evidence from School Tax Referenda
American Journal of Political Science bit.ly/1K5URiB

31.
Moynihan, Donald P. “Uncovering the Circumstances of Performance Information Use: Findings from an
Experiment.” Public Performance and Management Review. http://bit.ly/1A2IXTw
Moynihan, Donald P. and Stéphane Lavertu. 2012. “Does Involvement in Performance Reforms
Encourage Performance Information Use? Evaluating GPRA and PART.” Public Administration Review 72:
592-602 http://bit.ly/1H5Fa4J
Moynihan, Donald P. and Alex Kroll. 2014. Performance Management Routines that Work: AN Early
Assessment of the GPRA Modernization Act. La Follette School Working Paper No. 2014-005
http://bit.ly/1LFfoK4
Olsen, Asmus. 2015. Citizen (Dis)satisfaction: An Experimental Equivalence Framing Study. Public
Administration Review 75(3): 469-478 http://onlinelibrary.wiley.com/doi/10.1111/puar.12337/abstract
Nielsen, Poual Aees and Donald Moynihan. 2015. Elected Officials’ Assignment of Bureaucratic
Responsibility for Performance: A Romance of Leadership? http://bit.ly/1EI0MCJ
Nielsen, Poual Aees and Donald Moynihan. 2015b. Biases and Governing: Experimental Evidence from
Elected Officials Use of Performance Data. Paper to be presented at 2015 Public Management Research
Conference.
Rhee, Dong-Young. "The Impact of Performance Information on Congressional Appropriations." Public
Performance & Management Review 38.1 (2014): 100-124
http://www.tandfonline.com/doi/abs/10.2753/PMR1530-9576380105.
Van Ryzin, Greg and Oliver James. 2015. Incredibly Good Performance: . American Review of Public
Administration An Experimental Study of Source and Experimental Effects on the Credibility of Good
Government http://bit.ly/1EIshvN