Bath University taster event: Evidence-Based Decision Making

Improving decision-making
through evidence-based
management
University of Bath taster event – June 15, 2015 - London

Our mission for today:
By the end of this workshop you will be more concerned
about the way you typically make decisions as a
manager/leader/consultant, so you firmly resolve that,
within the next year, you will increase the ‘evidence-
base’ of your decisions by at least 25%.
You can do this, of course, by signing up for the
executive course ‘Evidence-based decision-making’ at
Bath University School of Management.

20 min EBDsc: What is it and why do you need it?
20 min Professional experience & judgment
10 min Organizational data
20 min Scientific evidence: mini case
Agenda

Exercise
Think about a decision you have been involved in
making. This decision should be one which:
 Was reasonably important for your organization
 Involved spending significant resources
 Involved several or more people
 Was made over a period of time (ie. weeks or
months)

Exercise
Discuss with your neighbor (2 min)
What exactly was the problem (or opportunity)?
How many alternative decision options were considered?
How much evidence was used, and from which sources
(scientific, organizational, experience, crystal ball?)
Was any attempt made to explicitly evaluate its quality or
trustworthiness?

Evidence based decision-making:
What is it?

Evidence-based decision making
Central Premise:
Decisions should be based on a
combination of critical thinking and
the ‘best available evidence‘.

Evidence?
outcome of scientific research,
organizational facts & figures,
benchmarking, best practices,
personal experience

All managers and leaders
base their decisions on
‘evidence’

But…many managers and
leaders pay little or no
attention to the quality of the
evidence they base their
decisions on

Trust me, 20 years of
management experience

Teach managers/leaders
how to critically evaluate the
validity, and generalizability of
the evidence and help them
find ‘the best available’
evidence

Evidence based decision
Professional
experience and
judgment
Organizational data,
facts and figures
Stakeholders’ values
and concerns
Scientific
research
outcomes
Ask
Acquire
Appraise
Aggregate
Apply
Assess
diagnosis intervention

Evidence based practice:
Where does it come from?

McMaster University Medical School, Canada
Medicine: Founding fathers
David Sackett Gordon Guyatt

1. Ask: translate a practical issue into an answerable question
2. Acquire: systematically search for and retrieve the evidence
3. Appraise: critically judge the trustworthiness of the evidence
4. Apply: incorporate the evidence into the decision-making process
5. Assess: evaluate the outcome of the decision taken
5 steps of EBmed

Evidence-Based Practice
1991 Medicine
1998 Education
2000 Social care, public policy
Nursing, Criminal justice,
Policing, Architecture, Conservation
2010 Management

Evidence-based decision-making
=
the use of critically appraised
evidence from multiple sources to
increase the likelihood of a
favourable outcome
Focus on the decision making process
Think in terms of probability

“Where the evidence is strong, we should act on it.
Where the evidence is suggestive, we should consider it.
Where the evidence is weak, we should build the knowledge
to support better decisions in the future.”
Jeffrey Zients, acting director of the
Office of Management and Budget and
President Obama’s Economic Advisor

Evidence-Based Decision-Making
Why do we need it?

1. Incompetent people benefit more from feedback than
highly competent people.
2. Task conflict improves work group performance while
relational conflict harms it.
3. Encouraging employees to participate in decision
making is more effective for improving organizational
performance than setting performance goals.
True or false?

How evidence-based are we?
 959 (US) + 626 (Dutch) HR professionals
 35 statements, based on an extensive body of evidence
 true / false / uncertain
HR Professionals' beliefs about effective human resource practices: correspondence
between research and practice, (Rynes et al, 2002, Sanders et al 2008)

Outcome: not better than random chance

Professional
experience and
judgment
facts and figures
and concerns
Scientific
research
outcomes
Ask
Acquire
Appraise
Aggregate
Apply
Assess

Thinking critical about
professional experience and judgment

Discuss with your neighbor
Why is a physician’s clinical experience,
as a rule, more trustworthy than
a manager’s professional experience?
60595857565453525150494847464544434241403938373635343332313029282726252423222120191817161514131211109876543210

Developing expertise
1. A sufficiently regular, predictable environment
2. Opportunities to learn regularities through prolonged
practice and feedback
The management domain is not highly
favorable to expertise!

Bounded rationality / prospect theory
System 1
 Fast
 Intuitive, associative
 heuristics & biases
System 2
 Slow (lazy)
 Deliberate,
 Reasoning
 Rational

Shepard’s tables
System 1: short cuts

System 1: dominant, cause necessary to survive
95%

 Seeing order in randomness
 Overconfidence bias
 Halo effect
 False consensus effect
 Group think
 Self serving attribution bias
 Sunk cost fallacy
 Cognitive dissonance reduction
System 1: cognitive errors
 Confirmation bias
 Authority bias
 Small numbers fallacy
 In-group bias
 Recall bias
 Anchoring bias
 Availability bias

1. Pattern recognition
2. Confirmation-bias
3. Groupthink
Cognitive errors that mess up decision making

We are predisposed to see order, pattern and causal
relations in the world.
Patternicity: The tendency to find meaningful patterns in
both meaningful and meaningless noise.
Error 1: pattern recognition

We are pattern seeking primates: association learning
Bias 1: pattern recognition

Points of impact of V-1 bombs in London

 A Type I error or a false positive, is
believing a pattern is real when it is not
(finding a non existent pattern)
 A Type II error or a false negative, is
not believing a pattern is real when it is
(not recognizing a real pattern)
Dr. Michael Shermer
(Director of the Skeptics Society)

 A Type I error or a false positive: believe that the
rustle in the grass is a dangerous predator when it is
just the wind (low cost)

 A Type II error or a false negative: believe that the
rustle in the grass is just the wind when it is a
dangerous predator (high cost)

 A Type I error or a false positive: believe that the
rustle in the grass is a dangerous predator when it is
just the wind (low cost)
 A Type II error or a false negative: believe that the
rustle in the grass is just the wind when it is a
dangerous predator (high cost)

superstitious
rituals
superstitious
rituals
more stress = more prone to type 1 errors

3. Groupthink
Cognitive errors

We are predisposed to selectively
search for or interpret information in
ways that confirms our existing beliefs,
expectations and assumptions, and
ignore information to the contrary.
In other words, we “see what we want to
see”
2. Confirmation bias

Example
You may believe that astrology actually
works. As a result of confirmation bias
you’ll remember only those instances
when when the prediction in the astrology
column came true and forget the majority
of the cases when the prediction was very
wrong. As a result you will continue to
believe astrology has some base in reality
2. Confirmation bias

Confirmation bias
Pattern recognition
Error 2: confirmation bias

McKinsey (1997 case study / 2001 book)

3. Groupthink
Errors

Groupthink:
Groupthink is a psychological
phenomenon that occurs within a
group of people, in which the desire
for harmony or conformity in the
group results in an incorrect or
irrational decision
Error 3: Groupthink

Lean Management / Lean Six Sigma
Self steering / autonomous teams
Agile working / New World of Working
Talent management
Employee engagement
Groupthink?

“I’ve been studying judgment for 45 years, and I’m no better
than when I started. I make extreme predictions. I’m over-
confident. I fall for every one of the biases.”
Bounded rationality

Professional
experience and
judgment
and concerns
Scientific
research
outcomes
Ask
Acquire
Appraise
Aggregate
Apply
Assess
facts and figures

1. financial data (cash flow, solvability)
2. business outcomes (ROI, market share)
3. customer/client impact (customer satisfaction)
4. performance indicators (occupancy rate, failure
frequency)
5. HR metrics (absenteeism, employee engagement)
6. marketing intelligence (brand awareness, customer
feedback)
7. ‘soft’ data (organizational culture, trust in senior
management, leadership style, commitment)
8. data from benchmarking
Types organizational evidence

Organizational facts and figures

People operate with beliefs
& biases. To the extent you
can reduce both and replace
them with data, you gain a
clear competitive advantage
Laszlo Bock (CHRO Google)
Organizational data

Can your organization correlate/regress
level of education
years of experience
productivity?
customer satisfaction?
failure frequency?
employee satisfaction?
employee turnover?
absenteeism?
+

In the next weeks,
before you make a decision, ask yourself:
 What exactly is the problem?
 What is the evidence available?
 Where does it come from (multiple sources?)
 How trustworthy is it?

 Identify the many other factors, rather than evidence, that shape
managerial decisions such as cognitive biases, politics, power,
management fads and the ideas of management gurus
 Distinguish between the four different types of evidence used in
evidence-based management
 Identify differences between your own and your organisation’s decision-
making and an evidence-based management approach to decision-
making
 Understand practical ways in which your and your organisation’s decision-
making can become more evidence-based and the potential benefits and
costs that may result.
 Learn how to critically evaluate the trustworthiness of evidence (including
assumed best practices, organisational data, findings from research etc.)
At the end of the course you will be able to

Bath University taster event: Evidence-Based Decision Making

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Bath University taster event: Evidence-Based Decision Making

Ähnlich wie Bath University taster event: Evidence-Based Decision Making (20)

Mehr von Center for Evidence-Based Management

Mehr von Center for Evidence-Based Management (17)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Bath University taster event: Evidence-Based Decision Making