My presentation from the recent RIMS Atlanta Educational Conference. The slides cover the most common biases we naturally inject into strategic decision making and risk assessment. In some cases awareness alone will serve as mitigation but the slides cover some direct response tactics as well.

1. Cognitive Bias
in
Risk-Reward Analysis
Damon Levine, CFA, CRCMP, MA
Atlanta RIMS Educational Conference
February 9, 2017

2. Disclaimer
• The author has not conducted original research on these biases nor
has he coined any terminology related to the biases
• The suggestions for mitigation or response to a particular cognitive
bias may or may not be effective depending on the details of the
specific circumstance, organization, or people involved
• No guarantees of improved risk identification, risk mitigation, risk
management, or performance are stated or implied
2

3. The “From the Hip” Pop Quiz (just answer!)
1. Estimate how many randomly selected people are needed to have 97%
likelihood of (at least) 2 people sharing a birthday? (e.g. “March 3”)
2. A bat and ball cost $1.10. The bat costs one dollar more than the ball.
How much does the ball cost?
3. A risk manager shows Management the 95th percentile point for
potential hurricane losses for a calendar year. This loss value is $150
million. An executive new to the company asks for the probability of
incurring $150 million in hurricane losses. Assuming the model is correct
what should the risk manager say?
3

4. A Thankless Job?
“Prevention is not easily perceived, measured, or rewarded; it is
generally a silent and thankless activity…
One can easily compute the costs [of mitigation] while the results are
hard to determine. How can one tell its effectiveness, whether the
measure was successful or if it just coincided with no particular
accident?
Job performance assessments in these matters are not just tricky, but
may be biased in favor of the observed ‘acts of heroism.’ History books
do not account for heroic preventive measures.”
-Nassim Taleb
4

5. Availability Bias
• In a randomly selected English word of three or more letters is it more
likely that the word starts with an R (“rope”), or that R is its third
letter (“park”)?
• For a question such as the one above (tough to answer quickly in a
purely scientific, well substantiated manner) we turn to heuristics and
use what’s available in our memory and experience such as facts,
impressions, anecdotes, intuition, etc.
• Many are able to recall/list more words beginning with R and
therefore reply that R more often appears as the beginning letter.
The “availability” to recall those examples leads to a bias, trusting
what’s “available” in memory as evidence, and the wrong answer
5

6. Hindsight Bias
• Hindsight bias “…after learning the eventual outcome, [subjects] give
a much higher estimate for the predictability of that outcome than
subjects who [consider] the outcome without advance knowledge.
Hindsight bias is sometimes called the I-knew-it-all-along effect.”
-Eliezer Yudkowsky, Machine Intelligence Research Institute
• Risk managers must not be swayed by emotion. A recent or severe
event tends to make our probability estimate larger than it should be
• Mitigation: remember the context and look at the appropriate time
horizon (at least capture where the conditions or environment are
relatively stable) and determine an empirical probability. Blend with
some amount of expert opinion and any new information. Revisit the
in-place assumptions about likelihood, causality and correlations prior
to the observed event and confirm, question, or revise those
concepts and parameters
6

7. Black Swan Challenge
1. The hypothetical event X is thought of as a “black swan” and its
annual probability is regarded as 1/50. Assuming year-to-year
independence, what is the approximate probability that event X occurs
at least once in the next 50 years?
A) 2% B) 18% C) 46% D) 50% E) 63%
2. A diversified company stochastically models annual earnings for each
of its 20 lines of business. Assuming independence between the lines,
what is the probability that at least 1 line will have actual earnings less
than its own 1st percentile value (a very bad result) ?
A) 3% B) 5% C) 12% D) 18% E) 23%
7

8. Hindsight & Availability Biases Cloud our View
of Black Swans
• We may fail to remember extreme events from the distant past that
would help inform our outlook regarding risk
• We may have vivid mental pictures of the devastation of recent “rare”
disasters and our severity estimates for future events can be warped
• It is possible to overestimate probability of black swans because of
recent and striking examples of risk manifestation
8

9. “Yep, I saw that coming!’
“Mistakenly believing that the past was predictable, people conclude
that the future is predictable. It has been said1 :
When we attempt to understand past events, we implicitly test the
hypotheses or rules we use both to interpret and to anticipate the
world around us. If, in hindsight, we systematically underestimate
the surprises that the past held and holds for us, we are subjecting
those hypotheses to inordinately weak tests and, presumably,
finding little reason to change them.” 2
1. Fischhoff, Baruch. 1982. “For Those Condemned to Study the Past: Heuristics and Biases in Hindsight.”
2. “Cognitive Biases Potentially Affecting Judgment of Global Risks”, Eliezer Yudkowsky, 2008.
9

10. On the Average…
• A bet has 2 outcomes: $10 with probability 0.7 and $100 with
probability 0.3. It has expected value or “average value” of:
0.7(10) + 0.3(100) or $37
• Averages (expected values) are sometimes useful and give an idea of
what the sample mean of a large set of observations might be
• In a game a fair 6-sided die is rolled. If a “2” or more is rolled, you
win $1M but if a “1” occurs, you pay $50,000. This game has an
expected value of $825,000. Would you play the game? What would
you pay to play? What about playing 1000 times before settlement?
10

11. A Misleading Average
• Each January 1, an investor uses leverage and derivatives to set up a
one-year time horizon position, P, whose maturity value at 12-31
depends on an underlying asset A
• There is very little upfront investment because of the leverage and use
of deep out-of-the-money options
• The asset A has a long data history and the average (mean) annual
price fluctuation is +1.4 (delta from Jan 1 to Dec 31)
• Position P is such that there is a large profit for price changes of at
least +1, a zero profit for a change between -2 and +1, and a very
large loss results when the change is worse than -2, i.e., (-∞ , -2)
• Does the average annual price change imply this is great strategy?
What role does risk appetite play here? Standard deviation?
11

12. Standard Deviation: A Remedy?
• Question: A quantity follows a specific, but unknown, statistical
distribution. It is of great interest to know when an observation three
(3) standard deviations below the mean may occur. How likely is this?
• Answer: We don’t know. (even knowing that the quantity follows
some named distribution does not allow us to answer)
• Better answer: Use Chebyshev’s inequality…
12

13. Chebyshev’s Inequality (one-sided version)
Chebyshev’s inequality is a nearly universal result which is powerful in
that very few assumptions are needed for its application. For any
random variable X with finite expected value (average) μ and finite non-
zero variance σ2 we have for any real number k>0:
P(X ≤ μ-kσ) ≤ 1/(1+k2)
Example with k=3: the probability of an observation being 3 or more
standard deviations below the mean is at most 1/(1 + 32) or 1/10. It is
important to note that this is a much larger probability than under a
normal distribution assumption.
13

14. Drawing different conclusions from the same information presented differently
[towergateinsurance.co.uk] 14

15. Framing Effect: Mitigation
• In public forums and the media, tone and terminology will influence
and be influenced by public opinion
• Separate facts from hearsay; separate facts from “expert opinion”
• Understand motives and conflicts of interest
• Remember the West Africa Ebola scare of 2014-2015
15

16. The Foggy Mirror
You emerge from the shower and stare into the foggy bathroom mirror.
You can just make out your reflection and, being in a playful mood,
outline your face with your finger. The resulting outline traced on the
mirror has a size which:
A) is approximately the same as your (actual) face
B) depends on the distance you stand from the mirror
C) is half the size of your face
D) is exactly the size of your face
16

17. Intuition: Be Careful of What You “Know”
• A bat and ball cost $1.10. The bat costs one dollar more than the ball.
How much does the ball cost? Solution: $0.05 (5 cents)
• “Mirrors Don’t Lie: Mislead? Oh, Yes.” - NY Times, July 22, 2008.
17

18. What was “Known” in the Past
• “Stocks have reached what looks like a permanently high plateau.”
– Irving Fisher, 3 days before the 1929 Black Thursday crash precipitated the Great
Depression
• Circa 1987: Portfolio insurance can protect investors: it will let them
get out with minimal damage if markets ever begin to fall. They would
simply sell ever-increasing numbers of futures contracts, a process
known as dynamic hedging. (Then…Black Monday)
• Circa 2006: housing prices always go up! (and therefore the GFC!)
• “At this juncture, however, the impact on the broader economy and
financial markets of the problems in the subprime market seems
likely to be contained.”
- Ben Bernanke, March 2007, in testimony to the Congressional committee
18

19. What’s “Known” Today?
• What do you assume or take as fact about your company’s risks,
businesses, policies and practices?
• What do management and the Board assume to be true?
• Does the security desk in the lobby really secure anything? Do people
know procedures to follow in emergencies? Do you have backup
resources and backup systems to stay in business? Run tabletops!
• Don’t question everything but…
19

20. …You Should Question…
• Key assumptions by those pitching a new product, venture, mitigation or solution
• Use and extrapolation of data; consider if it’s apples to apples, if the sample size is large
enough, if implicit assumptions about distribution or volatility are made
• Do people know who is a risk owner and who is responsible for mitigation?
• Are people aware of risk/safety policies or when/how to escalate risks?
• Why is this project or this team different from those in past failures?
• Why is this probability so low? Who is emotionally attached to the quality of a proposal,
merger, acquisition, resource, tactic or mitigation? Is an estimate made under duress?
20

21. We Should Also Question “Expert Opinion”
• As a society it seems we have a special fondness for expert forecasts
• In many cases such predictions are wrong to an embarrassing extent
and often could have been greatly improved by strong consideration
of the normal or “base rate” seen in the available data
• It is the temptation of too many experts (and laymen for that matter)
to ignore the generic situation once a particular case comes under
analysis
• The previous is described as the Base Rate Neglect or the base rate
fallacy
21

22. Base Rate Fallacy and the Acquisition
• an executive pushing for an acquisition projects that all sales growth,
profit and synergy projections will be met or exceeded over the next
ten years while the historic record would paint a much more somber
picture of post-acquisition performance
• Honest M&A experts sometimes say “synergies are never realized”
• Solution: run scenarios where sales do not follow the “hockey stick”,
synergies fail or come very late, and/or profit margin is not improved
by the (superior intellect of) new management
• Solution: avoid the emotional attachment…use valuation analysis
before the negotiation to determine a “walk-away” price 22

23. Don’t Go Away Mad, Just Go Away
Former CEO of Citibank Sandy Weill says “Knowing when to get out of
the game is a critical consideration...I’ve been in situations where we’ve
got an agreement, and as time goes by the other side sees you getting
anxious and raises the price,” he says.
“You have to be disciplined at that point, and it isn’t easy. Deal making
is akin to dating and falling in love. If you don’t think the behavior of
the other party is something you can live with from a cultural point of
view, you have to grit your teeth and simply say ‘No. We’re done.’
23

24. Base Rate and Additional Information: Hit-and-Run
Consider the following scenario and your gut response to the question:
A cab was involved in a hit-and-run accident at night. Two cab companies,
the Green and the Blue, operate in the city. (The Green company cabs are
indeed all green and all Blue cabs are blue) Assume the following data:
• 85% of the cabs in the city are Green and 15% are Blue.
• A witness identified the cab as Blue. The court tested the reliability of
the witness under the circumstances that existed on the night of the
accident and concluded that the witness is 80% accurate in color
identification (so is wrong 20% of the time).
What is the probability that the cab involved in the accident was Blue?
Example taken from Thinking, Fast and Slow by Daniel Kahneman
24

25. Hit-and-Run (continued)
• The answer is 41%
• The “base rate” information in this problem is the statement about
the percentage of each cab in the city. With no other assumptions we
must infer from it that Green cabs are in 85% of the accidents (hit-
and-run or otherwise).
• Bayes’ theorem is the right approach to combine that information
with the witness testimony. If the base rate is ignored, and it is by
many people, one comes up with a reply of 80%, which is very far
from the correct answer of 41%. (shown in appendix)
25

26. The Law of Small Numbers or the Sophomore Slump
• The “law of small numbers” refers to this misleading pattern which
drops out as we reach samples of sufficient size
• Most risk managers know the problems with small samples but still
may attach significance to the behavior seen in small samples
• Consider a baseball rookie whose first season batting average is .400.
This is a nearly unsustainable average and likely represents what will
later be regarded as the player’s full potential. His “sophomore”
season he bats .320. This is perhaps more representative of his skill
level but compared to the first season it is seen as a slump!
26

27. Sophomore Slump (cont’d)
• In many real-world situations, the occurrence of a value very far from
the average is less likely than one closer to the average
• If one of the “typical” (near-the-average) values occurs after one that
was seen as extreme (far from the average), this may be perceived as
some type of “reversion to the mean” and this is a common, but
misleading and inaccurate term, for this behavior.
• As mentioned, in sports, it is common to see a rookie’s stellar first
year followed by a more “normal” level of performance; in Australia
they call this the “second year syndrome”
27

28. Sophomore Slump (cont’d)
• If a bond manager has an incredible year and is among the top 1% of
managers across the country, there’s a good chance the next year will be
less “extreme” and will be closer to the average
• No matter what name is used for this behavior, risk managers should
understand it and not assume it’s a result of corrective action, praise, or
year-to-year (negative) auto-correlation
• Mitigation: don’t fall for “reversion to the mean”; understand likelihood of
values far from the mean…what shape is the distribution? Are there fat
tails? Is the distribution uniform around its mean? For product
“experiments” or studies, don’t rely on statistically small samples!
28

29. Trees and Forest (and the Portfolio View)
• Risk exposures are often analyzed in isolation and this is not
necessarily wrong for certain purposes
• Enterprise risk management (ERM), Management, and the Board
must be aware of a holistic or portfolio view of risk
• When considering stand-alone risk assessments, e.g., focused on
specific risk types or LOBs, one must not forget the enterprise view
and the concepts of risk aggregation, concentration, correlation and
diversification!
29

30. WYSIATI (“What you see is all there is”*)
• The analysis of a very specific situation may naturally lull one into
forgetting the bigger picture
• A True Story… A LOB’s risk-reward analysis shows very low volatility in
earnings, little chance of needing a capital infusion, and forecasts a modest
profit. A Board member suggests the LOB should be taking on more risk. He
forgot to consider what is happening in the rest of the company, and in
doing so, ignored the portfolio or aggregate view that is so sought after in
the ERM world. It was in fact this stable and low risk business that
tempered aggressive risk taking at the other business lines in the company!
(Note: he was, by all other measures, a very intelligent person!)
*term coined by Daniel Kahneman
30

31. The desire to do the opposite of what is requested or advised, due to a perceived threat to
freedom of choice [towergateinsurance.co.uk]
31

32. Mitigation of Reactance
• A message that conflicts with your original understanding can be hard
to swallow; try to put aside any pre-conceived notions or pride
• We may not want to admit we misunderstood, misjudged or were
unaware of key facts
• Before rejecting a proposed path, decide if your objection is tied to
your ego
32

33. Sunk Cost Fallacy
“It is commonly expected that individuals will reverse decisions or
change behaviors which result in negative consequences. Yet, within
investment decision contexts, negative consequences may actually
cause decision makers to increase the commitment of resources and
undergo the risk of further negative consequences.”
- “Knee-Deep in the Big Muddy: A Study of Escalating Commitment to a Chosen Course of Action”,
by Barry Staw. Organizational Behavior and Human Performance, Volume 16, Issue 1, June 1976,
Pages 27–44.
33

34. Sunk Cost Fallacy
• The Sunk Cost Fallacy often goes away when a forward-looking cost-benefit
analysis or risk-reward analysis is conducted
• Many fall into the trap!
o Nick Leeson: short straddle, Kobe earthquake, long-long future arbitrage, prison!
o Boston’s Big Dig (Central Artery/Tunnel Project): the most expensive highway project
in the US; originally scheduled to be completed in 1998 at an estimated cost of $2.8
billion, but was completed in December 2007, at a cost of over $14.6 billion
o Sony: continued participation in electronics after $8.5 billion in losses over 10 years
o After a heated and aggressive bidding war, Robert Campeau buys Bloomingdale's
with an estimated $600 million overpayment. The Wall Street Journal says "we're not
dealing in price anymore but egos." Campeau declared bankruptcy soon afterwards
34

35. On Board the Flying Bank
• The Swiss airline Swissair was once thought to be so financially stable
that it earned the title “the Flying Bank”
• Despite financial troubles in the late 90s it continued to execute the
Hunter Strategy by buying up bankrupt airlines all over Europe
• The management underestimated the dangers and difficulties in
acquisitions and investments of partially ailing airlines. So the Belgian
Sabena and the German LTU were taken despite the significant capital
requirements
35

36. On Board the Flying Bank (cont’d)
• The Hunter Strategy “was rooted in the arrogance of a company
desperate to punch above its weight on the international scene” –
Sepp Moser
• Aaron Hermann and Hussain Rammal cite groupthink as a
contributing factor in the collapse of Swissair: the belief that the
group is invulnerable and the belief in the morality of the group.
• They point out that before its failure, “the size of the company board
was reduced, subsequently eliminating industrial expertise and the
board members were lacking expertise in the field and had somewhat
similar background, norms, and values…the pressure to conform may
have become more prominent”
36

37. Mitigation of Groupthink
• To the extend possible seek out assessments, risk identification,
probabilities, forecasts, and parameter estimates in a one-on-one
environment; employ anonymous polling and/or surveys (or pick up
the phone!)
• Besides looking at individual results to assess majority views, one can
also use the spread or range of numerical risk assessments as a
measure of understanding of the risk itself or the volatility of the
“risky” quantity of interest: in other words, a wide variety of replies
may indicate there is not a great understanding of the risk or it may
simply indicate a risk with high volatility or uncertainty
37

38. Black Swans Revisited
• Consider a very unlikely event with annual probability 1/n. For
example, n might be 100 or 500
• In a given year the probability the event does not occur is 1-1/n
• Assuming year to year independence, the probability that it does not
occur over n years is (1-1/n)n
• As n gets larger this expression approaches 1/e ≈ 0.37 and therefore,
for large n, the probability the black swan event does occur at least
once in n years is approximately 1 –1/e ≈ 0.63 or 63%
• This works well even for “small” n such as 20
38

39. Black Swans Revisited
A diversified company models annual earnings using a stochastic
approach for each of its 20 lines of business. Assuming independence
between the lines, what is the probability that at least 1 line will have
actual earnings less than its own 1st percentile value?
Solution: P(no LOB has a result below its 1st %ile) is
0.9920 or 81.79%, and therefore, using the complement, we have:
P(at least one LOB has a result below its 1st %ile) = 1 – 81.79% or
18.21%
39

40. The Anchor Effect
• Risk quantification and financial forecasting often begin with a single
initial estimate
• The source for the value might be last year’s value, a result of deep
analysis, or could be based on intuition
• Regardless of the source, once the initial value is seen, it is very hard
to mentally move far away from it in subsequent consideration or
estimation of alternatives!
• That initial value, whether or not of high “quality”, tends to anchor
any future estimates to be close to that initial value
40

41. The MIT Mock Auction
• A professor of management science at MIT, Dan Ariely, conducted a
mock auction with his MBA students.
• A CFO magazine article summarizes his behavior experiment: “He
asked students to write down the last two digits of their Social
Security numbers, and then submit bids on such items as bottles of
wine and chocolate.”
• The half of the group with higher two-digit numbers bid “between 60
percent and 120 percent more” on the items!
41

42. The Anchor Effect (cont’d)
• Ariely says “people are good at setting relative values” but “it’s very
hard to figure out what the fundamental value of something is
whether it’s an accounting system, a company’s stock, or a CEO.”
• Mitigation: when possible, make the estimate or projection without
the benefit of another person’s estimate or “last year’s value”
• Make the following joke false!: How many actuaries does it take to
screw in a light bulb? Well, how many did it take last year?”
- “Avoiding Decision Traps”, by Edward Teach. Online at: http://ww2.cfo.com/human-capital-careers/2004/06/avoiding-decision-traps
42

43. Solution to Pop Quiz Question #3
• A risk manager shows Management the 95th percentile point for
potential hurricane losses for a calendar year. This loss value is $150
million. An executive new to the company asks for the probability of
incurring $150 million in hurricane losses. Assuming the model is
correct what should the risk manager say?
• Solution: about 0%, but the probability of incurring $150 million or
more in hurricane losses is 5% based on the model
• This is almost a version of WYSIATI type thinking as most of us know
the answer and some are simply too focused on the specifics
43

44. The Birthday Problem
Given “n” randomly selected people there is a probability of p(n) that
(at least) 2 people share a birthday as follows:
n p (n )
1 0.00%
5 2.70%
10 11.70%
20 41.10%
23 50.70%
30 70.60%
40 89.10%
50 97.00%
60 99.40%
70 99.90%
44

45. Appendix: Hit-and-Run Solution 1
One can use Bayes’ theorem in the form:
P(A|B) = P(B|A)P(A) / [ P(B|A)P(A) + P(B|A′)P(A′)] where A′ is
the complement of A.
One can define event A as “cab was blue” and event B as
“witness said cab was blue”; then the left hand side is:
P(cab was blue | witness said “Blue”) and using the rest of the
formula gives approximately 41.4%. It is necessary to make
use of the witness’s 80% accuracy.
45

46. Hit-and-Run Solution 2
actual car
green blue
85 green 68 17
15 blue 3 12
12/(12+17) = 41.4%
witness account
46

47. References and Additional Information
• Many of these concepts and some wording has been based on this author’s
whitepaper “ERM at the Speed of Thought: Mitigation of Cognitive Bias in
Risk Assessment” available here:
http://www.ermsymposium.org/2015/Additional_Research_Papers/Levine-05-05-15.pdf
• A great source for information on cognitive bias and a great read is
Thinking, Fast and Slow by Daniel Kahneman
• The website https://www.towergateinsurance.co.uk/liability-
insurance/cognitive-biases has a quick presentation of many biases,
complete with cartoons!
• Website: www.ermvalue.com Email: damonlevine239@yahoo.com
47