2. • Teaching Smart People How to Learn
• Double-loop learning
• Defensive reasoning
• How can you do better?
• Building an Effective Team
• Three types of diversity: social category, cognitive, and value
• Look for unshared information
• Let your people rest
• Even Swaps:A Rational Method for Making Trade-offs
• Consequences tables
• The Hidden Traps in Decision Making
• Anchoring, status quo bias, sunk cost fallacy, confirmation bias,
overconfidence in estimation and forecasting
Reading list
3. • Pay close attention to mental models -- they’re the
basis for everything
• A startup is a process for decision making and action
• The most important decision you will make is whose
advice to take
Today’s arguments
6. “[M]odels people have of themselves, others, the
environment, and the things with which they interact."
- Donald A. Norman.The Design of Everyday Things (1988)
What are mental models?
10. Mental models define how we think the world works,
but not necessarily how it actually works
- Me, just now
Mental models are necessarily personal
If a model doesn’t work for you, build a better one
When judging a model’s quality, focus on process, not outcome
What are mental models?
11. How do we form mental models?
Real world
Interpretation
Feedback
What a video camera would record.
The story we create in our mind.
Is our story confirmed or disconfirmed?
(usually we only ask the former)
13. “Insanity is repeating the same mistakes and expecting
different results.”
- Narcotics Anonymous. BasicText, pg. 11
(nope, not Einstein)
Single-loop learning
http://amonymifoundation.org/uploads/NA_Approval_Form_Scan.pdf
Want better results? Change your model
15. Learning loops in Product Design
Donald A. Norman.The Design of Everyday Things (1988).
What’s missing?
16. Donald A. Norman.The Design of Everyday Things (1988).
User feedback should alter the
product by altering the design model
Learning loops in Product Design
18. Strong sources of mental models
• Bayesian updating
• Physical laws (especially movement mechanics)
- Elasticity (springs)
- Friction
• Large and representative data sets (empirical
observation)
• Careful experimentation (seeking to disconfirm)
• Relevant analogy
19. • Abstract theory
• Personal experience
• Irrelevant analogy
• Repeated observations (small data sets)
• Single observation (single data point)
• Anecdote/inductive reasoning (Malcolm Gladwell)
• Opinion
Unfortunately, the less data we have,
the more heavily we weight it
Weak sources of mental models
20. Treat your models as hypotheses
Make sure they’re testable
Models that can’t be disproven aren’t models -- they’re beliefs
!
Actively seek to disprove them
Welcome disproof -- a model disproved is a lesson learned
!
Look for hidden assumptions
Treat secondhand data as assumptions until proven otherwise
!
Question their predictability
The same event may be evidence of many different hypotheses
Models don’t care about your loyalty
If a model doesn’t work, change it
22. What does rational decision making look like?
Herbert A. Simon, Administrative Behavior
1. Complete knowledge and anticipation of the
consequences of every choice
2. A valuation for each consequence
3. Awareness of all possible alternatives
23. What does actual decision making look like?
Herbert A. Simon, Administrative Behavior
1. At best, our understanding of consequences is
fragmentary
2. Values depend on context, and future context is
unknown
3. We never consider more than a small subset of
possible alternatives
24. Optimizing (rationality)
Searching through alternatives until we find the
best option.
Satisficing (bounded rationality)
Searching through alternatives until we find one
that’s acceptable given the constraints at play.
25. Satisficing != Poor decision-making
What satisficing is:
- A recognition of cognitive and situational limitations
- A means for making decisions under uncertainty
- A means for knowing when to move on
What satisficing is not:
- An excuse to be lazy
- A justification for poor process
26. The dark side of satisficing…
“People have basic physiological limitations of not
being able to handle complexity, of unconsciously
adapting to gradually changing conditions, of
conforming to group and organizational norms,
and of focusing on repetitive activities.”
Van DeVen. Central Problems in the Management of Innovation
27. Good decision process: PrOACT
• Problem
• Objectives
• Alternatives
• Consequences
• Tradeoffs
28. Problem
A good solution to a well-posed problem is almost always better
than an excellent solution to a poorly posed one
• Consider the trigger
- How problems present often bias our thinking — don’t let it!
- Ex: best mailing list software vs. best way to manage direct-mail
• Question latent constraints
- Problems often take unnecessary constraints for granted
- Ex: when to meet vs. how can we finish this
• Establish a workable scope
- Too narrow is bad. So is too broad.
- Ex: how to save on gas vs. how to save on vehicle costs
• Gather outside feedback
29. Objectives
Objectives are your decision criteria.They determine the what,
how, and how long.They are also your basis for explanation.
1. Write down all concerns you hope to address through your
decision, e.g., “don’t want to launch too late”.
2. Convert your concerns into succinct objectives, e.g., “minimize
time to market”.
3. Separate ends from means to establish fundamental objectives.
Ask “why?” until you reach something that’s important for its
own sake. (Five whys)
4. Clarify each objective. Ask “what do I mean by that?” Be
specific.
5. Test objectives to see if they capture your interests. Do they
form a good basis for explaining your decision to someone else?
30. Alternatives
Alternatives are your range of potential choices for pursuing your
objectives. You can’t choose an alternative you haven’t considered.
• Don’t box yourself in with limited alternatives. Poor
decisions often happen when you:
- Fall back on the default alternative
- Choose the first possible solution
- Choose from alternatives presented by others
- Wait until you’re stuck with what’s left
• Use your objectives—ask “How?” for each objective
• Challenge your assumptions
• Do your own thinking first
• Ask others for suggestions
• Create first, evaluate later
• Always be searching for alternatives
31. Alternatives
Tailor your alternatives to your problem.
• Process alternatives. Sometimes the best choice is a
process, e.g., voting, auctions, arbitration.
• Win-wins. Sometimes the key is to analyze someone
else’s decision problem and make sure yours gibes with it.
• Information-gathering. Reduce uncertainties by
identifying and collecting needed information.
• Time-buying. Deferring a decision is a decision.
32. Consequences & Tradeoffs
Be sure you understand the consequences of each choice. Once
you do, the decision may well be obvious. If not, make even swaps.
• Describe consequences with appropriate accuracy,
completeness, and precision.
• Build a consequences table.
• Make even swaps. (READINGS!!!)
33. “The fox knows many things;
the hedgehog one great thing.”
- Archilochus
http://www.etsy.com/listing/60007735/woodland-animal-pair-hedgehog-and-fox
Expert Prediction
34. Every feature
suggestion
opinion
piece of advice is a prediction
What does this have to do with startups?
Who should you listen to?
How much credence should you give?
39. "Freddie Mac and Fannie Mae are fundamentally sound.They're not in
danger of going under…I think they are in good shape going
forward."
- Barney Frank (D-Mass.) House Fin. Svcs. Comm. chairman, July 14, 2008
Placed into conservatorship in September
"I think you'll see [oil prices at] $150 a barrel by the end of the year"
- T. Boone Pickens, May 20, 2008
$100/bbl in May - $135/bbl in July - $38/bbl in November
“The subscription model of buying music is bankrupt. I think you
could make available the Second Coming in a subscription model and
it might not be successful.”
- Steve Jobs, Rolling Stone, Dec. 3, 2003
Spotify and Rdio would beg to differ
42. Fox-Experts Hedgehog-Experts
Fox-Dilettantes Hedgehog-Dilettantes
One model for thinking about advisors
Fox
Knows many things well
!
Hedgehog
Knows one thing well
!
Expert
Expert in the subject at hand
!
Dilettante
Expert in a related subject
(but not the one at hand)
When it comes to China, the Chinese Ambassador is an expert
and the British Ambassador is a dilettante
43. Tetlock, Philip E., Expert Political Judgment: How Good Is It? How Can We Know? (2005), fig. 3.4
Refers to political extremism
regardless of party
44. If advice is a prediction, then whose advice
deserves your attention?
Turns out that a lot of knowledge in
a single area is a dangerous thing
Short-term advice
1. Fox-Experts
2. Fox-Dilettantes
3. Hedgehog-Dilettantes
4. Hedgehog-Experts
Long-term advice
1. Fox-Dilettantes
2. Fox-Experts
3. Hedgehog-Dilettantes
4. Hedgehog-Experts
45. How to recognize a fox
• skeptical of deductive approaches to explanation and prediction
• disposed to qualify tempting analogies by noting disconfirming
evidence
• reluctant to make extreme predictions of the sort that start to
flow when positive feedback loops go unchecked by dampening
mechanisms
• worried about hindsight bias causing us to judge those in the
past too harshly
• prone to a detached, ironic view of life
• motivated to weave together conflicting arguments on
foundational issues in the study of politics, such as the role of
human agency or the rationality of decision making
Tetlock, Philip E. Expert Political Judgment: How Good Is It? How Can We Know? 2006.
46. Uncertainty stops most people in their tracks, but it’s
only by movement that uncertainty can be resolved
“Strong opinions, weakly held.”
- Paul Saffo
In the meantime, read widely
think deeply
focus on process
choose your advisors wisely
improve your model set
move forward.