Decision making with limited time and information

1. Proprietary & Confidential
Product World Forum
SEPTEMBER 18, 2018
Decision making
with limited time
& information

2. 2 Proprietary & Confidential
Poulomi Damany
Vice President, Data Products
Poulomi oversees the platforms and
model that power Credit Karma’s predictive,
data-driven recommendations.

3. 3 Proprietary & Confidential
More than 80 million members
Over 50% of millennials in the US
are on Credit Karma
SCREEN IMAGES SIMULATED

5. 5 Proprietary & Confidential
Thought
experiment:
What was the result/outcome?
What was the best and the worst
decision you made this year?

6. 6 Proprietary & Confidential
Good decision-making
isn’t actually about
the outcome
OUTCOME
DECISION
BADGOOD
GOODBAD
Drive drunk,
crash
Drive sober,
crash
Drive sober,
no crash
Drive drunk,
no crash
Good decisions don’t guarantee
good outcomes…. but they increase
their likelihood
...Bad outcomes don’t invalidate
good decisions

7. 7 Proprietary & Confidential
Our decisions are only
as good as our beliefs.
The more accurate our beliefs,
the better the foundations of
the decisions we make

8. 8 Proprietary & Confidential
- We hear something;
- We vet whether it is true or false
...but our beliefs
aren’t always
accurate
What we think we do:
- We hear something
- We believe it to be true
- Sometimes, later, we might vet it
What we actually do:

9. 9 Proprietary & Confidential
An approach
that works
for me
Making
decisions with
imperfect data

10. 10 Proprietary & Confidential
Will result in <this outcome>
We believe the <the capability>
We will know we have succeeded when
Start with a
Hypothesis
< we see metric>

11. 11 Proprietary & Confidential
Pre-mortem
Imagining success helps
you see all the things that
need to go right for your
decision to work out

12. 12 Proprietary & Confidential
Pre-mortem
Imagining failure forces
you to think about possible
failure paths and how you
could mitigate them

13. 13 Proprietary & Confidential
02 Precision is not needed - be directionally right,
and within the same order of magnitude
What do you have to believe?
Incorporate percentages or ranges of confidence
(take your ego out of the decision)
Calibrate
your beliefs
03
01
Get feedback - ask experts!!
04

14. 14 Proprietary & Confidential
Decision Types
Type 1 decision
“Some decisions are consequential and irreversible
or nearly irreversible — one-way doors — and these
decisions must be made methodically, carefully,
slowly, with great deliberation and consultation…”
- Jeff Bezos

15. 15 Proprietary & Confidential
Decision Types
Type 2 decision
“Type 2 decision are changeable, reversible —
they’re two-way doors. If you’ve made a suboptimal
Type 2 decision, you don’t have to live with the
consequences for that long.”
- Jeff Bezos

16. 16 Proprietary & Confidential
And relax…
most decisions are reversible

17. 17 Proprietary & Confidential
Example 1:
Set the default
to yes on a
privacy policy
consent
checkbox
SCREEN IMAGES SIMULATED
02 Pre-mortem
- x% adoption in y days
- User complaints, PR issues
01
03
Hypothesis
- It’s a free trial, people are used
to a default opt-out state
Calibrate your beliefs
- Confidence in default behavior
- Are you in the EU?
- App upgrade timelines

18. 18 Proprietary & Confidential
Example 2:
Identity
Monitoring
SCREEN IMAGES SIMULATED
02 Pre-mortem
- Increase trust, drive engagement
- Cost; monitor v/s protect
01
03
Hypothesis
- ID monitoring is a valuable extension
similar to credit monitoring
Calibrate your beliefs
- What’s the value split between
ID monitoring and protection?
- What do you have to believe to
activate a majority of our users?

19. “
19 Proprietary & Confidential
- Brandon Chu
Impact is created by those who are driving
lots of decisions quickly, and are right when
it’s really important to be.

20. 20 Proprietary & Confidential
THANK YOU
for your time & attention
For more information please contact me at:
poulomi.damany@creditkarma.com
@PDatCK