Recording of Dave Mathias (Beyond the Data) presenting at Twin Cities Product Conf 2019.

1. GoBeyondTheData.com
Product Conf 2019
Dave Mathias
Beyond the Data | @DaveMathias | @GoBeyondtheData

2. GoBeyondTheData.com
• Gaps in perception and knowing your unknowns
• Managing bias and uncertainty
• Balancing System 1 and System 2
• A tale of two data stories

3. GoBeyondTheData.com
Gaps in perception and knowing your unknowns

4. GoBeyondTheData.com
Source: Youtube https://www.youtube.com/watch?v=OQxmT2kVz-c
Data can be deceiving

5. GoBeyondTheData.com
Perception can be imperfect
Sources: https://www.verywellmind.com/cool-optical-illusions-2795841
https://www.wired.co.uk/article/optical-illusions-science-perception
Which circle is darker?
Which line is longer?

6. GoBeyondTheData.com
Source: YouTube https://www.youtube.com/watch?v=IGQmdoK_ZfY
We all have blind spots

7. GoBeyondTheData.com
So what do we really “know”?
Source: https://en.wikipedia.org/wiki/Johari_window
Known
Knowns
aka Arena
Known
Unknowns
aka Facade
Unknown
Unknowns
aka Unknown
Unknown
Knowns
aka Blind Spot
We know We don’t know
Others know
Not known
to others
Johari Window

8. GoBeyondTheData.com
Example Johari Window: Customer is the other and you are the product person
Source: https://en.wikipedia.org/wiki/Johari_window
Known
Knowns
aka Arena
Known
Unknowns
aka Facade
Unknown
Unknowns
aka Unknown
Unknown
Knowns
aka Blind Spot
We know We don’t know
Others know
Not known
to others
Johari Window
https://buyerblueprints.com/wp-content/uploads/2016/01/Screen-Shot-2016-01-21-at-11.03.07-AM.png
How can we build on
product positives? How
can we reduce product
negatives?
What new product ideas
based on customer
knowns?
What do we need to
communicate more and
better to customer?
What do we need to
research and discover
using data?

9. GoBeyondTheData.com
Managing bias and uncertainty

10. GoBeyondTheData.com
Identifying & managing uncertainty
• What is the level of uncertainty?
• 60%, 95%, 99%, 99.9% confident
• What is the impact of errors?
• Type 1 (false +) & Type 2 (false -)
• Are there ways to reduce this uncertainty?
• What are the tradeoffs of above?
• Communicate uncertainty to colleagues and
customers fairly but smartly
• Uncertainty will exist and it is managing what
you known against how well you know it
against outcomes
?
Type 1 Error
False Positive
Correct
Type 2 Error
False Negative
Correct
Reality
True False
Measured /
Perceived
True
False

11. GoBeyondTheData.com
Identifying & managing bias
• Understand the objective and its value and ask:
Who & how are the people impacted?
• Understand your data and remember:
Garbage in = garbage out!
• Does your population relate to people
impacted and ask:
What is the impact?
• Bias may exist but ask:
Is the alternative more biased?

12. GoBeyondTheData.com
Balancing System 1 and System 2

13. GoBeyondTheData.com
Shifting System 1 to System 2 thinking, sometimes
System 1 is the elephant: instinctual and powerful
System 2 is the rider: deliberate and discerning

14. GoBeyondTheData.com
A tale of two data stories

15. GoBeyondTheData.com
Photo: https://www.nhpr.org/post/youre-full-nh-parole-board-tough-talk-can-veer-profane
Story: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Risk Scores in Sentencing in Florida
• Fort Lauderdale implemented a private
company’s risk score to help it determine
who was more like for recidivism in
sentencing
• Model had 137 input questions many of
which were asked to defendant that some
clearly would proxy asking a person’s race
even though race not officially a factor
• Private “black box” model
• Result was the model appeared to
discriminate on race

16. GoBeyondTheData.com
Photo: http://www.chancebailbondsnj.com/is-bail-bonds-information-public-in-new-jersey/
Source: https://www.politico.com/states/f/?id=00000169-df3a-d48d-a57d-dfff7f270000
The New Jersey Bail Bond Model
• New Jersey decided that bail was
discriminating against poor and minorities
and at same time empowering the wealthy
but dangerous
• Decided an analytical model was a better
approach
• Communicated broadly and bipartisan
brought many together
• Result was a complete success with 40%
less people in jail two years later with
similar levels of people showing up for trial

17. GoBeyondTheData.com
Source: https://www.youtube.com/watch?v=cbtf1oyNg-8
Keep the human side of data in mind so you will avoid this…

18. GoBeyondTheData.com
In the end the human side of data is about understanding people,
managing risk, minimizing bias, and aligning incentives so people
can make thoughtful data-informed decisions

19. GoBeyondTheData.com
Always continue learning
• Understanding Data and Bias:
• Books: Weapons of Math Destruction, Dataclysm, Automating
Inequality
• Understanding Behavioral Science:
• Books: Nudge, Influence, Thinking Fast and Slow, Freakonomics
• Podcasts: Hidden Brain, Freakonomics, Behavioral Grooves
• Online: PeopleScience.maritz.com, BehavioralEconomics.com
• MeetUps: Behavioral Grooves, Behavior MN

20. GoBeyondTheData.com
Contact or follow me at:
o dave@gobeyondthedata.com | @DaveMathias | @GoBeyondtheData
o linkedin.com/in/davemathias1 | about.me/davemathias
o Check out Data Able podcast
Source: https://memegenerator.net/instance/55474775/fist-pump-baby-you-rock Source: https://pixabay.com/illustrations/question-mark-pile-questions-symbol-2492009/
Questions