The 17 Ways Machine Learning and AI Can Fail

the 17 ways ML / AI can
FAIL
Brian Lange, Partner + Data Scientist

TODAY
- A quick framework for the lifecycle of a
data product
- A multitude of failure modes we’ve seen
throughout that lifecycle
- Some ways those failures can be avoided

Lifecycle of a ML/AI product/project

conception
The idea for a product
or project is born.

conception experimentation
A data team tries to
make it happen.

conception experimentation productionizing
The experiment
becomes more reliable
and integrated.

conception experimentation productionizing usage
The product is
used by people
who didn’t
design it.

∞

conception experimentation productionizing usage ∞

Failure of Imagination
The opportunity never gets dreamed of in the ﬁrst place
How to avoid:
- Leadership that is aware and excited about
opportunities w/ data
- Data scientists are involved early in idea generation
- Data team has diverse backgrounds

Failure by Overimagination
The idea isn’t something that can actually be
done, even using creative/cutting-edge methods
How to avoid:
- Leadership is data literate
- Data team is involved in conception stage

Failure by Inability
There’s nobody at the organization with the skills to
even begin experimenting
How to avoid:
- Hire data scientists! Preferably self-starters

Failure of Collection
The data needed to make the project happen
doesn’t exist yet
How to avoid:
- Collect as much as possible from as early as
possible, or
- Be patient and let the data roll in after deﬁning
what you need

Failure of Access
exists, but something prevents us from
accessing it or we’re not aware of it.
How to avoid:
- Good data governance/documentation
- Hire data scientists skilled in pulling from many
possible types of sources

Failure of Access
exists, but something prevents us from
accessing it or we’re not aware of it.
How to avoid:
- High-ranking leadership supports the project, cuts
through red tape/gatekeepers

Failure of Investment
Funding/resources are cut before the experiment
reaches its conclusion
How to avoid:
- Leadership/data team knows how to budget for
data projects
- Project has enough potential value to justify risk/
investment

Failure of Data Richness
There isn’t enough data to train a model
eﬀectively, or the data we have doesn’t have
enough “signal” in it
How to avoid:
- This is hard. Don’t necessarily know until you try
- Experienced data scientists can at least screen out
the projects where there obviously isn’t enough

Failure of Rigor
The data team thinks they got great results, but
they’re not actually good.
How to avoid:
- Data team (and preferably leadership) are skeptical
and know how to properly use metrics

Failure of Interpretability
The model works great, but you have no idea why
How to avoid:
- This only matters if you need to know.
- Constraints around interpretability should be
deﬁned before experimentation

Failure of IT Cooperation
The infrastructure gatekeepers are busy or
aren’t cooperative
How to avoid:
- Have a great IT team and plan for their involvement
early on, or
- Empower data team to stand up their own
infrastructure and hire for those skills

Failure of Practicality
The thing you devise is too damn complicated
“We evaluated some of the new methods oﬄine but the additional accuracy
gains that we measured did not seem to justify the engineering eﬀort needed to
bring them into a production environment.”

Failure of Practicality
The thing you devise is too damn complicated
How to avoid:
- Deﬁne relevant context of usage before even
building the model, and keep it in mind
- Don’t spend too much time experimenting if you
have something that’s viable

Failure of Business Case
The infrastructure required to put your model into
production is too expensive compared to its utility
How to avoid:
- Focus on applications that are clearly high value
and may have many potential uses
- Hire skilled IT/data engineers who know how to
keep running costs low

Failure of Business Case
The infrastructure required to put your model into
production is too expensive compared to its utility
How to avoid:
- Leadership plays role of monitoring project viability
or
- Data team is business savvy and plays this role

Failure by Shifting Needs
You took so long that the business case changed
How to avoid:
- Iterate/sprint. Don’t go work on your project in
a closet for 9 months.
- Keep leadership involved with frequent quick
progress updates

Failure of Adaptation
Reality drifts, and the model doesn’t keep up
How to avoid:
- Have a plan for how the system is updated
- Research how often patterns change drastically in your data
- Monitor accuracy over time with periodic quality checks

Failure by Side Eﬀects
Your product has unintended consequences
Black defendants were often predicted to be at a higher risk of recidivism than
they actually were. Our analysis found that black defendants who did not
recidivate over a two-year period were nearly twice as likely to be misclassiﬁed as
higher risk compared to their white counterparts (45 percent vs. 23 percent).
How We Analyzed the COMPAS Recidivism Algorithm

How to avoid:
- Dedicate some time up front to exploring potential implications
- Consider where your training data comes from
- Write test code that looks at model outcomes across diﬀerent
groups of users

How to avoid:
- Avoid collecting unnecessary PII, and safeguard/encrypt what
you have

Failure of Morality
Your product works well, but towards an immoral purpose
How to avoid:
- Dedicate some time up front to exploring potential
implications
- Invest in learning about ethics in design, data, technology
- Don’t work for evil organizations

Failure of Design
Nobody uses your product
How to avoid:
- Use design methods to ensure your product solves a real need
- Focus on the interfaces and test them with users
- Experiment to determine how much people trust your product
and/or think it’s creepy

$
🕑
FAIL

$$$$$$$$
🕑 🕑 🕑 🕑 🕑 🕑
FAIL

Process choices to minimize failure
Put prototypes in front of real users to learn and reﬁne
Involve data team as early as possible
Prioritize ideas with clear value
Think about potential implications and context
Move fast, overcommunicate

conception experimentation productionizing usage (or testing)

Team choices to minimize failure
Leadership
- Recognize potential value of
data products and support
them with resources and
authority
- Are data literate enough to
evaluate work and contribute
ideas
- Keep tabs on team process
and help them navigate
shifting business realities
Data Team
- Creative and business savvy,
can generate ideas for new
data projects
- Diverse analytical
backgrounds
- Able to pull, merge, clean data
from a variety of sources
- Can accurately evaluate their
own work
- Have either design skills or
dedicated designers, to ensure
the usefulness of a product
IT
- Data literate, supportive,
and unconstrained, OR
provide self-serve resources
to a dedicated IT role on
data team
- Able to estimate and
minimize costs for
productionizing models
- Optionally, can assist data
team in more software-
heavy aspects

Culture choices to minimize failure
- Invest in training the team about ethics, privacy,
and security, and encourage conversations about it
- Encourage every role to focus on the needs of
individual users and the business
- Allow ideas for new projects to emerge from the
data team as well as leadership
- Make the default answer to “can I have that data/
computing power?” yes

Don’t let perfect be the enemy of good
- Checking all these boxes makes projects run smoother
- It’s exceedingly rare to check all these boxes
- Failure of Even Starting is a failure to learn or make
progress at all

Quick plugs
-Becoming a Data Scientist/Growing a Data
Science Team: Metis (thisismetis.com)
-Increasing Data Literacy in Leadership:
data-science-for-managers.datascope.co
-Hiring a data science team to help you
dream up projects and make them happen:
Datascope (datascope.co)

.co@bjlange
thanks!
questions?
brian [at] datascope.co
slides will be posted on Twitter later!

The 17 Ways Machine Learning and AI Can Fail

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Empfohlen

Empfohlen (20)

The 17 Ways Machine Learning and AI Can Fail