Dan Kuster led a talk at Sentiment Analysis Symposium discussing why businesses should consider adopting deep learning solutions. Key takeaways include simplicity, accuracy, flexibility, and some hacks for working with the tech.
About the Session:
Machine learning is becoming the tool of choice for analyzing text and image data. While traditional text processing solutions rely on the ability of experts to encode domain knowledge, machine learning models learn this directly from the data. Deep learning is a branch of machine learning that like the human brain quickly learns hierarchical representations of concepts, and it has been key to unlocking state-of-the-art results on a range of text and image classification tasks such as sentiment analysis and beyond.
In this session, we will show the impact of a deep learning based approach over NLP and traditional machine learning based methods for text analysis across key dimensions such as accuracy, flexibility, and the amount of required training data. Specifically, we will discuss how deep learning models are now setting the records for state-of-the-art accuracy in sentiment analysis. We will also demonstrate the flexibility of this approach by showing how the features learned by one model can be easily reused in different domains (e.g., handling additional languages, or predicting new categories) to drastically reduce the time to deployment. Finally, we will touch on the ability of this method to handle additional types of data beyond text, e.g, images, for maximum insight.
2. āAll good researchers will tell you
that the most promising direction is
the one they are currently pursuing.
If they thought something else was
more promising, they would be
doing that instead.ā
ā G. Hinton
3. What is deep learning?
ā¦a method for applying simple
mathematical functions to data.
4. web: search, facial recognition
smartphones: speech -> text
email: smart reply
mail: handwriting -> digits
cars: pedestrian detection
art & design: artistic style transfer
5. Wait, why now?
~1960ās (visual cortex is a deep neural network)
Simple neurons ā¾ hierarchical features ā¾ complex
~1990ās (computational models)
Neural networks ā¾ simple functions, applied piecewise
~ 2000ās (the internet + cheap storage)
Lots of data
~2012 (2 GPUs beat Googleās 16,000 CPU cluster)
Very fast and cheap parallel computing power
Deep neural networks ā mathematics + data
11. Maybe you are skeptical that deep learning
will have a lasting impactā¦
12. Maybe you are skeptical that mathematicsāØ
will have a lasting impact?
13. āThe enormous usefulness of
mathematics in the natural sciences is
something bordering the mysterious and
there is no rational explanation for it.ā
āEugene Wigner (1960) āØ
āThe Unreasonable Effectiveness of āØ
Mathematics in the Natural Sciencesā
Unreasonable Beneļ¬ts of Deep Learning
14. āā¦mathematical formulationā¦leads in an uncanny
number of cases to an amazingly accurate
description of a large class of phenomena.ā
āā¦the concepts of mathematics are not chosen for
their conceptual simplicityā¦but for their
amenability to clever manipulations and to
striking, brilliant arguments.ā
āEugene Wigner (1960) āØ
āThe Unreasonable Effectiveness of āØ
Mathematics in the Natural Sciencesā
Unreasonable Beneļ¬ts of Deep Learning
17. Simple? Compared to what?
ā¢ Expert systemsāØ
Domain expertise ā¾ think a lot ā¾ codify rules (e.g., 1700 pages of English grammar)āØ
More data, more pain.āØ
Previous wave of āA.I.ā (good rules can seem magical).
ā¢ Traditional machine learningāØ
Data ā¾ domain expertise ā¾ feature extraction ā¾ learned weightsāØ
Learn everything from scratch. āØ
Manual feature engineering, biased and tedious.āØ
More data helps!
ā¢ Deep neural networksāØ
Data ā¾ model ā¾ learned weightsāØ
End-to-end learning, directly from examples. Like we (humans) do.āØ
Can learn transferable features.āØ
More data really helps!
27. Content ļ¬ltering
(especially for user-generated content)
You have a brand
Your brand has an identity
(Disney vs. Calvin Klein)
Your audience might
have different sensibilities
than you do, about
what is appropriate
for your brand
Filter out the
inappropriate content at
your own custom threshold
46. Experiment:āØ
image features + text features
A man standing in
a ļ¬eld holding āØ
a small parachute
image
encoder
text
encoder
similarity(image, text)
49. Q: What problems can be solved with a
deep neural network?
A: If a human mind can do it in 1/10th of a second, a
deep neural network can probably do it well enoughā¦
assuming you have data!
50. āMany scientists (myself included) take a sadistic
pleasure in proving other people wrong.
ā Y. LeCun
51. The Unreasonable Benefits of Deep Learning:
simplicity, accuracy, flexibility, hacks
Questions?
Daniel
Ā Kuster,
Ā Ph.D.
Ā
@djkust @indicodata
52. Image credits:
Unsplash (backgrounds)
Google search, Facebook AI Research (DeepFace), nVidia, Gatys et al. (arXiv: 1508.06576)
Brigitewear International (Borat swimsuit)
Imgur (skeptical dogs)
Jack the cat
ā¦and the team at indico!
53. āA machine learning researcher,
a crypto-currency expert,
and an Erlang programmer
walk into a bar.
Facebook buys the bar for $27 billion.ā
āØ
@ML_Hipster