The advent of modern deep learning techniques has given organizations new tools to understand, query, and structure their data. However, maintaining complex pipelines, versioning models, and tracking accuracy regressions over time remain ongoing struggles of even the most advanced data engineering teams. This talk presents a simple architecture for deploying machine learning at scale and offer suggestions for how companies can get their feet wet with open source technologies they already deploy.

1. Deep Learning
at Scale

2. 👋 Hi, I’m Kern.
@kern
@KernCanCode

4. deep learning?!

5. AlphaGo achieves honorary 9 dan rank.

6. Google’s
DeepDream
or: scary fish-dog
things everywhere

7. deep learning is a kind
of machine learning.

8. machine learning is like
pattern recognition.
(kind of)

9. deep learning
has been applied to
images, text, audio,
video, and games
with great success!

10. DEEP
CONVOLUTIONAL
NEURAL
NETWORKS

11. 14.2 million images

12. Challenge Winners
top-5 classification error (lower is better)
0
0.075
0.15
0.225
0.3
0.036
0.05
0.074
0.112
0.153
0.258
0.282
2010 2011 2012 20142013 Human 2015

13. 0
0.075
0.15
0.225
0.3
0.036
0.05
0.074
0.112
0.153
0.258
0.282
2010 2011 2012 20142013 Human 2015
Challenge Winners
top-5 classification error (lower is better)

14. 0
0.075
0.15
0.225
0.3
0.036
0.05
0.074
0.112
0.153
0.258
0.282
2010 2011 2012 20142013 Human 2015
Challenge Winners
top-5 classification error (lower is better)
BEFORE
DEEP LEARNING
AFTER
DEEP LEARNING

15. 0
0.075
0.15
0.225
0.3
0.036
0.05
0.074
0.112
0.153
0.258
0.282
2010 2011 2012 20142013 Human 2015
Challenge Winners
top-5 classification error (lower is better)
}much wow
AlexNet
BEFORE
DEEP LEARNING
AFTER
DEEP LEARNING

16. 0
0.075
0.15
0.225
0.3
0.036
0.05
0.074
0.112
0.153
0.258
0.282
2010 2011 2012 20142013 Human 2015
Challenge Winners
top-5 classification error (lower is better)
}much wow
AlexNet
BEFORE
DEEP LEARNING
AFTER
DEEP LEARNING

18. how does
deep learning
work?

20. it’s easy!
lol just kidding

21. it’s calculus,
but you don’t need to
know much.

22. label score
dog 97.1%
shiba inu 2.5%
meme 0.4%
neural networks are classifiers

23. supervised learning
inferring a function from
labeled training data
Wikipedia

24. you provide
“ground truth”
examples and labels.

25. perceptrons
input output
weights

26. is this a dog?
T
F
features classes
weights

27. is this a dog?
T
F
features classes
weights

28. is this a dog?
T
F
features classes
weights

29. PERCEPTRONS ARE TOO LINEAR
WE MUST GO DEEPER

30. deep convolutional neural networks
affectionately known as “convnets”
input output
convolution,
activation,
& pooling layers fully-connected layers (perceptrons)

31. how are the optimal
weights determined?

32. stochastic gradient descent
find the minimum error

33. training takes a while.
anywhere from 8hrs to 2wks

34. how do you scale
deep learning?

35. data collection & cleaning
more clean data, the better
1

36. data collection & cleaning
more clean data, the better
1
2 model training & selection
anywhere from 8hrs to 2wks

37. data collection & cleaning
more clean data, the better
1
2 model training & selection
anywhere from 8hrs to 2wks
3 serving in production
with real-time or batch requests

38. data collection & cleaning
more clean data, the better
1
∞
rinse & repeat
keep models fresh with new data
2 model training & selection
anywhere from 8hrs to 2wks
3 serving in production
with real-time or batch requests

39. you’re gonna need GPUs
for NVIDIA CUDA & cuDNN

40. NVIDIA GeForce GTX TITAN X
is the gold standard

41. AWS G2 instances
mid-tier on-demand GPUs
Model GPUs vCPU Mem (GiB) SSD Storage (GB)
g2.2xlarge 1 8 15 1 x 60
g2.8xlarge 4 32 60 2 x 120

42. AWS G2 GPUs are also
great for model serving.

43. use HDFS + Spark
for data storage and processing
👍

44. TensorFlow with Keras
is a good choice.

45. from keras.models import Sequential
from keras.layers import Dense, Activation
model = Sequential()
model.add(Dense(output_dim=64, input_dim=100))
model.add(Activation("relu"))
model.add(Dense(output_dim=10))
model.add(Activation("softmax"))
model.compile(
loss='categorical_crossentropy',
optimizer='sgd',
metrics=['accuracy'])
model.fit(X_train, Y_train,
nb_epoch=5,
batch_size=32)

46. distributed training
with CaffeOnSpark
https://github.com/yahoo/CaffeOnSpark

47. it’s often better to
peer high-end GPUs
on a single machine.

48. parallelize training on
hyperparameters.
(e.g. learning rate, momentum)

49. hyperparamter optimization
with MOE, hyperopt, or custom scripts
hyperoptMOE custom scripts

50. wrapping up

51. get your feet wet
TensorFlow MNIST Walkthrough
bit.ly/pavlovtensor
Andrej Karpathy’s CS231n
bit.ly/pavlov231

52. suggested technologies
• Neural Network Libraries
• Caffe & CaffeOnSpark
• TensorFlow
• Torch
• Keras
• Hyperparameter Optimization
• MOE
• hyperopt
• Spearmint
• Infrastructure and Hardware
• Apache Spark & HDFS
• NVIDIA CUDA
• Amazon Web Services G2 instances
such scale
much wow

53. references
• icons by John Caserta, Liau Jian Jie, Garrett Knoll, Luboš Volkov, Noe Araujo from the Noun Project
• images from Andrej Karpathy
• Alex Kern, co-founder & CTO of
• we help you structure image & video w/ deep learning
• @KernCanCode on Twitter • @kern on GitHub
about me
• deep learning is great for many kinds of media
• you can scale a deep learning system on Spark & AWS
• get started @ bit.ly/pavlovtensor & bit.ly/pavlov231
in summary