7. !
big picture
"
when is it practical to use ML for iOS?
#
what's available to us?
$
end-to-end examples
!
8.
9. barriers to entry?
1. A large dataset
2. Access to high end compute power
3. PhD in machine learning
4. All the time in the world
...nope!
10. Is it practical for my app?
image classification
audio classification
speech recognition
text classification
gesture recognition
optical character recognition (OCR)
translation
voice synthesis
17. Can this be solved without ML?
if so, choose that
18. ML vs not ML
basic unit of solving problem = function ("model")
ML: enabling a machine to learn function on its own
classify sign language alphabet images
not ML: explicitly defining function
determining if a number is even/odd
19. If you decide to use ML
still go with the simplest solution
20. Why do ML (predictions) on mobile?
→ low latency user experience
→ user privacy
21. What's available from Apple?
image classification of 1000 common categories
→ trees, animals, food, vehicles, people
→ SqueezeNet (5 MB), MobileNet (17 MB), Inception
V3 (95 MB), ResNet50 (103 MB), VGG16 (554 MB)
scene classification of 205 categories
→ airport terminal, bedroom, forest, coast
→ Places205-GoogLeNet (25 MB)
22. If not, train custom ML model
step 1: use framework for training
TensorFlow, keras, Turi Create , Caffe, etc
⚠
warning, there are a lot of them
step 2: convert to .mlmodel format (OSS)
→ coremltools github.com/apple/coremltools
→ tf-coreml github.com/tf-coreml
30. End-to-end Process as a developer?
0. Define problem
1. Collect data
2. Train ML model
3. Convert to coreml .mlmodel
4. Import into Xcode project
5. Predict using Core ML (+Vision) framework
36. Quick Review: Deep Learning
neural network model with many layers
deep = many layers
-> deep neural network
Mobile Machine Learning 101: Glossary Jameson
Toole on Heartbeat blog
37.
38. sometime way back in B.C.
people used to train deep neural network from
scratch
39. still some (more recent) time in B.C.
people stand on the shoulders of giants' work
utilizing transfer learning
86. Attributions & Mentions (1/4)
Apple Machine Learning
WWDC 2017 Videos
TensorFlow for Poets Google codelabs tutorial
Apple coremltools GitHub repo
tf-coreml GitHub repo: TensorFlow->core ml
converter
87. Attributions & Mentions (2/4)
Heartbeat by fritz.ai blog: Machine Learning at the
edge
ASL Datasets
Kaggle Sign Language MNIST
Urban Sound Datasets, NYU CUSP
deeplearning.ai course: Data Augmentation
88. Attributions & Mentions (3/4)
Swift for TensorFlow GitHub repo
Dockerized Swift for TF GitHub repo, Alexis Gallager
themorningpaper by Adrian Colyer
OpenAI Research
"The Building Blocks of Interpretability" Google: C.
Olah et al
89. Attributions & Mentions (4/4)
"Strategically Ignorant" Devon Zuegel
"Transfer Learning of Temporal Information for Driver
Action Classification" J. Lemley et al
"Transfer Learning for Sound Classification"
TataLab
90. Further Learning (1/3)
fast.ai Deep Learning course
My Udacity Core ML course
machinethink,
!
ML for iOS blog by Matthijs
Hollemans
TensorFlow Dev Summit 2018 Videos
TensorFlow playground
91. Further Learning (2/3)
Building Mobile Apps w/ Tensor Flow Pete Warden
Neural Networks & Deep Learning Michael
Nielsen
Stanford's Computer Vision course (CS231n)
92. Further Learning (3/3)
"Distilling the Knowledge in a Neural Network"
Geoffrey Hinton et al.
"Transfer Learning - Machine Learning's Next
Frontier"
!
Sebastian Ruder
"Transfer learning for music classification and
regression tasks"
!
Keunwoo Choi et al.