Slides for a talk that was given at PyTorch meetup group at Facebook

1. What’s New in
PyTorch Land
By Sam Witteveen
@sam_witteveen

2. About me
• Google Developer Expert for Machine Learning & Deep
Learning
• Deep Learning - Language & Dialogue
• Multiple Startups- B2B, B2C
• Red Dragon AI

3. • Deep Learning Consulting and Prototyping
• Education & Training
• Products:
• Conversational Computing
• Natural Voice Generation - multiple languages
• Knowledge Base Creation & Reasoning

4. PyTorch - What’s New
• PyTorch 0.4
• PyTorch Road Map
• ONNX
• Fast.ai
• Skorch
• Others

5. PyTorch 0.4
• “Tensors” and “Variables” have merged
• Device agnostic code - no need for .cuda()
• Support for scalar Tensors
• Windows support

6. PyTorch 0.4
Notebook

7. PyTorch Roadmap
• Path to 1.0 and to Production
• Imperative and declarative
• Easy for research, easy for production
• Folding Caffe2 into PyTorch
• ONNX/Caffe for mobile support

8. PyTorch Roadmap
• Fused kernels for declarative graphs
• Tools, pre-trained models & Datasets
• ELF (game playing), Glow (compiler), FAISS etc
• Currently in use in Facebook
• Azure & AWS Support
• Coming in the next few months.

9. ONNX
(Open Neural Network Exchange)
• Converting models to Caffe, CNTK, others
• 0.4 adds RNN support for PyTorch
• TensorFlow support is experimental currently

10. ONNX - CoreML
https://github.com/onnx/onnx-coreml

11. ONNX
(Open Neural Network Exchange)

12. Fast.ai Library
• Wrapper for PyTorch
• Fast.ai - Jeremy Howard & Rachel Thomas
• Adds features beyond PyTorch Vision and Text
• Abstracts a lot of the lower level PyTorch code
• Wraps Pandas, Spacy, TorchVision, TorchText to create
a set of often used data and ML functions
• Focuses on data handling and cutting edge techniques

13. Fast.ai Library
• date handling, text look ups and embeddings, tabular
data, categorical data etc
• Latest things like Stochastic Weight Averaging
• Very opinionated lib, often well explained in the
courses
• Function names are not always the easiest to decipher
• Documentation ….

14. Fast.ai Docs

15. • A scikit-learn compatible neural network library
that wraps pytorch
• Supports 0.4
• Allows you to use SKL pipelines, grid search etc
• https://github.com/dnouri/skorch

16. Other interesting projects
• Pyro 0.2 - https://github.com/uber/pyro
• Wisdom - https://github.com/facebookresearch/
visdom
• Tensor Board - https://github.com/lanpa/
tensorboard-pytorch

17. Links
• https://pytorch.org/2018/04/22/0_4_0-migration-
guide.html
• https://pytorch.org/2018/05/02/road-to-1.0.html
• https://github.com/onnx/onnx
• https://github.com/onnx/tensorflow-onnx
• https://github.com/fastai/fastai
• https://github.com/dnouri/skorch

18. How to Choose?
• Worth learning both
• If your are building for production go with
TensorFlow
• If you are doing research or trying to make
something new work go with PyTorch

19. The End
Contact
email: s@maclea.ai
twitter: sam_witteveen
https://github.com/samwit/