"Collaboratively Benchmarking and Optimizing Deep Learning Implementations," a Presentation from General Motors
For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2017-embedded-vision-summit-bordoloi For more information about embedded vision, please visit: http://www.embedded-vision.com Unmesh Bordoloi, Senior Researcher at General Motors, presents the "Collaboratively Benchmarking and Optimizing Deep Learning Implementations" tutorial at the May 2017 Embedded Vision Summit. For car manufacturers and other OEMs, selecting the right processors to run deep learning inference for embedded vision applications is a critical but daunting task. One challenge is the vast number of options in terms of neural network models, frameworks (such as Caffe, TensorFlow, Torch), and libraries such as CUDA and OpenCL. Another challenge is the large number of network parameters that can affect the computation requirements, such the choice of training data sets, precision, and batch size. These challenges also complicate efforts to optimize implementations of deep learning algorithms for deployment. In this talk, Bordoloi presents a methodology and open-source software framework for collaborative and reproducible benchmarking and optimization of convolutional neural networks. General Motors' software framework, CK-Caffe, is based on the Collective Knowledge framework and the Caffe framework. GM invites the community to collaboratively evaluate, design and optimize convolutional neural networks to meet the performance, accuracy and cost requirements of a variety of applications – from sensors to self-driving cars.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie "Collaboratively Benchmarking and Optimizing Deep Learning Implementations," a Presentation from General Motors
Ähnlich wie "Collaboratively Benchmarking and Optimizing Deep Learning Implementations," a Presentation from General Motors (20)
Mehr von Edge AI and Vision Alliance
Mehr von Edge AI and Vision Alliance (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
"Collaboratively Benchmarking and Optimizing Deep Learning Implementations," a Presentation from General Motors
- 1. General Motors 1 Collaboratively Benchmarking and Optimizing Deep Learning Implementations Unmesh D. Bordoloi1, Massimo Osella1, Anton Lokhmotov2, Grigori Fursin2 May 2017 General Motors1 dividiti2
- 3. General Motors 3 • Towards global availability everywhere • Towards affordability by everyone Opportunities for optimization Among others, the high performance computing requirements is a factor
- 4. General Motors 4 Major components of autonomous car Radar Data Lidar Data Processing Sensor Fusion Camera Image Processing Scene Understanding Control Algorithms Deep learning has been identified as a key enabler for at least some of these functions Planning
- 5. General Motors 5 LATENCY THERMAL SAFETY SIZE Challenge: Computational Complexity
- 6. General Motors 6 Chaotic universe of deep learning DSP TensorFlow clBLAS AlexNet Many- core CPU Squeez eNet GoogleNet FPGA GM‘s world famous secret network CLBlast OpenBLAS cuBLAS (BVLC) cuDNN (BVLC) cuBLAS (NVIDIA) cuDNN (NVIDIA) cuBLAS fp16 (NVIDIA) cuDNN fp16 (NVIDIA) viennaCL libDNN- clBLAS libDNN- CLBlast libDNN- cuBLAS libDNN- viennaCL TensorRT TensorRT fp16 YOLO SSD Fast RCNN Mask RCNN RCNN VGG ResNet Caffe Torch Theano pyTorch CNTK GPU ASIC/ASIP Vendor 1 Vendor 2 Vendor n
- 7. General Motors 7 • The Computational Challenge for Self-Driving Cars • Collective Knowledge (CK) • Results • Outlook Outline of this talk
- 8. General Motors 8 Libraries Goal: CK (Collective Knowledge) to address the problem of benchmarking Hardware DSP Many- core CPU FPGA GPU CK clBLAS CLBlast Open BLAS cuBLAS (BVLC) cuDNN (BVLC) cuBLAS fp16 (NVIDIA) cuDNN fp16 (NVIDIA) viennaCL libDNN clBLAS libDNN- CLBlast libDNN- cuBLAS libDNN- viennaCL TensorRT TensorRT fp16 TensorFlowCaffe Torch Theano pyTorchCNTK Framework AlexNet Squeeze Net GoogleNet GM‘s world famous secret network YOLO SSD Fast RCNN Mask RCNN RCNN VGG ResNet Neural Net Model RESULTS KITTIImagenet GM dataset Dataset
- 9. General Motors 9 TECHNOLOGY • A technology for creating, sharing and re- using research artifacts such as workloads, datasets, tools, experimental results, experimental workflows, predictive models, etc. • Cross-platform • Customizable/extensible • Open-source • Lightweight Python2/3 package CK (Collective Knowledge) approach METHODOLOGY • Enables systematic and reproducible experimentation • Encourages artifact share and re-use • Involves the community to collaboratively find and explain unexpected behaviour • Crowdsourcing: benchmarking, design space exploration, optimization…
- 10. General Motors 10 • CK-Caffe is a CK based wrapper around the Caffe framework from Berkeley as well as the libraries associated with Caffe • Cross-platform: Linux, windows, android • Customizable: extensible with new models, datasets, forks of Caffe… • Open-source: https://github.com/dividiti/ck-caffe • Lightweight: Python2/3 package What is CK-Caffe?
- 11. General Motors 11 • GM, dividiti, and suppliers are using CK-Caffe for performance evaluation and design space exploration • We welcome the community to use and contribute • Go to https://github.com/dividiti/ck-caffe • (Or type in “CK-Caffe” or “cknowledge” in your search engine) • Follow the steps outlined in the guide, e.g. CK Caffe: Getting started
- 12. General Motors 12 • The Computational Challenge for Self-Driving Cars • Collective Knowledge (CK) • Results • Outlook Outline of this talk
- 13. General Motors 13 The design space evaluated (only examples) Platforms Networks Libraries • Nvidia GTX 1080 • Nvidia Tegra TX1 • … • … • ... • AlexNet • GoogleNet • SqueezeNet 1.0 • SqueezeNet 1.1 • GM Super Cool Net • … • cuDNN • fp16cuDNN • cuBLAS • fp16cuBLAS • libDNN • CLBlast • clBLAS • OpenBLAS • …
- 14. General Motors 14 Result 1: Impact of libraries
- 15. General Motors 15 Result 2: Redesigned neural nets
- 16. General Motors 16 Result 3: Impact of batch size
- 17. General Motors 17 Result 4: Impact of data precision
- 18. General Motors 18 Result 4: Impact of data precision
- 19. General Motors 19 Putting all optimizations together (on Tegra TX1)
- 20. General Motors 20 • The Computational Challenge for Self-Driving Cars • Collective Knowledge (CK) • Results • Outlook Outline of this talk
- 21. General Motors 21 • Collaborate in benchmarking/optimization: IP providers, chipmakers & OEMs can use CK-Caffe • GM, dividiti, several chip vendors • Provides validation to numbers shown in marketing slides • Results can be easily reproduced • And growing… • Student/community competitions • Academic community (University of Michigan) CK-Caffe for collaborative benchmarking/optimization
- 22. General Motors 22 • For applications/workloads like Deep Learning/CNN • What is the right kind of abstraction and models to represent workloads for real-time schedulability analysis? • For resources like GPU, FPGA, DSP • How can we formally analyze worst-case execution time (WCET), schedulability, utilization on such devices for deep learning networks? Open challenges for design automation / real- time systems community: examples
- 23. General Motors 23 • Links • http://cknowledge.org/ai.html • https://github.com/dividiti/ck-caffe • https://twitter.com/cruise Resources and Thanks!