“Deploying Deep Learning Applications on FPGAs with MATLAB,” a Presentation from MathWorks
•
0 gefällt mir•100 views
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2020/11/deploying-deep-learning-applications-on-fpgas-with-matlab-a-presentation-from-mathworks/ For more information about edge AI and computer vision, please visit: https://www.edge-ai-vision.com Jack Erickson, Principal Product Marketing Manager at MathWorks, presents the “Deploying Deep Learning Applications on FPGAs with MATLAB” tutorial at the September 2020 Embedded Vision Summit. Designing deep learning networks for embedded devices is challenging because of processing and memory resource constraints. FPGAs present an even greater challenge due to the complexity of programming in Verilog or VHDL, and the hardware expertise needed for prototyping on an FPGA. This talk illustrates a workflow to facilitate the design and deployment of these applications to FPGAs using pre-built bitstreams without the need for much hardware expertise. Starting with a pre-trained model trained either in MATLAB or any framework of your choice, Erickson demonstrates the workflow to prototype and deploy the trained network from MATLAB to an FPGA. He illustrates this flow using a deep learning network for image recognition, deploying it to the Xilinx MPSoC board for inference using APIs from MATLAB. This demonstrates how deep learning algorithm engineers can quickly explore different networks and their performance on an FPGA from MATLAB.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie “Deploying Deep Learning Applications on FPGAs with MATLAB,” a Presentation from MathWorks
Ähnlich wie “Deploying Deep Learning Applications on FPGAs with MATLAB,” a Presentation from MathWorks (20)
Mehr von Edge AI and Vision Alliance
Mehr von Edge AI and Vision Alliance (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
“Deploying Deep Learning Applications on FPGAs with MATLAB,” a Presentation from MathWorks
- 1. © 2020 MathWorks Deploying Deep Learning Application on FPGAs with MATLAB Jack Erickson Technical Marketing September 2020
- 2. © 2020 MathWorks Airborne Image Analysis Deep Learning Deployment on Embedded Devices 2 Autonomous Driving Industrial Inspection Medical Image Analysis Wireless Modulation Classification Radar Signature Classification
- 3. © 2020 MathWorks System Requirements Drive Network Design 3 Industrial Inspection Hardware/Software Engineers Systems Engineer Deep Learning Practitioner Camera specs Accuracy Latency Cost Power
- 4. © 2020 MathWorks Challenges of Deploying Deep Learning to FPGA Hardware: Convolution Each stride is an 11x11x3 matrix multiply-accumulate →105M floating-point multiply operations! 55 55 96 filters stride=4 224 224 11x11 96 filters of 11x11x3 of 32-bit parameters →140k bytes 11x11 →1.16M bytes of activations
- 5. © 2020 MathWorks Challenges of Deploying Deep Learning to FPGA Hardware conv 1 conv 2 conv 3 conv 4 conv 5 fc6 fc7 fc8input Total 140K 1.2M 3.5M 5.2M 1.8M 148M 64M 16M Parameters (Bytes) n/a 230 M 1.1M 728K 252K 252K 168K 16K 16K 4K Activations (Bytes) 588K 3.1 M 105M 223M 149M 112M 74M 37M 16M 4MFLOPs n/a 720 M Off-chip RAM Block RAM DSP Slices
- 6. © 2020 MathWorks Deploying Deep Learning to FPGA Hardware Requires Collaboration 140K 1.2M 3.5M 5.2M 1.8M 148M 64M 16M Parameters (Bytes) conv 1 conv 2 conv 3 conv 4 conv 5 fc6 fc7 fc8 n/a input Total 230 M 1.1M 728K 252K 252K 168K 16K 16K 4K Activations (Bytes) 588K 3.1 M 105M 223M 149M 112M 74M 37M 16M 4MFLOPs n/a 720 M Resize Acquire data Output / display Mem i/f Optimize • Network /layers • Fixed-point quantization • Processor micro-architecture
- 7. © 2020 MathWorks A Collaborative AI Workflow Model design and tuning Hardware accelerated training Interoperability AI Modeling Integration with complex systems System verification and validation System simulation System Design Data cleansing and preparation Simulation- generated data Human insight Data Preparation 7 Enterprise systems Embedded devices Edge, cloud, desktop Deployment I Iteration and Refinement
- 8. © 2020 MathWorks Design and Analyze Your Networks in MATLAB Classification Learner app to try different classifiers and find the best fit for your data set Deep Network Designer app to build, visualize, and edit deep learning networks Model design and tuning Hardware accelerated training Interoperability AI Modeling 8
- 9. © 2020 MathWorks 9 MATLAB Interoperates with Other AI Frameworks Caffe importer Keras importer Model design and tuning Hardware accelerated training Interoperability AI Modeling
- 10. © 2020 MathWorks Application logic Deploy from MATLAB to a Variety of Hardware Platforms 10 FPGA CPU GPU Enterprise systems Embedded devices Edge, cloud, desktop Deployment
- 11. © 2020 MathWorks FPGA Deployment from MATLAB
- 12. © 2020 MathWorks Application logic Get Started Prototyping on FPGA with Deep Learning HDL ToolboxTM Hardware support package Deep learning processor with I/O and external memory interfaces • Int8 or single precision • Supported boards: • Xilinx: ZCU102 or ZC706 • Intel: Arria10 SoC • http://mathworks.com/hardware-support.html FPGA Bitstream Layer control instructions Weights & Activations Fully Connected Module Convolution Module Processor Control Memory Access Activations Activations Activations
- 13. © 2020 MathWorks Defect Detection Example Application logic Pre-processing: Extract regions and resize Post-processing: Annotate and label Inference: Predict using trained network FPGA
- 14. © 2020 MathWorks Run Deep Learning on FPGA from MATLAB 14
- 15. © 2020 MathWorks Application logic Prototyping: Design Exploration and Customization 15 Layer control instructions Weights & Activations Re-train
- 16. © 2020 MathWorks Design Exploration and Customization 16
- 17. © 2020 MathWorks Optimizing Deep Learning Applications Requires Collaboration Systems Engineer Deep Learning Practitioner Hardware/Software Engineers Fully Connected Module Convolution Module Processor Control Memory Access Activations Activations Activations x +/- Σ 32 / x +/- Σ 8 / x +/- Σ 8 / x +/- Σ 8 / x +/- Σ 8 /
- 18. © 2020 MathWorks INT8 Quantization 18
- 19. © 2020 MathWorks Application logic Deep Learning HDL Toolbox 19 Layer control instructions Weights & Activations Re-train Modify network % Create target object hTarget = dlhdl.Target(…) % Create workflow object, using the target hW = dlhdl.Workflow(…); % Compile the network hW.compile; % Program the bitstream and deploy the compiled network and weights hW.deploy; % Run prediction [score, speed] = hW.predict(img, ‘Profile’, ‘on’);>> deepNetworkDesigner Parameters Speed 140 MB 18 fps 84 MB 45 fps 68 MB 139 fps Quantize >> deepNetworkQuantizer Generate HDL Iterate and Converge on Deep Learning FPGA Deployment from MATLAB
- 20. © 2020 MathWorks Application logic Deploy from MATLAB to a Variety of Hardware Platforms 20 FPGA CPU GPU Tune for system requirements Prototype from MATLAB Configure and generate RTL Deep Learning HDL Toolbox
- 21. © 2020 MathWorks Resource Slide Deep Learning Solutions in MATLAB https://www.mathworks.com/solutions/deep-learning.html Deep Learning HDL Toolbox https://www.mathworks.com/products/deep-learning-hdl.html Onramp: Deep Learning in MATLAB https://www.mathworks.com/learn/tutorials/deep-learning-onramp.html MathWorks FPGA Solutions Page https://www.mathworks.com/solutions/fpga-asic-soc-development.html 21