"Highly Efficient, Scalable Vision and AI Processors IP for the Edge," a Presentation from Cadence
•
2 gefällt mir•889 views
For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/cadence/embedded-vision-training/videos/pages/may-2019-embedded-vision-summit-desai For more information about embedded vision, please visit: http://www.embedded-vision.com Pulin Desai, Vision Product Marketing Director at Cadence, presents the "Highly Efficient, Scalable Vision and AI Processors IP for the Edge" tutorial at the May 2019 Embedded Vision Summit. This presentation describes the architecture of the latest Tensilica-based vision and AI processor family, and illustrates how easily vision algorithms (e.g., SLAM, 3D capture) and AI inference can be implemented on these processors. See how this low-power architecture simplifies development of a scalable vision and AI solution from low to high end for mobile, AR/VR, surveillance and automotive markets.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie "Highly Efficient, Scalable Vision and AI Processors IP for the Edge," a Presentation from Cadence
Ähnlich wie "Highly Efficient, Scalable Vision and AI Processors IP for the Edge," a Presentation from Cadence (20)
Mehr von Edge AI and Vision Alliance
Mehr von Edge AI and Vision Alliance (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
"Highly Efficient, Scalable Vision and AI Processors IP for the Edge," a Presentation from Cadence
- 1. 1 © 2019 Cadence Design Systems, Inc Highly Efficient, Scalable Vision and AI Processors IP for the Edge Pulin Desai Director Product Marketing Cadence Design System May 2019
- 2. 2 © 2019 Cadence Design Systems, Inc Tensilica® Products DNA Standalone AI processor for AI-at-the-edge inference CNN, RNN, regression Fusion • Always-alert sensor processing • Multi-purpose: - Low-end imaging - Audio, video, speech Vision • Image and vision pre-/post-processing • AI at the edge • SLAM, HDR Xtensa® Processor Automated User-Defined Customization (TIE) CustomBroad Range of Application-Specific DSPs HiFi Controllers/ Custom ISAs • High-performance DSPs, NPUs, CPUs • Application-specific data types • Custom ISA, special functions ConnX • Radar, lidar • Communications: - 5G/4G/LTE-A/LTE - WiFi, smart grid - V2X, IoT - Infrastructure Hello Hi • Audio pre- and post-processing • Voice trigger • Noise reduction, audio
- 3. 3 © 2019 Cadence Design Systems, Inc Image Sensors Driving Huge Market for Vision DSPs • Image sensor market growing at ~12% CAGR from 2018 to 2025 • Largest growth to come from: – Mobile : ~8% CAGR – Automotive : 36% CAGR – Other markets including AR/VR, security camera, drone and robotics growing at a healthy 12% CAGR Source: IBS 0 2000 4000 6000 8000 10000 12000 14000 2018 2019 2020 2021 2022 2023 2024 2025 Munits Mobile NB & Tablet Automotive Others
- 4. 4 © 2019 Cadence Design Systems, Inc Vision Processing Is Integral to Camera Use Cases AR/VR Headsets Automotive Drones and Robots Mobile Surveillance Cameras Market Trends 4 to 6 cameras Real-time response >20 Camera Camera resolution going from VGA to >1080P Higher field of view (FOV) Higher frame rates (Real time) 4K/greater resolution Higher FOV Real time at video rate 3X increase in dual cameras over the next 4 years Higher resolution sensors Higher frame rates Real time at video rate Increase in camera resolution: >4K Applications Inside-out tracking Eye tracking Gesture recognition In-car monitor (DMS) Autonomous navigation (path planning) 360° view 360° capture Autonomous navigation (path planning) AR map Face detection/recognition Measurements Night shot Visual analytics (face/people/object) detection Face recognition Low light capture Vision Processing SLAM Depth sensing, Noise reduction De-warping SLAM HDR processing Depth sensing Image stitching De-warping SLAM HDR processing Depth sensing Image stitching De-warping SLAM HDR processing Depth sensing Multi sensor computational photography Noise reduction HDR processing Object tracking Noise reduction
- 5. 5 © 2019 Cadence Design Systems, Inc Simultaneous Localization and Mapping (SLAM) Challenges What Is SLAM? • A technique used in the robotics, drone, mobile and automotive markets to automatically construct or update a map of an unknown environment • SLAM is used in the AR/VR market for inside-out tracking • CPUs are great for general-purpose usage and prototyping • Lower performance and higher power consumption • GPUs have a variety of modalities for parallel processing, which can improve performance and help meet real-time requirements • GPUs are also power hungry and generate a lot of heat • Blocks are based on classical computer vision (CV) approaches • Relies heavily on a variety of linear algebra and matrix operations and is computationally heavy • Today these blocks are typically implemented on CPUs or GPUs Challenges
- 6. 6 © 2019 Cadence Design Systems, Inc AR Map Use Case Navigation Assistant Store finder AR Map with Visual Positioning
- 7. 7 © 2019 Cadence Design Systems, Inc AI Inference Workloads Vary by End Market Surveillance Automotive High to Mid-end mobile AR/VR Data Center Automotive (towards autonomous) Low-end mobile Smart home Intelligent devices (IoT) Processing Power Pick the right inference platform for the market—one size does NOT fit all! <0.5 TMAC/sec 1TMAC/sec to 4TMAC/sec 10s of 100s TMAC/sec
- 8. 8 © 2019 Cadence Design Systems, Inc Role of Vision DSP in the Camera Pipeline ISP AI Processing 2D/3D Noise Reduction HDR/WDR Super Resolution Image Stitching SLAM Classification Object Detection Segmentation Post Processing (YUV) Sensor Sensor RGIR RGBW Pre-ISP Processing Bayer Noise Reduction Image/Vision Processing Neural Net Processing Different Sensor Support = Specialized Low Power Vision and AI Processing Usually a HW Block
- 9. 9 © 2019 Cadence Design Systems, Inc Introducing the New Tensilica® Vision Q7 DSP for Vision and AI VisionandAIPerformance Time Vision P5 Vision Vision + AI Vision P6 Vision + AI Vision Q6 1.5X P6 Vision + AI Vision Q7 2X Q6
- 10. 10 © 2019 Cadence Design Systems, Inc Introducing Tensilica Vision Q7 DSP: 6th-Generation Vision and AI DSP * Tensilica® Vision Q7 DSP performance is relative to the Vision Q6 DSP Follow on to Vision Q6 DSP for vision and AI: Up to 2X performance for vision/AI applications, including floating-point performance Up to 2X GMAC/mm2 and GFLOPS/mm2 Up to 1.7X higher TOPS in same area Delivers up to 1.82TOPS
- 11. 11 © 2019 Cadence Design Systems, Inc AI/SLAM/Vision Kernel Improvement on Tensilica Vision Q7 DSP Up to 2X performance improvements on Tensilica® Vision Q7 DSP * Tensilica Vision Q7 DSP performance is relative to Vision Q6 DSP 0 0.5 1 1.5 2 2.5 PerformancerelativetoVisionQ6
- 12. 12 © 2019 Cadence Design Systems, Inc Tensilica Vision Q7 DSP: 6th-Generation Vision and AI DSP Relative numbers presented are in comparison with Tensilica® Vision Q6 DSP Vision Improvements AI Improvements Floating-Point Performance • Up to 2X performance on SLAM kernels with special ISA and special SLAM package • 2X 8x8 and 16x16 MAC compared to Vision Q6 DSP • Additional imaging, computer vision instruction (ISA) enhancements • 2X 8x8 MAC and 16x16 MAC compared to Vision Q6 DSP • 512 8-bit MAC and 128 16-bit MAC • 2X half-precision floating-point operations compared to Vision Q6 for AI applications • Support for on-the-fly decompression of weights • 2X (32) single-precision floating-point operations compared to Vision Q6 DSP for vision applications including SLAM • 2X (64) half-precision floating-point operations compared to Vision Q6 DSP for AI applications
- 13. 13 © 2019 Cadence Design Systems, Inc Tensilica Vision Q7 DSP: 6th-Generation Vision and AI DSP Relative numbers presented are in comparison with Tensilica® Vision Q6 DSP Backward Compatible with Vision P6 and Q6 • Vision Q7 DSP is a superset of Vision P6 and Q6 DSPs • Preserves software investment with easy migration • Custom instructions using Tensilica Instruction Extension (TIE) language iDMA Enhancements • New iDMA features to improve data movement • 3D iDMA • Data compression on iDMA • 4 channels ISO 26262 Readiness • IP and tools designed for ISO 26262 certification • Customers can generate ISO 26262-compliant optimized DSP and design SoC • Customers can add custom TIE instructions while maintaining ISO26262 certification
- 14. 14 © 2019 Cadence Design Systems, Inc SLAM Improvements on Tensilica® Vision Q7 DSP SLAM Sections Sensors Key Point Detection and Matching Pose Estimation Path Estimation and Loop Closure Bundle Adjustment Algorithm Options • Monocular, • Stereo, RGB-D • ORB, Shi and Tomasi, SIFT, SURF • RANSAC • SVD • Triangulation • Local bundle Adjustment • Displacement estimation • Loop closure detection • Path tuning • Global bundle adjustment • Pose-graph optimization Data Types • Fixed point • Single-precision floating point Vision Q7 Improvements • Added instructions with specific hardware package • 2X single-precision floating point compared to Vision Q6 DSP • Added instructions with hardware package for linear algebra functions • SLAM library optimized for Vision Q7 DSP
- 15. 15 © 2019 Cadence Design Systems, Inc Tensilica® Vision Q7 DSP Architecture / block diagram Integrated DMA (iDMA) Multiple AXi interfaces Dedicated AXi interface for iDMA (128/256) Data (1024) and Instruction Memory (128) Instruction (128) and Scalar Data Cache (32/128)
- 16. 16 © 2019 Cadence Design Systems, Inc Tensilica Solutions for Vision and AI Surveillance Automotive High to Mid -end mobile AR/VR Data Center Automotive (towards autonomous) Low-end mobile Smart home Intelligent devices (IoT) Processing Power Pick the right inference platform for the market—One size does NOT fit all! <0.5 TMAC/sec 1TMAC/sec to 4 TMAC/sec 10s of 100s TMAC/sec Vision Q7 Vision Q7 DNA 100 Tensilica® AI Processor +
- 17. 17 © 2019 Cadence Design Systems, Inc Multi-Core Solution with Tensilica Vision Q7 DSP Tensilica® Vision Q7 DSP + DNA 100 is ideal for products with both vision and AI experiences AXI4 Interconnect Vision Q7 Inst RAM Data RAM Vision Q7 Inst RAM Data RAM Vision Q7 Inst RAM Data RAM AXI4 Interconnect Vision Q7 Inst RAM Data RAM DNA100 Multi-Core Vision Q7 DSP Multi-Core Vision Q7 DSP and DNA 100 Processor
- 18. 18 © 2019 Cadence Design Systems, Inc Android Neural Network App AI Software Platform Tensilica VisionP6/Q6/Q7/DNA 100 Network Optimizer Neural Network Descriptor Tensilica® Neural Network Compiler Network Analyzer/Quantizer to 8b and 16b Target-Specific Library Selection DMA and Tile Manager Android Neural Network Runtime Android Neural Network API Android Neural Network HAL Tensilica IP Neural Network Driver Tensilica VisionP6/Q6/Q7/DNA 100 Lite Cadence® Tool Cadence SW Runtime User Code Tensilica Processor Android NN API Code
- 19. 19 © 2019 Cadence Design Systems, Inc Tensilica®: Comprehensive Vision Software Solutions Full ecosystem of software frameworks and compilers for all vision programming styles Linux Host CPU (Xtensa or Other) OpenVx/OpenCL Host Code Cadence Compiler / Tool Cadence SW library / Runtime User Code Cadence Tensilica DSP Vision + AI Enhanced Vision P6/Q6/Q7 DSP OpenCV based Imaging Library (lib “XI”) Embedded C/C++ OpenVx GraphHalide Tile Manager OpenVx Toolkit Xtensa C/C++ Compiler (XCC) XTOS (Single Thread) or XOS (Multithread) or Commercial RTOS OpenVx Runtime OpenCL BIFL library DMA Manager (libidma) OpenCL OpenCL Compiler (LLVM) OpenCL Runtime OpenCL BIFL library OpenCL Runtime Halide Compiler
- 20. 20 © 2019 Cadence Design Systems, Inc Tensilica® Vision and AI DSP Partner Ecosystem • Face and voice authentication • Face detection • WDR (wide dynamic range) • Super video-image stabilization • VSLAM • Super-resolution zoom, HDR • Camera processing • Live beautify • HDR / low-light enhance • Facial recognition • Dual-camera solutions • VSLAM • Convolutional neural networking • Imaging algorithm expertise • Imaging and vision experts • Low light enhancement • Advanced noise reduction • Face detection • VSLAM • Real-time 3D capture • VSLAM • System vendor and software partner • Surveillance • Car recognition • Face recognition • HDR, bokeh • Face / object recognition • VSLAM
- 21. 21 © 2019 Cadence Design Systems, Inc Summary Vision DSP and AI Processor Trends • Market needs both vision DSPs and AI accelerators • Image/vision processing requirements are increasing: new experiences, higher resolution and more sensors • SoCs are designed with different HW blocks for specific tasks: CPU, GPU, Vision DSP, AI acceleration Vision Q7 DSP • Up to 2X vision/AI/SLAM performance compared to the Tensilica® Vision Q6 DSP • Delivers 1.82 TOPS performance, 1.7X higher TOPS compared to the Vision Q6 DSP in the same area • Complete vision software: Lib, OpenCL, Halide • Complete AI software platform – ANN and Tensilica® Neural Network Compiler to meet market needs
- 22. 22 © 2019 Cadence Design Systems, Inc Resources 22 https://ip.cadence.com Embedded Vision Summit “Fundamentals of Monocular SLAM” Fundamentals Track Dr. Shrinivas Gadkari Tuesday, May 21, 4:20 PM - 5:25 PM
- 23. 23 © 2019 Cadence Design Systems, Inc Thank you 23