For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/06/optimization-techniques-with-intels-openvino-to-enhance-performance-on-your-existing-hardware-a-presentation-from-intel/
Nico Galoppo, Principal Engineer (substituting for Ansley Dunn, Product Marketing Manager), and Ryan Loney, Technical Product Manager, both of Intel, present the “Optimization Techniques with Intel’s OpenVINO to Enhance Performance on Your Existing Hardware” tutorial at the May 2022 Embedded Vision Summit.
Whether you’re using TensorFlow, PyTorch or another framework, Galoppo and Loney show you optimization techniques to enhance performance on your existing hardware. With the OpenVINO Toolkit, built on the foundation of OneAPI, developers can utilize their own AI model or leverage one of the hundreds of pre-trained models available across vision and audio use cases.
In this presentation, you’ll learn how the Neural Network Compression Framework provides optimal model training templates for performance boosts while preserving accuracy, and how the Model Optimizer reduces complexity and makes model conversion faster. Other areas explored by Galoppo and Loney include auto device discovery to enable automatic load balancing and how to optimize for latency or throughput based on your workload.
30. Supported Devices
RUNTIME
GPU
CPU
APPLICATION
AUTO PLUGIN
CPU PLUGIN GPU PLUGIN
AUTO
AUTO chooses the device
AUTO sets device config
based on hints
AUTO handles the exec
logic on multiple devices
CPU
GPU
MYRIAD
AUTO
HETERO
MULTI
compiled_model = core.compile_model(model=model, device_name="CPU")
30
35. • Set-and-forget optimization knob – choose latency or throughput.
• Completely portable between the devices
• Throughput hint drives device-specific optimizations
• Even works for AUTO!
Portable Performance Hints
35
36. Automatic Batching
36
• How do you choose a good batch size?
• Let the OpenVINO™ runtime decide for you!
• No need to batch requests, runtime will do that too!
• Batching can improve throughput with select devices and models.
45. Try it yourself!
Learn more and download
at openvino.ai
Complete the Intel® Edge
AI Certification!
• Supercharge your career
• Approx 20 hours of video + quizzes +
coding
46. Deep Dive Session
Thursday, May 19
Intel AI Developer Expo—Let's Build Something Wonderful
Together
Session: 3:00 – 5:30 pm
Reception: 5:30 - 7:30 pm
Location: Room 209/210
Afternoon snack and post-session Reception will be
provided