Onnx at lf oss na 20200629 v5
•Als PPTX, PDF herunterladen•
0 gefällt mir•138 views
ONNX - the emerging standard for interoperable and optimized AI inference and training. A graduated project of the Linux Foundation Artificial Intelligence - best practice open source - true multi-vendor open governance in a foundation.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ă„hnlich wie Onnx at lf oss na 20200629 v5
Ă„hnlich wie Onnx at lf oss na 20200629 v5 (20)
Mehr von ISSIP
Mehr von ISSIP (20)
KĂĽrzlich hochgeladen
KĂĽrzlich hochgeladen (20)
Onnx at lf oss na 20200629 v5
- 1. ONNX: Past, Present, and Future … with use cases Jim Spohrer (IBM), Prasanth Pulavarthi (Microsoft) Monday June 29, 2020 11:30am - 12:20pm Central Time
- 2. Today’s Speakers 6/23/2020 2 Introduction to ONNX: Past, Present, Future Jim Spohrer (IBM) – Director, Cognitive Opentech Group/CODAIT Linux Foundation AI – Technical Advisory Council Chairperson ONNX Steering Committee Member https://www.ibm.com/opensource/centers/codait/ https://www.linkedin.com/in/spohrer/ @JimSpohrer ONNX in Practice: Why we use it and how you can too Prasanth Pulavarthi (Microsoft) – Principal Program Manager, AI Platform ONNX Steering Committee Member, ONNX Co-Founder https://www.linkedin.com/in/prasanthpulavarthi/
- 3. Introduction to ONNX: Past, Present, and Future 6/23/202 0 3 › Past: Why ONNX? › Quick Review: ONNX Website Tour › https://onnx.ai/ › Present: Growth of Community and Tools › Quick Review: April 2020 ONNX Community Meeting › https://tinyurl.com/onnx-april2020 › Future: You, AI Landscape and ONNX! › Quick Review: Call to Action › https://landscape.lfai.foundation/
- 6. ONNX Website Tour https://onnx.ai/ 6/23/2020 6 › Home Page › News › About › Getting Started › Supported Tools › GitHub › Gitter
- 7. April 2020 Community Meeting https://tinyurl.com/onnx-april2020 6/23/2020 7 › Welcome and Updates › Partners › IBM Chief Data Office › Huawei Mindspore › Microsoft Runtime Optimizations › Xilinx FINN › UC Santa Cruz Genomics › Microsoft Azure OCR › Mathworks › SIGs and Working Groups › Some Highlights
- 9. One Year Growth 96/23/2020 From March 2019 – April 2020 As of June 22, 2020 1548 162 8.5K 1.5K 148 35
- 11. You, AI Landscape, and ONNX https://landscape.lfai.foundation/ 6/23/2020 11 › > 250 GitHub Projects › > 1.4M stars › > $12T Market Cap of companies › > $5B Investment in startups › ONNX key to interoperability and optimizations
- 12. ONNX in Practice: Why we use it and how you can too 126/23/2020
- 13. Microsoft 365 AI/ML at Microsoft | Research Edge
- 14. Common problems impacting ML productivity • Inference latency is too high to put into production • Training in Python but need to deploy into a C#/C++/Java app • Model needs to run on edge/IoT devices • Same model needs to run on different hardware and operating systems • Need to support running models created in several different frameworks • (more recently) Training very large models takes too long
- 15. ONNX Runtime https://onnx.ai @onnxai ONNX https://onnxruntime.ai @onnxruntime Compatible with PyTorch, TensorFlow, Keras, SciKit-Learn, and more
- 16. Training framework Deployment target CPU GPU FPGA NPU Improving ML deployment productivity
- 17. Training framework Deployment target CPU GPU FPGA NPU Improving ML deployment productivity ONNX Runtime Freedom to use tool of choice Strong performance and compatibility with platforms and accelerators
- 18. Speech with ONNX Runtime Speech Services power a vast landscape of products and services at MSFT. From Office, Cortana to Xbox and Bing, the ML models service hundreds of millions of requests a month. ONNX Runtime powers inferencing for Speech at high scale in production environments including Cognitive Services, on-premise solutions, and micro services Agility: 10x reduction in time to productize new models Performance: 10% accuracy and latency improvements
- 19. Azure Cognitive Services using ONNX Runtime
- 20. Azure Kinect with ONNX Runtime Desktop scenario Body Tracking SDK is installed on your PC Reduced the First Frame Processing Time by 7.8x on the GTX 1070 with ONNX Runtime with CUDA execution provider. Body Tracking SDK Azure Kinect developer kit for tracks bodies in 3D with advanced AI sensors that use sophisticated computer vision and speech models. IoT scenario (in progress) Body Tracking SDK is installed on Jetson TX2 (ARM CPU + Nvidia GPU)
- 21. ONNX model portability with ONNX Runtime model.onnx TensorRTOpenVINO model.onnx CUDA model.onnx
- 22. WindowsML with ONNX Runtime • Windows API for machine learning inference • Input models are ONNX models for broad framework support • ONNX Runtime as engine with DirectML acceleration
- 23. Azure Customers with ONNX Runtime Financial ISV • Uses AI for economic scenario modeling • Train models in Python with SciKit-Learn and PyTorch but production environment is pure C# • ONNX Runtime with C# API was a good fit (bonus 2x speedup)
- 24. Transformer Inferencing › Hugging Face provides popular transformer models, like BERT, GPT2, etc. › Can be trained with either PyTorch or TensorFlow › Hugging Face module transformers.convert_graph_to_onnx exports ONNX models › ONNX Runtime does inferencing with speedup whether you are using CPU or GPU
- 25. New: Transformer Training › Integrates with PyTorch (and TensorFlow) to accelerate training and fine- tuning of large transformer models › Incorporates latest algorithms and techniques such as DeepSpeed/ZeRO and Parasail/Adasum › Used by Office, Visual Studio, and others at Microsoft › Available as preview now 256/23/2020
- 26. Getting Started
- 27. Pre-trained ONNX models: https://onnx.ai/models
- 28. ML.NET Creating an ONNX model
- 29. Examples: Model Conversion python -m tf2onnx.convert --input frozen_model.pb --inputs input_batch:0, lengths:0 --outputs top_k:1 --fold_const --opset 8 --output deepcc.onnx import sklearn import skl2onnx initial_type = [('float_input', FloatTensorType([1, 4]))] onnx_model = skl2onnx.convert_sklearn(pipe, initial_types=initial_type) with open("logreg_iris.onnx", "wb") as f: f.write(onnx_model.SerializeToString())
- 31. Using ONNX Runtime C# …… also available for C, C++, Java, and JavaScript (Node.js)
- 32. Get involved
- 33. ONNX has open governance › Annual Steering Committee election › Technical decisions made by SIGs and Working Groups › All meetings open to everyone › Calendar: https://onnx.ai/calendar › GitHub: https://github.com/onnx › Gitter: https://gitter.com/onnx › Mailing List: https://lists.lfai.foundation/g/onnx-announce 336/23/2020
- 34. Thank You! Q & A 346/23/2020 Resources https://onnx.ai https://onnxruntime.ai
- 35. Legal Notices › The Linux Foundation, The Linux Foundation logos, and other marks that may be used herein are owned by The Linux Foundation or its affiliated entities, and are subject to The Linux Foundation’s Trademark Usage Policy at https://www.linuxfoundation.org/trademark-usage, as may be modified from time to time. › Linux is a registered trademark of Linus Torvalds. Please see the Linux Mark Institute’s trademark usage page at https://lmi.linuxfoundation.org for details regarding use of this trademark. › Some marks that may be used herein are owned by projects operating as separately incorporated entities managed by The Linux Foundation, and have their own trademarks, policies and usage guidelines. › TWITTER, TWEET, RETWEET and the Twitter logo are trademarks of Twitter, Inc. or its affiliates. › Facebook and the “f” logo are trademarks of Facebook or its affiliates. › LinkedIn, the LinkedIn logo, the IN logo and InMail are registered trademarks or trademarks of LinkedIn Corporation and its affiliates in the United States and/or other countries. › YouTube and the YouTube icon are trademarks of YouTube or its affiliates. › All other trademarks are the property of their respective owners. Use of such marks herein does not represent affiliation with or authorization, sponsorship or approval by such owners unless otherwise expressly specified. › The Linux Foundation is subject to other policies, including without limitation its Privacy Policy at https://www.linuxfoundation.org/privacy and its Antitrust Policy at https://www.linuxfoundation.org/antitrust-policy. each as may be modified from time to time. More information about The Linux Foundation’s policies is available at https://www.linuxfoundation.org. › Please email legal@linuxfoundation.org with any questions about The Linux Foundation’s policies or the notices set forth on this slide. 6/23/2020 35
Hinweis der Redaktion
- Link: https://sched.co/c3WZ Abstract: https://events.linuxfoundation.org/open-source-summit-north-america/program/schedule/ ONNX: Past, Present, and Future ONNX is now a graduated project in Linux Foundation AI. Are you a developer looking to operationalize machine learning models from different sources without compromising performance? Are you a data scientist who wishes there was a way to use the machine learning framework you want without worrying about how to deploy it to a variety of end points on cloud and edge? We'll describe ONNX, which provides a common format supported by many popular AI frameworks and hardware. Learn about ONNX and its core concepts and find out how to create ONNX models using frameworks like TensorFlow, PyTorch, and SciKit-Learn. We'll explain how to deploy models to cloud or edge using the high-performance, cross-platform ONNX Runtime, which leverages accelerators like NVIDIA TensorRT. Come learn how ONNX is being used in other LF AI projects, as well as about ONNX Work Groups that you can participate in. Finally, this talk will include case studies of Microsoft teams improving latency and reducing costs, thanks to ONNX. Speakers: Jim Spohrer IBM IBM Cognitive Opentech Group Jim Spohrer directs IBM’s open source Artificial Intelligence developer ecosystem effort. He led IBM Global University Programs, co-founded Almaden Service Research, and was CTO Venture Capital Group. After his MIT BS in Physics, he developed speech recognition systems at Verbex (Exxon) before receiving his Yale PhD in Computer Science/AI. In the 1990’s, he attained Apple Computers’ Distinguished Engineer Scientist and Technologist role for next generation learning platforms. With over ninety publications and nine patents, he received the Gummesson Service Research award, Vargo and Lusch Service-Dominant Logic award, Daniel Berg Service Systems award, and a PICMET Fellow for advancing service science. Prasanth Pulavarthi Microsoft AI Platform at Microsoft Prasanth Pulavarthi is Principal Program Manager, AI Platform and Co-Founder of ONNX. With nearly two decades of leading software development projects, Prasanth has a MS Computer Science from Stanford. ONNX is a now a graduated projected in Linux Foundation Artificial Intelligence. Monday June 29, 2020 11:30am - 12:20pm Conference Room 10  AI/ML/DL hosted by LF AI, Machine and Deep Learning (Framework, Libraries, Platform, Tools) Skill Level Entry level
- Nick Pentreath Presentation:
- What is ONNX? Open Neural Network Exchange Advantages Jagreet Kaur Gill | Data Science | 3 mins read | May 07, 2019 https://www.xenonstack.com/blog/onnx/
- Intel UpSquare
- Here are some of the popular frameworks that support conversion to ONNX. For some of these, like PyTorch, ONNX format export is built in natively, and for others, like Tensorflow or Keras, there are separate installable packages that handle the conversion. Support is available for popular models including object detection such as Mask RCNN and Faster RCNN, speech, and NLP including BERT and Transformer models.