A recap of the major Azure announcements from the Microsoft Build conference in May 2018, with e a deep dive into some of the newest AI and Machine Learning features. Watch the live webinar recording at: https://info.microsoft.com/ww-ondemand-TopAzureTakeawaysMicrosoftBuild.html

1. Top Azure Takeaways
from Microsoft Build
Azure webinar series

2. Welcome
How do I ask a question?
If you have a technical or content-related question, please use the Q&A window
We will address the questions as they come in
Can I view this presentation after the webinar?
Yes, this presentation is being recorded
A link to the recorded presentation will be sent to the email address you used to register

3. Tim Heuer @timheuer
Cloud Developer Advocate
David Smith @revodavid
Cloud Developer Advocate
Our
speakers

4. What did you miss at BUILD?
Project Brainwave
• FPGA: Field-programmable
gate arrays
• Hardware accelerated
inference
• Circuit board quality
detection
Model deployment
• Customvision.ai export
• ONNX
• Containers
• Azure AI Gallery
Azure ML packages
• Forecasting
• Computer Vision
• Text Analytics
• ML.net
1

5. What did you miss at BUILD?
AI Lab
• Drawing Bot
• JFK Files
• AirSim
• Drone Search and Rescue
AI for Accessibility
• Seeing AI
• Translator
• Custom Voice
• Custom Vision
• Vison AI Dev Kit
And more…
• Other announcements you
may have missed.
2

6. Top Azure Takeaways from Microsoft BUILD
Project Brainwave
1

7. Breakthroughs in deep learning
demand real-time AI
Deep neural networks have enabled major
advances in machine learning and AI
Computer vision
Language translation
Speech recognition
Question answering
And more…
Problem
DNNs are challenging to serve and deploy
in large-scale online services
Heavily constrained by latency, cost, and power
Size and complexity outpacing growth of commodity CPUs
Convolutional Neural Networks
ht-1 ht ht+1
xt-1 xt xt+1
ht-1 ht ht+1
yt-1 yt yt+1
Recurrent Neural Networks

8. Azure ML Hardware Accelerated Models, powered by Brainwave
• Create and deploy trained AI models as webservices
• Millisecond-scale round-trip inference with accelerated ResNet50
ResNet 50 on FPGA
(8 billion calculations per image)
Record speed:
<1.8 ms per image

9. Microsoft Build // Jabil Case Study:
Project BrainwaveMay 7-9, 2018 // Seattle, WA

11. FPGAs for ultra-fast inferencing
DATA BUILD TRAIN DEPLOY
Stored on
Azure Premium Storage
Azure Machine
Learning
Jabil Classification Model
Transfer learning
ResNet-50
Circuit board images
Jupyter Notebook
Azure Machine Learning
Ultra-fast Inferencing
using FPGAs
ResNet-50

13. Project Brainwave
Resources
WATCH: BRK3202 Hyperscale hardware: ML at scale on top of Azure + FPGA
https://mybuild.microsoft.com/sessions
Background on Field Gate Programmable Arrays (FPGA):
https://www.microsoft.com/en-us/research/project/project-catapult/
Documentation: deploying a model with FPGA:
https://cda.ms/tS
Github samples, and link to request quota:
http://aka.ms/aml-real-time-ai

14. Top Azure Takeaways from Microsoft BUILD
AI Model Deployment
2

15. ONNX
Open Neural Network Exchange Format
What is ONNX?
• Open standard to exchange trained deep neural network models
• Jointly developed by Microsoft and Facebook
• Natively supported as an export by many frameworks: Caffe2, Microsoft
Cognitive Toolkit, MXNet, and PyTorch
• Converters for CoreML and Tensorflow
• Supported by Windows ML: GPU-accelerated inference on Windows 10
• Supports a variety of DNN architectures (and other predictive models, too)
The flexibility to run your trained AI model anywhere:
on devices, on the cloud, or on the edge.
onnx.ai

16. Azure AI
Gallery:
models
gallery.azure.ai/models

17. Customvision.ai

18. “Not Hotdog”
customvision.ai

19. AI Model Deployment
Resources
WATCH: THR3106 Train your Computer Vision model in the cloud and export it to run
anywhere
https://mybuild.microsoft.com/sessions
Get Started with Custom Vision:
https://cda.ms/tW
Background on ONNX:
https://onnx.ai/
Overview of Windows ML:
https://cda.ms/tV

20. Top Azure Takeaways from Microsoft BUILD
Azure ML Packages
3

21. Forecasting
Text Analytics
Computer Vision
Azure ML Packages for Python
Python packages for Computer Vision, Forecasting and Text Analytics
Easily build and deploy models on Azure Machine Learning
Provides high level APIs for data preparation, augmentation, training,
evaluating and deployment.
Model experimentation and comparison, run history, model
management and deployment through Azure ML

22. Computer Vision Simplified with Azure ML Packages
After  Code example – Azure ML Package
for Computer Vision
Transfer learning example using the AML
Package for Computer Vision.
Before  Code example – Keras
Training a small CNN, screenshots taken from the file
“cifar10_cnn.py” in the Keras github repository.

23. Proven & Extensible
Open Source
Developer Focused
Join at github.com/dotnet/machinelearning
ML.NET
Cross-platform Open Source Machine learning framework for .NET
Extensively used across Microsoft: Windows, Bing, Azure
High productivity throughout the entire ML workflow
Extensible to other frameworks (TensorFlow, CNTK…)

24. ML.NET usage at Microsoft
+ more!
Windows 10
Power Point
Excel
Bing

25. Azure ML Developer Libraries
Resources
WATCH: BRK3226 What's new with Azure Machine Learning
WATCH: BRK3203 Introducing ML.NET
https://mybuild.microsoft.com/sessions
Azure ML packages for Python
https://cda.ms/tY
ML.NET
http://dot.net/ml

26. Top Azure Takeaways from Microsoft BUILD
Azure AI Lab
4

27. Azure AI Lab
Complete implementations of AI applications
http://ailab.microsoft.com
Inspired by real-life problems
Implemented using Microsoft Cognitive Services: Vision, Speech, Language and Search
Try out applications online
Source code provided in Github
Easy to deploy your own instance to Azure

28. Azure
AI
Lab
JFK Files
Search the JFK files using Azure Cognitive Search
Data extraction, natural language processing (NLP), and image processing.
https://github.com/Microsoft/AzureSearch_JFK_Files

29. Azure
AI
Lab
Drones: Search and Rescue training with AirSim
Goal: train AI-powered drones to recognize targets for search-and-rescue
Train recognizer using real objects in controlled environment with Custom Vision
Simulate training data for drone control with AirSim (simulator based on Unreal)
https://github.com/Microsoft/AirSim

31. Azure
AI
Lab
Drawing Bot
Generative Adversarial Network to generate an image based on your description of a bird
Implementation of AttGAN in PyTorch, running in AKS (Azure Container Service)
drawingbot.azurewebsites.net

32. github.com/taoxugit/AttnGAN

33. Azure AI Lab
Resources
WATCH: BRK3218 Microsoft AI overview for developers
https://mybuild.microsoft.com/sessions
Azure AI Lab Projects:
ailab.microsoft.com
Microsoft Cognitive Services
https://cda.ms/tX

34. Top Azure Takeaways from Microsoft BUILD
AI for Accessibility
5

35. //
A new $25 million, 5-year program
aimed at harnessing the power of AI
to amplify human capability for the
more than one billion people around
the world with disabilities. Together,
we can create a more inclusive world.
AI for Accessibility

37. Microsoft translator and custom speech
Custom Speech Model Microsoft Translator

39. Vision AI Dev Kit
http://www.visionaidevkit.com

40. AI for Accessibility
Resources
AI for Accessibility
https://cda.ms/v2
WATCH: BRK2601 You Can’t Start a Fire Without a Spark: AI for Accessibility
WATCH: BRK3222 Computer Vision Made Easy: Consume or Build Your Own State-of-the-Art
Models And Use Them Everywhere
https://mybuild.microsoft.com/sessions
Microsoft Translator
https://www.microsoft.com/translator
Getting Started with Speech Service API
https://cda.ms/tZ
Microsoft Custom Speech Service
https://cda.ms/v0
Vision AI Dev Kit early access preview
http://www.visionaidevkit.com

41. Top Azure Takeaways from Microsoft BUILD
And more…

42. And so much more…
Azure Conversational AI
Develop together with Visual Studio and VS Code live sharing
Notepad now supports Linux line endings
Custom Excel functions
Azure Kubernetes Service general availability
…

43. Resources
Explore more BUILD presentations:
https://developer.microsoft.com/en-us/events/build/content
Azure Cognitive Services: Overview and Tutorials
https://cda.ms/v1
Free Azure subscription, with $200 free credits:
https://cda.ms/tT

44. Q&A

45. Tim Heuer @timheuer
Cloud Developer Advocate
David Smith @revodavid
Cloud Developer Advocate
Our
speakers

46. Thank you for joining us.

47. The power of Deep Learning on FPGA
Performance Flexibility Scale
Rapidly adapt to evolving ML
Inference-optimized numerical precision
Exploit sparsity, deep compression
Excellent inference at low batch sizes
Ultra-low latency | 10x < CPU/GPU
World’s largest cloud investment in FPGAs
Multiple Exa-Ops of aggregate AI capacity
Runs on Microsoft’s scale infrastructure
Low cost
$0.21/million images on Azure FPGA

48. Azure Cognitive Services
Powerful prebuilt AI models exposed as API services
Simple REST APIs with .NET, Java, Python, Node SDKs
Easily customize for highest accuracy
Train in the cloud and deploy anywhere
Vision
Speech
Language
Conversation
Bing Search
Knowledge

49. Docker container Export
You own
Prediction Endpoint
(Docker container)
Local or on-prem
Azure App Service