- What is Deep Learning? - What is DevOps?

2. WIFI: awsDevDay | PASS: CodeHappy
U P N E X T :
Deploy Deep Learning
Models on Amazon ECS

3. T H A N K S T O O U R F R I E N D S A T :

4. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Yash Pant, | AWS Solution Architect | Amazon AI
September 12, 2017
Deploy Deep Learning
Models on Amazon ECS

5. Why are we here today?
Front-End team
Data Engineering team
Analysts / DS team
DevOps team
Business Problem
Data
ML Model
ML Application
Deep Learning is Great
The Missing Link:
• How do we
operationalize it?
• How do we scale it?

6. What to Expect from the Session
• What is Deep Learning?
• Demo: Build a Deep Learning model with AWS
• What is DevOps?
• Demo: Build a CI/CD toolchain for deploying AI model

7. Significantly improve many applications on multiple domains
“deep learning” trend in the past 10 years
image understanding speech recognition natural language
processing
…
Deep Learning
autonomy

8. Deep Learning at Amazon

9. AWS AI Services

10. Demo – Building an AI model with AWS

11. What is DevOps?

12. What is DevOps?
• Cultural philosophies
• Practices
• Tools

13. DevOps Culture
• Dev & Ops coming together
• No more “silos”
• Shared responsibility
• Ownership
• Visibility and communication

14. DevOps Practices
• Microservices architecture
• Moving away from large “monolithic”
architectures to smaller decoupled services

15. DevOps Practices
• Continuous Integration
• Continuous Delivery & Deployment

16. DevOps Practices
• Infrastructure as Code
• Model your AWS resources using code

17. DevOps Practices
• Monitoring and Logging
• Track and analyze metrics and logs
• Understand real-time performance of
infrastructure and application

18. Reliability
Benefits of DevOps
Speed
Scale
Rapid DeliveryImproved Collaboration
Security

19. AWS Services: DevOps Solution for AI
Applications
• AWS CodePipeline
• AWS CodeBuild
• AWS CloudFormation
• Amazon ECS
• Amazon ECR

20. Fully managed build service that compiles source code,
runs tests, and produces software packages
Scales continuously and processes multiple builds
concurrently
You can provide custom build environments suited to
your needs via Docker images
Only pay by the minute for the compute resources you
use
Launched with CodePipeline and Jenkins integration
AWS CodeBuild

21. Continuous delivery service for fast and
reliable application updates
Model and visualize your software release
process
Builds, tests, and deploys your code every time
there is a code change
Integrates with third-party tools and AWS
AWS CodePipeline

22. Easy way to create and manage a collection of
related AWS resources.
Provisioning and update them in an orderly
and predictable fashion.
Templates concisely capture resource
relationships.
You can easily view your AWS resources and
their relationships, and arrange their layout so
that the diagram makes sense to you.
AWS CloudFormation

23. Amazon ECS
A highly scalable, high performance container
management service that supports Docker
containers
Allows you to easily run applications on a
managed cluster of Amazon EC2 instances
Eliminates the need for you to install, operate,
and scale your own cluster management
infrastructure

24. Amazon ECR
A fully-managed Docker container registry that
makes it easy for developers to store, manage,
and deploy Docker container images
Integrated with Amazon EC2 Container Service
(ECS), simplifying your development to
production workflow.
Amazon ECR eliminates the need to operate your
own container repositories or worry about scaling
the underlying infrastructure.

25. A CI/CD architecture for deploying Deep Learning
Applications

26. Demo - Deployment

27. Amazon + Intel
• 10+ year partnership
• Joint development
• Shared customer passion
• High performance + low costs
• World class supply chain
CLOUD &
DATA
CENTER
THINGS &
DEVICES
AWS IOT Alexa Voice
Services
Amazon EC2 Amazon S3

28. Bigger Data Better Hardware Smarter Algorithms
Why Now?
Image: 1000 KB / picture
Audio: 5000 KB / song
Video: 5,000,000 KB / movie
Transistor density doubles
every 18 months
Cost / GB in 1995: $1000.00
Cost / GB in 2015: $0.03
Advances in algorithm
innovation, including neural
networks, leading to better
accuracy in training models

29. Optimized Deep Learning Environment
Fuel the development of vertical solutions
Deliver excellent deep learning environment
Develop deep networks across frameworks
Maximum performance on Intel architecture
EC2
Intel® Math Kernel Library (Intel® MKL)

30. Elastic Compute Cloud (EC2)
C4 Instances
• “Highest performing processors and the lowest price/compute
performance in EC2”1
• Excellent inference / fine-tuning performance
• Vilynx
• Deep learning for video content extraction
• Supports various companies: CBS, TBS, etc.
•
1https://aws.amazon.com/ec2/instance-types/https://www.stlmag.com/news/st-louis-app-pikazo-will-turn-your-profile-picture/
• Pikazo app
• Transforms photos into artistic render

31. Elastic Compute Cloud (EC2)
C4 Instances
c4.8xlarge On-Demand:
• $1.675/hr
GoogleNet inference:
• batch size 32
• 262 ims/sec = 3.8 ms/im
• 1 million images costs
$1.77
Spot prices are cheaper
OS: Linux version 3.13.0-86-generic (buildd@lgw01-51) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #131-Ubuntu SMP Thu May 12 23:33:13 UTC 2016.
MxNet Tip of tree: commit de41c736422d730e7cfad72dd6afc229ce08cf90, Tue Nov 1 11:43:04 2016 +0800. MKL 2017 Gold update 1
6.1 2.4 1.2 0.8
679.5
262.5
79.7 73.9
0
200
400
600
800
AlexNet GoogLeNet v1 ResNet-50 Inception v3
Images/Sec
c4.8xlarge MXNet Inference
No MKL MKL

32. Learn more
AWS AI Services: https://aws.amazon.com/amazon-ai/
AWS DevOps Tools: https://aws.amazon.com/products/developer-tools/
AWS Container Service: https://aws.amazon.com/ecs/
DevOps for AI Solution Details:
https://aws.amazon.com/blogs/ai/deploy-deep-learning-models-on-
amazon-ecs/

33. Thank You!