For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2019-embedded-vision-summit-sant-chu For more information about embedded vision, please visit: http://www.embedded-vision.com Nishita Sant, Computer Vision Manager, and Greg Chu, Senior Computer Vision Scientist, both of GumGum, present the "Creating Efficient, Flexible and Scalable Cloud Computer Vision Applications: An Introduction" tutorial at the May 2019 Embedded Vision Summit. Given the growing utility of computer vision applications, how can you deploy these services in high-traffic production environments? Sant and Chu present GumGum’s approach to the infrastructure for serving computer vision models in the cloud. They elaborate on a few aspects, beginning with modularity of computer vision models, including handling images and video equivalently, creating module pipelines, and desiging for library agnosticism so we can leverage open source developments. They also discuss inter-process communication—specifically, the pros and cons of data serialization, and the importance of standardized data formats between training and serving data, which lends itself to automated feedback from serving data for retraining and automated metrics. Finally, they discuss GumGum's approaches to scaling, including a producer/consumer model, scaling triggers and container orchestration. They illustrate these aspects through examples of image and video processing and module pipelines.

1. © 2019 GumGum
Creating Efficient, Flexible, and
Scalable Cloud Computer Vision
Applications: An Introduction
Nishita Sant, Greg Chu
GumGum
May 2019

2. © 2019 GumGum
GumGum Overview
GumGum is an artificial intelligence
company with a particular focus in computer vision
2

3. © 2019 GumGum
Agenda
3
1. Why Computer Vision in the Cloud?
2. Computer Vision System Design
• Modularity
• Inter Process Communication
• Scaling
3. Putting it All Together
4. Key Takeaways

4. © 2019 GumGum
Why Computer Vision in the Cloud?
4
• Reliability
• Latency
• Hardware and Software Resources
• Power, Size and Cost Constraints
• Volume of Data/ Throughput
It depends!

5. © 2019 GumGum 5
Design Principles

6. © 2019 GumGum
Design Principles
6
03 Scalability
01 Modularity/ Flexibility 02 Inter Process Communication

7. © 2019 GumGum
Design Principles: 01 Modularity/Flexibility
7

8. © 2019 GumGum
Design Principles: 01 Modularity/Flexibility
8
Red-Ford-Mustang
Not-Red-Ford-Mustang DenseNet
Red Ford Mustang

9. © 2019 GumGum
Design Principles: 01 Modularity/Flexibility
9
Blue-BMW-Z4
Not-Blue-BMW-Z4
Blue BMW Z4
DenseNet

10. © 2019 GumGum
Design Principles: 01 Modularity/Flexibility
10
Computer Vision Modules
Input
ML-based: CNNs, LSTMs, SVMs
Traditional: Feature Matching
Hybrid: Feature Matching + SVM
Heuristic: Design Logic
Output
• Vehicle
• Ford Mustang
• Red
Queue/Load
Balancer
Auto Scaling
Group

11. © 2019 GumGum
Objects
Design Principles: 01 Modularity/Flexibility
11
Computer Vision Module Pipelines
Make/Model Dominant Color

12. © 2019 GumGum
Design Principles: 02 Inter Process Communication
12

13. © 2019 GumGum
Design Principles: 02 Inter Process Communication
13
JSON for IPC
Property : Car
Region : List of points
describing a contour
Property : Car, Ford, Mustang, Red
Property : Tire, Black
Sub-region : List of
points describing a
contour within
another contour

14. © 2019 GumGum
Design Principles: 02 Inter Process Communication
14
Data collection/annotation/verification
Re/train model
Serving

15. © 2019 GumGum
Design Principles: 02 Inter Process Communication
15
Data Serialization
Pros Cons
Safe contract between
producers & consumers
Schema changes
requires careful effort
~10x reduced file size Overhead for serialization
code/registry
Cleaner data
Schema evolution

16. © 2019 GumGum
Design Principles: 03 Scalability
16

17. © 2019 GumGum
Design Principles: 03 Scalability
17

18. © 2019 GumGum
Design Principles: 03 Scalability
18

19. © 2019 GumGum
Design Principles: 03 Scalability – Scaling Triggers
1. # of messages
• e.g. SQS & CloudWatch, ALB
2. # of messages & average CPU utilization
• e.g. SQS & custom logic in Lambda
3. Container management solutions
• e.g. K8s/ECS
19
/
+

20. © 2019 GumGum
Putting It All Together:
How we serve our CV at GumGum
20

21. © 2019 GumGum 21

22. © 2019 GumGum 22

23. © 2019 GumGum 23

24. © 2019 GumGum
Multivitamin:
A Python framework for serving ML models
https://github.com/gumgum/multivitamin
24

25. © 2019 GumGum 25
Multivitamin: Data Flow within 1 Application

26. © 2019 GumGum
Multivitamin: Code Sample
26

27. © 2019 GumGum 27

28. © 2019 GumGum
NHL sponsor & placement detection
28
Sample output:

29. © 2019 GumGum
Key Takeaways
29
• Efficiency
• Inter Process Communication
• Modularity
• Scalability
• Client Requirements

30. © 2019 GumGum
Thank You
30

31. © 2019 GumGum
Resources
• Computer Vision: At the Edge or In the Cloud? It Depends.
• Keras Wiki, Keras Documentation, Github – Keras
• MXNet
• PyTorch
• Open Neural Network Exchange
• Caffe2
• TensorFlow
• Swagger - API Development Tool
• Amazon ElasticSearch
• Amazon ElasticBeanstalk
• Spring Framework for Java Platform
• Densly Connected Convolutional Networks
• Three reasons why apache avro data serialization is a good choice
• Apache Avro Schema 1.8.1

32. © 2019 GumGum
About GumGum
70%
Of Fortune 100
Companies
240+
Gumgummers
16
Offices On
16 Continents
99.4%
CAGR Since
2012

33. © 2019 GumGum
Computer Vision Applications
CONTENT
ENHANCES AD
Ad creative built to
incorporate image
content
Localized detection of
objects, people, or
body parts using
Computer Vision
…in Augmented
Advertising

34. © 2019 GumGum
Sports Sponsorship Measurement

35. © 2019 GumGum
Social Listening
GUMGUM SOCIAL
Ingest social posts with visual content
from firehose of Twitter, Instagram, etc.
Detect presence and location of brand
logos or other objects
Analytics dashboard, interact with
influencers

36. © 2019 GumGum
Design Principles
ACCURACY SPECS CAN BE CASE SPECIFIC
Example: Brand Safety
in Digital Advertising
High Recall/Low
Precision = Risks
Minimized
High Recall/Low Precision
= Inventory Reduced

37. © 2019 GumGum
Design Principles : Modularity/Flexibility
Inter-Process Communication

38. © 2019 GumGum
Design Principles: Modularity/Flexibility
Inter-Process Communication