4. 4
What is Deep Learning?
▪Deep Learning is not “Black Magic”
▪Deep Learning is using common statistics tools
▪Deep Learning is using iterative convergence process to
find a local minimum point which minimize the loss (“best
local solution”)
5. 5
What is Deep Learning ?
Gradient Descent !
Neural Networks Convolutional Neural Networks
Back Propagation!?
X Impossible Math?
X
6. 6
Deep Learning for system eng. & Architects
▪Set of Tools
▪Simple library
▪Development process
▪Non-standard programming
8. 8
The tools
▪Most common tools:
▪Classifier
▪Detector (Box Detectors)
▪Segmentor
▪Instance Segmentor
▪Time series networks
9. 9
Classifiers - 2
▪Answer on the existence of X (in image?)
▪Number classifier 0-9 (first CNN)
▪Common objects (Car, Chair, Cat…)
▪Cancer / No Cancer
▪Gun shot / broken glass
▪Most classifiers use some form of CNNs
10. 10
Classifiers
▪Single classifier could classify 100 objects
▪Most classifiers use COCO (~80 classes) as default
▪Example of La-net
▪First CNN for digits
▪Over 20 years old!!!
11. 11
BOX Detectors
▪Box detectors detects the location of the object
▪Used to detect:
▪People, face, corona mask
▪car, road signs
▪cats
12. 12
BOX Detectors
▪Box detectors could detect more than one class
▪Box detectors could detect overlapping objects
▪Known box detectors:
▪YOLO3 (most used) YOLO5 (Latest common model)
▪Yolo2PP, YOLOR (Fast)
▪YOLACT-EDGE (fast and accurate)
▪SSD / SSD2 / Mobile-SSD
13. 13
Segmentor and Multi-Instance Segmentor
▪Segmentor
▪defines class for each pixel
▪Pixel(X,Y) belongs to cat
▪FCN-8 basic implementation
▪Does not separate overlapping objects!!!
▪Common Segmentor U-Net
15. 15
Multi-Instance Segmentor - Implementation
▪Common / Simple implementation – MRCNN
▪Detectoron2 - Good implementation by Facebook
16. 16
(Time) series network
▪Runs on sequential data
▪Number of visitors in site
▪ECG data
▪Solar spot data
▪Single variable Data
▪Multi Variable
▪Example: LSTM, RNN
20. 20
Data
▪Data gathering (Initial)
▪Cameras / datasets
▪Data generation
▪Graphic engine (Unity / Omnviverse)
▪GAN
▪Data exploration
▪Anomaly detection
▪Class bias
▪Errored data
▪Data cleaning
▪Setting data pipes
21. 21
Training
▪Model selection Consideration
▪Performance constraints (run under < 1 TFLOPS)
▪Accuracy requirements
▪Model
▪Our own network
▪Existing network with/out weights
▪Training
▪No weights - from scratch
▪ Weights – Transfer learning
▪Hyper-param optimization
▪learning rate, decay, batch size, optimizer
22. 22
KPI – Measuring the results
▪Classifier accuracy
▪Percentage detected out of total objects
▪False detection
▪Detector accuracy
▪Classifier accuracy +
▪Box Accuracy
▪Segmentor Accuracy
▪Class accuracy
▪Segmentation accuracy – IoU (Intersection of Union)
▪Instance segmentor accuracy…..
23. 23
Model Optimization
▪For which platform? CPU, GPU, TPU?
▪What type of GPU/TPU?
▪How many cores?
▪What type of cores (FP32, FP16, Int 16, Int8)
▪What batch to use?
24. 24
Deployment and Automation
▪How to install the environment?
▪Python
▪Libraries
▪SDKs?
▪In Docker ?
▪How to automate the training and deployment platforms?
27. 27
Dataset Challenges
▪Existence – does the data exists somewhere
▪Faces, numbers, clothing, animals, cars, fonts
▪Can we use the data to train freely?
▪Datasets Has license like code
▪Does it fit my needs?
▪Data is RGB we need IR
▪Data is from close range we need distant range
▪Good plants we need sick plants
▪Cancer non-cancer image ration is 1:4.3*10^5
▪Data fine but too little
28. 28
Datasets
▪Know used datasets
▪MNIST – numbers
▪MNIST fashion – clothing
▪COCO – most used Dataset
▪Kaggle
▪Many dataset exists in Kaggle
▪Some closed for competition many open
▪Buy dataset
▪Create your own
▪Take images (10000? automate?)
▪Tag on your own / External service
30. 30
Deployment and Automation
▪For code we have Git and Git lab for CI/CD
▪Can we use it for Deep Learning
▪Requirements
▪Manage data gathering
▪Store data versioning
▪Create and store model versioning
▪Automatic training + KPI Measurements
▪Automatic deployment