SlideShare ist ein Scribd-Unternehmen logo
1 von 64
Downloaden Sie, um offline zu lesen
06 image features
Legal Notices and Disclaimers
This presentation is for informational purposes only. INTEL MAKES NO WARRANTIES,
EXPRESS OR IMPLIED, IN THIS SUMMARY.
Intel technologies’ features and benefits depend on system configuration and may require
enabled hardware, software or service activation. Performance varies depending on system
configuration. Check with your system manufacturer or retailer or learn more at intel.com.
This sample source code is released under the Intel Sample Source Code License
Agreement.
Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
Copyright © 2018, Intel Corporation. All rights reserved.
2
06 image features
Image Features
Image features are interesting locations in an image.
We want to be able to apply…
• Feature detection
Finding regions, when moved, that cause maximal variation
• Features description
Building a good context around a feature
We use this information about these interesting locations for a wide variety of uses.
Feature Detection
Background feature matches any background cell.
Foreground feature matches any foreground cell.
L-edge matches any vertical edge.
T-edge matches any horizontal edge.
But, CORNER only has one match!!!
Uses of Image Features: Matching
Classification (Object Recognition)
What is this object?
It is a Crayon
Tracking of Features in Motion
Image reference: https://commons.wikimedia.org/wiki/File:Madison_Square_Garden_food_court.jpg
Image Stitching
Object Detection
Image from OpenCV documentation: https://docs.opencv.org/3.3.0/d7/d8b/tutorial_py_face_detection.html
Techniques for Image Features
Techniques that can be used for any of these use-cases:
• Corners
• HOG
• SIFT
• SURF
• FAST
• BRIEF
• However, these techniques are typically developed by a research group to solve
one use case.
06 image features
Corners
An edge is indicated by a strong derivative.
• This is helpful to find an edge, but all features along that edge may look the same.
• We want a better way to identify an interesting feature.
Corners are edges in two orthogonal directions.
• Hopefully, a corner will have enough information to identify our feature over multiple
images, or frames.
Harris Corners
Harris
Corners
Corner on
Original
Image
Harris Corners
Used for sparse feature matching.
Use a Gaussian weighted window.
• Allows detection when there is in-plane rotation of an interesting feature
• Down-weights edge-like features
Compute a scalar interest measure.
Find local maxima.
Harris Corners
Harris corners are defined as second-order derivatives.
We create a second-derivative Hessian image.
A corner is identified when this matrix has two large eigenvalues.
Look at eigenvalues of the autocorrelation matrix.
• Eigenvectors tell us the orientation of a matrix.
• Eigenvalues tell us the scale in those directions.
• Either minimum eigenvalue, or something more complicated like:
𝜆0 𝜆1 − 0.06 𝜆0 𝜆1
2
Harris Corners and Shi-Tomasi
Shi-Tomasi modified the Harris corner.
In both cases we take the:
• Correlation between gradients in x and
y directions
Harris used:
Shi-Tomasi used:
• Invariant to rotation (because of eigenvalues)
𝜆0 𝜆1 − 0.06 𝜆0 𝜆1
2 < 𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑
𝜆0 < 𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑
Sub-Pixel Methods
Harris and Shi-Tomasi corners are good for feature recognition and used…
• When you are extracting geometric measurements
• When you need higher resolution feature recognition
If I am a…
Biologist trying to generate a 3D reconstruction of a blood vessel
Or a marine determining the location of a target in a satellite image
I need sub-pixel coordinates.
Sub-Pixel Methods
If we know an approximate (pixel level) location Capprox of a corner we can
find a better corner Cimproved by:
• Sampling a region of points P around Capprox.
• Setting up a series of equations that capture the directional similarities
between each p of P and Cimproved
Note, we don’t know Cimproved!
• Solve the equations. The result is Cimproved.
• Repeat this process.
Sub-Pixel Methods
Similarity measure has these important properties:
• Similarity(0,any) = Similarity(any,0) = 0
• Similarity(perpendicular directions) = 0
For us, our directions are:
1. The gradient at a region point p
2. The direction from C_approx to p
Sub-Pixel Methods
When the similarity between the direction of the gradient and the direction from
C_approx to p is 0
• The directions must be perpendicular.
In turn, this means that the direction lies along an edge
• Multiple edges meet at a corner!
The similarity measure is the dot product between the gradient at p and the direction
from Capprox to p.
06 image features
Histogram of Oriented Gradients
Developed to detect pedestrians in a scene
Techniques focus on detection accuracy instead of computing speed and efficiency.
• Details about an object are determined as accurately as possible.
HOGs are a distribution of the directions of the gradients.
The histogram confers distribution
The oriented gradients confer direction of gradient
Histogram of Oriented Gradients
Features produced by HOG are the histograms from the cells.
Constants are Tuned for Recognizing Humans
HOG is described in terms of one ROI, but many steps can be precomputed
on the whole image.
Because HOG was originally developed for pedestrian recognition, many
HOG ROIs are humans.
Humans are basically tall rectangles.
Therefore, we will make patches around an ROI by using a…
Patch aspect of 2 (tall) to 1 (wide)
Cell size of 8 x 8 because it captures human features like eyes
You Can Use HOG on Other, Non-Human,
Features
Image
ROI
Frames for Normalization
Cells
Tree image from: https://commons.wikimedia.org/wiki/Christmas_tree#/media/File:Piazza_Portanova_Natale_2008.jpg
HOG: An Example
Step 1: Take the image and create a patch of the ROI with a fixed aspect ratio of 1:2.
 Essentially, you are cropping the image around the ROI.
Patch (or ROI)
HOG: An Example
Step 2: Calculate the horizontal and vertical gradients for each color in the patch.
– Convolve with [-1,0,1] kernels
– For instance, you can use Sobel for this step
HOG: An Example
Step 3: Consider the component-wise gradients in polar coordinates.
– Take x, y gradients and convert to Q
• x, y  r, Q [graphic]
• At every pixel, take max r over the three color channels
• r is magnitude
• Associated direction is the theta of that max r
HOG: An Example
Step 4: The patch is divided into 8 x 8
cells, from which we make a histogram
of gradients.
Each bucket, or bin, are angles (Q)
– Q = 0, 20, 40, ..., 160
• This gives us nine total bins
– We assign the magnitude (r) of a pixel to its
angle (th) bin.
– For angles between bins, we divide it
proportionally to distance from the bins.
Direction (Q) Magnitude (r)
0 20 40 60 80 100 120 140 160
HOG: An Example
Step 5: Create a mega-cell and associated mega-histogram with 36
total values.
• Use a sliding 16 x 16 window (of four histograms each time you slide the
window).
• Normalize this window with respect to L2 norm to create one block of
features.
• Slide the window over 8 pixels and repeat normalization.
• Some pixels will be repeated normalized, but with respect to different
neighborhoods extending differed directions.
Features per Patch: Some Simple Math
The result is 3,780 total features for one patch.
• On a 128 x 64 image we have 16 x 8 cells that are 8 x 8
• We have 15 x 7 mega-cells
• Which gives us 105 blocks of 36 features
06 image features
Scale Invariant Feature Transform (SIFT)
Is a feature detection and description method that is
invariant to scale
• Particularly helpful when analyzing larger corners
Scale-space filtering is used to detect keypoints with
different scales.
• Uses difference of Gaussians (DoG) to approximate
Laplacian of Gaussians (LoG).
• Finds maxima with respect to space and scale.
Space is across an image
Scale is up and down a DoG pyramid
SIFT: Feature Detection
First, we need to find candidate keypoints (feature detection).
We do this by finding extrema versus neighbors in a Gaussian pyramid.
This is on the same scale but can be high or lower resolution.
Then, we find candidates by filtering out points with low contrast and on edges.
SIFT: Feature Description
Now that feature detection is complete, we need to describe the feature.
• The neighborhood is proportional to level of the extrema
• Estimate orientation in neighborhood around the keypoint
• Compute a histogram of gradients in this neighborhood
• Normalize histogram
SIFT: Matching
Lastly, we will use the feature description to match the feature.
• We will use the Euclidean distance between keypoint descriptors
Speeded Up Robust Features (SURF)
Similar to SIFT, but relies on several approximations and tricks to speed the processes.
Uses a box filter to approximate Gaussian smoothing.
• Integral images are easy to compute
Hessian matrix is used to find both extrema and scale.
• Matrix of mixed second-order derivatives
SURF: Feature Detection
Computed via its determinant
• Eigenvalues are like scale factors
• Determinant is like a volume
Multiply the eigenvalues together like dimensions of a cube
• The determinant of the matrix of partial second-order derivatives! (LOL)
SURF: Feature Description
Uses wavelet responses for orientation and description
• In turn, these calculations are sped-up by the use of the integral image.
Feature from Accelerated Segment Test
(FAST)
High-speed corner detector is even faster than SURF.
Imagine the corner of a building against a blue sky.
• From around 9 o'clock to 6 o'clock
The sky will be blue.
The center point will be building color (tan, brown, and so on).
FAST
Uses a trivial first test to rule out the big arc.
• Checks for 12 o’clock and 6 o’clock, then 3 o’clock and 9 o’clock, for sameness.
• Finds corners
But there is no real descriptor built in to FAST.
• FAST is not good with high-level noise.
Identifies points where much of an arc around it is all the same color (and different from
the center).
Binary Robust Independent Elementary
Features (BRIEF)
When we are looking at many features, feature description takes up too much memory
and time.
• The descriptors can be too verbose.
• They are often hard to learn from, hard to match, and have inefficient space use.
• There are various means to compress this data: PCA, hashing, and so on.
• Some methods will apply hashing after feature description.
This still requires too much memory
• BRIEF is designed to solve this problem.
BRIEF
BRIEF uses hashing to convert vectors to binary strings.
• The strings are then often compared with Hamming distance: the number of
characters that are different for two strings.
Fast and efficient process as it is binary (off/on)
BRIEF goes directly from pixels to binary numbers.
• Fixed set of pairs of points (pi, pj)
Selected once and permanently
Sampled from a random distribution
• The binary values
pi pj = 1
pi pj = 0
Example Hamming distances:
• 100→011: distance 3
• 010→111: distance 2
Source: Wikipedia
Image source: https://upload.wikimedia.org/wikipedia/commons/6/6e/Hamming_distance_3_bit_binary_example.svg
Oriented Fast and Rotated BRIEF (ORB)
Developed by OpenCV labs and is not patented like SIFT and SURF
Enhances FAST keypoint detectors
Adds orientation
Enhances BRIEF descriptors
Integrates orientation information into BRIEF
Oriented FAST and Rotated BRIEF (ORB)
Uses FAST to find keypoints.
Applies Harris corner measure to pick keypoints.
• Finds top N points among the keypoints
• Uses a pyramid to produce multiscale features
Builds BRIEF descriptors by incorporating keypoint orientation.
06 image features
Feature Matching: Matching Strategy
We have seen many techniques for feature detection and description.
How do we match them?
First, we need to pick a matching strategy.
Brute force
 Compare all features against each other
 Quadratic time
Feature Matching: Matching Strategy
Use a maximal distance threshold with a Euclidean distance metric.
Simplest procedure
Thresholds can be difficult to set
Need to optimize against false positives/negatives
Use nearest neighbor in feature space.
Nearest neighbor distance ratio
Feature Matching: Indexing Structure
Efficient strategies need an indexing structure.
Multidimensional hashing
New features are hashed into buckets
Buckets are matched between images
Return buckets like me (family address, check members of family)
Locally sensitive hashing
Indexes features to unions of independently computed hashing functions
Feature Matching: Indexing Structure
Parameter-sensitive hashing
• More sensitive to distribution points
Mulitdimensional search trees
• kd-trees
Divides features into alternating
axis-aligned hyperplanes
Slicing
• Series of 1D binary searches
Metric tree Example KD tree in three dimensions
Source: Wikipedia
Image source: https://upload.wikimedia.org/wikipedia/commons/b/b6/3dtree.png
Evaluation of Algorithms: Accuracy
Determining the accuracy of our feature-matching algorithm:
Need to evaluate its accuracy
• Percentage of correct prediction of all predictions
• 95 percent accuracy is a good job
Example
• With 54 percent Democrats and 46 percent Republicans
Predict party using votes
How many times did I get it right?
Evaluation of Algorithms: Accuracy
Predicting low probability events
Example:
• 1 percent have leukemia, 99 percent are healthy
• Predict leukemia using health records and tests
• Imagine stupidest possible answer (Healthy = Always)
99 percent accuracy
You will be correct 99 percent of the time, but you will NOT catch sick
people
USELESS!
Evaluation Using a Confusion Matrix
Bastoncini
Box
(Predicted)
Not -
Bastoncini
Box
(Predicted)
Row-Wise
Accuracy
(Correctness)
Bastoncini
Box
(Actual)
27 6 81.81
Not -
Bastoncini
Box
(Actual)
10 57 85.07
Overall
Accuracy
83.44
Confusion Matrix
Positive Negative
Positive True positive
(TP)
False negative
(FN)
Negative False positive
(FP)
True negative
(TN)
Predicted
(our claim, diagnosis, our result)
Actual
(real-world,actual
sickness,relevance)
Sensitivity and Specificity
P N
P TP FN TP/
(TP+FN)
N FP TN TN/
(TN+FP)
Sensitivity
Specificity
Predicted
Actual
Precision and Recall
𝐹1 = 2 ∗
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 ∗ 𝑅𝑒𝑐𝑎𝑙𝑙
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑅𝑒𝑐𝑎𝑙𝑙P N
P TP FN TP/
(TP+FN)
N FP TN
TP/
(TP+FP)
Predicted
Actual
Precision
Recall
Evaluation Using a Confusion Matrix
Bastoncini
Box
(Predicted)
Not -
Bastoncini
Box
(Predicted)
Accuracy
Bastoncini
Box
(Actual)
27 6 81.81
Not -
Bastoncini
Box
(Actual)
10 57 85.07
Overall
Accuracy
83.44
Precision = 27/37 = 73.0%
Recall= 27/33 = 81.8%
How to Tune Precision and Recall
Google.
Forget Recall. There are millions of relevant search results. Catching all of them is not
important: Most people only look at the first page of results.
Precision is way more important. If one of those ten links in the first page is not
relevant to the search, you’ll lose customers.
How to Tune Precision and Recall
HIV Tests.
Recall is very, very important. We want to catch all sick people. Imagine an HIV+
person taking the test and being told they are HIV-. Worst outcome. Nope.
Precision is also important. Telling someone that they are HIV+ when they are not is
a devastating mistake. Not as bad as letting HIV+ slip through, but we still cannot turn
the knob all the way to recall without a care about this, either.
How to Tune Precision and Recall
United States Legal System.
Higher Precision at the cost of Recall. Burden of proof. Reasonable doubt =
Acquittal. These mean that the threshold for classifying someone with the criminal label
is set very high.
High precision means making sure everyone sent to jail is indeed guilty.
High recall means making sure all criminals go to jail. The legal system is ready to pay
the cost of letting some criminals go to ensure no innocents go to jail.
Receiver Operating Characteristic
Tune Precision and Recall: Thresholds
Lower threshold: Ease criteria for calling it positive.
Can catch more positives by accepting edge cases.
higher recall, lower precision; higher true positive rate,
higher false positive rate.
Higher threshold: Only call it positive if absolutely sure.
To make most positive calls right, give up on
lower recall, higher precision; lower true positive rate, lower
false positive rate.
Area Under Curve
An evaluation of a classification algorithm
• including all possible thresholds

Weitere ähnliche Inhalte

Was ist angesagt?

Machine learning and_nlp
Machine learning and_nlpMachine learning and_nlp
Machine learning and_nlpankit_ppt
 
Generative Adversarial Networks : Basic architecture and variants
Generative Adversarial Networks : Basic architecture and variantsGenerative Adversarial Networks : Basic architecture and variants
Generative Adversarial Networks : Basic architecture and variantsananth
 
Introduction to Deep learning and H2O for beginner's
Introduction to Deep learning and H2O for beginner'sIntroduction to Deep learning and H2O for beginner's
Introduction to Deep learning and H2O for beginner'sVidyasagar Bhargava
 
Machine Learning - Neural Networks - Perceptron
Machine Learning - Neural Networks - PerceptronMachine Learning - Neural Networks - Perceptron
Machine Learning - Neural Networks - PerceptronAndrew Ferlitsch
 
Deep learning simplified
Deep learning simplifiedDeep learning simplified
Deep learning simplifiedLovelyn Rose
 
Basics of Machine Learning
Basics of Machine LearningBasics of Machine Learning
Basics of Machine LearningPranav Challa
 
Ml5 svm and-kernels
Ml5 svm and-kernelsMl5 svm and-kernels
Ml5 svm and-kernelsankit_ppt
 
Decision Trees
Decision TreesDecision Trees
Decision TreesStudent
 
Support Vector Machines- SVM
Support Vector Machines- SVMSupport Vector Machines- SVM
Support Vector Machines- SVMCarlo Carandang
 
Machine Learning using Support Vector Machine
Machine Learning using Support Vector MachineMachine Learning using Support Vector Machine
Machine Learning using Support Vector MachineMohsin Ul Haq
 
Dimensionality reduction
Dimensionality reductionDimensionality reduction
Dimensionality reductionShatakirti Er
 
Image processing
Image processingImage processing
Image processingmaheshpene
 
Chapter 1 introduction (Image Processing)
Chapter 1 introduction (Image Processing)Chapter 1 introduction (Image Processing)
Chapter 1 introduction (Image Processing)Varun Ojha
 
SIFT vs other Feature Descriptor
SIFT vs other Feature DescriptorSIFT vs other Feature Descriptor
SIFT vs other Feature DescriptorNisar Ahmed Rana
 
07 dimensionality reduction
07 dimensionality reduction07 dimensionality reduction
07 dimensionality reductionMarco Quartulli
 
Variational Autoencoders For Image Generation
Variational Autoencoders For Image GenerationVariational Autoencoders For Image Generation
Variational Autoencoders For Image GenerationJason Anderson
 
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...Simplilearn
 

Was ist angesagt? (20)

Machine learning and_nlp
Machine learning and_nlpMachine learning and_nlp
Machine learning and_nlp
 
Generative Adversarial Networks : Basic architecture and variants
Generative Adversarial Networks : Basic architecture and variantsGenerative Adversarial Networks : Basic architecture and variants
Generative Adversarial Networks : Basic architecture and variants
 
Introduction to Deep learning and H2O for beginner's
Introduction to Deep learning and H2O for beginner'sIntroduction to Deep learning and H2O for beginner's
Introduction to Deep learning and H2O for beginner's
 
Machine Learning - Neural Networks - Perceptron
Machine Learning - Neural Networks - PerceptronMachine Learning - Neural Networks - Perceptron
Machine Learning - Neural Networks - Perceptron
 
Deep learning simplified
Deep learning simplifiedDeep learning simplified
Deep learning simplified
 
Basics of Machine Learning
Basics of Machine LearningBasics of Machine Learning
Basics of Machine Learning
 
SVM
SVMSVM
SVM
 
Ml5 svm and-kernels
Ml5 svm and-kernelsMl5 svm and-kernels
Ml5 svm and-kernels
 
Decision Trees
Decision TreesDecision Trees
Decision Trees
 
Support Vector Machines- SVM
Support Vector Machines- SVMSupport Vector Machines- SVM
Support Vector Machines- SVM
 
Machine Learning using Support Vector Machine
Machine Learning using Support Vector MachineMachine Learning using Support Vector Machine
Machine Learning using Support Vector Machine
 
Ppt shuai
Ppt shuaiPpt shuai
Ppt shuai
 
Dimensionality reduction
Dimensionality reductionDimensionality reduction
Dimensionality reduction
 
Image processing
Image processingImage processing
Image processing
 
Chapter 1 introduction (Image Processing)
Chapter 1 introduction (Image Processing)Chapter 1 introduction (Image Processing)
Chapter 1 introduction (Image Processing)
 
SIFT vs other Feature Descriptor
SIFT vs other Feature DescriptorSIFT vs other Feature Descriptor
SIFT vs other Feature Descriptor
 
07 dimensionality reduction
07 dimensionality reduction07 dimensionality reduction
07 dimensionality reduction
 
Variational Autoencoders For Image Generation
Variational Autoencoders For Image GenerationVariational Autoencoders For Image Generation
Variational Autoencoders For Image Generation
 
Lesson 39
Lesson 39Lesson 39
Lesson 39
 
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...
 

Ähnlich wie 06 image features

Various object detection and tracking methods
Various object detection and tracking methodsVarious object detection and tracking methods
Various object detection and tracking methodssujeeshkumarj
 
Michal Erel's SIFT presentation
Michal Erel's SIFT presentationMichal Erel's SIFT presentation
Michal Erel's SIFT presentationwolf
 
Introduction to Real Time Rendering
Introduction to Real Time RenderingIntroduction to Real Time Rendering
Introduction to Real Time RenderingKoray Hagen
 
Computer Vision descriptors
Computer Vision descriptorsComputer Vision descriptors
Computer Vision descriptorsWael Badawy
 
Practical Digital Image Processing 3
 Practical Digital Image Processing 3 Practical Digital Image Processing 3
Practical Digital Image Processing 3Aly Abdelkareem
 
Cahall Final Intern Presentation
Cahall Final Intern PresentationCahall Final Intern Presentation
Cahall Final Intern PresentationDaniel Cahall
 
Feature detection and matching
Feature detection and matchingFeature detection and matching
Feature detection and matchingKuppusamy P
 
Practical Digital Image Processing 4
Practical Digital Image Processing 4Practical Digital Image Processing 4
Practical Digital Image Processing 4Aly Abdelkareem
 
Passive stereo vision with deep learning
Passive stereo vision with deep learningPassive stereo vision with deep learning
Passive stereo vision with deep learningYu Huang
 
Image feature extraction
Image feature extractionImage feature extraction
Image feature extractionRishabh shah
 
EFFECTIVE INTEREST REGION ESTIMATION MODEL TO REPRESENT CORNERS FOR IMAGE
EFFECTIVE INTEREST REGION ESTIMATION MODEL TO REPRESENT CORNERS FOR IMAGE EFFECTIVE INTEREST REGION ESTIMATION MODEL TO REPRESENT CORNERS FOR IMAGE
EFFECTIVE INTEREST REGION ESTIMATION MODEL TO REPRESENT CORNERS FOR IMAGE sipij
 
Fisheye Omnidirectional View in Autonomous Driving
Fisheye Omnidirectional View in Autonomous DrivingFisheye Omnidirectional View in Autonomous Driving
Fisheye Omnidirectional View in Autonomous DrivingYu Huang
 
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017StampedeCon
 
An automatic algorithm for object recognition and detection based on asift ke...
An automatic algorithm for object recognition and detection based on asift ke...An automatic algorithm for object recognition and detection based on asift ke...
An automatic algorithm for object recognition and detection based on asift ke...Kunal Kishor Nirala
 
cohenmedioni.ppt
cohenmedioni.pptcohenmedioni.ppt
cohenmedioni.pptChinnuDS
 
Chapter10 image segmentation
Chapter10 image segmentationChapter10 image segmentation
Chapter10 image segmentationasodariyabhavesh
 

Ähnlich wie 06 image features (20)

Various object detection and tracking methods
Various object detection and tracking methodsVarious object detection and tracking methods
Various object detection and tracking methods
 
Michal Erel's SIFT presentation
Michal Erel's SIFT presentationMichal Erel's SIFT presentation
Michal Erel's SIFT presentation
 
Introduction to Real Time Rendering
Introduction to Real Time RenderingIntroduction to Real Time Rendering
Introduction to Real Time Rendering
 
Computer Vision descriptors
Computer Vision descriptorsComputer Vision descriptors
Computer Vision descriptors
 
SIFT.ppt
SIFT.pptSIFT.ppt
SIFT.ppt
 
SIFT.ppt
SIFT.pptSIFT.ppt
SIFT.ppt
 
PPT s11-machine vision-s2
PPT s11-machine vision-s2PPT s11-machine vision-s2
PPT s11-machine vision-s2
 
Practical Digital Image Processing 3
 Practical Digital Image Processing 3 Practical Digital Image Processing 3
Practical Digital Image Processing 3
 
Cahall Final Intern Presentation
Cahall Final Intern PresentationCahall Final Intern Presentation
Cahall Final Intern Presentation
 
Feature detection and matching
Feature detection and matchingFeature detection and matching
Feature detection and matching
 
Practical Digital Image Processing 4
Practical Digital Image Processing 4Practical Digital Image Processing 4
Practical Digital Image Processing 4
 
Passive stereo vision with deep learning
Passive stereo vision with deep learningPassive stereo vision with deep learning
Passive stereo vision with deep learning
 
Image feature extraction
Image feature extractionImage feature extraction
Image feature extraction
 
SIFT
SIFTSIFT
SIFT
 
EFFECTIVE INTEREST REGION ESTIMATION MODEL TO REPRESENT CORNERS FOR IMAGE
EFFECTIVE INTEREST REGION ESTIMATION MODEL TO REPRESENT CORNERS FOR IMAGE EFFECTIVE INTEREST REGION ESTIMATION MODEL TO REPRESENT CORNERS FOR IMAGE
EFFECTIVE INTEREST REGION ESTIMATION MODEL TO REPRESENT CORNERS FOR IMAGE
 
Fisheye Omnidirectional View in Autonomous Driving
Fisheye Omnidirectional View in Autonomous DrivingFisheye Omnidirectional View in Autonomous Driving
Fisheye Omnidirectional View in Autonomous Driving
 
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
The Search for a New Visual Search Beyond Language - StampedeCon AI Summit 2017
 
An automatic algorithm for object recognition and detection based on asift ke...
An automatic algorithm for object recognition and detection based on asift ke...An automatic algorithm for object recognition and detection based on asift ke...
An automatic algorithm for object recognition and detection based on asift ke...
 
cohenmedioni.ppt
cohenmedioni.pptcohenmedioni.ppt
cohenmedioni.ppt
 
Chapter10 image segmentation
Chapter10 image segmentationChapter10 image segmentation
Chapter10 image segmentation
 

Mehr von ankit_ppt

Deep learning summary
Deep learning summaryDeep learning summary
Deep learning summaryankit_ppt
 
08 neural networks
08 neural networks08 neural networks
08 neural networksankit_ppt
 
01 foundations
01 foundations01 foundations
01 foundationsankit_ppt
 
Text similarity measures
Text similarity measuresText similarity measures
Text similarity measuresankit_ppt
 
Text generation and_advanced_topics
Text generation and_advanced_topicsText generation and_advanced_topics
Text generation and_advanced_topicsankit_ppt
 
Nlp toolkits and_preprocessing_techniques
Nlp toolkits and_preprocessing_techniquesNlp toolkits and_preprocessing_techniques
Nlp toolkits and_preprocessing_techniquesankit_ppt
 
Latent dirichlet allocation_and_topic_modeling
Latent dirichlet allocation_and_topic_modelingLatent dirichlet allocation_and_topic_modeling
Latent dirichlet allocation_and_topic_modelingankit_ppt
 
Intro to nlp
Intro to nlpIntro to nlp
Intro to nlpankit_ppt
 
Ml9 introduction to-unsupervised_learning_and_clustering_methods
Ml9 introduction to-unsupervised_learning_and_clustering_methodsMl9 introduction to-unsupervised_learning_and_clustering_methods
Ml9 introduction to-unsupervised_learning_and_clustering_methodsankit_ppt
 
Ml8 boosting and-stacking
Ml8 boosting and-stackingMl8 boosting and-stacking
Ml8 boosting and-stackingankit_ppt
 
Ml6 decision trees
Ml6 decision treesMl6 decision trees
Ml6 decision treesankit_ppt
 
Ml4 naive bayes
Ml4 naive bayesMl4 naive bayes
Ml4 naive bayesankit_ppt
 
Lesson 3 ai in the enterprise
Lesson 3   ai in the enterpriseLesson 3   ai in the enterprise
Lesson 3 ai in the enterpriseankit_ppt
 
Ml3 logistic regression-and_classification_error_metrics
Ml3 logistic regression-and_classification_error_metricsMl3 logistic regression-and_classification_error_metrics
Ml3 logistic regression-and_classification_error_metricsankit_ppt
 
Ml2 train test-splits_validation_linear_regression
Ml2 train test-splits_validation_linear_regressionMl2 train test-splits_validation_linear_regression
Ml2 train test-splits_validation_linear_regressionankit_ppt
 
Ml1 introduction to-supervised_learning_and_k_nearest_neighbors
Ml1 introduction to-supervised_learning_and_k_nearest_neighborsMl1 introduction to-supervised_learning_and_k_nearest_neighbors
Ml1 introduction to-supervised_learning_and_k_nearest_neighborsankit_ppt
 
Lesson 2 ai in industry
Lesson 2   ai in industryLesson 2   ai in industry
Lesson 2 ai in industryankit_ppt
 
Lesson 5 arima
Lesson 5 arimaLesson 5 arima
Lesson 5 arimaankit_ppt
 

Mehr von ankit_ppt (20)

Deep learning summary
Deep learning summaryDeep learning summary
Deep learning summary
 
08 neural networks
08 neural networks08 neural networks
08 neural networks
 
01 foundations
01 foundations01 foundations
01 foundations
 
Word2 vec
Word2 vecWord2 vec
Word2 vec
 
Text similarity measures
Text similarity measuresText similarity measures
Text similarity measures
 
Text generation and_advanced_topics
Text generation and_advanced_topicsText generation and_advanced_topics
Text generation and_advanced_topics
 
Nlp toolkits and_preprocessing_techniques
Nlp toolkits and_preprocessing_techniquesNlp toolkits and_preprocessing_techniques
Nlp toolkits and_preprocessing_techniques
 
Latent dirichlet allocation_and_topic_modeling
Latent dirichlet allocation_and_topic_modelingLatent dirichlet allocation_and_topic_modeling
Latent dirichlet allocation_and_topic_modeling
 
Intro to nlp
Intro to nlpIntro to nlp
Intro to nlp
 
Ml9 introduction to-unsupervised_learning_and_clustering_methods
Ml9 introduction to-unsupervised_learning_and_clustering_methodsMl9 introduction to-unsupervised_learning_and_clustering_methods
Ml9 introduction to-unsupervised_learning_and_clustering_methods
 
Ml8 boosting and-stacking
Ml8 boosting and-stackingMl8 boosting and-stacking
Ml8 boosting and-stacking
 
Ml7 bagging
Ml7 baggingMl7 bagging
Ml7 bagging
 
Ml6 decision trees
Ml6 decision treesMl6 decision trees
Ml6 decision trees
 
Ml4 naive bayes
Ml4 naive bayesMl4 naive bayes
Ml4 naive bayes
 
Lesson 3 ai in the enterprise
Lesson 3   ai in the enterpriseLesson 3   ai in the enterprise
Lesson 3 ai in the enterprise
 
Ml3 logistic regression-and_classification_error_metrics
Ml3 logistic regression-and_classification_error_metricsMl3 logistic regression-and_classification_error_metrics
Ml3 logistic regression-and_classification_error_metrics
 
Ml2 train test-splits_validation_linear_regression
Ml2 train test-splits_validation_linear_regressionMl2 train test-splits_validation_linear_regression
Ml2 train test-splits_validation_linear_regression
 
Ml1 introduction to-supervised_learning_and_k_nearest_neighbors
Ml1 introduction to-supervised_learning_and_k_nearest_neighborsMl1 introduction to-supervised_learning_and_k_nearest_neighbors
Ml1 introduction to-supervised_learning_and_k_nearest_neighbors
 
Lesson 2 ai in industry
Lesson 2   ai in industryLesson 2   ai in industry
Lesson 2 ai in industry
 
Lesson 5 arima
Lesson 5 arimaLesson 5 arima
Lesson 5 arima
 

Kürzlich hochgeladen

Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Amil baba
 
EPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptxEPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptxJoseeMusabyimana
 
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfSummer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfNaveenVerma126
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxrealme6igamerr
 
ChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai searchChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai searchrohitcse52
 
A Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software SimulationA Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software SimulationMohsinKhanA
 
Phase noise transfer functions.pptx
Phase noise transfer      functions.pptxPhase noise transfer      functions.pptx
Phase noise transfer functions.pptxSaiGouthamSunkara
 
Test of Significance of Large Samples for Mean = µ.pptx
Test of Significance of Large Samples for Mean = µ.pptxTest of Significance of Large Samples for Mean = µ.pptx
Test of Significance of Large Samples for Mean = µ.pptxHome
 
Design of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxDesign of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxYogeshKumarKJMIT
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical SensorTanvir Moin
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvementVijayMuni2
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS Bahzad5
 
ASME BPVC 2023 Section I para leer y entender
ASME BPVC 2023 Section I para leer y entenderASME BPVC 2023 Section I para leer y entender
ASME BPVC 2023 Section I para leer y entenderjuancarlos286641
 
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Apollo Techno Industries Pvt Ltd
 
The relationship between iot and communication technology
The relationship between iot and communication technologyThe relationship between iot and communication technology
The relationship between iot and communication technologyabdulkadirmukarram03
 
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Sean Meyn
 
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratoryدليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide LaboratoryBahzad5
 
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....santhyamuthu1
 
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfRenewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfodunowoeminence2019
 

Kürzlich hochgeladen (20)

Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
 
EPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptxEPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptx
 
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfSummer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
 
ChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai searchChatGPT-and-Generative-AI-Landscape Working of generative ai search
ChatGPT-and-Generative-AI-Landscape Working of generative ai search
 
A Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software SimulationA Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software Simulation
 
Phase noise transfer functions.pptx
Phase noise transfer      functions.pptxPhase noise transfer      functions.pptx
Phase noise transfer functions.pptx
 
Test of Significance of Large Samples for Mean = µ.pptx
Test of Significance of Large Samples for Mean = µ.pptxTest of Significance of Large Samples for Mean = µ.pptx
Test of Significance of Large Samples for Mean = µ.pptx
 
Design of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptxDesign of Clutches and Brakes in Design of Machine Elements.pptx
Design of Clutches and Brakes in Design of Machine Elements.pptx
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical Sensor
 
Landsman converter for power factor improvement
Landsman converter for power factor improvementLandsman converter for power factor improvement
Landsman converter for power factor improvement
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
 
ASME BPVC 2023 Section I para leer y entender
ASME BPVC 2023 Section I para leer y entenderASME BPVC 2023 Section I para leer y entender
ASME BPVC 2023 Section I para leer y entender
 
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
 
The relationship between iot and communication technology
The relationship between iot and communication technologyThe relationship between iot and communication technology
The relationship between iot and communication technology
 
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
 
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratoryدليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
 
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....
 
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdfRenewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
Renewable Energy & Entrepreneurship Workshop_21Feb2024.pdf
 
Lecture 4 .pdf
Lecture 4                              .pdfLecture 4                              .pdf
Lecture 4 .pdf
 

06 image features

  • 2. Legal Notices and Disclaimers This presentation is for informational purposes only. INTEL MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. Check with your system manufacturer or retailer or learn more at intel.com. This sample source code is released under the Intel Sample Source Code License Agreement. Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. Copyright © 2018, Intel Corporation. All rights reserved. 2
  • 4. Image Features Image features are interesting locations in an image. We want to be able to apply… • Feature detection Finding regions, when moved, that cause maximal variation • Features description Building a good context around a feature We use this information about these interesting locations for a wide variety of uses.
  • 5. Feature Detection Background feature matches any background cell. Foreground feature matches any foreground cell. L-edge matches any vertical edge. T-edge matches any horizontal edge. But, CORNER only has one match!!!
  • 6. Uses of Image Features: Matching
  • 7. Classification (Object Recognition) What is this object? It is a Crayon
  • 8. Tracking of Features in Motion Image reference: https://commons.wikimedia.org/wiki/File:Madison_Square_Garden_food_court.jpg
  • 10. Object Detection Image from OpenCV documentation: https://docs.opencv.org/3.3.0/d7/d8b/tutorial_py_face_detection.html
  • 11. Techniques for Image Features Techniques that can be used for any of these use-cases: • Corners • HOG • SIFT • SURF • FAST • BRIEF • However, these techniques are typically developed by a research group to solve one use case.
  • 13. Corners An edge is indicated by a strong derivative. • This is helpful to find an edge, but all features along that edge may look the same. • We want a better way to identify an interesting feature. Corners are edges in two orthogonal directions. • Hopefully, a corner will have enough information to identify our feature over multiple images, or frames.
  • 15. Harris Corners Used for sparse feature matching. Use a Gaussian weighted window. • Allows detection when there is in-plane rotation of an interesting feature • Down-weights edge-like features Compute a scalar interest measure. Find local maxima.
  • 16. Harris Corners Harris corners are defined as second-order derivatives. We create a second-derivative Hessian image. A corner is identified when this matrix has two large eigenvalues. Look at eigenvalues of the autocorrelation matrix. • Eigenvectors tell us the orientation of a matrix. • Eigenvalues tell us the scale in those directions. • Either minimum eigenvalue, or something more complicated like: 𝜆0 𝜆1 − 0.06 𝜆0 𝜆1 2
  • 17. Harris Corners and Shi-Tomasi Shi-Tomasi modified the Harris corner. In both cases we take the: • Correlation between gradients in x and y directions Harris used: Shi-Tomasi used: • Invariant to rotation (because of eigenvalues) 𝜆0 𝜆1 − 0.06 𝜆0 𝜆1 2 < 𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 𝜆0 < 𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑
  • 18. Sub-Pixel Methods Harris and Shi-Tomasi corners are good for feature recognition and used… • When you are extracting geometric measurements • When you need higher resolution feature recognition If I am a… Biologist trying to generate a 3D reconstruction of a blood vessel Or a marine determining the location of a target in a satellite image I need sub-pixel coordinates.
  • 19. Sub-Pixel Methods If we know an approximate (pixel level) location Capprox of a corner we can find a better corner Cimproved by: • Sampling a region of points P around Capprox. • Setting up a series of equations that capture the directional similarities between each p of P and Cimproved Note, we don’t know Cimproved! • Solve the equations. The result is Cimproved. • Repeat this process.
  • 20. Sub-Pixel Methods Similarity measure has these important properties: • Similarity(0,any) = Similarity(any,0) = 0 • Similarity(perpendicular directions) = 0 For us, our directions are: 1. The gradient at a region point p 2. The direction from C_approx to p
  • 21. Sub-Pixel Methods When the similarity between the direction of the gradient and the direction from C_approx to p is 0 • The directions must be perpendicular. In turn, this means that the direction lies along an edge • Multiple edges meet at a corner! The similarity measure is the dot product between the gradient at p and the direction from Capprox to p.
  • 23. Histogram of Oriented Gradients Developed to detect pedestrians in a scene Techniques focus on detection accuracy instead of computing speed and efficiency. • Details about an object are determined as accurately as possible. HOGs are a distribution of the directions of the gradients. The histogram confers distribution The oriented gradients confer direction of gradient
  • 24. Histogram of Oriented Gradients Features produced by HOG are the histograms from the cells.
  • 25. Constants are Tuned for Recognizing Humans HOG is described in terms of one ROI, but many steps can be precomputed on the whole image. Because HOG was originally developed for pedestrian recognition, many HOG ROIs are humans. Humans are basically tall rectangles. Therefore, we will make patches around an ROI by using a… Patch aspect of 2 (tall) to 1 (wide) Cell size of 8 x 8 because it captures human features like eyes
  • 26. You Can Use HOG on Other, Non-Human, Features Image ROI Frames for Normalization Cells Tree image from: https://commons.wikimedia.org/wiki/Christmas_tree#/media/File:Piazza_Portanova_Natale_2008.jpg
  • 27. HOG: An Example Step 1: Take the image and create a patch of the ROI with a fixed aspect ratio of 1:2.  Essentially, you are cropping the image around the ROI. Patch (or ROI)
  • 28. HOG: An Example Step 2: Calculate the horizontal and vertical gradients for each color in the patch. – Convolve with [-1,0,1] kernels – For instance, you can use Sobel for this step
  • 29. HOG: An Example Step 3: Consider the component-wise gradients in polar coordinates. – Take x, y gradients and convert to Q • x, y  r, Q [graphic] • At every pixel, take max r over the three color channels • r is magnitude • Associated direction is the theta of that max r
  • 30. HOG: An Example Step 4: The patch is divided into 8 x 8 cells, from which we make a histogram of gradients. Each bucket, or bin, are angles (Q) – Q = 0, 20, 40, ..., 160 • This gives us nine total bins – We assign the magnitude (r) of a pixel to its angle (th) bin. – For angles between bins, we divide it proportionally to distance from the bins. Direction (Q) Magnitude (r) 0 20 40 60 80 100 120 140 160
  • 31. HOG: An Example Step 5: Create a mega-cell and associated mega-histogram with 36 total values. • Use a sliding 16 x 16 window (of four histograms each time you slide the window). • Normalize this window with respect to L2 norm to create one block of features. • Slide the window over 8 pixels and repeat normalization. • Some pixels will be repeated normalized, but with respect to different neighborhoods extending differed directions.
  • 32. Features per Patch: Some Simple Math The result is 3,780 total features for one patch. • On a 128 x 64 image we have 16 x 8 cells that are 8 x 8 • We have 15 x 7 mega-cells • Which gives us 105 blocks of 36 features
  • 34. Scale Invariant Feature Transform (SIFT) Is a feature detection and description method that is invariant to scale • Particularly helpful when analyzing larger corners Scale-space filtering is used to detect keypoints with different scales. • Uses difference of Gaussians (DoG) to approximate Laplacian of Gaussians (LoG). • Finds maxima with respect to space and scale. Space is across an image Scale is up and down a DoG pyramid
  • 35. SIFT: Feature Detection First, we need to find candidate keypoints (feature detection). We do this by finding extrema versus neighbors in a Gaussian pyramid. This is on the same scale but can be high or lower resolution. Then, we find candidates by filtering out points with low contrast and on edges.
  • 36. SIFT: Feature Description Now that feature detection is complete, we need to describe the feature. • The neighborhood is proportional to level of the extrema • Estimate orientation in neighborhood around the keypoint • Compute a histogram of gradients in this neighborhood • Normalize histogram
  • 37. SIFT: Matching Lastly, we will use the feature description to match the feature. • We will use the Euclidean distance between keypoint descriptors
  • 38. Speeded Up Robust Features (SURF) Similar to SIFT, but relies on several approximations and tricks to speed the processes. Uses a box filter to approximate Gaussian smoothing. • Integral images are easy to compute Hessian matrix is used to find both extrema and scale. • Matrix of mixed second-order derivatives
  • 39. SURF: Feature Detection Computed via its determinant • Eigenvalues are like scale factors • Determinant is like a volume Multiply the eigenvalues together like dimensions of a cube • The determinant of the matrix of partial second-order derivatives! (LOL)
  • 40. SURF: Feature Description Uses wavelet responses for orientation and description • In turn, these calculations are sped-up by the use of the integral image.
  • 41. Feature from Accelerated Segment Test (FAST) High-speed corner detector is even faster than SURF. Imagine the corner of a building against a blue sky. • From around 9 o'clock to 6 o'clock The sky will be blue. The center point will be building color (tan, brown, and so on).
  • 42. FAST Uses a trivial first test to rule out the big arc. • Checks for 12 o’clock and 6 o’clock, then 3 o’clock and 9 o’clock, for sameness. • Finds corners But there is no real descriptor built in to FAST. • FAST is not good with high-level noise. Identifies points where much of an arc around it is all the same color (and different from the center).
  • 43. Binary Robust Independent Elementary Features (BRIEF) When we are looking at many features, feature description takes up too much memory and time. • The descriptors can be too verbose. • They are often hard to learn from, hard to match, and have inefficient space use. • There are various means to compress this data: PCA, hashing, and so on. • Some methods will apply hashing after feature description. This still requires too much memory • BRIEF is designed to solve this problem.
  • 44. BRIEF BRIEF uses hashing to convert vectors to binary strings. • The strings are then often compared with Hamming distance: the number of characters that are different for two strings. Fast and efficient process as it is binary (off/on) BRIEF goes directly from pixels to binary numbers. • Fixed set of pairs of points (pi, pj) Selected once and permanently Sampled from a random distribution • The binary values pi pj = 1 pi pj = 0 Example Hamming distances: • 100→011: distance 3 • 010→111: distance 2 Source: Wikipedia Image source: https://upload.wikimedia.org/wikipedia/commons/6/6e/Hamming_distance_3_bit_binary_example.svg
  • 45. Oriented Fast and Rotated BRIEF (ORB) Developed by OpenCV labs and is not patented like SIFT and SURF Enhances FAST keypoint detectors Adds orientation Enhances BRIEF descriptors Integrates orientation information into BRIEF
  • 46. Oriented FAST and Rotated BRIEF (ORB) Uses FAST to find keypoints. Applies Harris corner measure to pick keypoints. • Finds top N points among the keypoints • Uses a pyramid to produce multiscale features Builds BRIEF descriptors by incorporating keypoint orientation.
  • 48. Feature Matching: Matching Strategy We have seen many techniques for feature detection and description. How do we match them? First, we need to pick a matching strategy. Brute force  Compare all features against each other  Quadratic time
  • 49. Feature Matching: Matching Strategy Use a maximal distance threshold with a Euclidean distance metric. Simplest procedure Thresholds can be difficult to set Need to optimize against false positives/negatives Use nearest neighbor in feature space. Nearest neighbor distance ratio
  • 50. Feature Matching: Indexing Structure Efficient strategies need an indexing structure. Multidimensional hashing New features are hashed into buckets Buckets are matched between images Return buckets like me (family address, check members of family) Locally sensitive hashing Indexes features to unions of independently computed hashing functions
  • 51. Feature Matching: Indexing Structure Parameter-sensitive hashing • More sensitive to distribution points Mulitdimensional search trees • kd-trees Divides features into alternating axis-aligned hyperplanes Slicing • Series of 1D binary searches Metric tree Example KD tree in three dimensions Source: Wikipedia Image source: https://upload.wikimedia.org/wikipedia/commons/b/b6/3dtree.png
  • 52. Evaluation of Algorithms: Accuracy Determining the accuracy of our feature-matching algorithm: Need to evaluate its accuracy • Percentage of correct prediction of all predictions • 95 percent accuracy is a good job Example • With 54 percent Democrats and 46 percent Republicans Predict party using votes How many times did I get it right?
  • 53. Evaluation of Algorithms: Accuracy Predicting low probability events Example: • 1 percent have leukemia, 99 percent are healthy • Predict leukemia using health records and tests • Imagine stupidest possible answer (Healthy = Always) 99 percent accuracy You will be correct 99 percent of the time, but you will NOT catch sick people USELESS!
  • 54. Evaluation Using a Confusion Matrix Bastoncini Box (Predicted) Not - Bastoncini Box (Predicted) Row-Wise Accuracy (Correctness) Bastoncini Box (Actual) 27 6 81.81 Not - Bastoncini Box (Actual) 10 57 85.07 Overall Accuracy 83.44
  • 55. Confusion Matrix Positive Negative Positive True positive (TP) False negative (FN) Negative False positive (FP) True negative (TN) Predicted (our claim, diagnosis, our result) Actual (real-world,actual sickness,relevance)
  • 56. Sensitivity and Specificity P N P TP FN TP/ (TP+FN) N FP TN TN/ (TN+FP) Sensitivity Specificity Predicted Actual
  • 57. Precision and Recall 𝐹1 = 2 ∗ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 ∗ 𝑅𝑒𝑐𝑎𝑙𝑙 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑅𝑒𝑐𝑎𝑙𝑙P N P TP FN TP/ (TP+FN) N FP TN TP/ (TP+FP) Predicted Actual Precision Recall
  • 58. Evaluation Using a Confusion Matrix Bastoncini Box (Predicted) Not - Bastoncini Box (Predicted) Accuracy Bastoncini Box (Actual) 27 6 81.81 Not - Bastoncini Box (Actual) 10 57 85.07 Overall Accuracy 83.44 Precision = 27/37 = 73.0% Recall= 27/33 = 81.8%
  • 59. How to Tune Precision and Recall Google. Forget Recall. There are millions of relevant search results. Catching all of them is not important: Most people only look at the first page of results. Precision is way more important. If one of those ten links in the first page is not relevant to the search, you’ll lose customers.
  • 60. How to Tune Precision and Recall HIV Tests. Recall is very, very important. We want to catch all sick people. Imagine an HIV+ person taking the test and being told they are HIV-. Worst outcome. Nope. Precision is also important. Telling someone that they are HIV+ when they are not is a devastating mistake. Not as bad as letting HIV+ slip through, but we still cannot turn the knob all the way to recall without a care about this, either.
  • 61. How to Tune Precision and Recall United States Legal System. Higher Precision at the cost of Recall. Burden of proof. Reasonable doubt = Acquittal. These mean that the threshold for classifying someone with the criminal label is set very high. High precision means making sure everyone sent to jail is indeed guilty. High recall means making sure all criminals go to jail. The legal system is ready to pay the cost of letting some criminals go to ensure no innocents go to jail.
  • 63. Tune Precision and Recall: Thresholds Lower threshold: Ease criteria for calling it positive. Can catch more positives by accepting edge cases. higher recall, lower precision; higher true positive rate, higher false positive rate. Higher threshold: Only call it positive if absolutely sure. To make most positive calls right, give up on lower recall, higher precision; lower true positive rate, lower false positive rate.
  • 64. Area Under Curve An evaluation of a classification algorithm • including all possible thresholds

Hinweis der Redaktion

  1. Image url?
  2. Image reference: https://commons.wikimedia.org/wiki/File:Madison_Square_Garden_food_court.jpg
  3. Image from OpenCV documentation: https://docs.opencv.org/3.3.0/d7/d8b/tutorial_py_face_detection.html
  4. Note: One use case, for example HOG, was designed for the use of pedestrian detection and many of the parameters are specific to finding humans in an outdoor scene. This fixes many aspects of the problem…for example, the size of the humans is basically constant and also the size of faces (etc.). You can think about this as “learning” being fit by a problem, and may need to refit to use a technique in another context.
  5. The term autocorrelation here refers to aligning products of image intensities (like a statistical covariance matrix – but without subtracting the means). Likewise, in signal processing (which is where image processing inherits the term from), the term “autocorrelation” does not imply centering and scaling (that is, you don’t subtract the mean and you don’t divide by standard deviation).
  6. Technical note: The “similarity measure” is the dot-product between the two measures (sum of element-wise multiplications and/or length of a * length of b * cos of angle between them. (The angle part is most important.)
  7. Technical note: The “similarity measure” is the dot-product between the two measures (sum of element-wise multiplications and/or length of a * length of b * cos of angle between them. (The angle part is most important.)
  8. Tree image from: https://commons.wikimedia.org/wiki/Christmas_tree#/media/File:Piazza_Portanova_Natale_2008.jpg
  9. Image url?
  10. Image url?
  11. Image source: https://upload.wikimedia.org/wikipedia/commons/6/6e/Hamming_distance_3_bit_binary_example.svg
  12. [reference: Sz 225-234]
  13. [reference: Sz 225-234]
  14. [reference: Sz 225-234]
  15. Image source: https://upload.wikimedia.org/wikipedia/commons/b/b6/3dtree.png
  16. 5, 10, 12*, 11, 16, 23, 19*, 30-32, 35, 37, 38 from CEM
  17. Sensitivity - True positive rate Specificity = True negative rate
  18. F1 = Their harmonic mean