2. Need for image data managementNeed for image data management
► The process of digitisation does not in itselfThe process of digitisation does not in itself
make image collections easier to manage.make image collections easier to manage.
Some form of cataloguing and indexing isSome form of cataloguing and indexing is
still necessary – the only difference beingstill necessary – the only difference being
that much of the required information canthat much of the required information can
now potentially be derived automaticallynow potentially be derived automatically
from the images themselves.from the images themselves.
► While it is perfectly feasible to identify aWhile it is perfectly feasible to identify a
desired image from a small collection simplydesired image from a small collection simply
by browsing, more effective techniques areby browsing, more effective techniques are
needed with collections containingneeded with collections containing
thousands of items which need some form ofthousands of items which need some form of
access by image content.access by image content.
3. What is CBIR?What is CBIR?
Process of retrieving desired images fromProcess of retrieving desired images from
a large collection on the basis of featuresa large collection on the basis of features
(such as colour, texture and shape) that(such as colour, texture and shape) that
can be automatically extracted from thecan be automatically extracted from the
images themselves.images themselves.
Also known as query by image contentAlso known as query by image content
((QBICQBIC) and content-based visual) and content-based visual
information retrieval (information retrieval (CBVIRCBVIR))
4.
5. ► ““Content-based” means that the search willContent-based” means that the search will
analyze the actual contents of the image.analyze the actual contents of the image.
The term 'content' in this context might referThe term 'content' in this context might refer
to colors, shapes, textures, or any otherto colors, shapes, textures, or any other
information that can be derived from theinformation that can be derived from the
image itself.image itself.
► Indexing is often used as identifying featuresIndexing is often used as identifying features
within an image; with indexing datawithin an image; with indexing data
structures we here mean structures to speedstructures we here mean structures to speed
up the retrieval of features within imageup the retrieval of features within image
collections.collections.
6. Practical applications of CBIRPractical applications of CBIR
► Crime preventionCrime prevention
► The militaryThe military
► Intellectual propertyIntellectual property
► Architectural and engineering designArchitectural and engineering design
► Fashion and interior designFashion and interior design
► Journalism and advertisingJournalism and advertising
► Medical diagnosisMedical diagnosis
► Geographical information and remote sensing systemsGeographical information and remote sensing systems
► Cultural heritageCultural heritage
► Education and trainingEducation and training
► Home entertainmentHome entertainment
► Web searchingWeb searching
7. Query techniquesQuery techniques
a)Query by examplea)Query by example
Query by example is a query technique thatQuery by example is a query technique that
involves providing the CBIR system with aninvolves providing the CBIR system with an
example imageexample image that it will then base its searchthat it will then base its search
upon. The underlying search algorithms mayupon. The underlying search algorithms may
vary depending on the application, but resultvary depending on the application, but result
images should all share common elements withimages should all share common elements with
the provided example.the provided example.
8. b) Semantic retrievalb) Semantic retrieval
The ideal CBIR system from aThe ideal CBIR system from a user perspectiveuser perspective
would involve what is referred to aswould involve what is referred to as semanticsemantic
retrieval.This type of open-ended task is veryretrieval.This type of open-ended task is very
difficult for computers to perform.difficult for computers to perform.
Current CBIR systems therefore generally makeCurrent CBIR systems therefore generally make
use of lower-level features like texture, color,use of lower-level features like texture, color,
and shape, although some systems takeand shape, although some systems take
advantage of very common higher-level featuresadvantage of very common higher-level features
like faces.like faces.
9. Other query methods includeOther query methods include
browsing for example images, navigatingbrowsing for example images, navigating
customized/hierarchical categories,customized/hierarchical categories,
querying by image region (rather than thequerying by image region (rather than the
entire image), querying by multiple exampleentire image), querying by multiple example
images, querying by visual sketch, queryingimages, querying by visual sketch, querying
by direct specification of image features,by direct specification of image features,
and multimodal queries (e.g. combiningand multimodal queries (e.g. combining
touch, voice, etc.).touch, voice, etc.).
c)Other query methodsc)Other query methods
10. Semantic representation for imageSemantic representation for image
retrievalretrieval
It is very difficult to manually annotate images andIt is very difficult to manually annotate images and
videos as well as retrieve raw data. The mainvideos as well as retrieve raw data. The main
problem in semantic labeling is a wide variation inproblem in semantic labeling is a wide variation in
visual appearance within objects of the samevisual appearance within objects of the same
class.class.
An alternative approach is to build a statisticalAn alternative approach is to build a statistical
learning system that starts with a minimallearning system that starts with a minimal
knowledge and simple descriptors. Users thenknowledge and simple descriptors. Users then
train the systemtrain the system via formative feedback before itvia formative feedback before it
can work with large databases.can work with large databases.
11. Feature extraction using SVDFeature extraction using SVD
In linear algebra, theIn linear algebra, the singular valuesingular value
decompositiondecomposition ((SVDSVD) is an important) is an important
factorization of a rectangular real or complexfactorization of a rectangular real or complex
matrix, with several applications in signalmatrix, with several applications in signal
processing and statistics.processing and statistics.
StatementStatement
SupposeSuppose MM is anis an mm-by--by-nn matrix whosematrix whose
entries come from the fieldentries come from the field KK, which is either the, which is either the
field of real numbers or the field of complexfield of real numbers or the field of complex
numbers. Then there exists a factorization of thenumbers. Then there exists a factorization of the
formform
12. wherewhere
UU is anis an mm-by--by-mm unitary matrix overunitary matrix over KK,,
Σ isΣ is mm-by--by-nn diagonal matrix withdiagonal matrix with
nonnegative numbers on the diagonalnonnegative numbers on the diagonal
V*V* denotes the conjugate transpose ofdenotes the conjugate transpose of V,V,
anan nn-by--by-nn unitary matrix overunitary matrix over KK..
Such a factorization is called aSuch a factorization is called a singular-valuesingular-value
decompositiondecomposition ofof MM..
13. MatrixMatrix VV contains a set of orthonormalcontains a set of orthonormal
"input" or "analysing" basis vector directions for"input" or "analysing" basis vector directions for
MM..
Matrix U contains a set of orthonormal
"output" basis vector directions for M.
Matrix Σ contains the singular values, which
can be thought of as scalar "gain controls" by
which each corresponding input is multiplied to
give a corresponding output.
The diagonal matrix Σ is uniquely determined
by M.
14. Classification of data classes usingClassification of data classes using
support vector machine (SVM)support vector machine (SVM)
SVMsSVMs are a set of related supervisedare a set of related supervised
learning methods used for classification .learning methods used for classification .
Viewing input data as two sets ofViewing input data as two sets of
vectors in anvectors in an nn-dimensional space, an-dimensional space, an
SVM will construct a separatingSVM will construct a separating
hyperplane in that space, one whichhyperplane in that space, one which
maximizes themaximizes the marginmargin between the twobetween the two
data sets.data sets.
15. To calculate the margin, two parallelTo calculate the margin, two parallel
hyperplanes are constructed, one onhyperplanes are constructed, one on
each side of the separatingeach side of the separating
hyperplane, which are "pushed uphyperplane, which are "pushed up
against" the two data sets.against" the two data sets.
In general the larger the margin theIn general the larger the margin the
better the generalization error of thebetter the generalization error of the
classifier.classifier.
16.
17. In the case of SVM, a data point is viewedIn the case of SVM, a data point is viewed
as aas a pp-dimensional vector and we can-dimensional vector and we can
separate such points with aseparate such points with a pp − 1-− 1-
dimensional hyperplane. This is called adimensional hyperplane. This is called a
linear classifier.linear classifier.
There are many hyperplanes that mightThere are many hyperplanes that might
classify the data. We pick the hyperplaneclassify the data. We pick the hyperplane
so that the distance from the hyperplane toso that the distance from the hyperplane to
the nearest data point is maximized.the nearest data point is maximized. such asuch a
linear classifier is known as alinear classifier is known as a maximum-maximum-
margin classifier.margin classifier.
18. A set of features that describes one case is calledA set of features that describes one case is called
aa vector.vector. The vectors near the hyper plane are theThe vectors near the hyper plane are the
support vectorssupport vectors.. The distance between the dashed
lines shown in the figure is called the margin.
19.
20. Formalization of SVMFormalization of SVM
We are given some training data, a set of points of theWe are given some training data, a set of points of the
formform
where thewhere the cici is either 1 or −1, indicating the class tois either 1 or −1, indicating the class to
which the point belongs. Each is awhich the point belongs. Each is a pp-dimensional-dimensional
real vector. We want to give the maximum-marginreal vector. We want to give the maximum-margin
hyperplane which divides the points havinghyperplane which divides the points having cici = 1 from= 1 from
those havingthose having cici = − 1. Any hyperplane can be written as= − 1. Any hyperplane can be written as
the set of points satisfyingthe set of points satisfying
21. The vector is a normal vector: it is perpendicular toThe vector is a normal vector: it is perpendicular to
the hyperplane.the hyperplane.
The parameter determines the offset of theThe parameter determines the offset of the
hyperplane from the origin along the normal vector .hyperplane from the origin along the normal vector .
We want to choose the andWe want to choose the and bb to maximize theto maximize the
margin.margin.
22. The hyperplanes can be described by the equationsThe hyperplanes can be described by the equations
andand
If the training data are linearly separable, we canIf the training data are linearly separable, we can
select the two hyperplanes of the margin in a wayselect the two hyperplanes of the margin in a way
that there are no points between them and then try tothat there are no points between them and then try to
maximize their distance.maximize their distance.
By using geometry, we find the distance betweenBy using geometry, we find the distance between
these two hyperplanes is , so we want tothese two hyperplanes is , so we want to
minimize . As we also have to prevent data pointsminimize . As we also have to prevent data points
falling into the margin, we add the followingfalling into the margin, we add the following
constraint:constraint:
23. Contd…….Contd…….
for eachfor each ii eithereither
for of the first class orfor of the first class or
for of the second class.for of the second class.
We can put this together to get the optimizationWe can put this together to get the optimization
problem: choose to minimize subject toproblem: choose to minimize subject to
24. Contd….Contd….
► The optimization problem presented is difficult toThe optimization problem presented is difficult to
solve because it depends on ||solve because it depends on ||ww||, the norm of||, the norm of
w'w', which involves a square root., which involves a square root.
► It is possible to alter the equation by substitutingIt is possible to alter the equation by substituting
||||ww|| with without changing the solution|| with without changing the solution
.This is a quadratic programming (QP).This is a quadratic programming (QP)
optimization problem. More clearly, minimizeoptimization problem. More clearly, minimize
, subject to, subject to
25. Experimental work and resultsExperimental work and results
► We have taken a database consisting of 20We have taken a database consisting of 20
different classes of images each classdifferent classes of images each class
consisting of 72 images.consisting of 72 images.
► The different classes of images that wereThe different classes of images that were
taken in the database are as shown below:taken in the database are as shown below:
27. Consider that the image of class that is shown below isConsider that the image of class that is shown below is
given as querygiven as query
And the images that are retrieved for theAnd the images that are retrieved for the
query if the first ten relevant images are retrieved are asquery if the first ten relevant images are retrieved are as
follows:follows:
28. The top eight images among the retrievedThe top eight images among the retrieved
images are relevant to the given query and theimages are relevant to the given query and the
last two are not.last two are not.
Thus each image of the database is given asThus each image of the database is given as
query and the number of relevant imagesquery and the number of relevant images
retrieved amongst the total images that fall intoretrieved amongst the total images that fall into
the zone of query is calculated.the zone of query is calculated.
29. ConclusionConclusion
• The classification is done based on theThe classification is done based on the
ratio of number of images that fall into theratio of number of images that fall into the
relevant category.relevant category.
• The final result is 64.985%.The final result is 64.985%.
• The number of relevant images retrievedThe number of relevant images retrieved
and the total number of images that falland the total number of images that fall
into a particular zone concerned to theinto a particular zone concerned to the
given query images from different classesgiven query images from different classes
of database are tabulated.of database are tabulated.
