Apidays New York 2024 - The value of a flexible API Management solution for O...
Gray Image Coloring Using Texture Similarity Measures
1. Gray Image Coloring Using Texture
Similarity Measures
by
E. Noura Abd El-Moez Semary
Thesis Submitted in accordance with the requirements of
The University of Monofiya for the degree of
Master of Computers and Information 1
( Information Technology )
2. Thesis summary on:
Gray Image Coloring Using Texture
Similarity Measures
Prof. Mohiy Prof. Nabil Dr.Waiel .S.
Supervised by:
.M.Hadhoud .A.Ismail Al-Kilani
Presented by
E. Noura Abd El-Moez Semary
For Master degree in Computers and Information
IT department, Faculty of Computers and information,
Menofia University
3. :ملخص رسالة بعنوان
تلوين الصور الرمادية بإستخدام معايير تشابه
النسجة
د. وائل شوقي أ.د. نبيل عبد الواحد أ.د. محي محمد
:تحت إشراف
الكيلني إسماعيل هدهود
:مقدم من
م . نورا عبد المعز السباعي سمري
للحصول على درجة الماجستير في الحاسبات والمعلومات
قسم تكنولوجيا المعلومات - كلية الحاسبات و المعلومات - جامعة المنوفية
4. Outlines
Outlines
Introduction Introduction
Automatic
Automatic coloring in the literature coloring in the
literature
TRICS ‘Texture Recognition based Image‘Texture
TRICS
Recognition
Coloring System’ based Image
Coloring
Results System’
Results
Conclusion Conclusion
Future work
Future work
4
5. Introduction
Gray image principles
Outlines
Introduction
Automatic
coloring in the
literature
TRICS ‘Texture
Recognition
based Image
Coloring
System’
Gray values Results
Conclusion
Future work
.
.
.
.
.
.
.
.
.
.
.
0
50
100
150
200
250
255
5
7. Introduction
Coloring Problem
There are two definitions to describe the gray value as an equation of
the three basic components of RGB color model (red, green, blue):
1: Intensity (most common used)
Gray = (Red + Green + Blue) /3
2: Luminance (NTSC standard for luminance)
Gray = (0.299 × Red) + (0.587 × Green) + (0.114 × Blue)
RGB Color R, G, B values Gray value Gray Color
87 ,150 ,100 128
147, 87, 149 128
149, 147, 87 128
THERE IS NO MATHEMATICAL
FORMULA
TO CONVERT FROM GRAY TO RGB 7
9. Introduction
Coloring Types
1 . Hand coloring
Adobe Photoshop
and Paintshop Pro
Layers
Changing Hue
BlackMagic, photo
colorization software,
version 2.8, 2003
9
10. Introduction
Coloring Types
2 . Semi automatic
coloring
Pseudocoloring is a
common example for
semi automatic
coloring technique
10
11. Introduction
Coloring Types
Outlines
3 . Automatic coloring Introduction
Automatic
i. Transformational coloring coloring in the
literature
ii. Matched image coloring TRICS ‘Texture
Recognition
iii. User selected coloring based Image
Coloring
System’
Results
Conclusion
Future work
11
12. Automatic coloring in the literature
1. Transformational Coloring
Outlines
A transformation function Tk is applied on
Introduction
Automatic
the intensity value of each pixel coloring in the
Ig(i,j)
literature
resulting in the chromatic value Ick(i,j) for
TRICS ‘Texture
Recognition
channel k based Image
Coloring
System’
Results
Ic k (i, j ) = Tk [ Ig (i, j )] Conclusion
Future work
12
13. Automatic coloring in the literature
1. Transformational Coloring
Al-Gindy et al * system.
× Results have unnatural look
* A. N. Al-Gindy, H. Al Ahmad, R. A. Abd Alhameed, M. S. Abou Naaj and P. S. Excell
’Frequency Domain Technique For Colouring Gray Level Images’ 2004 found in
www.abhath.org/html/modules/pnAbhath/download.php?fid=32 13
14. Automatic coloring in the literature
2. Matched image coloring
The most similar pixel color is transferred to the corresponding gray
one by the color transfer technique proposed by E.Reinhard*;
l α β
* Reinhard, E. Ashikhmin, M., Gooch B. And Shirley, P., Color Transfer between Images, IEEE Computer
Graphics and Applications, September/October 2001, 34-40 14
15. Automatic coloring in the literature
2. Matched image coloring
“Global matching procedure” of T. Welsh et al*
“Local color transfer” of Y. Tai et al**.
× All these algorithms fail, when different colored
regions have similar intensities
* T. Welsh, M. Ashikhmin, K. Mueller. “Transferring color to greyscale images.” In Proceedings of the
29th Annual Conference on Computer Graphics and interactive Techniques, pp 277–280, 2002
** Y. Tai, J. Jia, C. Tang ‘Local Color Transfer via Probabilistic Segmentation by Expectation-
maximization‘, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
(CVPR'05), Volume 1, pp. 747-754, 2005
15
16. Automatic coloring in the literature
2. Matched image coloring
Welsh et al proposed also another technique to improve
the coloring results when the matching results are not
satisfying. It was achieved by asking users to identify
and associate small rectangles, called “swatches” in both
the source and destination images to indicate how
certain key colors should be transferred
16
17. Automatic coloring in the literature
3. User selection coloring
Outlines
Introduction
Automatic
coloring in the
literature
TRICS ‘Texture
Recognition
based Image
User selection coloring gives high quality colors Coloring
× User dependent color quality System’
Results
× Time-consuming Conclusion
× Colorization must be fully recomputed for any slight change Future work
in the
initial marked pixels
* A. Levin, D. Lischinski, Y. Weiss. “Colorization using optimization.” ACM Transactions on Graphics,
Volume 23, Issue 3, pp.689–694, 2004 17
18. TRICS System
Research Objectives
Outlines
To simulate the human vision in coloring
Introduction
process Automatic
coloring in the
literature
To be fully automatic coloring system TRICS ‘Texture
Recognition
To spend so little execution time as possible
based Image
Coloring
as a basic requirement for video coloring.
System’
Results
Conclusion
Future work
18
19. TRICS System
Structure
Gray image
1 2 A
Segmentation
Features extraction Segmentation
(Joint, wavelets, laws,…) (Mean Shift, K-Mean, FCM,..)
Segmented image,
Clusters
3 4
Classification B
Features extraction Classification
(Co-occurrence, Tamura, Wavelets energies) (K-NN classifier,..)
Samples features
Class labels
Classes Hues Database
5 6 7 Coloring
C
Convert image to Set Hue, Saturation, and Convert to
HSV channels Brightness RGB
19
Colored image
20. TRICS System
Structure 1. Segmentation Stage
Feature extraction: (Pixel based )
1. pixel position
2. pixel intensity
3. texture features
wavelets coefficients
Laws kernels coefficients.
20
21. TRICS System
Structure 1. Segmentation Stage
1. Wavelets coefficients
× Quarter the image size.
Up sampling
Upper level construction
21
24. TRICS System
Structure 1. Segmentation Stage
Segmentation technique:
Mean Shift *
K-mean (Fast k-mean) **
Adaptive Fast k-mean
* D. Comaniciu and P. Meer. ‘Mean shift: A robust approach toward feature space
analysis.’ PAMI, 24(5):603–619, May 2002
** C.Elkan, ‘Using the triangle inequality to accelerate k-Means.’ In Proc. of ICML
2003. pp 147--153 24
25. Mean Shift Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
25
26. Mean Shift Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
26
27. Mean Shift Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
27
28. Mean Shift Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
28
29. Mean Shift Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
29
30. Mean Shift Region of
interest
Center of
mass
Mean Shift
vector
Objective : Find the densest region
30
31. Mean Shift Region of
interest
Center of
mass
Objective : Find the densest region
31
33. Fast (Accelerated) K-mean*
Lemma 1: Let p be a point and
let c1 and c2 be centers.
c1
If E(c1,c2) ≥ 2E(p,c1) then
E(p,c2) ≥ E(p,c1).
p
Lemma 2: Let p be a point and
let c1 and c2 be centers. Then
E(p,c2) ≥ max{0,E(p,c1) – E(c1,c2)} c2
* C.Elkan, ‘Using the triangle inequality to accelerate k-Means.’ In Proceedings of the 20th ICML,
Washington DC, 2003. pp 147--153 33
34. TRICS System
Structure 1. Segmentation Stage
2. Fast K-mean :
with spatial features
× structured segmentation
Increase no. clusters
k=3 k=9
34
35. TRICS System
Structure 1. Segmentation Stage
2. Fast K-mean :
without spatial features
× scattered regions of same cluster
disjoint region separation
1 1 1 2
2 3
1 4
3 5
before after
35
36. TRICS System
Structure 1. Segmentation Stage
2. Fast K-mean :
× Small regions (noise)
Small regions elimination
Original Before After
36
37. TRICS System
Structure 1. Segmentation Stage
3. Adaptive Fast K-mean :
Clusters number generation
Minimum region size estimation
Fully automatic segmentation technique
37
38. TRICS System
Structure 1. Segmentation Stage
a. Clusters number generation :
• CEC “Combined Estimation Criterion”*:
•If the VRC index, for k clusters, is smaller f n
than 98 of the VRC index, for k-1 clusters, WCSS = ∑∑ ( xij − mij ) 2
the CEC is not satisfied.
i =1 j =1
f n
•If the VRC index, for k clusters, is larger
than 102 of the VRC index for k-1 clusters, or TSS = ∑∑ ( xij − M i ) 2
if k=1, the CEC is satisfied. i =1 j =1
BCSS = TSS − WCSS
•If the VRC index, for clusters, is smaller than
102 but larger than 98 of the VRC index for (n − k ) BCSS
clusters, the CEC is satisfied only if TSS for VRC =
k-1 clusters is smaller than 70 of TSS for k (k − 1)WCSS
clusters.
* D.Charalampidis, T.Kasparis, ‘Wavelet-Based Rotational Invariant Roughness Features for
Texture Classification and Segmentation’. IEEE Transactions on Image Processing.Vol.11.No.8
38
August 2002
39. TRICS System
Structure 1. Segmentation Stage
Start k=1
Gray Image
Fast k-mean
k=k+1 Calculate CEC Segmented Image
Yes
CEC Satisfied?
No
Stop…
Clusters number =k 39
40. TRICS System
Structure 1. Segmentation Stage
b. Minimum region size estimation :
• Split the disjoint regions.
• Count all regions size.
• Sort regions size and calculate the step between them.
• Select the regions size of step more than the largest
image dimension.
• Consider the minimum region size.
40
41. TRICS System
Structure 1. Segmentation Stage
Original gray wavelets Laws
A- F- T- E- EM-
Image Features Clusters Regions
time time time time time
19 12 42 31 59
Wavelets 4 8
sec sec min sec sec
size = 170×256
M=500, 10
EM = 324 11 3 min 31 59
Laws 4 7
sec sec 36 sec sec
sec
A-time : adaptive fast k-mean time, F-time: fixed k fast k-mean time, T-time: traditional k-
mean , E-time: elimination time , EM-time : Elimination time with estimating minimum size 41
region
42. TRICS System
Structure Database set1
The training set consists of 32 classes of Brodatz texture
database
Each image has a size of 256 × 256. Each image was
mirrored horizontally and vertically to produce a 512 ×512
image.
The image is split into 16 images of size 128 ×128.
256 × 256 512 × 512 16 × 128 × 128
42
43. TRICS System
Structure Database set2
The training set consists of 9 classes ‘cloud, sky, sea, sand,
tree, grass, stone, water, and wood’
Each class has number of samples from 12 to 25 samples.
These samples are taken from real natural images as random
64x64 rectangles.
43
44. TRICS System
Structure Database
Database record:
Sample
Class (level1,level2)
58 Features (6 Moment statistics, 4 Co-ocurance measures, 3
Tamura, and “ 15 wavelets mean, 15 wavelets variance , 15
wavelets energy” for five levels wavelets decomposition)
Hue
44
45. TRICS System
Structure 2. Classification Stage
Feature extraction (Region based)
Rectangular region:
1. Maximum rectangle
2. 64 x 64 rectangle
× Arbitrary shape
Padding rectangle*
* Ying Liu, Xiaofang Zhou, Wei-Ying Ma, ‘Extracting Texture Features
from Arbitrary-shaped Regions for Image Retrieval‘. 2004 IEEE
International Conference on Multimedia and Expo., Taipei, Jun. 2004
45
46. TRICS System
Structure 2. Classification Stage
Feature extraction (Region based)
Region based features:
GLCM * measures (Energy, Entropy, Inertia, Homogeneity )
Tamura * (Coarseness, Contrast , Directionality’)
Wavelets coefficients for 5 levels
Mean and variance **
Energy ***
* P.Howarth, S.Ruger,: Evaluation of texture features for content-based image retrieval. In:
proceedings of the International Conference on Image and Video Retrieval, Springer-Verlag (2004)
326–324
** O. Commowick – C. Lenglet – C. Louchet, ‘Wavelet-Based Texture Classification and Retrieval’
2003 found in http://www.tsi.enst.fr/tsi/enseignement/ressources/mti/classif-textures/
*** Eka Aulia, ‘Hierarchical Indexing For Region Based Image Retrieval’, Master thesis of Science in 46
Industrial Engineering, Louisiana State University and Agricultural and Mechanical College, May 2005
47. TRICS System
Structure 2. Classification Stage
GLCM and Tamura
× Scale variant features, not suitable for natural
textures
47
48. TRICS System
Structure 2. Classification Stage
Wavelets mean and variance :
× Values were very scattered and the results
were not accurate for most cases.
Wavelets energies :
Classification accuracy of 92% using “leave-
one-out” (each (sub) image is classified one
by one so that other (sub) images serve as
the training data) method
48
49. TRICS System
Structure 2. Classification Stage
a. Classification technique
KNN classifier with (k=1,k=5,k=10,k=20)
Distance Metric is L2 “Euclidean distance”
1
2
2
E ( I , J ) = ∑ I (i ) − J (i )
i
k=5 gives accuracy up to 94% using “N-fold “ (the
collection of (sub) images is divided into N disjoint
sets, of which N-1 serve as training data in turn and
the Nth set is used for testing)
49
51. TRICS System
Structure 3. Coloring Stage
Color model conversion
HSV/HSB color model
Change in Saturation Change in Brightness Change in Hue
Hue = 0, Luminance=0.5 Hue = 0, Saturation = 1 Sat=1, Luminance=1
HSI/HLS color model
Change in Saturation Change in Luminance Change in Hue
Hue = 0, Luminance=0.5 Hue = 0, Saturation = 1 Sat=1, Luminance=1
51
59. Results and Conclusion
Results
Misclassified results
2 classification levels:
•if the KNN results in 5 classes “grass, sea, water, grass, sea”
•The traditional solution is the class of the grass.
•The 2 levels classification solution is sea.
•(Sea and water), (trees and grass), (sky and clouds) and (wood
and stone) are considered as one class in level one.
59
61. Results and Conclusion
Comparisons
Local color transfer
Outlines
Introduction
Automatic
coloring in the
literature
Global Image Matching TRICS ‘Texture
Recognition
based Image
Coloring
System’
Results
Conclusion
Future work
61
62. Results and Conclusion
Conclusion
Outlines
We proposed a new computer coloring technique
that simulates the human vision in this area.
Introduction
Automatic
coloring in the
The proposed coloring system is contributed for
literature
TRICS ‘Texture
coloring gray natural scenes. Recognition
based Image
Coloring
System’
The execution time of TRICS is minimized using
Results
Conclusion
Fast k-mean segmentation technique and the
Future work
results are enhanced by splitting the disjoint regions
and by eliminating small regions.
62
63. Results and Conclusion
Conclusion
Clusters number generation algorithm and the
minimum region size estimation algorithm
increase the professionalism of the system but
also increases the time of the execution. And by
using both of them TRICS becomes a fully
unsupervised intelligent recognition based
coloring system.
HSV coloring model is very suitable for our
system and the coloring results have good
natural look.
63
64. Results and Conclusion
Conclusion
Outlines
We consider our proposed system structure as
Introduction
an abstract structure for building any more Automatic
coloring in the
intelligent coloring systems for any other types of
literature
images TRICS ‘Texture
Recognition
based Image
Coloring
System’
Our proposed system results perform the other
Results
coloring systems. Conclusion
Future work
64
65. Future work
Gray image
Segmentation A Outlines
Features extraction Segmentation
(Joint, wavelets, laws,…) (Mean Shift, K-Mean, Introduction
FCM,..)
Automatic
Segmented image,
Clusters coloring in the
literature
Classification B TRICS ‘Texture
Features extraction Classification
(Co-occurrence, Tamura, Wavelets energies) (K-NN classifier,..) Recognition
based Image
Samples features Coloring
Class labels System’
Classes Hues Database
Results
Conclusion
Coloring
Future work
C
Convert image to Set Hue, Saturation, and Convert to
HSV channels Brightness RGB
65
Colored image
66. Future work
Intelligent System for Classifying the image
Gray image
Segmentation A
Features extraction Segmentation
(Joint, wavelets, laws,…) (Mean Shift, K-Mean,
FCM,..)
Segmented image,
Clusters
Classification B
Features extraction Classification
(Co-occurrence, Tamura, Wavelets energies) (K-NN classifier,..)
Samples features
Adaptive Learning Class labels SOFM
Classes Hues Database
Coloring
C
Convert image to Set Hue, Saturation, and Convert to
HSV channels Brightness RGB
66
Colored image
67. Future work
Segmentation and classification stages
are research areas and any improvement
will increase the accuracy of the system.
Using different types of features and
training set enables the system for
coloring images like manmade images,
indoors, and people photos .
67
68. List Of Publications
Noura A.Semary, Mohiy M. Hadhoud, W. S. El-Kilani, and Nabil A.
Ismail, “Texture Recognition Based Gray Image Coloring”, The
24th National Radio Science Conference (NRSC2007), pp. C22,
March 13-15, 2007, Faculty of Engineering, Ain-Shams Univ.,
Egypt.
68