rapport de stage iot

1. Ministère de l’Enseignement
Supérieur et de la Recherche
Scientifique
Université de Carthage
Ecole Polytechnique de Tunisie
IjJ.Ë@ ð úÍAªË@ ÑJʪJË@ èP@Pð
ùÒʪË@
h
. A£Q
¯
éªÓAg
.
HAJJ
®JÊË
éJ‚ñJË@
éƒPYÖÏ@
Engineering Internship Report
Classification and detection of skin types using computer
vision
July 1st
to August 30th
2022
Hosting organization: Vistasy clinic
Elaborated by: Nassim Hammami
Third year engineering student at Ecole Polytechnique de Tunisie
Supervised by: Dr. Ahmed Badaoui Badis
Computer Vision Engineer and co-founder of Vistasy clinic
Academic year
2022-2023
Rue Elkhawarezmi BP 743 La Marsa 2078
Tel: 71 774 611 – 71 774 699 Fax: 71 748 843
Site Web: www.ept.rnu.tn
2078 úæ QÖÏ@ 743 H
. .  ú×PP@ñmÌ'@ i
. îE
71 748 843 :»A®Ë@ 71 774 699 - 71 774 611 :­KAêË@
www.ept.rnu.tn :H
. @ñË@ ©
¯ñÓ

2. Chapter 1
Project framework
1.1 Introduction
In this chapter, we introduce the general framework of this project by explain-
ing the main existing challenges and problems that have pushed to find an engineering
solution. also we are going to define some dermatological and technical notions which we
will often talk about in my project.
1.2 Startup presentation
VISTASY is a Startup Providing High Quality Simulations for different fields.
The simulations quality is based on a combination of advanced cutting edge technology
in both Artificial Intelligence (Computer Vision) and Embedded Systems, Allowing us to
provides during consultation a High Quality Simulation without using the Standard 3D
reconstructions.
1.3 Problem Statement
On the one hand, smooth, luminous and healthy skin is everyone’s dream to
maintain self-confidence. Good skin reflects that the owner is very concerned about their
personal health, as skin health is a useful marker of a person’s current state of health.
On the other hand, cosmetic products are a double-edged sword. Indeed, to
obtain an effective result with these products, it is necessary to use the product adapted
to our type of skin.
3

3. Figure 1.5: The Local Binary Gray Level Co-occurrence Matrix concept
1.5.4 Support Vector Machine
The objective of the support vector machine algorithm is to find a hyperplane
in an N-dimensional space(N:the number of features) that distinctly classifies the data
points. To separate the two classes of data points, there are many possible hyperplanes
that could be chosen. Our objective is to find a plane that has the maximum margin,
i.e the maximum distance between data points of both classes. Maximizing the margin
distance provides some reinforcement so that future data points can be classified with
more confidence.
Figure 1.6: Support Vector in Two dimensions
Nassim Hammami 8 Engineering internship report

4. 1.5.5 Skin types
1.5.5.1 Normal skin
A well-balanced complexion is referred to as normal skin. This skin type does
not suffer from excessive dryness or oiliness. Acne, uneven skin tone, and large pores
are also not issues. This type of skin, also known as eudermic, indicates good blood
circulation and consistent sebum production.
Eudermic: the scientific term for healthy skin.
Result: Normal skin won’t feel tight or oily and it will have few imperfections.
Normal skin is characterized by:
• Limited sensitivity
• Few blemishes or imperfections
• Small pores
• Radiance and good circulation
Figure 1.7: Examples of normal skin
Nassim Hammami 9 Engineering internship report

5. 1.5.5.5 Sensitive skin
Sensitive skin is inherited. Thinner, more delicate skin and blood vessels close
to the surface are common characteristics, which means you may get red more frequently.
Sensitive skin is a common condition that causes skin reactions such as redness
and itching. Most people with sensitive skin experience itching, burning, and stinging in
patches of skin on a regular or irregular basis. Sensitive skin is easily treated and may
necessitate a visit to the dermatologist.
The symptoms of sensitive skin include the following:
• Redness and irritation
• Uneven texture
• Itchiness
Figure 1.12: Examples of sensitive skin
Nassim Hammami 13 Engineering internship report

6. Chapter 2
First approach: Data mining
approach
2.1 Introduction
In this approach, we built an intelligent system capable of classifying facial skin
types using the concept of data mining.
This approach uses discrete wavelet transform (DWT), gray level co-occurrence
matrix (GLCM), local binary pattern (LBP) and binary local gray level co-occurrence
matrix (LBGLCM) to extract the features contained in the face image and uses a method
2.2 Data Mining
2.2.1 Description of the datasets
The basic dataset that we have is composed of 5 classes, which represents each
skin type. For each folder, we have a number of photos between 15 and 20.The number
of images per skin type is as follows:
• Normal: 20
• Oily : 20
• Dry : 20
• Combination : 20
• Sensitive : 16
14
we developed as a classifier to determine facial skin type.

7. In an effort to get a more accurate model, we seek to focus only on the region
of interest in the images, which is why we created other datasets.
2.2.1.1 face segmentation dataset
In this dataset, we transformed the image into two regions, a black region which
represents the non-facial region and the colored region which represents the facial region.
To obtain this result we used the landmarks provided by the Mediapipe library.
Figure 2.1: Face landmarks given by mediapipe library
Figure 2.2: Example of face segmentation
Nassim Hammami 15 Engineering internship report

8. Figure 2.5: Examples of results from Algorithm 1
Algorithm 2: GLCM features significance determination
for Skin in ( Skins ) do
; // skin:Normal, Oily, Dry ... //
for d in ( Distances ) do
for θ in ( Angles ) do
S ← 0;
for Image in ( Skin ) do
GLCM ← GLCM(Image, d, θ)
Contrast ← GLCM.Contrast; Correlation ← GLCM.Correlation
Homogeneity ← GLCM.Homogeneity
Dissimilarity ← GLCM.Dissimilarity
Energy ← GLCM.Energy
# − − − − − feature_best_parameters − − − −−
MinDistancefeature ← min(
for feature in [Features] do
( [
Normal(featureList(d, θ)) − feature|,
|Dry(featureList(d, θ)) − feature|,
|Oily(featureList(d, θ)) − feature|,
|Combination(featureList(d, θ)) − feature|,
|Sensitive(featureList(d, θ)) − feature|
] )
Right_skin=Skin(MinDistancefeature)
if skin = Right_skin then
S ← S + 1
end
end
significance = S / N
featuresig[(d, θ)] ← significance
end
end
end
Skins[skin] ← featuresig
end
Nassim Hammami 19 Engineering internship report

9. 2.3 Results of the first approach
The model is evaluated using the accuracy. After the testing we obtain the
following results.
2.3.1 Normal skin
Scores for each image
’oily’: 0, ’normal’: 10, ’dry’: 4, ’combination’: 0, ’sensitive’: 8
’oily’: 1, ’normal’: 10, ’dry’: 4, ’combination’: 0, ’sensitive’: 5
’oily’: 0, ’normal’: 9, ’dry’: 3, ’combination’: 5, ’sensitive’: 3
’oily’: 0, ’normal’: 10, ’dry’: 2, ’combination’: 2, ’sensitive’: 6
’oily’: 4, ’normal’: 6, ’dry’: 10, ’combination’: 1, ’sensitive’: 2
’oily’: 0, ’normal’: 10, ’dry’: 1, ’combination’: 3, ’sensitive’: 5
’oily’: 2, ’normal’: 10, ’dry’: 2, ’combination’: 1, ’sensitive’: 5
’oily’: 1, ’normal’: 11, ’dry’: 1, ’combination’: 6, ’sensitive’: 1
’oily’: 3, ’normal’: 9, ’dry’: 1, ’combination’: 2, ’sensitive’: 2
’oily’: 1, ’normal’: 11, ’dry’: 1, ’combination’: 5, ’sensitive’: 1
’oily’: 1, ’normal’: 7, ’dry’: 4, ’combination’: 0, ’sensitive’: 3
’oily’: 2, ’normal’: 3, ’dry’: 6, ’combination’: 0, ’sensitive’: 8
’oily’: 0, ’normal’: 10, ’dry’: 1, ’combination’: 3, ’sensitive’: 6
’oily’: 2, ’normal’: 4, ’dry’: 5, ’combination’: 1, ’sensitive’: 5
’oily’: 0, ’normal’: 8, ’dry’: 2, ’combination’: 1, ’sensitive’: 5
’oily’: 0, ’normal’: 7, ’dry’: 2, ’combination’: 1, ’sensitive’: 2
’oily’: 1, ’normal’: 11, ’dry’: 3, ’combination’: 3, ’sensitive’: 1
’oily’: 1, ’normal’: 11, ’dry’: 1, ’combination’: 5, ’sensitive’: 1
’oily’: 1, ’normal’: 10, ’dry’: 1, ’combination’: 0, ’sensitive’: 9
Final accuracy : 85%
Nassim Hammami 25 Engineering internship report

10. Chapter 3
Second approach: Deep learning
approach
3.1 Introduction
In this approach we developed a deep learning model based on convolutional
neural network. we used the fastai library developed by M.Jeremy Howard.
3.2 Dataset description
In this approach we worked with a database that have a number of photos
between 900 and 1000 image per skin type.
3.3 the stages of classification
3.3.1 Creating datablocks and dataloaders
DataBlocks are blueprints on how to assemble data.Think of them as a pipeline
about how to manage your entire dataflow.
DataLoader are extensions of Pytorch’s DataLoader Class but with more func-
tionality and flexibility. They help as to investigate, clean, change and prepare our data
before, during and after building our deep learning model.we always need a DataLoader
to build your fastai model.This are the steps and the tricks that we used to create our
dataloaders:
1. A trick often used in image classification is data augmentation. This means
one creates more data by manipulating existing data. For images this means that the
31

11. Figure 3.2: Loading the model
3.4.2 Find the optimal learning rate
In this step, the goal is to find the best learning rate that a) avoids overshooting
during stochastic gradient descent, and b) converges as a fast as possible. We are looking
for the spot in the graph where the line has the steepest slope. This means that the model
is improving the most with that learning rate.
Figure 3.3: finding the learning rate of the model
3.5 Fitting the learner
Model fitting is a measure of how well a machine learning model generalizes to
similar data to that on which it was trained.
Nassim Hammami 33 Engineering internship report

12. Figure 3.4: Fitting the model
3.6 Evaluate the results
In this step we see which classes our classifier is having the most identifying
correctly. We can have a look at the confusion matrix. It tells us which classes the model
mixes up.
Figure 3.5: The confusion matrix
Nassim Hammami 34 Engineering internship report

13. Bibliography
[B1] Sekolah Tinggi Bali, classification of facial skin type using discrete wavelet
transform, contrast, local binary pattern and support vector machine.
[B2] Dr. Punal, M. Arabi, Mrs. Gayathri Joshi, Rohith. N. Reddy, Anusha
S R, Categorising Normal Skin, Oily Skin and Dry Skin using 4-Connectivity and 8-
Connectivity Region Properties
[B3] Koushik Chandrasekaran, “2D-Discrete Wavelet Transformation and its ap-
plications in Digital Image Processing using MATLAB”,
[B4] Pawanpreet Kaur Harra, Deepak Aggarwal, Hybrid Approach for Face
Recognition Using DWT and LBP.
[B5] Ridha Ilyas Bendjillali, Mohammed Beladgham and Khaled Merit, Im-
proved Facial Expression Recognition Based on DWT Feature for Deep CNN.
[B6] Rafael C. Gonzalez, Richard E. Woods, Digital Image Processing, Fourth
edition.
[B7] Andreas Savakis, T. Tanaka, “Discrete wavelet transform core for image
processing applications”.
37