A report for drowsiness detection system

1. Driver’s Drowsiness Detection System Using
OpenCV and Python.
A Project Report (Project Work I)
Submitted in partial fulfillment of requirement of the
Degree of
BACHELOR OF TECHNOLOGY in
ELECTRONICS COMMUNICATION
BY
PurvanshJain
EN18EL301133
Underthe guidanceof
Ms. Divya Chouhan
Department of Electronics Engineering
Faculty of Engineering
MEDI-CAPS UNIVERSITY, INDORE- 453331
Aug - Dec 2021

2. Report Approval
The project work “Driver’s Drowsiness Detection System Using
OpenCV and Python” is hereby approved as a creditable study of an
engineering/computerapplication subjectcarried out and presentedin a
manner satisfactory to warrant its acceptance as a prerequisite for the
Degree for which it has been submitted.
It is to be understood that by this approval the undersigned does not
endorse or approve any statement made, opinion expressed, or
conclusiondrawn therein;but approvethe “Project Report”only for the
purposefor which it has been submitted.
Internal Examiner
Name: Ms. Divya Chouhan
Designation:AssistantProfessor
Affiliation:Medi-CapsUniversity

3. Declaration
I hereby declare that the project entitled “Driver’s Drowsiness
Detection System Using OpenCV And Python” submitted in partial
fulfillment for the award of the degree of Bachelor of Technology in
‘Electronics and Communication Department’ completed under the
supervision of Ms. Divya Chouhan, Assistant Professor in
Electronics Engineering Department Faculty of Engineering, Medi-
Caps University Indoreis an authenticwork.
Further, I declarethat the contentof this project work, in full or in parts,
has neither been taken from any other source nor has been submitted to
any otherInstituteorUniversityforthe award of any degreeor diploma.
Purvansh Jain
19 Nov 2021

4. Certificate
I, Ms. Divya Chouhan certify that the project entitled “Driver’s
Drowsiness Detection System Using OpenCV And Python”
submitted in partial fulfillment for the award of the degree of Bachelor
of Technology by PurvanshJain is the record carried out by him under
my/ourguidanceand that the work has not formed the basis of award of
any other degreeelsewhere.
PurvanshJain
EN18EL301133
Ms. Divya Chouhan
Electronics Engineering Department
Medi-Caps University, Indore
Dr. Ajay Kulkarni
Head of the Department
Electronics Engineering Department
Medi-Caps University, Indore

5. Acknowledgments
I would like to express my deepest gratitudeto HonourableChancellor,
Shri R C Mittal, who has provided me with every facility to
successfullycarry out this project, and my profoundindebtednessto Dr.
Dilip K. Patnaik Vice-Chancellor, and Dr. D.K. Panda Pro Vice-
Chancellor, Medi-Caps University, whose unfailing support and
enthusiasm have always boostedup my morale. I also thank, Dr. Suresh
Jain Dean, Faculty of Engineering, Medi-Caps University, for giving
me a chance to work on this project.I would also liketo thank my Head
of the Department Dr. Ajay Kulkarni for his continuous
encouragement for the bettermentof the project.
It is their help and support, due to which we became able to complete
the design and technical report. Without theirsupport,this report would
not have been possible.
Purvansh Jain
B.Tech. IV Year
Enrolment No: En18EL301133
Department of Electronics and Communication
Faculty of Engineering
Medi-Caps University, Indore

6. ABSTRACT
In recent years driver fatigue is one of the major causes of vehicle
accidents in the world. A direct way of measuring driver fatigue is
measuringthestateof thedriver i.e., drowsiness.So, it is very important
to detect the drowsiness of the driver to save life and property. This
project is aimed towards developing a prototype of a drowsiness
detection system. This system is a real-time system that captures image
continuously and measures thestateoftheeyeaccordingto thespecified
algorithm and gives a warning if required. Though there are several
methodsfor measuringthe drowsiness this approach is completelynon-
intrusive which does not affect the driver in any way, hence giving the
exact conditionofthedriver.Fordetection ofdrowsiness, theperclosure
value of the eye is considered. So, when the closure of the eye exceeds
a certain amount then the driver is identified to be sleepy. For
implementing this system several OpenCV libraries are used including
Dlib.

7. Table of Contents
Page No.
Report Approval ii
Declaration iii
Certificate iv
Acknowledgment v
Abstract vi
List of Figures vii
Chapter 1 Introduction 1
1.1 Introduction 1
1.2 Literature Review 2
1.3 Background of Study 2
1.4 Significance Of this Project 3
1.5 Problem Statement 3
1.6 Objectives 4
1.7 Relevancy 4
Chapter 2 Work Plan and Process 5
2.1 Methodology 5
2.2 Description of tools used in the project 7
Chapter 3 Results and Outputs 9
3.1 Experimental Results 9
Conclusion 10
References 11

8. List of Figures
Sr. No. Figure Page no.
1. Figure 1.1: Reference Image 14
2. Figure 2.1: Flow chart showing the entire process
of a drowsiness detection system
15
3. Figure 2.2: Landmarks on Closed and open eye 15
4. Figure 2.3: Landmarks as depicted by dlib facial
predictor
19
5. Figure 2.4: Facial landmarks associated with
mouth
20
6. Figure 3.1: System output with eyes open 25
7. Figure 3.2: System output while sleepy 26
8. Figure 3.3: System output while drowsy
Abbreviations
SVM Support Vector Machines
EEG Electroencephalography
ECG Electrocardiography
EAR Eye Aspect Ratio

9. Chapter-1
Introduction
1.1 Introduction
The attention position of driver degrades because of lower sleep, long nonstop
driving or any other medical condition like brain diseases etc. Several checks
on road accidents says that around 30 percent of accidents are caused by
fatigue of the motorist. When driver drives for further than normal period for
mortal also inordinate fatigue is caused and also results in frazzle which drives
the motorist to sleepy condition or loss of knowledge.
Drowsiness is a complex miracle which states that there's a drop in cautions
and conscious situations of the driver. Though there's no direct measure to
descry the drowsiness but several other indirect approaches can be used for
this purpose.
Figure 1.1: ReferenceImage

10. 1.2 LiteratureReview
JayD. Fuletra and Dulari Bosamiya (1) has bandied several drowsiness
discovery approaches, we concluded that different ways will be. Applicable
depending on the circumstances. Although EEG- grounded results are effective,
it isn't practicable for motorists to wear electrodes. The approach grounded on
Artificial Neural Networks is straightforward, but neurons is the very best
option, if we want a better outgrowth. One of the most popular strategies among
experimenters is image processing.
Vahid Kazemi and Sullivan Josephine, (6) in their paper bandied about. the 300
faces in the wild challenge which is aimed to examine different datasets of
images for facial milestones identification.
Tereza Soukupova and Jan Cech (7) in their paper enforced the facial landmark
discovery to descry eye blinks in real time through training their machine with
different datasets with the help of a SVM classifier.
1.3 Backgroundof Study
In India, there's an increase in the number of cases of business accidents .
Involving buses and heavy vehicles similar as motorcars, lorries, and
exchanges each time. One of the primary causes contributing to business
accidents is doziness and weariness. Driving in this situation can have
disastrous consequences because it impairs the motorist's judgement and
focus. Motorists can avoid falling asleep at the wheel if they make way similar
as carrying enough sleep before driving, drinking coffee, or stopping for a
break when drowsy. However, indeed when motorists are apprehensive that
they are fatigued, they constantly refuse to take one of these conduct and .
Continue driving. Thus, detecting doziness is important as one of the way to

11. help the road accidents. This design proposed that yawning and eyes
discovery is the egregious signs of fatigue and doziness.
1.4 SignificanceOf This Project
In India, doziness and weariness are the leading causes of business accidents.
As a result, Motorist Doziness Discovery Using Webcam is being enforced in
order to limit and reduce the number of accidents involving motorcars,
exchanges, and lorries. When an auto mobilist is in a drowsy state, it identifies
the pointers of frazzle and informs them.
1.5 Problem Statement
Current doziness discovery systems, similar as Electroencephalography (EEG)
and Electrocardiography (ECG), which descry brain frequency and measure
heart meter, independently, bear complex calculation and precious outfit that's
uncomfortable to wear while driving and isn't suitable for driving conditions.
A drowsiness detection system that uses a camera in front of the motorist is
more ideal for use, but the physical signals that indicate doziness must first be
linked in order to develop a dependable and accurate drowsiness detection
algorithm. During discovery of the eyes and mouth region, issues crop due to
lighting intensity and while the motorist tilts their face left or right.
As a result, the thing of this design is to review all once exploration and styles,
and also present a system for detecting doziness using videotape or webcam. It
examines the videotape images that have been captured and devises a system
that can examine each frame of the videotape.

12. 1.6 Objectives
The design aims to achieve the following
• The purpose of this study was to look at the physical goods of
prostration and doziness.
• To produce a system that detects prostration and doziness by closing
eyes and yawning.
1.7 Relevancyof the Project
This design is applicable to the perpetration since fatigue and doziness
motorists contribute to the chance of road accidents. Numerous researches
have been conducted to apply safe driving systems in order to reduce
road accidents. Detecting the motorist’s alertness and doziness is an effective
way to help road accidents. With this system, motorists who are drowsy will
be advised by an alarm to regulate knowledge, attention and attention of the
motorists. This will help to reduce the number of road accidents.
This design is an active content that's still being enhanced and bettered by
inquiries and can be applied in numerous areas similar as detecting the
attention-position of scholars in classrooms and lectures. This is also
applicable to the three author’s field of study since it requires the author to
apply and combine the knowledge of electronics, programming and
algorithms.

13. Chapter-2
Work Plan and Process
This chapter will discuss the approach used to achieve the project's objectives
and will take a closer look at how the project is implemented. In order to
complete this project, it will be necessary to analyse each stage. Each choice
and achievement of the method used in this project will be discussed in detail at
each level until the project is completed. The software used in this study is the
Anaconda IDE and PyCharm IDE. The algorithms utilised to detect the face,
eyes, and mouth area are OpenCV, Dlib.
2.1 Methodology
Figure 2.1: Flow chart showing entire process of drowsinessdetectionsystem

14. The proposed technique is primarily based on eye blinking and yawning of driver
which can be behavioural measures. The aim of this project is to detect closed
eyes or opened mouth, this is yawning and to alert the driver. This is done by
placing a camera or recording device in front of the driver and capturing real time
video continuously using OpenCV and dlib. The application is executed in python
and processing is done in laptop’s camera.
The work is mostly divided into three steps:
1. Face, eye and mouth detection
2. Eye closure detection
3. Open Mouth detection
A. Face, eye and mouth detection
In this step face is detected by the use of dlib library. Shape predictor that is
implemented in dlib library is used to find facial landmarks. The Predictor gives
68 landmark points that can be applied to localize regions of face such as eyes,
eyebrows, nose, ears and mouth. Figure 1 gives the facial landmarks that can be
detected via dlib. Therefore, by applying facial landmarks detection eyes and
mouth can be localized and detected.
B. Eye Closure Detection
Each eye is characterized by 6 coordinates as in figure 2.2. An equation called
Eye Aspect Ratio (EAR) which reflects the relation between width and height of
coordinators can be derived.
The distance between vertical eye landmarks is computed in numerator and those
of horizontal eye landmarks are calculated in the denominator using the formula
for Euclidean distance.
Eye Aspect Ratio = ||𝑝2−𝑝6||+||𝑝3−𝑝5||
2||𝑝1−𝑝4||
where p1,p4 are endpoints of an eye,p2 and p3 are upper eyelid points , p6 and p5
are lower eyelid points.
This ratio of eye landmark distances can be used to determine whether a person is
blinking or not.
Figure 2.2: LandmarksonClosedandopeneye

15. C. Open Mouth Detection
Yawning is characterized by wide opening of mouth. Facial landmarks can be
used to detect an open. mouth. Mouth is characterized with the help of 20
coordinates as shown in figure 2.3. Using these coordinates, the distance between
lip is calculated which is the difference between top lip and bottom lip and this
lip distance is used to determine whether the driver’s mouth is open. If the lip
distance is greater than a threshold amount the subject is determined to be
yawning and an alert is given accordingly.
Mouth Aspect Ratio =
||𝑝2−𝑝8||+||𝑝3−𝑝7||+||𝑝4−𝑝6||
2||𝑝1−𝑝5||
Figure2.4: Facial landmarks associated with mouth
Figure 2.3: Landmarksas depictedby dlibfacial predictor

16. 2.2 Descriptionof tool used in the project
Following optimized tools and image processing libraries are used by author for
implementation of presented algorithm.
Open CV:
OpenCV (Open-source Computer Vision) is the Swiss Army knife of computer
vision. It has a wide range of modules that can help us with a lot of computer vision
problems. But perhaps the most useful part of OpenCV is its architecture and
memory management. It provides you with a framework in which you can work
with images and video in any way you want, using OpenCV’s algorithms or your
own, without worrying about allocating and reallocating memory for your images.
Open CV libraries and functions are highly optimized and can be used for real time
image and video processing. OPENCV’s highly optimized image processing
function are used by author for real time image processing of live video feed from
camera.
DLib:
Dlib is a modern C++ toolkit containing machine learning algorithms and tools for
creating complex software in C++ to solve real world problems. It is used in both
industry and academia in a wide range of domains including robotics, embedded
devices, mobile phones, and large high performance computing environments.
Dlib's open-source licensing allows you to use it in any application, free of charge.
Open-Source Dib library is used by author for implementation of CNN (Neural
Networks). Highly optimized pre-learned facial shape predictor and detectors
functions are used by author for detection of facial landmarks. Facial landmarks
were further used for extracting eye coordinates.
Python: Python is an object-oriented programming language created by Guido
Rossum in 1989. It is ideally designed for rapid prototyping of complex

17. applications. It has interfaces to many Operating system calls and libraries and is
extensible to C or C++. Many large companies use the Python programming
language include NASA, Google, YouTube, BitTorrent, etc. Python is widely used
in Artificial Intelligence, Natural Language Generation, Neural Networks and other
advanced fields of Computer Science. Python had deep focus on code readability.
Python language is used by author due to his cross-platform compatibility as main
coding language for algorithm. OpenCV and Dlib libraries are integrated in python
interpreter for using readymade optimized functions.
PyCharm: Open-source community version of PyCharm platform is used as main
coding editor .

18. Chapter-3
Results
3.1 ExperimentalResults
The proposed method was initially implemented using a laptop with attached
webcam of 15 frames per second. The system was tested. Figure 3.2 shows
the output when the subject is yawning. The visual output contains eye and
mouth aspect ratio and the audio output is an alert message which include
the signal Sleepy. Facial landmarks are detected and displayed as shown in
figure 3.1. In figure 3.1 the eyes of subject are open and corresponding EAR
value, MAR value and status is displayed in turn an alert alarm was also
generated as audio output.
Figure 3.1 : System output witheyes open Figure 3.2: System output while sleepy
Figure 3.3: System output while drowsy

19. Conclusion
This project looks at how to detect tiredness in a driver in real time by looking at
eye closure and yawning. This technology has the advantage of detecting
tiredness early on and sounding an alarm before an accident occurs. The use of
OpenCV is considered to be more suitable for this application based on the design
of the proposed work because it meets the relevant requirements such as cost,
power, and size. Face, eye, and mouth are easily detected by this technology, and
these are captured using a webcam. The technology can detect whether the eyes
and mouth were open or closed during monitoring. A warning signal will be
issued if the eyes have been closed for an extended period of time or if yawning
is detected.

20. References
[1]Jay D. Fuletra and Dulari Bosamiya, “A Survey on Driver’s Drowsiness
Detection Techniques”, International Journal on Recent and Innovation
Trends in Computing and Computation, Volume: 1, Issue: 1
[2]M. Ramzan, H. U. Khan, S. M. Awan, A. Ismail, M. Ilyas and A. Mahmood,
"A Survey on State-of-the-Art Drowsiness Detection Techniques," in IEEE
Access, vol. 7, pp. 61904- 61919, 2019.
[3]Mohamad-Hoseyn Sigari, Muhammad-Reza Pourshahabi, Mohnsen Soryani
and Mahmood Fathy , “A Review on Driver Face Monitoring Systems for
Fatigue and Distraction Detection”, International Journal of Advanced
Science and Technology Vol.64.pp 73-100
[4]Bappaditya Mandal, Liyuan Li, Gang Sam Wang and JIe Lin, “Towards
Detection of Bus Driver Fatigue Base on Robust Visual Analysis of Eye
State”, IEEE Transactions on Intelligent Transportation Systems, vol 18, No.
3, March 2017
[5]Vahid Kazemi and Sullivan Josephine, “One Millisecond Face Alignment
with an Ensemble of Regression Trees”, 27th
IEEE Conference on Computer
Vision and Pattern Recognition, CVPR 2014, Columbus, United States, 23
June 2014 through 28 June 2014
[6]Christos Sagonas , Georgios Tzimiropoulos, Stefanos Zafeiriou and Maja
Pantic, “300 Faces in-the-Wild Challenge: The First Facial Landmark
Localization Challenge, IEEE International Conference on Computer Vision
Workshop, 2013
[7]Tereza Soukupova and Jan Cech , “Real-Time Eye Blink Detection using
Facial Landmarks”, 21st Computer Vision Winter Workshop, Luke Cehovin,
Rok Mandeljic, Vitomir Struc (eds.) Rimskke Toplice, Slovenia, February 3-
5 2016.
[8]https://www.freecodecamp.org/news/smilfie-autocapture-selfies-by-
detecting-a-smile-using-opencv-andpython-8c5cfb6ec197
[9] S. Pandey, “Study Tonight,” 12 August 2021. [Online]. Available:
https://www.studytonight.com/post/dlib-68-points-face-landmark-
detection-with-opencv-and-python. [Accessed 2021].
[10] D. Pandey, “Medium.com,” Analytics Vidhya, april 2021. [Online].
Available: https://medium.com/analytics-vidhya/eye-aspect-ratio-
ear-and-drowsiness-detector-using-dliba0b2c292d706. [Accessed
2021].