Kristian Dokic, Domagoj Sulc and Dubravka Mandusic. Collecting big data in cinemas to improve recommendation systems - a model with three types of motion sensors. (ICDEc 2021)

1. Collecting big data in cinemas to
improve recommendation systems - a
model with three types of motion
sensors
Kristian Dokic, Domagoj Sulc, Dubravka Mandusic

2. Introduction
• recommendation system helps consumers to identify
attractive content and most video-on-demand services are
supported by some RS
• they can be divided into two types: personalised and non-
personalised systems
• Personalised systems use consumer profile data to suggest
video content to consumers,
• non-personalised systems propose video content based on
other data types.
• What about cinemas ?

3. RS in cinemas ?
• cinema operators have less opportunity to use
recommendation systems
• non-selective methods is to show advertisements
before the screening itself
• use a loyalty system that is typically available online or
by using a mobile application, where cinema
operators can have much higher precision from RS
• when cinema operators use loyalty systems, they also
know the place in the cinema where the user sat while
watching the movie
• this allows them to measure the user's body
movements while watching the film

4. Body movements when watching... ?
• The association of emotions and unconscious body and
face movements
• unconscious body movements of cinema viewers could
be recorded while watching a movie
• analyze and draw certain conclusions about cinema
viewer preferences
• collect a certain amount of data and then try to find
correlations
• Model for data collecting is proposed

5. Recommendation systems based on
conscious choices
• The authors try to achieve better results of
recommendation systems, but the data source they use
is based on the conscious choices of system users
• In this paper, a model is set up that assumes that certain
knowledge is also stored in the unconscious movements
of the user's body while watching a movie.
• How to get this data and how to process it is the essence
of this paper.

6. Involuntary body movement research
• Involuntary body movements in response to a stimulus
have been studied in psychology for quite some time.
• Montepare et al. studied the possibility of recognising
emotions based on body movements.
• Deal et al. found that there is relatively little research
dealing with the detection of emotions based on body
movements.
• Sogon and Izard explored gender differences in
emotion recognition based on body movements.

7. Body movement sensing and measuring
• Face is the most interesting to researchers
• Camera is a very commonly used device for measuring
body movement
• there are several other sensors for the same purpose
that have been described in the literature
(accelerometer, gyroscope, global positioning system,
magnetometer, electromyography, resistive flex and
pressure sensing textile and environmental context
sensors )

8. Acceptable sensors for body movement
measuring in a cinema
• Camera ? NO.
• Body attached ? NO.
• WHICH SENSORS ?
a) microwave radar sensor based on
the Doppler effect
b) passive infrared receiver sensor
c) ultrasonic distance sensor

9. Microwave radar sensors
• emit microwave signal and
compare the echo with the
transmitted signal and calculate
the object's distance based on
the Doppler effect
• device serves as a motion
detector and the output is a
logical zero or one, depending
on whether the movement is
registered

10. Passive infrared receiver sensor
• electronic sensor that measures
infrared light emitted from
objects around so it detects the
movement of people

11. Ultrasonic distance sensor
• ultrasonic distance sensor is
based on the principle of
echolocation
• The sensor emits a short
ultrasound signal, and registers
the echo of the emited signal
• distance of the object can be
calculated based on the elapsed
time.

12. Device development
• Sensors are connected to the
Arduino Uno development
board, to which the button is
also attached
• A sampling time of one second
was defined, and testing could
begin
• board collects data from all
three sensors and forward it
over a serial connection to a
computer

13. Location of the sensors and device
• the best place for the sensors is
on the left or right side of the
armchair in the cinema hall
• sensors are sufficiently
protected

14. Data collecting
• development board has the function of a signal concentrator
with three different sensors
• collected data were not always harmonized
• it was decided to generate a neural network that could,
based on the collected data, determine whether the cinema
spectator is moving on an armchair or not
• a button is connected to the Arduino Uno board, the values
of which are also sent to the computer
• neural networks belong to supervised learning, the button is
used to collect data on a person's movement by pressing a
button each time he or she moves. In this way, data were
collected for neural network training.

15. Data processing and results
• One thousand samples were collected in just over 16
minutes and computer received four numbers every
second
• These 1000 samples were used to train the neural
network
• Edge impulse service was used to train the network

16. Trained neural network
• The neural network consists of
input, output and two fully
connected layers with 30 neurons
• model is only 3 kB in size
• very high accuracy was achieved in
the viewer idle mode (98%)
• accuracy of predicting the viewer's
movement is 72%, while the neural
network will register 28% of
situations when the viewer moves
in an armchair as idle

17. Discussion and conclusion
• application of the proposed system in cinemas requires first of all the
installation of the system in cinema armchairs
• cinema spectators must have the choice of whether or not they want
data about their unconscious movements to be stored, and it is up to
cinemas to argue to cinema visitors the benefits of collecting data
using the system described
• proposed model represents only the first step in the use of big data in
cinemas
• we have reached a technological level in recent years that allows
cheap processing of large amounts of data and finding knowledge in
them