1. A Utility Model for Sensory Experience
Christian Timmerer, Benjamin Rainer, and Markus Waltl
Alpen-Adria-Universität Klagenfurt (AAU)  Faculty of Technical Sciences (TEWI)  Department of Information
Technology (ITEC)  Multimedia Communication (MMC)  Sensory Experience Lab (SELab)
http://research.timmerer.com  http://blog.timmerer.com 
http://selab.itec.aau.at/mailto:christian.timmerer@itec.uni-klu.ac.at
5th International Workshop on Quality of Multimedia Experience (QoMEX)
5th July, 2013
Acknowledgments. This work was supported in part by the EC in the context of the ALICANTE (FP7-ICT-248652),
SocialSensor (FP7-ICT-287975), and QUALINET (COST IC 1003) projects and partly performed in the Lakeside Labs research cluster at AAU.

2. Outline
• Introduction / Motivation / Goal
• Methodology and Subjective Evaluation
• Utility Model for Sensory Experience
• Instantiation and Validation
• Conclusions
July 5th, 2013 QoMEX 2013 2

3. Introduction
• Multimedia services enriched by sensory effects like
wind, vibration and light
• Previous studies showed
– Enhancement of QoE for genres like action, documentary,
commercial, and sports
– Can compensate for lower video quality (lower bitrate)
– Influence the users’ emotions and intensity
• Active emotions are decreased
July 5th, 2013 QoMEX 2013 3

4. Motivation / Goal
• Current QoE models focus on single modalities
(or simple combinations)
• In previous studies we focused on a holistic
approach
– What is the influence of single sensory effects?
– Which sensory effect has the highest impact?
• Find a utility model to estimate QoE based on the
individual effects
– Complementary to existing models for audio-visual
content without sensory effects
July 5th, 2013 QoMEX 2013 4

5. Methodology
• Subjective evaluation ➜ generic utility model ➜ instantiation and
validation (based on data from previous studies)
• 32 Participants
– 6 female and 26 male
– Age 20 – 47
– 90% were aged between 20 – 31
• Stimuli
– Four video sequences from our dataset
including the sensory effects
• Two sports sequences (Berrecloth
and Pastranas)
• Two action sequences (2012 and Tron)
– Length 10 seconds, resolution 720p
– Each video sequence was presented in
eight configurations
– One action video sequence (Tron) was
selected as training sequence
July 5th, 2013 QoMEX 2013 5
ID Configuration
1 No sensory effects
2 Light only
3 Wind only
4 Vibration only
5 Light and wind
6 Light and vibration
7 Wind and vibration
8 Light, wind, and vibration

6. Methodology (cont’d)
• Single Stimulus Continuous Quality Scale
– Slider with a continuous scale from 0 (very low) to 100 (very high) with an
initial position at 50 (medium)
• Test setup
– Introduction
– Pre-questionnaire
– Training phase
– Main evaluation
– Post-questionnaire
• Training phase with one video sequence in all eight configurations
– To eliminate or at least reduce the surprise effect
• Test configurations were presented in randomized order
July 5th, 2013 QoMEX 2013 6

7. Results of Subjective Evaluation
Pastranas Berrecloth
July 5th, 2013 QoMEX 2013 7
MOS & 95% CI

8. Major Findings
• Sensory effects enhance the QoE
• Some sensory effects have a higher impact on
the QoE than other single sensory effects
– Vibration compared to light and/or wind
• Highest QoE with all three sensory effects
present
July 5th, 2013 QoMEX 2013 8

9. A Utility Model for Sensory Experience
• Proposed utility model:
• The model represents single sensory effects as
binary variables
– Present or not
• The model should allow to estimate the QoE with
sensory effects from the QoE without sensory
effects
July 5th, 2013 QoMEX 2013 9

10. Instantiation of the Utility Model
• The instantiation takes light, wind, and
vibration into account:
• Model fitted to the data by Multivariate Linear
Regression (MLR) with least square estimator
by taking the assessed MOS into account
July 5th, 2013 QoMEX 2013 10

11. Validation of the Utility Model
July 5th, 2013 QoMEX 2013 11

12. Validation of the Utility Model (cont’d)
July 5th, 2013 QoMEX 2013 12

13. Validation of the Utility Model (cont’d)
• Analyzed the impact of the variables on the
model by the squared sample correlation
– Gives an estimate on the variability that the variables
introduce in the model
• The highest variability or impact in the model is
represented by the variable depicting the
vibration (73%)
– Strengthens the finding that vibration has the highest
impact on the QoE in terms of MOS points
July 5th, 2013 QoMEX 2013 13

14. Conclusions
• Influence of single sensory effects and their
combination provides insights on how single and
combined sensory effects influence the QoE
• Vibration effects have the highest influence on
the QoE compared to other single sensory effects
(or combinations)
• We presented a first utility model for sensory
experience + instantiation thereof with light,
wind, and vibration effects
July 5th, 2013 QoMEX 2013 14

15. Thank you for your attention
... questions, comments, etc. are welcome …
Ass.-Prof. Dipl.-Ing. Dr. Christian Timmerer
Klagenfurt University, Department of Information Technology (ITEC)
Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIA
christian.timmerer@itec.uni-klu.ac.at
http://research.timmerer.com/
Tel: +43/463/2700 3621 Fax: +43/463/2700 3699
© Copyright: Christian Timmerer
15July 5th, 2013 QoMEX 2013