1.
Assessing the Influence on User
Experience of Web Interface
Interactions Across Different Devices
ERKKI SAARNIIT, TALLINN UNIVERSITY
JUNE 7TH, 2016
MASTER’S THESIS – FINAL DEFENCE

2.
Structure of the Presentation
1. Problem
2. Research questions
3. Hypothesis
4. Literature review
5. Research design
6. Study
7. Results
8. Analysis
9. Conclusion
10. Answers to the reviewer’s questions

3.
Problem
The problem arose from professional experience with web design:
Designers would like to create web-based solutions that
would work on different devices, but the user experience
may differ across the devices, which may complicate the
work of designers.

4.
Research Questions
The research sought answers to
how users perceive different web interactions
and
whether there are any differences in the perceptions of
interactions among laptops and smartphones.

5.
Hypotheses
Hypothesis 1. It is possible to design web interactions that provide
homogeneous user experience across different devices.
Hypothesis 2. User experience evaluation can provide instructions
for designers.

6.
Literature Review
Goal: to define the research
methodology.
Areas studied:
• User experience evaluation
• Creation of web interactions
• Web design characteristics
• Challenges faced when designing for
web

7.
Literature Review: UX Evaluation
Methods
Various UX evaluation methods were analysed in the literature review:
User Experience Questionnaire (UEQ), System Usability Scale (SUS), Post Study System
Usability Questionnaire (PSSUQ), AttrakDiff.
AttrakDiff was found to be the most relevant for the research:
1. It helps to assess the user’s feelings about an interaction with a
questionnaire, providing quantitativedata.
2. It helps designers to understand user experience on a detailed level (word
pair assessments), but also based on different groups of qualities
(pragmatic qualities, hedonic qualities, attractiveness).
3. It provides a proven and tested methodology used in a many researches.

8.
Research Design
Individual Situation
Product Time
Diversity in UX
Diversity in UX. Source based on Karapanos 2010.
Constant
Constant
Products A B C D:
2 designs on laptop , 2
designs on smartphone
Different participants
Experimental research design

9.
Research Design
• Population was limited to office workers in Estonian bigger cities, in age
groups 20-65 and who use computer and smartphone on a daily basis.
• Limitation to set a focus due to the research problem: the author had seen a problem with
web based interactions used by office workers.
• In case the hypotheses are valid, the same methodology can be applied to other groups.
• Data collection via AttrakDiff questionnaire:
Pragmatic qualities
Hedonic qualities -
individual
Hedonic qualities -
stimulation
Attractiveness

10.
Study: Stimuli
1. Preliminary survey was conducted to choose an interaction for the main study.
2. Grouping interaction was chosen because it was considered the most problematic
interaction.
3. Two interaction designs with the same task – arrange products into two groups.
“button to group” “drag and drop”
=

11.
Study: Procedure and Setting
1. A research system was created where interactions could be completed and afterwards
assessed using AttrakDiff questionnaire.
2. Research was conducted at the work places of study participants.
3. 3 devices were used:
◦ 1 smartphone and 1 laptop
for interactions
◦ 1 laptop for questionnaire

12.
Study: Results
• 17 study participants were involved (gender and age requirements + back-up).
• Each completed 4 interactions – 2 on laptop and 2 on smartphone – and filled
4 questionnaires with 28 word pair assessments.
• 1904 evaluations of Attrakdiff word pairs were done.
• All collected data was applicable and in high quality due to system’s validation
rules.
• Unplanned observations were made during the sessions.

13.
Analysis: Individual Analysis
In this participant’s case “drag and drop” interaction was assessed similarly across devices. “Button to
group” solution received rather different assessments compared between laptop and smartphone.
PQ HQ-I HQ-S ATT
pragmatic qualities hedonic qualities - identification hedonic qualities - stimulation attractiveness
-3-2-10123
Episode 1 ("button to group" / laptop) Episode 2 ("drag and drop" / laptop)
Episode 3 ("button to group" / mobile) Episode 4 ("drag and drop" / mobile)
−0.4 −0.2 0.0 0.2 0.4 0.6
−0.4−0.20.00.20.40.6
PC1
PC2
EP1
EP2
EP3
EP4
−2 0 2 4
−2024
PQ
HQ−IHQ−S
ATT
“drag and drop”
“button to group”
similar
across devices
different
across devices

14.
Analysis: Comparison
of Pragmatic and
Hedonic Qualities
superfluous too task-oriented
hedonic quality (HQ)
pragmatic quality (PQ)
too self-oriented self-oriented desired
task-orientedneutral
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
Episode 2
PQ;HQ; 2,0168; 1,0882 Episode 4
PQ;HQ; 2,4538; 1,0084
Episode 1
PQ;HQ; 0,9832; -0,3277
Episode 3
PQ;HQ; 1,0924; -0,4328
“button to group”
“button to
group”
“drag and
drop”
“drag and drop”
• A comparison of pragmatic
and hedonic qualities helps
designers to design towards,
for example, a more desired
or a more task-oriented
solution.
• “Drag and drop” design was
more desired, whereas
“button to group” was more
task-oriented.
confidence
intervals

15.
Analysis: Comparison of AttrakDiff Word Pairs
unruly manageable
isolating connective
unprofessional professional
tacky stylish
cheap premium
alienating integrating
separates me brings me closer
unpresentable presentable
conventional inventive
unimaginative creative
cautious bold
conservative innovative
dull captivating
undemanding challenging
ordinary novel
unpleasant pleasant
ugly attractive
disagreeable likeable
rejecting inviting
bad good
repelling appealing
discouraging motivating
HQ-IHQ-SATT
technical human
complicated simple
impractical practical
cumbersome straightforward
unpredictable predictable
confusing clearly structured
unruly manageable
isolating connective
unprofessional professional
tacky stylish
cheap premium
alienating integrating
separates me brings me closer
unpresentable presentable
conventional inventive
unimaginative creative
cautious bold
conservative innovative
dull captivating
PQHQ-IHQ-S
-3 -2 -1 0 1 2 3
“button to group” “drag and drop” “button to group” “drag and drop”

16.
Analysis: Comparison of AttrakDiff Groups
1. Assessments of “drag and drop” solution were more homogeneous than “button to group”.
2. Pragmatic qualities were assessed higher than hedonic qualities for all interactions.
3. “Drag and drop” received higher scores than “button to group”.
PQ HQ-I HQ-S ATT
pragmatic qualities hedonic qualities - identification hedonic qualities - stimulation attractiveness
-3-2-10123
Episode 1 ("button to group" / laptop) Episode 2 ("drag and drop" / laptop)
Episode 3 ("button to group" / mobile) Episode 4 ("drag and drop" / mobile)
−20 −10 0 10 20
−15−10−5051015
episodes' mapping
Dim 1
Dim2
EP1
EP2
EP3
EP4
“drag and drop”
“drag and drop”
“button to group”

17.
Conclusion
1. The design of web interactions has influence on user experience.
2. User experience is very individual.
3. An interaction with the same task, but with different design creates
different user experience.
4. Different interaction designs provide different user experience across
devices.
5. User experience qualities (pragmatic and hedonic qualities and
attractiveness) were homogeneous across devices in case of a “drag and
drop” solution.

18.
Hypothesis Evaluation
1. It is possible to design web interactions that provide
homogeneous user experience across different devices by using a
“drag and drop” grouping interaction.
2. User experience evaluation can provide instructions for designers
by using AttrakDiff model.

19.
Limitations and further studies
Population, devices and interactions were limited to a specific group. The
conclusions are not validated among designers.
Further studies:
• Different geographical locations, age groups, type of work, former user
experience can be studied.
• Devices (tablets, smart TVs, smartwatches).
• Other web interactions.
• Other types of grouping interactions.
The research conducted for the thesis can be applied to other similar studies.

20.
Questions from the reviewer
Siddharth Nakul Gulati

21.
Question 1. Only ACM was searched for literatures. There are many
journals such as MIS quarterly among others which might not be
indexed in ACM; then based on this is the literature review exhaustive?
1. ACM Digital Library + Google Scholar were used (as referred on page 18 of
the thesis, chapter 2.1).
2. MIS Quarterly is covered by Google Scholar search.
3. Google Scholar is a recognised source for scientific research. Example:
“Aesthetics of Interaction – A Literature Synthesis” by Lenz, Diefenbach and
Hassenzahl (2014).

22.
Question 2. On what basis were only 12 people chosen for the pilot
study? Why the figure “12” specifically? Could you also back this with
prior research as to why a certain figure was chosen!
1. 17 people were chosen for the study.
2. There were the following criteria set for the participant:
◦ 5 age groups (20-29, 30-39, 40-49, 50-59, 60+)
◦ 2 genders (male, female)
◦ 5x2=10 people as minimum had to be chosen to cover the variety of age and gender.
◦ Additional 7 people were chosen for back up in case the data quality is not sufficient with the initial
10 people or in case the sample does not provide significant proof at the confidence level of 95%.
3. The analysis of the sample had to provide significant proof at the confidence level of 95%.
The given sample provided it.
4. Generalisationto the population can be provided in case of a larger sample (covered in the
limitations chapter of the thesis). The goal of this research was to test if homogeneous or
different user experience can be created in different devices. Such sample size was sufficient
to test it.

23.
Question 3. Why does a participant who lives in a bigger city given
importance in a pilot study. In a small village in Estonia where someone
works from home, they could also use different devices such as laptops
and smartphones for the most part of the day?
1. No, such participants who live in a bigger city do not have bigger importance.
2. It was the author’s choice to have a focus in this study specifically for bigger cities, office
workers and people who use computer and smartphone on a daily basis.
3. All the limitations applied to the research are thoroughly described in the Participants (3.3)
and Limitations (7.2) chapters.

24.
Question 4. How do you account for ecological validity in your
research? Are your results empirically valid?
1. The results are empirically valid, considering the limitations applied in the
research.
2. The limitations applied to the research were the following:
◦ Population
◦ Devices
◦ Interactions
3. The limitations applied to the research are described in the Limitations
chapter (7.2) of the thesis.

25.
Thank you!