Presentation done by Aleksander Groth and Daniel Haslwanter, during "Responsive website design" workshop, of the ENTER2015 eTourism conference.

1. ENTER 2015 Research Track Slide Number 1
Perceived Usability, Attractiveness
and Intuitiveness of Responsive
Mobile Tourism Websites.
Aleksander Groth, Daniel Haslwanter
Management Center Innsbruck, Austria
Department Management, Communication & IT (MCiT)
aleksander.groth@mci.edu, dan.haslwanter@mci4me.at
http://www.mci.edu

2. ENTER 2015 Research Track Slide Number 2
Agenda
• Theoretical background.
• Methodology.
• Results.
• Discussion.

3. ENTER 2015 Research Track Slide Number 3
THEORETICAL BACKGROUND.

4. ENTER 2015 Research Track Slide Number 4
Responsive Web Design (RWD) (I)
• Methodology introduced to help realizing the
vision of a "One Web" (Gardner, 2011).
• A RWD approach alters the layout of the website
based upon the viewport of the device,
transforming static websites into responsive,
adjustable, and fluid layouts, which are much
more flexible in handling elements and
automatically rearrange them accordingly
(Bohyun, 2013).

5. ENTER 2015 Research Track Slide Number 5
Responsive Web Design (RWD) (II)
• RWD effectively adjusts the content and layout to
the context of the device and ensures that users
have a better and richer viewing experience
(Gardner, 2011).

6. ENTER 2015 Research Track Slide Number 6
Usability & Mobile Devices (I)
• “The extent to which a product can be used by
specified users to achieve specified goals with
effectiveness, efficiency, and satisfaction in a
specified context of use.” - ISO 9241-11
• Usability therefore stands as a basic evaluation
criterion of a technical system (Brau & Sarodnick,
2006) - a very functional ‚way-of-doing-things‘.

7. ENTER 2015 Research Track Slide Number 7
Usability & Mobile Devices (II)
• Mobile devices are considered personal, portable,
and immediate. (Wessels et al., 2011)
• With the appearance of smartphones, an
increasing number of users are accessing the
mobile Internet via their phone on-the-go, leaving
more stationary and familiar settings, like at
home or at work (Church & Oliver, 2011).

8. ENTER 2015 Research Track Slide Number 8
Usability & Mobile Devices (III)
• Nielsen and Budiu (2013) compared conversion
rates when studying e-commerce websites,
defining them as “the percentage of visiting users
who end up taking a desired action, observing
differences, depending on the used device”.
• Results showed that desktop computers have a
3.5% rate compared to mobile phones with only
1.4%.

9. ENTER 2015 Research Track Slide Number 9
Usability & Mobile Devices (IV)
• Explanations by Nielsen and Budiu (2013):
– Mobile user experience must be horrible, as mobile
sales could be 2.5 times higher if mobile websites
would be as easy to use as desktop sites.
– Assumption that there is no commitment to invest in
mobile design because mobile users do not account for
very much business.

10. ENTER 2015 Research Track Slide Number 10
Attractiveness & Intuitiveness (I)
• An experience that is created when using a
system has also implications for certain non-
functional aspects within a user interaction
(Hassenzahl, Kort, Law, Roto, & Vermeeren,
2009).
• “A person's perceptions and responses that result
from the use or anticipated use of a product,
system or service.” - ISO 9241-210

11. ENTER 2015 Research Track Slide Number 11
Attractiveness & Intuitiveness (II)
• Studies show that the attractiveness of a product
does play an important role in perceiving usability
(Chawda, Craft, Cairns, Heesch, & Rüger, 2005)
• In mobile computing, attractiveness does have the
highest influence on usability ratings, followed by
effectiveness and efficiency (Quinn & Tran, 2010)
• This implies that an attractive phone could have a
high usability rating, even when scoring low on either
effectiveness or efficiency.

12. ENTER 2015 Research Track Slide Number 12
Attractiveness & Intuitiveness (III)
• Developers design interfaces and create
emotions, harvesting on already learned
interaction gestures (Tanimura & Ueno, 2013).
• Responsive design can be seen here as a natural
extension, acknowledging the change in user
behaviour.

13. ENTER 2015 Research Track Slide Number 13
Attractiveness & Intuitiveness (IV)
• Intuitive interaction is defined as a non-challenging
cognitive process and can be narrowed down to mainly
information-based activities within a specific context of a
task, goal, user, environment, and technical system
(Hurtienne, Mohs, Meyer, Kindsmüller, & Habakuk Israel,
2006).
• A technical system may be classified as intuitive,
whenever an either natural or non-conscious utilization
with (or without) a user’s pre-experience leads to an
effective interaction (Diefenbach & Ullrich, 2011).

14. ENTER 2015 Research Track Slide Number 14
METHODLOGY.

15. ENTER 2015 Research Track Slide Number 15
Research Hypotheses
The research hypotheses were:
•H1: There is a general difference in the perceived
usability of the two websites between desktop
computers and smartphones.
•H2: A stricter approach to responsive design will have
an effect on the perceived usability of the participants.
•H3: A stricter approach to responsive design will have
an effect on the user experience of the participants.

16. ENTER 2015 Research Track Slide Number 16
Websites
Website A Website B

17. ENTER 2015 Research Track Slide Number 17
Test setting (Rubin, 1994)

18. ENTER 2015 Research Track Slide Number 18
Applied Measures
• Perceived Usability
– System Usability Scale, (Brooke J., 1996)
• Perceived Satisfaction & Promotion
– Net Promoter Score, (Reichheld, 2003)
• Perceived Intuitiveness
– INTUI, (Ullrich, D., & Diefenbach, S., 2010)
• Perceived Attractiveness
– AttrakDiff2, (Hassenzahl, M., Burmester, M., Koller, F.,
2003)

19. ENTER 2015 Research Track Slide Number 19
Usability Study
• 20 Participants
(14 male and 6 female - age-group 16-29)
• Two websites – www.tirol.at (strict adherence to RWD)
and www.oetztal.com (minor adherence to RWD)
• Two sessions – one on smartphone and one on desktop
computer
• A/B testing, (Brau & Sarodnick, 2006; Sauro & Lewis,
2012)
• 10 tasks, five on each website version
• Two weeks break between the sessions

20. ENTER 2015 Research Track Slide Number 20
Methodology
Day 1 Day 2
2 weeks
Smart A
Smart B
Web A
Web B
Group 1
(10 persons)
Group 2
(10 persons)
Web A
Web B
Smart A
Smart B
Group 1
(10 persons)
Group 2
(10 persons)

21. ENTER 2015 Research Track Slide Number 21
Tasks overview
• Within the experiment, participants had to
accomplish five tasks:
– three information-seeking and
– two function-based.
– All five were classified according to their levels of difficulty
(easy, medium, difficult) and the degree of scrolling (easy,
medium, heavy).
• None of the participants had any previous
experiences with the selected websites (Raptis et al.,
2013).

22. ENTER 2015 Research Track Slide Number 22
Tasks in detail
• Task 1: Subscribe to the newsletter of the website (easy, light scrolling)
• Task 2: Inform yourself about the Aqua Dome. Please note down the address
and phone number. (easy, light scrolling)
• Task 3: Inform yourself about the Hiking Tours in Tirol - “Adlerweg”. / Inform
yourself about the Hiking Tours in Ötztal - “Ötztal-Trek”. Please note down,
how much elevation / how many kilometres the tour comprises. (medium,
light scrolling)
• Task 4: Inform yourself about the National Parks. What is the duration in
hours of the hiking tour to the Trelebitschsee / Frischmannhütte in the
National Park “Hohe Tauern”? (difficult, medium scrolling)
• Task 5: Please book a vacation using your own criteria on the website, using a
budget of 1500€. Define your trip first using the following attributes: Date of
Arrival/Departure, City/Village, Category, and Number of adults/children.
(difficult, heavy scrolling)

23. ENTER 2015 Research Track Slide Number 23
RESULTS.

24. ENTER 2015 Research Track Slide Number 24
Results – Perceived Usability SUS
Figure 1: Figure from a comparison of the adjective ratings, acceptability scores, and school grading scales, in relation to
the average SUS score. (Bangor et al., 2009, p. 121)
Website version Geometric
mean
Standard
deviation
Lower
bound
(95 %)
Upper
bound
(95 %)
Website A - Smartphone 64.06 19.97 58.03 76.72
Website A - Desktop Computer 73.58 15.15 68.27 82.48
Website B - Smartphone 62.91 19.29 56.67 75.03
Website B - Desktop Computer 77.35 15.21 71.80 85.95

25. ENTER 2015 Research Track Slide Number 25
Perceived Satisfaction &
Promotion
Net Promotor Score (NPS)
Website Version NPS SD Mean (95 %)
Website A - Smartphone -40 2.52 6.0
Website A - Desktop Computer -5 1.94 7.1
Website B - Smartphone -45 2.82 5.4
Website B - Desktop Computer -10 1.72 7.3

26. ENTER 2015 Research Track Slide Number 26
Results – Perceived Intuitiveness (I)
INTUI

27. ENTER 2015 Research Track Slide Number 27
Results – Perceived Intuitiveness (II)

28. ENTER 2015 Research Track Slide Number 28
AttrakDiff2
Results - Perceived
Attractiveness (I)

29. ENTER 2015 Research Track Slide Number 29
AttrakDiff2
Results - Perceived
Attractiveness (II)

30. ENTER 2015 Research Track Slide Number 30
H1
• Measure:
– Website version on perceived usability (SUS)
– Website version on effortlessness (INTUI)
• Post-hoc (Fischer LSD): Significant differences between Website B-
Smart and Website B-Desktop (p=0.022), but none between the
versions of website A (p=0.158).
• Post-hoc (Bonferoni): Significant differences between the Website B-
Smart and Website B-Desktop (p=0.006), but none between the
versions of website A (p=1.000).
• H1 rejected

31. ENTER 2015 Research Track Slide Number 31
H2
• Measure:
– Website version on perceived usability (SUS)
– Website version on effortlessness (INTUI)
• Post-hoc (Fischer LSD): No significant differences between
the smartphone versions of A and B (p=0.829).
• Post-hoc (Bonferoni): No significant differences between
the smartphone versions of A and B (p=1.000).
• H2 rejected.

32. ENTER 2015 Research Track Slide Number 32
H3
• Measure:
– Website version on overall Intuitiveness (INTUI)
– Website version on magical experience (INTUI)
• Kruskal-Wallis for INT: No significant differences between
the versions were found (p=0.256).
• Kruskal-Wallis for X: No significant differences between
the versions were found (p=0.065).
• H3 rejected

33. ENTER 2015 Research Track Slide Number 33
DISCUSSION.

34. ENTER 2015 Research Track Slide Number 34
Summary of results

35. ENTER 2015 Research Track Slide Number 35
Conclusion (I)
• Strong reductions on visual features can have
negative consequences for the user experience and
may fail to create emotion.
• On the other hand, reductions can lead to
improved usability and increased overall
satisfaction.
• RWD is a possible way to enhance perceived usability,
but a noticeable trade-off towards being too
pragmatic (or boring) is likely to be created and
smartphone users will be negatively affected by it.

36. ENTER 2015 Research Track Slide Number 36
Conclusion (II)
• The increasingly growing mobile user group needs
to be addressed with greatest care, employing a
user-centred design process that relies on
established models and guidelines.
• The challenge of successfully implementing a
mobile website will be to find an adequate
balance between aesthetical aspects and efforts
to optimize the usability and compatibility
according to a device’s requirements.

37. ENTER 2015 Research Track Slide Number 37
Thank you!
Aleksander Groth & Daniel Haslwanter
Department Management, Communication & IT
Management Center Innsbruck
Universitätsstrasse 15, 6020 Innsbruck, Austria
Phone: +43 512 2070 -3523, Fax: -3599
www.mci.edu/mcit